Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

跑了2个小时自己挂了 #2

Closed
netaxcess opened this issue May 10, 2017 · 3 comments
Closed

跑了2个小时自己挂了 #2

netaxcess opened this issue May 10, 2017 · 3 comments

Comments

@netaxcess
Copy link

2017/05/10 10:57:55 All getters finished.
2017/05/10 10:59:28 Chan: 144, IP: 275
2017/05/10 11:01:28 Chan: 0, IP: 277
panic: runtime error: index out of range

goroutine 8554 [running]:
panic(0x6daf40, 0xc042006060)
C:/Go/src/runtime/panic.go:500 +0x1af
github.com/henson/ProxyPool/getter.IP66(0x0, 0x0, 0x0)
E:/golang/src/github.com/henson/ProxyPool/getter/66ip.go:20 +0x759
main.run.func1(0xc042172060, 0xc04222a4b0, 0x768160)
E:/golang/src/github.com/henson/ProxyPool/main.go:66 +0x32
created by main.run
E:/golang/src/github.com/henson/ProxyPool/main.go:71 +0x16b
exit status 2

@henson
Copy link
Owner

henson commented May 10, 2017

因为限制了mgo数据库连接数,所以数据库并行处理效率变慢,队列容量激增,超出上限

ipChan := make(chan *models.IP, 1000)

把1000这个设置调高,比如2000就可以了

@henson henson closed this as completed May 10, 2017
@netaxcess
Copy link
Author

达到2000的时候。也会挂吧。好像报的是henson/ProxyPool/getter/66ip.go:20 +0x759
是不是body = strings.Split(body, "c.js'></script>")[1] ,当没采集到IP的时候。这个应该溢出了吧。

@henson
Copy link
Owner

henson commented May 10, 2017

不是,是超出队列容量了

panic: runtime error: index out of range

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants