Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

线程没有正确结束? #41

Closed
yumexupanic opened this issue Aug 25, 2017 · 4 comments
Closed

线程没有正确结束? #41

yumexupanic opened this issue Aug 25, 2017 · 4 comments

Comments

@yumexupanic
Copy link

yumexupanic commented Aug 25, 2017

链接大概有5-10k左右 如下:

List<HttpRequest> requests;
 GeccoEngine.create()
                .pipelineFactory(springPipelineFactory)
                .classpath("xxx")
                .interval(2000)
                .thread(thread)
                .start(requests).start();

运行了一段时间过后 项目报错

java.lang.OutOfMemoryError: Unable to create new native thread

观察了下系统线程

已经跑完的链接 run 方法并没有被结束 ,请问下爬虫跑完为什么没有让线程停止呢? 谢谢了。

@xtuhcy
Copy link
Owner

xtuhcy commented Aug 25, 2017

start()方式启动engine线程不会释放的,会一直监听队列。run()方式启动engine,如果队列里没有需要抓取的内容就会停止线程

@yumexupanic
Copy link
Author

也就是说 run() 方式爬虫跑完一个链接就停止一个线程, 而 start() 跑完并不会释放 是这个意思吗?

@yumexupanic
Copy link
Author

用 start() 方式不会自动释放 能不能手动进行释放? 这样的话 跑定时任务一直不结束线程 第一次还行 后面几次都直接累积了 一直到系统最大线程 服务又挂掉

@xtuhcy
Copy link
Owner

xtuhcy commented Aug 25, 2017

定时任务用run()的方式

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants