Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

批量化的长时间扫描，会出现卡住不动的情况。 #30

Closed
vinaquino opened this issue Apr 13, 2023 · 2 comments
Closed

批量化的长时间扫描，会出现卡住不动的情况。 #30

vinaquino opened this issue Apr 13, 2023 · 2 comments

Comments

@vinaquino
Copy link

目标几十万域名，扫着扫着就卡住不动了，我也在定位中，大家也帮看下。
@vinaquino vinaquino reopened this Apr 14, 2023
@vinaquino
Copy link
Author

2023-04-14 09 31 28
因url量有点大，我是直接从redis每次取100条url，把线程对象提到循环外面。
经常会在跑大概半小时后，出现卡住不动的情况。

@vinaquino
Copy link
Author

2023-04-16 16 20 19
我把整个流程大改后，去除了future阻塞，采用固定线程处理，仍然会出现卡死的情况。个人猜测也有可能是个别处理流程出现时间过长的情况，最后还是改回作者的代码继续使用，并且添加reture超时时间result(timeout=number)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@vinaquino