New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

任务重复 #10

Closed

yun17 opened this issue May 20, 2017 · 1 comment

Collaborator

yun17 commented May 20, 2017 •

edited

您在文档里提到，开两个beat会导致任务重复。
而在上一次beat开启的任务还没完成的情况下，又重新放入任务会导致任务。这在任务特别多而性能不是很好的情况下一定会出现。

建议：将wb_data.set_weibo_comment_crawled(weibo_data.weibo_id) 这行代码加入到excute_comment_task方法中（已经提了一个pr）。
#11

ps:代码中没有判断该微博的转发或评论是否爬取完毕

The text was updated successfully, but these errors were encountered:

Collaborator Author

yun17 commented May 20, 2017

已提交pr

yun17 closed this as completed

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment