Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

由于任务拥塞,导致定时登录任务没有机会不执行 #42

Closed
ResolveWang opened this issue Aug 7, 2017 · 0 comments
Closed
Labels

Comments

@ResolveWang
Copy link
Member

本来项目中配置的是20小时定时登录一次,但是由于所有任务都会抢占资源,所以导致定时登录任务可能不会执行,从而系统错误判定为无cookie可用而停掉项目。

一个理想的解决方法是任务有优先级,当任务堆积的时候,让登录任务优先执行。但事实上使用celery+redis的方案,celery官方并没有给定优先级任务,所以只有用别的方法来解决。目前我能想到的有两种,欢迎大家提供更好的方法。

1.让登录任务单独在一个或者多个节点运行,这样就没有其他任务和登录任务抢占资源。

2.让想执行登录和抓取任务的节点,同时起两个worker,一个执行登录任务,另外一个执行抓取任务。

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

1 participant