Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Stop facade being bottlenecked by core worker processes #2329

Merged
merged 3 commits into from Apr 20, 2023

Conversation

IsaacMilarky
Copy link
Contributor

@IsaacMilarky IsaacMilarky commented Apr 19, 2023

Description

  • Facade tasks should be placed in a separate pool of processes than the core tasks because a significant portion of the facade tasks do not require GitHub API calls to work. Before, the core tasks would hog all the processes because they were sleeping due to rate limit error handling.
  • Create an extra celery worker to handle facade tasks along with a respective rabbitmq queue.
  • Amend function to clear rabbitmq queue to also now clear the new facade queue
  • Amend celery routing to route all facade tasks to be routed to the facade worker.

Signed commits

  • Yes, I signed my commits.

Signed-off-by: Isaac Milarsky <imilarsky@gmail.com>
Signed-off-by: Isaac Milarsky <imilarsky@gmail.com>
@sgoggins sgoggins merged commit efbdea4 into dev Apr 20, 2023
1 check passed
@IsaacMilarky IsaacMilarky deleted the stop-facade-bottleneck branch April 20, 2023 17:59
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants