Have a separate worker queue for GitHub jobs

[frankie567]

This morning, we had some timeout and rate limit errors on lot of GitHub sync jobs, which delayed processing of new jobs by almost an hour:

It happens rarely but we have critical jobs (like Stripe subscriptions events) which should be handled quickly.

To avoid this, we could have a separate queue with a dedicated process only for GitHub sync, while leaving the other for critical jobs.

[zegl]

(this is not a blocker for your idea above)

Agreed, and truth be told! A lot of these GitHub syncs can probably be removed. We're barely using issue_references and issue_dependencies, and they are a majority of all jobs executed!

[frankie567]

Great call, @zegl. I even think it could be the first thing to consider before thinking about complex things like I propose above :) I'll open an issue for this!