-
Notifications
You must be signed in to change notification settings - Fork 188
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Allow que to be started without listen/notify #395
Conversation
Thanks for this! No idea why this is not already configurable. If you can add some specs I'll be happy to approve and merge this. :) |
Ability to disable listen/noify on CLI resulting in jobs being picked up purely from polling. We have seen issues with listen/notify usage at scale.
@oeoeaio I've updated the commit with a spec for the new option. Ready for review. |
Sorry for the delay! Out of curiosity, what issues were you seeing with listen/notify at scale? |
@oeoeaio Thanks for merging!
Hey @ZimbiX, we have quite a high throughput of jobs in our system. Average of 150 jobs/s peaking to about 700 jobs/s at times. We had issues where listen/notify queue would fill quickly and when full subsequent listen/notify calls would fail. I believe there were issues where long running transactions prevented the queue from being emptied if the transactions covered items in the queue. I feel listen/notify is ideal for systems with lower job throughput with sporadic events and you need low latency. In a system with high throughput like ours, poll mode works fine. |
For us, we're experimenting with pgcat and it'd doesn't yet support LISTEN/NOTIFY so this is a good fallback :) |
Ability to disable listen/notify on CLI resulting in jobs being picked up purely from polling. We have seen issues with listen/notify usage at scale.