Ramifications of Lowering dequeue_timeout

[briantsaunders]

I'll start by saying I'm a big fan of pgqueuer, keep up the good work! I have a question on the ramifications of lowering the dequeue_timeout. I was recently playing around with the execute_after parameter and noticed that when I set it to say 10 seconds it wouldn't fire until 30 seconds and when I set it to say 45 seconds it wouldn't fire until 1 minute. I noticed that if I adjusted the dequeue_timeout to 10 seconds and set execute_after to 10 seconds it would fire after 10 seconds.

Does lowering the dequeue_timeout having any impact on the performance of pgqueuer? Should this typically be avoided unless absolutely necessary? Cheers and thanks!

[janbjorge]

Thanks for the kind words—and for digging into this!

Under the hood, the QueueManager uses two mechanisms to decide when to pull jobs:

1. LISTEN/NOTIFY wakes it up immediately when you NOTIFY the channel after enqueueing.
2. A periodic poll every dequeue_timeout to look for work that’s become ready (i.e. jobs whose execute_after timestamp has just passed).

---

Why we need that periodic poll

Postgres’s LISTEN/NOTIFY is a best-effort broadcast, not a guaranteed message queue. Missing notifications can occur if the listener temporarily disconnects (for example, during a network hiccup or failover). The dequeue_timeout safety-net ensures that, even if a notification is dropped or delayed, the manager will re-scan the queue and pick up any ready jobs on its next timeout.

---

Why execute_after fires late

If you set execute_after = 10s but leave the default dequeue_timeout = 30s, your job “becomes ready” at 10 s, but the manager only polls again at 30 s—so it doesn’t actually dequeue until then. By lowering the timeout to 10 s, you told it to re-check every 10 s, so your job ran might run at 10 s, but the worst case would be 10s(?).

---

Impact of lowering dequeue_timeout

• Latency vs. load trade-off. Smaller timeouts improve scheduler accuracy but increase CPU wake-ups and DB traffic.
• Practical floor. Going much below a few seconds yields diminishing returns and noticeable overhead.

---

Recommendations

• Stick with the 30 s default unless you need sub-30 s precision.
• If you do lower it, monitor your database load and CPU usage.

Let me know if you’d like to dive deeper.

[briantsaunders]

Thanks for the reply! I'll stick with the default dequeue timeout of 30 seconds for now. Cheers!