Skip to content

feat(queue): POST /queues/:name/purge to drain stuck PENDING jobs (refs #13)#15

Merged
hacka0wi merged 1 commit into
mainfrom
fix/13-queue-purge-and-script-timeout
May 7, 2026
Merged

feat(queue): POST /queues/:name/purge to drain stuck PENDING jobs (refs #13)#15
hacka0wi merged 1 commit into
mainfrom
fix/13-queue-purge-and-script-timeout

Conversation

@hacka0wi

@hacka0wi hacka0wi commented May 7, 2026

Copy link
Copy Markdown
Member

Summary

Adds an operator endpoint to clear stuck PENDING jobs from a queue without dropping the queue itself.

POST /api/queues/:name/purge
→ {"success":true, "deleted": 42}

PG backend only — Redis/Kafka/RabbitMQ users have native purge in their broker UIs.

Why

After fixing a flow's code, messages enqueued under the old broken code keep retrying on the visibility timeout cycle (5 min × max_attempts = 15 min) until they hit the DLQ. New enqueues sit behind them. Today the only mitigation is "drop the queue and recreate it" — destructive and loses queue config (concurrency, backoff, etc).

Scope

  • ✅ Deletes status='PENDING' rows only
  • ❌ Does NOT touch status='PROCESSING' (in-flight) — those age out via visibility timeout if the handler dies
  • ❌ Does NOT touch DLQ entries

Refs

Refs #13 — this is a mitigation, not a root-cause fix. Posted analysis as a comment on that issue: the reporter's "stale cached flow code" hypothesis doesn't match the code (loadFlowConfig does a fresh SELECT on every delivery); the most likely root cause is messages stuck in the retry loop. Purge is the operator escape hatch.

Test plan

  • Engine builds (go build ./...)
  • go vet clean
  • CI green
  • Manual: enqueue 5 jobs → POST /queues/:name/purge → all 5 deleted, queue config intact

Operators currently have no way to clear bad messages from a queue
short of dropping the entire queue. After updating a flow's code to
fix a runtime bug, messages enqueued under the old broken code keep
retrying on a 5-minute visibility cycle until max_attempts hits the
DLQ — that's ~15 minutes of latency before fresh enqueues even get
picked up.

Purge deletes only PENDING rows (status='PENDING'). PROCESSING rows
are intentionally left alone so we don't yank a row out from under an
in-flight worker; those age out naturally via visibility timeout if
their handler dies. PG backend only — Redis/Kafka/RabbitMQ users have
their broker UIs for this.

Returns the deleted row count so an operator can confirm.

Refs #13. This is a mitigation, not a root-cause fix: see comment on
the issue for analysis of the reporter's "stale cached code" hypothesis
(loadFlowConfig already SELECTs fresh on every delivery; the more likely
root cause is messages stuck in retry loop).
@hacka0wi hacka0wi merged commit 362f1c3 into main May 7, 2026
8 checks passed
@hacka0wi hacka0wi deleted the fix/13-queue-purge-and-script-timeout branch May 7, 2026 17:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant