Skip to content

v5.7.0: Implement detection and recovery for stalled processing jobs

Choose a tag to compare

@thorewi thorewi released this 28 Aug 07:55
· 8 commits to master since this release
This introduces a mechanism to handle background jobs that become stuck
in the `PROCESSING` state due to unexpected consumer termination
(e.g., SIGKILL, OOM killer, container crash). Without this, such jobs
would remain unprocessable indefinitely, potentially blocking entire
`serialGroup` queues.

A "reaper" periodically scans for `PROCESSING` jobs whose `updated_at`
timestamp is older than `stalledJobTimeout` and transitions them to
`TEMPORARILY_FAILED`, allowing them to be reprocessed.

A "heartbeat" mechanism is also added: long-running jobs can signal
their liveliness by updating `updated_at`. This can be triggered
manually via `heartbeat()` or automatically if `BackgroundQueueMiddleware`
is installed on the application's DBAL connection.

This ensures job resilience and prevents permanent blockages within
serial groups, improving the robustness of the queue system.