Stalled Queue monitoring

[anchovy]

What are you trying to do?

We've just had an issue where our demon watching the queue failed when our site had some unexpected downtime. Be great if this plugin could also check to see if the queue has stalled.

What's your proposed solution?

Look at the next available queued job and if it hasn't been processed in x minutes send out a email - maybe this could be check every 30 (configurable) minutes so if the queue has stalled it doesn't flood the specified users email address, or even better, slack (via a webhook).

Additional context

No response

[engram-design]

Yeah, a great point, I agree that hits us often too. I believe we'll need to fail the job that's "stuck" in order for it to be restarted, but that should be fine.

Good call on the webhook handling too, that'll be a nice addition.

[anchovy]

Sweet / I was sort of meaning monitoring the queue to ensure that it's being run.

So if the mechanism that's running or watching the queue failed - cron/demon (or something else) fails and the queue isn't being processed at all we get alerted.

[engram-design]

Ah right, we can bundle that in too, but for example that process will need to be set on a schedule (cron/daemon), and if that's the thing that's failing, it may also fail to trigger our checking. But we'll see what we can do!