Bug: Worker gets stuck working on a task indefinitely since 1.12.0

[sarnikowski]

Issues

What OS are you using?

Ubuntu 20.04.1 LTS (Focal Fossa)

What version of Dramatiq are you using?

1.12.0

What did you do?

Enqueued jobs on a queue, on which a worker is listening.

What did you expect would happen?

That the worker does not get stuck running a task indefinitely.

What happened?

The worker gets stuck with a task running at 100% CPU, but fails to ever complete the task. I have to manually kill the process and restart it. The following warning is emitted in the logs:

[2021-11-05 12:21:37,834] [PID 8] [Thread-2] [dramatiq.brokers.redis._RedisConsumer] Found 'None' message in message cache for queue 'miner-low'. This may be a bug in dramatiq (related to #266), please report it.

[Bogdanp]

Would you be able to dump the contents of your queue? Either here or share it with me by e-mail. It would be really helpful to see what the data looks like.

[sarnikowski]

Yes, of course. I've sent you an e-mail.

[Bogdanp]

Thanks again for sharing the DB! I think this is now properly fixed on master (my previous fix for this problem had the right idea, but the wrong implementation!). Please give it a try and let me know if you run into any problems.

[sarnikowski]

No problem ! Thanks for the quick fix. I will let you know if the issue persists.

[mwek]

Could we get a 1.12.1 release (or 1.13.0) with this fix? We hit this in production, and are downgrading to 1.11.0 as a consequence.

[Harinib-Kore]

I got the same problem . how to fix it