Skip to content

Add support for Python Dramatiq background task / job library#127

Merged
carlosantoniodasilva merged 11 commits intomainfrom
ca-dramatiq
Feb 16, 2026
Merged

Add support for Python Dramatiq background task / job library#127
carlosantoniodasilva merged 11 commits intomainfrom
ca-dramatiq

Conversation

@carlosantoniodasilva
Copy link
Copy Markdown
Member

@carlosantoniodasilva carlosantoniodasilva commented Feb 12, 2026

https://dramatiq.io

request catcher samples: FastAPI + Dramatiq

CleanShot 2026-02-12 at 18 05 26@2x CleanShot 2026-02-12 at 18 05 00@2x

request catcher samples: Django + Dramatiq

CleanShot 2026-02-12 at 18 06 41@2x CleanShot 2026-02-12 at 18 07 06@2x

Dramatiq uses two separate Redis keys to handle this: one contains the
message ids, the other contains the actual messages we need to fetch.
It can be done with a separate django-dramatiq integration so it can be
a little more involving, thus a sample app to help test and show that
setup.
This is not really useful/relevant, especially because most of the time
people will have set up judoscale_dramatiq before adding any
actors/tasks, so there'll be an empty list of queues anyway. That's the
behavior I'm seeing, but it works on collection afterwards just fine.
The setup for FastAPI / Flask is basically the same usual one for doing
it standalone, so it doesn't seem to warrant an additional section, but
for Django there's some different way of doing things, if you're using
django-dramatiq (which presumably most are), so add explicit docs.
@carlosantoniodasilva carlosantoniodasilva self-assigned this Feb 12, 2026
Comment on lines +54 to +57
if message_id := self.redis.lindex(queue_key, -1):
if isinstance(message_id, bytes):
message_id = message_id.decode()
if payload := self.redis.hget(msgs_key, message_id):
Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this 2-step / Redis calls to read the actual message payload might result in a possible race condition if a worker happens to grab the same job/message we fetched, so when we get to read the payload it's not there anymore...

In that case we might end up reporting a 0 latency... so I'm wondering if we should perhaps try again (or 2-3 times) if we get no payload, so it tries on the next message_id and so on?

Perhaps it's such an edge case that won't be a problem in real usage, since it's very unlikely and likely self-corrects on the next report.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think since it's self-corrects, we don't need to worry about it.

@carlosantoniodasilva carlosantoniodasilva marked this pull request as ready for review February 12, 2026 21:09
Copy link
Copy Markdown
Member

@adamlogic adamlogic left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Awesome!! 🙌

Comment on lines +54 to +57
if message_id := self.redis.lindex(queue_key, -1):
if isinstance(message_id, bytes):
message_id = message_id.decode()
if payload := self.redis.hget(msgs_key, message_id):
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think since it's self-corrects, we don't need to worry about it.

@carlosantoniodasilva carlosantoniodasilva merged commit 4d36e72 into main Feb 16, 2026
5 checks passed
@carlosantoniodasilva carlosantoniodasilva deleted the ca-dramatiq branch February 16, 2026 12:51
carlosantoniodasilva pushed a commit that referenced this pull request Feb 16, 2026
## [1.13.0](v1.12.0...v1.13.0) (2026-02-16)


### Features

* Add support for Python Dramatiq background task / job library ([#127](#127)) ([4d36e72](4d36e72))


Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants