You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
I think the reasoning behind it might be similar to that why traefik was chosen to replace nginx: it's better yet quite familiar.
Use case(s) / visualization(s)
Dramatiq compares favorably with Celery and others:
Dramatiq
Celery
Huey
RQ
Simple implementation
Yes
No [3]
Yes
Yes
Automatic retries
Yes
No
Yes
No
Reliable delivery
Yes
No [1]
No
No
Locks and rate limiting
Yes
No
Yes
No
Task prioritization
Yes
No [4]
No
No [4]
Delayed tasks
Yes
Yes [2]
Yes
No
Cronlike scheduling
No [5]
Yes
Yes
No
Chaining / Pipelining
Yes
Yes
Yes
No
Result storage
Yes
Yes
Yes
Yes
Code auto-reload
Yes
No
No
No
RabbitMQ support
Yes
Yes
Yes
No
Redis support
Yes
Yes
Yes
Yes
In-memory broker support
Yes
No
Yes
No
Greenlet support
Yes
Yes
Yes
No
Footnotes:
Celery acks tasks as soon as they’re pulled by a worker by default. This is easy to change, but a bad default. Dramatiq doesn’t let you change this: tasks are only ever acked when they’re done processing.
Celery has poor support for delayed tasks. Delayed tasks are put on the same queue that is used for normal tasks and they’re simply pulled into worker memory until they can be executed, making it hard to autoscale workers by queue size. Dramatiq enqueues delayed tasks on a separate queue and moves them back when they’re ready to be executed.
Celery’s source code is spread across 3 different projects (celery, billiard and kombu) and it’s impenetrable. Its usage of runtime stack frame manipulation leads to heisenbugs.
(1, 2) Celery and RQ don’t support task prioritization. You have to deploy multiple sets of workers in order to prioritize queues. Dramatiq lets you prioritize down to the individual actor level.
For cron-like scheduling functionality, you can combine Dramatiq with APScheduler or Periodiq.
The text was updated successfully, but these errors were encountered:
Thanks for taking the time to write this, very well written!
I've heard about Dramatiq, but I've never get the chance to use it in the production environment. If the integration is not too complicated we could probably consider it as an option, but testing becomes increasingly complicated as we add more options.
So why not, provided it's something that folks are after and it's not too complicated to maintain.
(As a side note, someone suggested to include Django-Q (#2191) which I declined at the time, if we add the option Celery/Dramatiq, we could reconsider it)
investigate https://github.com/pricingassistant/mrq
very interesting project, much more speed compare with Celery, too have IO support, dashboard and etc., used redis+mongo
have some unique feature, for example use simple raw task when don't important order get tasks from store (super performance in hot projects). Have very good dashboard (charts, logs of tasks, rerun tasks etc.). Not depend on Django, can easy use in Flask example or on pure project. Simple sources, can overide task run.
i use in production, very cool, try (use git clone not release download!)
Description
I'd like to propose support for Dramatiq which is a pumped-up alternative for Celery.
Rationale
Main reasons behind Dramatiq are pointed out nicely in documentation:
I think the reasoning behind it might be similar to that why traefik was chosen to replace nginx: it's better yet quite familiar.
Use case(s) / visualization(s)
Dramatiq compares favorably with Celery and others:
Footnotes:
The text was updated successfully, but these errors were encountered: