I needed to integrate a distributed task queue with a Rust web application and so far I found batch-rs and oppgave which solve similar problems to this. However, I wanted a solution that would not use RabbitMQ and that would offer some of the same capabilities as rq (namely monitoring and saving of results) which is one of my goto tools in Python.
The result is
brokkr. It is backed by a Redis server and ends up a bit more complex than push / pull queues such as ost (Ruby) and oppgave in that it provides a
Worker implementation, observability of current workers and tasks (wip, you can already query the Redis server directly) and the ability to fetch task results from another process / machine. These features come at the cost of slightly lower reliability in the face of network failure compared to the simple reliable queues pattern.
brokkr is not supposed to a port of rq though. Initially, I aim to only implement what I need for a given use case (See TODO for planned features) in order to keep things simple. More complex features (such as job dependencies) will be considered but are not part of the initial roadmap.
Installation & Usage
To use brokkr, add this to your Cargo.toml:
[dependencies] brokkr = "0.1"
See the examples and documentation in the meantime.
Running the tests
As tests do some actual work against a Redis backend, they'll need it to be available. If you have docker installed, you can easily run a development redis server with:
docker run --name redis -p 6379:6379 -d redis:alpine
If you run the redis server on a url / port that is not
redis://127.0.0.1/, set the
BROKKR_URL environment variable before running the tests.
BROKKR_URL=redis://[:password@]host:port/db cargo test