Stefan Arentz, December 2017
Tiny Queue Service (TQS) is a small pragmatic queue service with modest performance that you can self host.
If the following sounds good to you then this project may be compatible with your needs:
- Very easy to deploy via Docker
- Most operations take a few milliseconds on modest hardware, so a single instance can usually handle hundreds of queue operations per second
- Simple REST API that lets you GET and POST messages
- (Optional) Simple authentication via pre configured API keys
- Zero configuration. Run and go.
- No broker topology, no redundancy, no enterprise features.
This project was inspired by Redis and Amazon's Simple Queue Service.
TODO Python Code TODO
Either pull the latest stable version from the Docker Hub:
docker pull tqs-server:latest
Or build your own copy from source:
git clone https://github.com/st3fan/tqs-server.git
cd tqs-server
docker build -t tqs-server:latest .
If you want to see how this works, run it in the foreground with the database in /tmp/tqs
and no authentication:
docker run --rm -v /tmp/tqs:/data/tqs -p 8080:8080 tqs-server:latest
To start an instance in the background:
docker run -d -v /var/lib/tqs:/data/tqs -e API_TOKEN=YourSecretToken -p 8080:8080 tqs-server:latest
This will make a TQS instance available on host port 8080 and store
the database file in var/lib/tqs
.
If you expose TQS publicly (or even via say an internal Digital Ocean
or AWS address), it is highly recommended that you set an
TQS_API_TOKEN
and put an SSL enabled server like Caddy in front of
it.
Basically the following:
git clone https://github.com/st3fan/tqs-server.git
cd tqs-server
python3 -m venv env
source env/bin/activate
pip install -r requirements.txt
python tqs.py --logging=debug
Tornado auto reloads the code on change so you can hack away and see your changes on save.
source env/bin/activate
pip install pytest pytest-tornasync
pytest
This service is built in Python on top of Tornado and SQLite. Tornado and SQLite are a good match; because of Tornado's asynchronous single thread model, there is a strong guarantee that all database operations are executed serially. This is a speed compromise to get a simple design.
Speed, other than "reasonable", was never a goal for this project. If you need to go crazy fast then this may not be the right project for you. Redis may then be a better solution.
Here are some performance numbers for the available API endpoints:
TODO Some Numbers TODO
These tests were run on a small Digital Ocean instance (512 MB).
Note that I do not have numbers to compare against. I simply do not care about that. This project was created because I was unable to find something simple to fit my needs and not to compete with other solutions.