Simple disk-based job queue
Python
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Permalink
Failed to load latest commit information.
zdisq
.gitignore
README.md
bootstrap.py
buildout.cfg
setup.py

README.md

zdisq

Simple job queue server and client built on top of ZeroMQ and SQLite.

Requirement

  • Linux or Mac OS X
  • Python 2.6+ (with sqlite3 module in the standard library)
  • libzmq 2.1.11+
  • pyzmq 2.1.11+

Note: Compiling libzmq and pyzmq requires necessary build toolchain and Python header files. libzmq also requires uuid-dev package on Debian-based systems.

Deployment

git clone git://github.com/riobard/zdisq.git
cd zdisq
python bootstrap.py
bin/buildout
bin/zdisqd    # the server
bin/zdisqc    # the client

Job Lifecycle

                 (timeout)
             +-----------------+
             |                 |
             |                 |
    PUT      V       POP       |    DEL   
  -------> [NEW] ----------> [WIP] -----> *poof*

Protocol

Generic request and response header

Request:

[ 1-byte protocol version]
[16-byte request key]
[ 3-byte request command]
...

Response:

[ 1-byte protocol version]
[16-byte request key]       # NULL bytes if no corresponding request
[ 3-byte request command]   # NULL bytes if no corresponding request
[ 3-byte response status code]
...

PUT command

Request:

"PUT"
[4-byte priority]
[4-byte TTR]
[Variable-length tag]
[Variable-length payload]

Response:

"200"
[16-byte job key]

POP command

Request:

"POP"
[TTL]
[tag]

Response:

"200"
[16-byte key]
[Variable-length payload]

or

"204"
("No Content")

DEL command

Request:

"DEL"
[16-byte job key]

Response:

"200"
[16-byte job key]

or

"404"
[16-byte job key]
("Not Found")

Generic error response

Response:

"5XX"     # where X is a digit
([Variable-length error message])