Skip to content
This repository

simple, lightweight message queue

Fetching latest commit…

Octocat-spinner-32-eaf2f5

Cannot retrieve the latest commit at this time

Octocat-spinner-32 bench benchmark updates August 26, 2012
Octocat-spinner-32 bin makes a dist version of HEAD August 21, 2012
Octocat-spinner-32 docs
Octocat-spinner-32 include
Octocat-spinner-32 src
Octocat-spinner-32 tests
Octocat-spinner-32 .gitattributes don't export .gitignore August 21, 2012
Octocat-spinner-32 .gitignore basic framework August 08, 2012
Octocat-spinner-32 CMakeLists.txt
Octocat-spinner-32 ChangeLog
Octocat-spinner-32 LICENSE readable readme, and license August 13, 2012
Octocat-spinner-32 README.md
README.md

Darner

Darner is a very simple message queue server. Unlike in-memory servers such as redis, Darner is designed to handle queues much larger than what can be held in RAM. And unlike enterprise queue servers such as RabbitMQ, Darner keeps all messages out of process, relying instead on the kernel's virtual memory manager via log-structured storage.

The result is a durable queue server that uses a small amount of in-resident memory regardless of queue size, while still achieving remarkable performance.

Darner is based on Robey Pointer's Kestrel simple, distributed message queue. Like Kestrel, Darner follows the "No talking! Shhh!" approach to distributed queues: A single Darner server has a set of queues identified by name. Each queue is a strictly-ordered FIFO, and querying from a fleet of Darner servers provides a loosely-ordered queue. Darner also supports Kestrel's two-phase reliable fetch: if a client disconnects before confirming it handled a message, the message will be handed to the next client.

Compared to Kestrel, Darner boasts much higher throughput, better concurrency, an order of magnitude better tp99, and uses an order of magnitude less memory. But Darner has less configuration, and far fewer features than Kestrel. Check out the benchmarks!

Darner is used at Wavii, and is written and maintained by Erik Frey.

Installing

You'll need build tools, CMake, Boost, and LevelDB/snappy to build Darner. The following works on Ubuntu 12.04:

sudo apt-get install -y build-essential cmake libboost-all-dev libsnappy-dev libleveldb-dev

Then you can fetch and install Darner:

git clone git://github.com/wavii/darner.git
cd darner
cmake . && make && sudo make install

Running

Make a directory for Darner to store its queues, say /var/spool/darner/, then run Darner like so.

user@home:~$ darner -d /var/spool/darner/
[INFO] 2012-Aug-13 03:59:41.047739: darner: queue server
[INFO] 2012-Aug-13 03:59:41.048051: build: Aug 26 2012 (15:47:48) v0.1.2 (c) Wavii, Inc.
[INFO] 2012-Aug-13 03:59:41.048132: listening on port: 22133
[INFO] 2012-Aug-13 03:59:41.048507: data dir: /var/spool/darner/
[INFO] 2012-Aug-13 03:59:41.048798: starting up

Voila! By default, Darner listens on port 22133.

Protocol

Darner follows the same protocol as Kestrel, which is the memcache protocol.

Currently missing from the Darner implementation but TODO: FLUSH, FLUSH_ALL, DELETE, and some stats.

Something went wrong with that request. Please try again.