Skip to content
This repository

HTTPS clone URL

Subversion checkout URL

You can clone with HTTPS or Subversion.

Download ZIP

a simple durable fanout pub/sub queueing system built with Redis + Node.js

tree: 240cf67c9f
README.md

About

QDis is a simple fanout pub/sub queue built using Redis and Node.JS. Simplicity is it's goal and advantage.

QDis Diagram

Test Status Build Status

Performance

It's able to handle multiple 1000s of messages per second on a MacBook Air when Redis isn't in Append-Only Log mode. In Append-Only Log performance will be dominated by the disk write times, ie the cost of durability.

Publishing

To publish you use the Redis API, first you start a transaction using MULTI then LPUSH to a list, then PUBLISH on a channel to announce something new is available. Run EXEC to execute the transaction.

Example

MULTI
LPUSH pub one
PUBLISH pub 1
EXEC

Subscriptions

First you need to setup your subscription queue using the restful API:

curl -X POST http://localhost:6380/subscribe/pub_queue/sub_queue

This creates a 'queue' called sub_queue. Every message that gets published to pub_queue also gets published in 'sub_queue'.

This means that there's a Redis Pub/Sub channel called 'sub_queue' which can be subscribed to. This publication is a notice which tells you there's something new to be read from from the 'sub_queue' Redis list. You're expected to LPOP from the 'sub_queue' list whenever you're ready to consume another message.

Every once and a while you'll probably want to check there's nothing to be consumed in 'sub_queue' regardless of publication notices. This should never happen if you're subscription socket is working.

To unsubscribe:

curl -X POST http://localhost:6380/unsubscribe/pub_queue/sub_queue

To list subscriptions:

curl -X GET http://localhost:6380/subscriptions

Durability

Message durability is handled entirely inside Redis. It's a beautiful piece of software. Configure it for your desired level of durability.

Scalability

One advantage of this simple approach is that it's very easy to customize and adapt QDis/Redis. Eventually I'll write a bridging daemon which can easily fanout across hosts. It should also be possible to use Redis replication to create emergency failover queues for high-availability.

Something went wrong with that request. Please try again.