Clojure RabbitMQ client with a focus on HA Configurations.
for a more detailed description of the publisher and consumer APIs.
This project takes much inspiration from, and was heavily influenced by, Beetle, a HA solution for Ruby and RabbitMQ.
Default configuration is for exchanges, queues and bindings to be durable.
Publishers are a list of broker configurations. Publishers are registered with logical names. Publishing is performed to all the brokers in the configuration. At least 1 broker must be published to or the client library will fail, throwing an exception. This ensures reliable message publishing.
Publish connections that encounter transient errors, a timeout or a broker failure, are passed off to an agent in order to be reconnected asynchronously from the code that is utilizing the publisher. In cases where a broker fails, this allows application code to continue publishing to the remaining brokers. When the connection is re-established, the publisher will begin using the connection again. The goal is to provide redundancy for message delivery as well as robustness (recovery) in the event a broker goes off line and comes back on-line.
Consumers are registered as with a logical name. A consumer is an amqp endpoint configuration (vhost, exchange, queue, bindings) plus event handler functions that follow the RabbitMQ Java Consumer interface. Consumers can be started / stopped by invoking ‘add-consumer’ with a given registered name. This allows pools of consumers to be grown or shrunk interactively.
By default, consumers attempt to reconnect if they are shut down unexpectedly. Reconnect attempts will occur (default is 250ms) until the connection can be re-established or until they are explicitly shut down.
To run the example code, you’ll need to run a few services: RabbitMQ, Redis and HA Proxy. Each of these can be executed in a seperate terminal with a Rake task from the project’s root directory.
Clone the git repo:
git clone firstname.lastname@example.org:kyleburton/teporingo.git
Execute each of the following in a separate terminal:
The First RabbitMQ Broker:
The Second RabbitMQ Broker:
A Redis Master Server:
Each of these will download and build the required services. Once these services are running you can start the consumers and then the publisher.
To run the consumers:
For the publisher:
With this executing you should see the publisher publishing message and the consumers processing (and deduplicating messages). At any time you can kill one of the RabbitMQ brokers (with a CTRL-C, followed by an ‘a’), publishing should continue unabated, and one of the consumers should continue to process messages. The other consumer will continually attempt to re-establish a connection to the failed broker until you bring it back up.
Bring the aborted broker back on-line and you should see the publisher and failed consumer recover and begin processing again.
Copyright (©) 2011 Kyle R. Burton <email@example.com>
Eclipse Public License 1.0 (http://opensource.org/licenses/eclipse-1.0.php)