High Availability AMQP Messaging With Redundant Queues
Clone or download
Type Name Latest commit message Commit time
Failed to load latest commit information.
doc updated doc Feb 12, 2016
etc renamed rake redis:start{1,2} to redis.start:{master,slave} Mar 3, 2016
examples undo unrelated commit Jun 27, 2018
features support switching with only a subset of clients responding Feb 17, 2018
go/src/github.com/xing/beetle bumped version to 2.1.2 Jun 18, 2018
lib bumped version to 2.2.1 Aug 7, 2018
script respect my access rights Jan 12, 2013
test Subscriber exits properly on possible authentication failures. Aug 7, 2018
tmp renamed rake redis:start{1,2} to redis.start:{master,slave} Mar 3, 2016
.gitignore ms precision for settings and exponential back off Nov 20, 2017
.gitmodules update realpath submodule so travis can access the repo Sep 24, 2017
.travis.yml updated software version to use on travis Aug 7, 2018
DEAD_LETTERING.md Preserve RabbitMQ 2.x requeueing behaviour via dead letter queues Jul 14, 2015
Dockerfile beetle command has been replaced by a go version Aug 4, 2017
Gemfile beetle command has been replaced by a go version Aug 4, 2017
MIT-LICENSE started documentation Feb 17, 2010
Makefile use go get github.com/davecgh/go-spew... to simplify Makefile Aug 8, 2018
README.rdoc updated README Sep 25, 2017
REDIS_AUTO_FAILOVER.rdoc beetle command has been replaced by a go version Aug 4, 2017
RELEASE_NOTES.rdoc bumped version to 2.2.1 Aug 7, 2018
Rakefile clean tmp files before running cucumber tasks Sep 24, 2017
beetle.gemspec updated gemspec to silence rubygems warnings Aug 7, 2018
create_release.sh releases don't have draft status anymore Aug 28, 2017



High Availability AMQP Messaging with Redundant Queues


Beetle grew out of a project to improve an existing ActiveMQ based messaging infrastructure. It offers the following features:

  • High Availability (by using multiple message broker instances)

  • Redundancy (by replicating queues)

  • Simple client API (by encapsulating the publishing/ deduplication logic)

More information can be found on the project website.



# configure machines

Beetle.config do |config|
  config.servers = "broker1:5672, broker2:5672"
  config.redis_server = "redis1:6379"

# instantiate a beetle client

b = Beetle::Client.new

# configure exchanges, queues, bindings, messages and handlers

b.configure do
  queue :test
  message :test
  handler(:test) { |message| puts message.data }


b.publish :test, "I'm a test message"




Beetle ships with a number of example scripts.

The top level Rakefile comes with targets to start several RabbitMQ and redis instances locally. Make sure the corresponding binaries are in your search path. Open four new shell windows and execute the following commands:

rake rabbit:start1
rake rabbit:start2
rake redis:start1
rake redis:start2


To set up a redundant messaging system you will need

  • at least 2 AMQP servers (we use RabbitMQ)

  • at least one Redis server (better are two in a master/slave setup, see REDIS_AUTO_FAILOVER.rdoc)

Test environment

For testing purposes, you will need a MySQL database with the database `beetle_test` created. This is needed to test special cases in which Beetle handles the connection with ActiveRecord:

mysql -e 'create database beetle_test;'

You also need a Redis instance running. The default configuration of Redis will work:


If you want to run the integration tests you need GO installed and you will need to build the beetle binary. We provide a Makefile for this purpose, so simply running


should suffice.

Gem Dependencies

At runtime, Beetle will use

For development, you'll need

For tests, you'll need


Stefan Kaes, Pascal Friederich, Ali Jelveh, Bjoern Rochel and Sebastian Roebke.

You can find out more about our work on our dev blog.

Copyright © 2010-2015 XING AG

Released under the MIT license. For full details see MIT-LICENSE included in this distribution.


  1. Fork it

  2. Create your feature branch (`git checkout -b my-new-feature`)

  3. Commit your changes (`git commit -am 'Add some feature'`)

  4. Push to the branch (`git push origin my-new-feature`)

  5. Create new Pull Request

Don't increase the gem version in your pull requests. It will be done after merging the request, to allow merging of pull requests in a flexible order.

How to release a new gem version

Update RELEASE_NOTES.rdoc!

We use semantic versioning and create a git tag for each release.

Edit `lib/beetle/version.rb` and `go/src/github.com/xing/beetle/version.go` to set the new version number (`Major.Minor.Patch`).

In short (see semver.org for details):

  • Major version MUST be incremented if any backwards incompatible changes are introduced to the public API.

  • Minor version MUST be incremented if new, backwards compatible functionality is introduced to the public API. It MUST be incremented if any public API functionality is marked as deprecated.

  • Patch version MUST be incremented if only backwards compatible bug fixes are introduced.

Then use `rake release` which will create the git tag and upload the gem to github.com:

bundle exec rake release

The generated gem is located in the `pkg/` directory.

In order to build go binaries and upload the docker container with the beetle GO binary to docker hub, run

make release

This will upload the go binaries to github.com/xing/beetle/ and push the beetle container to hub.docker.com/r/xingarchitects/gobeetle/.


make tag push TAG=vX.X.X

to tag and push the container with a specific version number.