Skip to content
This repository has been archived by the owner on Jul 16, 2020. It is now read-only.

Home

Jump to bottom
squaremo edited this page Jun 9, 2011 · 31 revisions

0MQ plugin for RabbitMQ

The purpose of this plugin is to provision 0MQ sockets that relay messages into RabbitMQ. Each 0MQ socket type is given an idiomatic encoding in the AMQP broker model, which means AMQP clients can interoperate.

Among other things, this means you can use RabbitMQ as a device in a 0MQ network, gaining rabbity goodness such as persistence and monitoring. You can use 0MQ as an interconnect to create federations of RabbitMQ brokers. RabbitMQ can serve as a gateway from 0MQ to other protocols such as AMQP, STOMP or XMPP. You can also use different 0MQ language bindings to access your RabbitMQ server from languages/platforms where there are no AMQP clients available.

Prerequisites

You have to have both RabbitMQ server, 0MQ and Erlang binding for 0MQ installed.

For instructions concerning download and installation of RabbitMQ Server look here:

http://www.rabbitmq.com/server.html

For instructions concerning download and installation of 0MQ look here:

http://www.zeromq.org/docs:procedures

For instructions concerning download and installation of Erlang binding for 0MQ look here:

http://www.zeromq.org/bindings:erlang

Installation of r0mq plugin

Copy the files from the downloads section of this repository into the RabbitMQ plugin directory as explained here:

http://www.rabbitmq.com/admin-guide.html#installing-plugins

Configuration

The mapping between 0MQ sockets and RabbitMQ exchanges and queues is managed by configuration. Each instance of a mapping is called a “service”.

The configuation file should be located in RabbitMQ configuration directory as explained here:

http://www.rabbitmq.com/install.html#configfile

Here is an example, given as a complete RabbitMQ config file:

  [{r0mq,
      [{services,
        [{<<"PIPELINE">>, push, "tcp://127.0.0.1:5557"},
         {<<"PIPELINE">>, pull, "tcp://127.0.0.1:5558"}]}]}]

Each service entry in the configuration is a triple consisting of a “rendez-vous point”, specification of what type of 0MQ sockets are going to connect to that rendez-vous point and the 0MQ-style address to bind the rendez-vous point to.

Rendez-vous point corresponds either to AMQP exchange or AMQP queue, depending on the messaging pattern. For req/rep pattern it’s the name of the request queue. For pub/sub pattern it’s the name of the exchange that serves as an entry point for the published messages. For pipeline (push/pull) pattern it corresponds to the shared queue to pass messages through.

For detailed diagrams of AMQP wiring corresponding to individual messaging patterns have a look at dedicated pages:

It’s not common for a broker to actively connect to another node on the network, however, it may come handy in federation scenarios. In such cases 0MQ address in the triple can be replaced by a pair consisting of bind or connect keyword and the 0MQ-style address:

  [{r0mq,
      [{services,
        [{<<"PIPELINE">>, pull, {connect, "tcp://127.0.0.1:5558"}}]}]}]

Examples

Subdirectory examples contains some simple examples you can use to get familiar with r0mq.

Individual examples are written in Python so you have to install Python binding for 0MQ to be able to run them:

http://www.zeromq.org/bindings:python

Each example contains a plugin configuration file (“rabbitmq.config”) you have to copy to RabbitMQ configuration directory as explained here:

http://www.rabbitmq.com/install.html#configfile

The examples are:

  1. reqrep: This example illustrates how to do RPC-style communication. rep.py is the service. It waits for a message from the client and responds with “World!” message. rep.py is the client. It sends “Hello!” message and prints out the reply. You can run as many services and clients in parallel as you wish.
  2. pipeline: This example shows how RabbitMQ broker can be used to gather messages from one processing step in the pipeline and forward them to the next step of the pipeline. push.py represents the previous step in the pipeline. It sends one message per second the the broker. pull.py represents the next processing step. It reads messages from the broker and prints them out. You can run as many pushers and pullers in parallel as you wish. The messages are load-balanced between individual pullers.
  3. pubsub: This example shows how to distribute messages to multiple clients. pub.py is the publisher. It sends one message per second. sub.py is the subscriber. It subscribes for the messages and prints them out. You can run as many publishers and subscribers in parallel as you wish.

Additionally, each example contains Java programs witch implement exactly the same functionality. These are true AMQP clients and you’ll need to install RabbitMQ Java Client to be able to compile and run them:

http://www.rabbitmq.com/java-client.html

Building the plugin from source

The makefile fetches and builds 0MQ Erlang binding (erlzmq) for you. It assumes that libzmq has been put in /usr/local; change ZMQ_PATH in deps/erlzmq/Makefile if you put it somewhere else.

At the minute RabbitMQ plugins expect to be built from a directory in rabbitmq-public-umbrella, and this one is no exception. For detailed instructions on the process have a look here:

http://www.rabbitmq.com/plugin-development.html

To build the plugin do the following:

    cd rmq-0mq
    make

The resulting Erlang archives will be located in dist subdirectory.

Known limitations

Lack of socket backpressure

Currently we use “active” mode for erlzmq sockets, whereby 0MQ messages are delivered straight to Erlang process mailboxes. This means that sending processes can happily send a million messages a second and fill available memory at the broker. Ideally we would propagate backpressure to the 0MQ socket so that RabbitMQ can make sure messages, if they need to rest at the broker, are safely in queues. We’re working on fixing this limitation.

Message persistence off by default

We default to using AMQP’s delivery-mode=1, i.e., non-persistent messages. This means they won’t be written to disk if they can’t be delivered straight away. We’re going to make it an option in the config to turn persistence on for a particular socket.

Only AMQP message bodies are sent between brokers

When using the bridge as a means of federation between Rabbits, only the message body is sent over 0MQ; at the other end, it’s reconstituted with defaults (so it will lose headers, delivery mode, and so on). We’re thinking of adding an option for “whole message” serialisation — the bridge would serialise and deserialise AMQP messages to and from the 0MQ payloads.

Clone this wiki locally