Skip to content

Commit

Permalink
add rabbitmq-priority-queue to rabbitmq`s deps
Browse files Browse the repository at this point in the history
original rev: 72d20292b9b7e

Change-Id: Id7cb3a1b670804da7c7db188c2aa8db8758b54ca
  • Loading branch information
siplabs committed May 7, 2015
1 parent 9c8a772 commit 663df29
Show file tree
Hide file tree
Showing 8 changed files with 1,529 additions and 0 deletions.

Large diffs are not rendered by default.

@@ -0,0 +1,28 @@
include ../umbrella.mk

RABBITMQCTL=../rabbitmq-server/scripts/rabbitmqctl
TEST_TMPDIR=$(TMPDIR)/rabbitmq-test
OTHER_NODE=undefined
OTHER_PORT=undefined

start-other-node:
rm -f $(TEST_TMPDIR)/rabbitmq-$(OTHER_NODE)-pid
RABBITMQ_MNESIA_BASE=$(TEST_TMPDIR)/rabbitmq-$(OTHER_NODE)-mnesia \
RABBITMQ_PID_FILE=$(TEST_TMPDIR)/rabbitmq-$(OTHER_NODE)-pid \
RABBITMQ_LOG_BASE=$(TEST_TMPDIR)/log \
RABBITMQ_NODENAME=$(OTHER_NODE) \
RABBITMQ_NODE_PORT=$(OTHER_PORT) \
RABBITMQ_CONFIG_FILE=etc/$(OTHER_NODE) \
RABBITMQ_PLUGINS_DIR=$(TEST_TMPDIR)/plugins \
RABBITMQ_PLUGINS_EXPAND_DIR=$(TEST_TMPDIR)/$(OTHER_NODE)-plugins-expand \
../rabbitmq-server/scripts/rabbitmq-server >/tmp/$(OTHER_NODE).out 2>/tmp/$(OTHER_NODE).err &
$(RABBITMQCTL) -n $(OTHER_NODE) wait $(TEST_TMPDIR)/rabbitmq-$(OTHER_NODE)-pid

cluster-other-node:
$(RABBITMQCTL) -n $(OTHER_NODE) stop_app
$(RABBITMQCTL) -n $(OTHER_NODE) reset
$(RABBITMQCTL) -n $(OTHER_NODE) join_cluster rabbit-test@`hostname -s`
$(RABBITMQCTL) -n $(OTHER_NODE) start_app

stop-other-node:
$(RABBITMQCTL) -n $(OTHER_NODE) stop
@@ -0,0 +1,100 @@
# Overview

This plugin adds support for priority queues to RabbitMQ.

# Downloading

You can download a pre-built binary of this plugin from
http://www.rabbitmq.com/community-plugins.html.

# Building

You can build and install it like any other plugin (see
[the plugin development guide](http://www.rabbitmq.com/plugin-development.html)).

# Declaring priority queues

Once the plugin is enabled, you can declare priority queues using the
`x-max-priority` argument. This argument should be an integer
indicating the maximum priority the queue should support. For example,
using the Java client:

Channel ch = ...;
Map<String, Object> args = new HashMap<String, Object>();
args.put("x-max-priority", 10);
ch.queueDeclare("my-priority-queue", true, false, false, args);

You can then publish prioritised messages using the `priority` field
of basic.properties. Larger numbers indicate higher priority.

There is a simple example using the Java client in the `examples` directory.

## Caution

While this plugin implements priority queues in terms of standard
queues, it does not support converting between a priority queue and a
standard queue, and the on-disc format is somewhat different. This has
the following implications:

* _It is dangerous to disable the plugin when durable priority queues exist_;
the broker will fail to start again. Remember that on broker upgrade
non-bundled plugins like this one need to be reinstalled.
* It is similarly dangerous to enable the plugin if you have declared
durable queues with an `x-max-priority` argument without it. I have no
idea why you'd do that, since you wouldn't get priority queues, but
it would also lead to broker crashes on startup.
* Priority queues can only be defined by arguments, not policies. Queues can
never change the number of priorities they support.

## Argument equivalence

RabbitMQ does not have a way for plugins to validate queue
arguments. Therefore the usual equivalence-checking that happens with
arguments does not happen here:

* You can declare a queue with `x-max-priority` and then declare it
again without `x-max-priority`; no error will result.
* Conversely, you can declare a queue without `x-max-priority` and then
declare it again with `x-max-priority`; again no error will result,
(but the queue will not become a priority queue).
* You can declare a queue with an `x-max-priority` argument which is not
an integer. The plugin will ignore this argument.

# Behaviour

The AMQP spec is a bit vague about how priorities work. It says that
all queues MUST support at least 2 priorities, and MAY support up to
10. It does not define how messages without a priority property are
treated.

In contrast to the AMQP spec, RabbitMQ queues by default do not
support priorities. When creating priority queues using this plugin,
you can specify as many priority levels as you like. Note that:

* There is some in-memory and on-disc cost per priority level per
queue, so you may not wish to create huge numbers of levels.
* The message `priority` field is defined as an unsigned byte, so in
practice priorities should be between 0 and 255.

Messages without a priority property are treated as if their priority were
0. Messages with a priority which is higher than the queue's
maximum are treated as if they were published with the maximum priority.

## Interaction with other features

In general priority queues have all the features of standard RabbitMQ
queues: they support persistence, paging, mirroring, and so on. There
are a couple of interactions that should be noted though:

* Messages which should expire (as at
http://www.rabbitmq.com/ttl.html) will still only expire from the
head of the queue. This means that unlike with normal queues, even
per-queue TTL can lead to expired lower-priority messages getting
stuck behind non-expired higher priority ones. These messages will
never be delivered, but they will appear in queue statistics.

* Queues which have a max-length set (as at
http://www.rabbitmq.com/maxlength.html) will, as usual, drop
messages from the head of the queue to enforce the limit. This means
that higher priority messages might be dropped to make way for lower
priority ones, which might not be what you would expect.
@@ -0,0 +1,51 @@
package com.rabbitmq.examples;

import com.rabbitmq.client.AMQP.BasicProperties;
import com.rabbitmq.client.Channel;
import com.rabbitmq.client.Connection;
import com.rabbitmq.client.ConnectionFactory;
import com.rabbitmq.client.DefaultConsumer;
import com.rabbitmq.client.Envelope;
import com.rabbitmq.client.MessageProperties;

import java.io.IOException;
import java.util.HashMap;
import java.util.Map;
import java.util.concurrent.CountDownLatch;

public class PriorityQueue {
private static final String QUEUE = "my-priority-queue";

public static void main(String[] argv) throws Exception {
ConnectionFactory factory = new ConnectionFactory();
Connection conn = factory.newConnection();
Channel ch = conn.createChannel();

Map<String, Object> args = new HashMap<String, Object>();
args.put("x-max-priority", 10);
ch.queueDeclare(QUEUE, true, false, false, args);

publish(ch, 0);
publish(ch, 5);
publish(ch, 10);

final CountDownLatch latch = new CountDownLatch(3);
ch.basicConsume(QUEUE, true, new DefaultConsumer(ch) {
@Override
public void handleDelivery(String consumerTag, Envelope envelope, BasicProperties properties, byte[] body) throws IOException {
System.out.println("Received " + new String(body));
latch.countDown();
}
});

latch.await();
conn.close();
}

private static void publish(Channel ch, int priority) throws IOException {
BasicProperties props = MessageProperties.PERSISTENT_BASIC.builder().priority(priority).build();
String body = "message with priority " + priority;
System.out.println("Sent " + body);
ch.basicPublish("", QUEUE, props, body.getBytes());
}
}
@@ -0,0 +1,14 @@
RELEASABLE:=true
DEPS:=rabbitmq-server rabbitmq-erlang-client rabbitmq-test
FILTER:=all
COVER:=false
WITH_BROKER_TEST_COMMANDS:=rabbit_test_runner:run_in_broker(\"$(PACKAGE_DIR)/test/ebin\",\"$(FILTER)\")
STANDALONE_TEST_COMMANDS:=rabbit_test_runner:run_multi(\"$(UMBRELLA_BASE_DIR)/rabbitmq-server\",\"$(PACKAGE_DIR)/test/ebin\",\"$(FILTER)\",$(COVER),\"/tmp/rabbitmq-multi-node/plugins\")

# NB: we cannot use PACKAGE_DIR in the body of this rule as it gets
# expanded at the wrong time and set to the value of a completely
# arbitrary package!
$(PACKAGE_DIR)+pre-test:: $(PACKAGE_DIR)+dist
rm -rf /tmp/rabbitmq-multi-node/plugins
mkdir -p /tmp/rabbitmq-multi-node/plugins/plugins
cp -p $(UMBRELLA_BASE_DIR)/rabbitmq-priority-queue/dist/*.ez /tmp/rabbitmq-multi-node/plugins/plugins

0 comments on commit 663df29

Please sign in to comment.