storm-amqp-spout: AMQP input source for Storm
storm-amqp-spout allows a Storm topology to consume an AMQP exchange as an input source. It currently provides one class:
an implementation of
backtype.storm.topology.IRichSpoutthat connects to an AMQP broker and consumes the messages published to a specified AMQP exchange.
The Javadocs can be found at http://code.rapportive.com/storm-amqp-spout.
To produce a jar:
$ mvn package
To install in your local Maven repository:
$ mvn install
To use in your
<project> <!-- ... --> <dependencies> <!-- ... --> <dependency> <groupId>com.rapportive</groupId> <artifactId>storm-amqp-spout</artifactId> <version>0.0.3-SNAPSHOT</version> </dependency> <!-- ... --> </dependencies> <!-- ... --> </project>
This is very early software. It may break and the API is liable to change completely between releases. Pull requests, patches and bug reports are very welcome.
This should not currently be used where guaranteed message processing is
required, because it binds to the exchange using a temporary queue when the
on the spout. This means it will only receive messages published to the
exchange after the call to
open(), and if the spout worker restarts or the
topology is killed, it will not receive any messages published while the worker
or topology is down.
For the same reason, this spout cannot currently be distributed among multiple workers (each worker gets its own exclusive queue, so multiple workers would each receive their own copy of every message).
Improvements are planned to overcome both these limitations and support guaranteed message processing, distributed across any number of workers. These improvements may require API changes (e.g. to specify the name of an existing queue to consume, rather than an exchange to bind to).
AMQPSpout has been tested with RabbitMQ 2.3.1 and 2.6.1. It should probably work with other
versions and other AMQP brokers.