Browse files

Readme and bump to 0.0.1

  • Loading branch information...
1 parent b167e40 commit d3661c40b3ed86d2570049c2e4044fdf6ac5f729 @samstokes samstokes committed Sep 29, 2011
Showing with 82 additions and 1 deletion.
  1. +81 −0 README.markdown
  2. +1 −1 pom.xml
81 README.markdown
@@ -0,0 +1,81 @@
+# storm-amqp-spout: AMQP input source for Storm #
+storm-amqp-spout allows a Storm topology to consume an AMQP exchange as an
+input source. It currently provides one class:
+ * [AMQPSpout](
+ an implementation of
+ [`backtype.storm.topology.IRichSpout`](
+ that connects to an AMQP broker and consumes the messages published to a
+ specified AMQP exchange.
+You'll need to provide a
+to tell `AMQPSpout` how to interpret the messages and turn them into Storm
+tuples. See e.g. [storm-json]( if
+your messages are JSON.
+## Documentation ##
+The Javadocs can be found at []().
+## Usage ##
+To produce a jar:
+ $ mvn package
+To install in your local Maven repository:
+ $ mvn install
+To use in your `pom.xml`:
+ <!-- ... -->
+ <dependencies>
+ <!-- ... -->
+ <dependency>
+ <groupId>com.rapportive</groupId>
+ <artifactId>storm-amqp-spout</artifactId>
+ <version>0.0.1</version>
+ </dependency>
+ <!-- ... -->
+ </dependencies>
+ <!-- ... -->
+## Caveats ##
+This is very early software. It may break and the API is liable to change
+completely between releases. Pull requests, patches and bug reports are very
+This should not currently be used where guaranteed message processing is
+required, because of two limitations:
+1. Uses a temporary queue to bind to the specified exchange when the topology
+on the spout, so it will only receive messages published to the exchange after
+the call to `open()`, and if the spout worker restarts or the topology is
+killed, it will not receive any messages published while the worker or topology
+is down.
+2. Currently auto-acks all consumed messages with the AMQP broker, and does not
+implement Storm's reliability API, so if processing a message fails it will
+simply be discarded.
+Limitation 1 also means this spout cannot currently be distributed among
+multiple workers (each worker gets its own exclusive queue, so multiple
+workers would each receive their own copy of every message).
+Improvements are planned to overcome both these limitations and support
+guaranteed message processing, distributed across any number of workers.
+These improvements may require API changes (e.g. to specify the name of an
+existing queue to consume, rather than an exchange to bind to).
+`AMQPSpout` has been tested with RabbitMQ. It should probably work with other
+AMQP brokers.
2 pom.xml
@@ -2,7 +2,7 @@
- <version>0.0.1-SNAPSHOT</version>
+ <version>0.0.1</version>

0 comments on commit d3661c4

Please sign in to comment.