Permalink
Browse files

Discuss caveats when a message fails to be processed.

  • Loading branch information...
1 parent cecc796 commit 28387271598e7454d7e7eddd135d956215846e59 @samstokes samstokes committed Oct 31, 2011
Showing with 15 additions and 0 deletions.
  1. +9 −0 README.markdown
  2. +6 −0 src/main/java/com/rapportive/storm/spout/AMQPSpout.java
View
9 README.markdown
@@ -66,6 +66,15 @@ while the task remains down. See [SharedQueueWithBinding][] to declare a
shared queue that allows for guaranteed processing. (For prototyping, an
[ExclusiveQueueWithBinding][] may be simpler to manage.)
+This does not currently handle malformed messages very well: the spout worker
+will crash if the provided [Scheme][] fails to deserialise a message.
+
+This does not currently support retrying messages in the event of transient
+failure to process: any message which the topology fails to process will simply
+be dropped. This is to prevent infinite redelivery in the event of
+non-transient failures (e.g. malformed messages, though see previous caveat!).
+This will probably be made configurable in a future release.
+
## Compatibility ##
AMQPSpout has been tested with RabbitMQ 2.3.1 and 2.6.1. It should probably work with other
View
6 src/main/java/com/rapportive/storm/spout/AMQPSpout.java
@@ -35,6 +35,10 @@
* {@link com.rapportive.storm.amqp.ExclusiveQueueWithBinding} may be
* simpler to manage.)</p>
*
+ * <p><strong>N.B.</strong> this does not currently handle malformed messages
+ * (which cannot be deserialised by the provided {@link Scheme}) very well:
+ * the spout worker will crash if it fails to serialise a message.</p>
+ *
* <p>This consumes messages from AMQP asynchronously, so it may receive
* messages before Storm requests them as tuples; therefore it buffers messages
* in an internal queue. To avoid this buffer growing large and consuming too
@@ -180,6 +184,8 @@ public void close() {
* redelivery in the event of non-transient failures (e.g. malformed
* messages). However it means that messages will <em>not</em> be retried
* in the event of transient failures.</p>
+ *
+ * <p><strong>TODO</strong> make this configurable.</p>
*/
@Override
public void fail(Object msgId) {

0 comments on commit 2838727

Please sign in to comment.