Skip to content
No description or website provided.
Branch: master
Clone or download
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
src/test
.gitignore
README.md
pom.xml

README.md

Problem

Assume a queue of messages. Each message belongs to exactly one group. The goal is to execute messages of the same group in sequence. Messages of different groups are allowed to be executed in parallel.

All solutions rely on Apache Camel 2.23.x, a widely used integration framework.

Solutions

[A] ActiveMQ - Message groups

Note that this does not support safe dynamic rebalancing of consumers without external synchronization. We can reset a message group (by setting JMSXGroupSeq to -1), but we must first make sure that there are no pending messages for that particular group.

[B] RabbitMQ - Consistent hash exchange

Note that this does not support safe dynamic rebalancing of consumers without external synchronization. We can alter the routing topology by adding/removing a queue, but we must first make sure that there are no pending messages at all.

[C] RabbitMQ/ActiveMQ & sticky load balancer & in-memory SEDA queues

Note that this does not support safe dynamic rebalancing of consumers.

Also note that messages could be lost after a service crash or restart! A message will be acknowledged once it has been added to the SEDA queue, as the route ends at that point. In theory, we could alter the implementation to manually acknowledge the message after successful asynchronous execution, but we currently face the following limitations in Apache Camel:

  • RabbitMQ:
    • Asynchronous processing is currently not supported.
    • Manual acknowledgment is currently not supported.
  • ActiveMQ:
    • Asynchronous transactions are scheduled for Camel 3.0, as documented here.
    • As an alternative to asynchronous transactions, we could use asynchronous processing (which can be enabled via the asyncConsumer=true property, as documented here) and the INDIVIDUAL_MESSAGE acknowledgment mode, as documented here. Unfortunately, the INDIVIDUAL_MESSAGE acknowledgment mode is currently not supported, as reported here.
      • Note that we cannot use the CLIENT_ACKNOWLEDGE acknowledgment mode, as Spring's MessageListenerContainer acknowledges messages synchronously in the message callback, as documented here. See also here and here.

[D] Kafka

This is not (yet) implemented here. This should be fairly trivial, as consumer groups are natively supported.

This should support safe dynamic rebalancing of consumers.

Additional resources

You can’t perform that action at this time.