POC Make all queues on Rabbitmq quorum queue when quorum option is en… #2065
Conversation
| QueueArguments.Builder builder = configuration.workQueueArgumentsBuilder(); | ||
| configuration.getQueueTTL().ifPresent(builder::queueTTL); |
There was a problem hiding this comment.
Are those options compatible?
There was a problem hiding this comment.
Seems not, good catch
There was a problem hiding this comment.
Indeed TTL seems to work with quorum queue cf: https://www.rabbitmq.com/docs/quorum-queues#queue-and-per-message-ttl-since-rabbitmq-310
Those parameters are not compatible with quorum queues
| .autoDelete(AUTO_DELETE) | ||
| .autoDelete(!AUTO_DELETE) |
There was a problem hiding this comment.
the key queue seems volatile and can accept message lost to me -> Maybe we do not need to apply quorum queue on key queues?
After James restarts, a key queue would not be consumed anymore. So I think we should keep autoDelete if possible to avoid flushing RabbitMQ with unused key queues.
There was a problem hiding this comment.
But thinking more about that, not applying quorum queue to the key queues could again break James if a RabbitMQ node is down...
There was a problem hiding this comment.
Likely we can write a task to clean up inactive RabbitMQ queues (no consumers)
There was a problem hiding this comment.
Hmm normally key queues should be cleaned up by James upon James shutdown: https://github.com/apache/james-project/blob/master/event-bus/distributed/src/main/java/org/apache/james/events/KeyRegistrationHandler.java#L140
Do not Guess. Do not use the word maybe. If needed experiment. How does this break things? Reject upon publish? Loose the notification? Are we able to recover connection loss afterward?
How about "not clean shutdown?"
In the best of the world I would prefer we do not come down to that... Also keep in mind that RabbitMQ pubsub architecture (keys) could be replaced by redis. Do that work presently allows RabbitHA when using Redis on the pubsub infrastructure? |
|
Work moved here => #2191 Closing this. |
Queue TTL still can apply on quorum queues -> dangling key queues would be cleared by RabbitMQ. I just checked: we already apply the queue TTL on the ephemeral queues. |
…abled