Trigger hot reload for reactive messaging

[mswiderski]

Description
Trigger hot reload of the application when new Kafka message is received, would be great to have it supported not only for Kafka but all transports supported by reactive messaging

Relates to discussion thread https://groups.google.com/d/msgid/quarkus-dev/5F5FFC95-3C70-4756-8191-13847D63D540%40gmail.com?utm_medium=email&utm_source=footer

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you!
We are doing this automatically to ensure out-of-date issues does not stay around indefinitely.
If you believe this issue is still relevant please put a comment on it on why and if it truly needs to stay request or add 'pinned' label.

[mswiderski]

I'd say this is still very relevant to have a complete store on developer workflow when using reactive messaging.

[gsmet]

See also #2709 .

[emmanuelbernard]

@cescoffier is that still something to be done? Or is it done? Just for cleanup purposes

[gsmet]

No, it's not done and we had several people asking for it.

[cescoffier]

@Ladicek that could be an interesting one, if you are interested.

The idea is to check for "reload" when we receive a message. There are multiple ways to do it.

1. Protocol specific

Basically, for Kafka, we can reproduce what we do for Kafka Streams. We introduce an interceptor and trigger the hot reload. To avoid too many verifications, we introduce a grace period.

I'm not sure we would be able to do the same for AMQP (there is no interceptors support in the client)

2. Protocol agnostic

This is more complicated. We can imagine introducing an interceptor in the connectors to do this.
Connectors are beans, so we can "intercept" them in dev mode, and introduce the reload logic when the connector emits a message. Something like:

PublisherBuilder pb = connector.getPublisherBuilder(Config config);
return pb.peek(m -> triggerReloadIfNeeded());

We just need to be sure to not lose the inflight message (that can be the tricky part).

[Ladicek]

I guess we want this for all protocols, not just Kafka (but if Kafka is easier, perhaps we can just start there). And, as mentioned above, not just when receiving a message, but perhaps also when emitting a message. Interesting one indeed :-) I'll finish some unfinished tasks and take a look at this. Nice opportunity to get deeper into Quarkus internals (hope I won't regret it :-) ).

[Ladicek]

I have a little bit of a prototype here: https://github.com/Ladicek/quarkus/commits/reactive-messaging-live-reload It seems to function well for instrumentation-based reload, with Kafka. Need to verify with other brokers and full restart, and then of course write some tests (gonna be interesting).

[Ladicek]

I've pushed something that behaves, I believe, reasonably well -- both with instrumentation and full restart (tested with Kafka and AMQP on Artemis). I'm pretty sure there are some small windows for message loss, but frankly, I don't think that's a big deal in dev mode. I still need to figure out how to write tests.

[cescoffier]

Awesome, did you open a PR?

[cescoffier]

Data loss should be mitigated by missing ack - so basically, it should be still in the queue / topic

[Ladicek]

Need to create some tests, PR will obviously follow.