Custom metrics/logging integration point

[nikoncode]

I want to build custom metrics over handler execution time and result (success/unsuccess).

After exploring spring-kafka codebase I found kind of hook-based micrometer built-in implementation (org.springframework.kafka.listener.KafkaMessageListenerContainer.ListenerConsumer#doInvokeRecordListener). But end-user cannot customize it.

Why it's so strict? I mean spring-kafka has various integration points: interceptor (before-execution), error handlers (in case of error) but no handler to connect them all together (even using ThreadLocal).

I expect something like this:

interface Hook {
   void before(ConsumerRecord);
   void after(ConsumerRecord);
   void error(ConsumerRecord, Exception);
}

And existing micrometer integration can work through this interface as well.

If there is any way how to handle all these events exists, please let me know.

P.S.: in our production, we can partially solve this problem by using aspects, but in case of multiple topics in KafkaListener we are not able to extract topic name because of using value injection.

[garyrussell]

I believe we can add additional (default) methods to the existing RecordInterceptor for this.

[nikoncode]

@artembilan @garyrussell BTW absence of this integration ruins migration to spring-kafka inside our organization since we cannot integrate tracing and metrics because of that.

As I understand this one will be addressed in 2.7.0 which will be included only in spring boot 2.5.x.

Does it possible to backport this change into 2.6.x to give us possibility to use it earlier than boot 2.5.x.

We have tried multiple ways how to bypass that including AOP, but it's not possible to get information about topic (in case of wildcard subscription) if you are using plain POJO instead of ConsumerRecord.

[artembilan]

Well, technically Spring Boot 2.5 is due on May 20th: https://github.com/spring-projects/spring-boot/milestone/201 - just a couple weeks! If even we back-port that fix into 2.6.x, it is going to happen around the same time anyway.

saying that I see @garyrussell has scheduled this for back-porting, but then remove that label. Perhaps there was a reason to consider this feature as a breaking change somehow...

[garyrussell]

I don't remember why I decided not to back-port - I'll take a look to see how hard it might be.

[nikoncode]

@artembilan @garyrussell we have big amount of services, that's why it might be hard to migrate anyway, moreover we use spring cloud which is not ready yet for 2.5

@garyrussell please, It will be fantastic.

[garyrussell]

Backported as ffa7e80 - 2.6.8 is currently scheduled for May 17th.