Backport Kafka client 2.0.0 in branch 2.1.x

[bgK]

Would it be possible to add support for the Kafka clients 2.0.0 in branch 2.1.x ?

Or do you think branch 2.1.x can work with the 2.0.0 Kafka clients by overriding the dependencies ?

Kafka 2.0.0 has some interesting bug fixes for us, and we'd like to upgrade without necessarily upgrading the rest of the Spring ecosystem.

[garyrussell]

Exactly which changes are you looking for?

It has not been tested but it might work.

However, the deprecated blocking APIs (e.g. poll(long)) will still be used (instead of the non-blocking poll(Duration)).

We do not intend to backport support for 2.0.0 to the 2.1 line (non-blocking APIs).

We expect to release the next milestone of 2.2 (including the changes for 2.0.0 non-blocking APIs) next week, with the release due next month or early October.

[bgK]

Thanks for your answer. We are mostly interested in the fix for KAFKA-6446 so the producers don't block indefinitely if for some reason the brokers fail to respond with a producerId (It has happened to us with the brokers properly started).

I'll try overriding the dependencies and report back here.

[bgK]

Overriding the dependencies to use kafka-clients 2.0.0 seems to work well enough for our purpose, except the embedded Kafka broker due to a hardcoded version number:

spring-kafka/spring-kafka-test/src/main/java/org/springframework/kafka/test/rule/KafkaEmbedded.java

Line 102 in f9f7d9c

if (clientVersion.startsWith("1.1.")) {