Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update Apache Kafka client libraries to 3.0.0 #11735

Merged
merged 1 commit into from
Oct 5, 2021
Merged

Conversation

xvrl
Copy link
Member

@xvrl xvrl commented Sep 22, 2021

Release notes:
https://downloads.apache.org/kafka/3.0.0/RELEASE_NOTES.html
https://blogs.apache.org/kafka/entry/what-s-new-in-apache6

On the consumer side, I did not see anything notable that would affect Kafka ingestion,
other than maybe the consumer group timeout increase.

On the producer side the client now defaults to acks=all and idempotent producer which
could affect the KafkaEmitter extension

@FrankChen021
Copy link
Member

FrankChen021 commented Sep 24, 2021

Since Kafka 3.0 officially deprecates the support of Java 8 which is the official version used by Druid, do we still need to upgrade the Kafka client ?

@FrankChen021
Copy link
Member

By the way, Kafka 4.0 plans to drop support of Java 8. As Kafka ingestion is a core feature in Druid, what's roadmap for Druid to officially support JDK9+ ? @gianm

@xvrl
Copy link
Member Author

xvrl commented Sep 30, 2021

@FrankChen021 Kafka 3.x still supports Java 8 even if it is deprecated, so we should't have to worry about this immediately.
Regarding Java 11 compatibility, there is an outstanding issue #5589 where this is getting discussed. It might be worth to bring that thread back alive. Many people run Druid fine with Java 11 though.

@xvrl xvrl merged commit bc3b038 into apache:master Oct 5, 2021
@xvrl xvrl deleted the kafka-3.0 branch October 5, 2021 17:23
@xvrl
Copy link
Member Author

xvrl commented Oct 5, 2021

we should mention the acks=all change in the release notes in case this affects users using the KafkaEmitter extensions.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants