Join GitHub today
GitHub is home to over 40 million developers working together to host and review code, manage projects, and build software together.Sign up
KAFKA-8788: Optimize client metadata handling with a large number of partitions #7192
Credit to @lbradstreet for profiling the producer with a large number of partitions.
Committer Checklist (excluded from commit message)
changed the title
MINOR: Compute `brokersMap` once in `MetadataResponse.topicMetadata()`
Aug 11, 2019
In the long term, I don't think we need MetadataResponse#TopicMetadata any more.
I think the better fix of just using the generated classes is actually not that much harder. If you leave out test code, there are not that many places that use MetadataResponse#TopicMetadata. KafkaApis, the MetadataCache, the Metadata class, and KafkaAdminClient#listTopics.
@cmccabe Unfortunately, the generated API is mutable and harder to use/read (no support for
I think we should merge this as it is to trunk and 2.3. And make further changes in trunk, if needed.
One job passed, two failed with a single unrelated flaky test each:
Merging to master and 2.3.