KAFKA-3640: Reduce the latency of topic metadata requests #1303
Conversation
| new TopicMetadata(topic, Seq.empty[PartitionMetadata], ErrorMapping.InvalidTopicCode) | ||
| } | ||
| if (topic == GroupCoordinator.GroupMetadataTopicName) { | ||
| createGroupMetadataTopic() |
There was a problem hiding this comment.
Love the cleanup. Thank you.
|
First, thanks for the cleanup. It looks fantastic. Did you do before / after tests for latency, or is this "common sense optimization"? (either is fine, but if you have numbers, do share...) And last but not least, should this go into 0.10.0 branch? It looks like a harmless refactoring, but I have to admit that anything that touches the metadata cache is a bit scary and we may want to give this more time to "bake". @ijuma - your thoughts? |
|
@gwenshap This is a backport of a change that is already in trunk and 0.10.0 (KAFKA-2073). The backport is more conservative as if doesn't use the new Java request classes. The PR for KAFKA-2073 includes some benchmark numbers and this backport has been tested in a large deployment where it caused a significant reduction in latency. One of the improvements is the one you mentioned. The other is that we avoid unnecessary collection copies in |
|
LGTM. |
Changes based on KAFKA-2073 to reduce the latency of metadata requests. Author: Flavio Junqueira <fpj@apache.org> Reviewers: Gwen Shapira Closes #1303 from fpj/KAFKA-3640
|
@fpj This was merged by Gwen. Can you please close the PR? (only PRs to trunk are auto-closed sadly). |
Changes based on KAFKA-2073 to reduce the latency of metadata requests.