MINOR: reduce Metadata log volume when last seen epoch is not updated #12541
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Context ------- Since updating to kafka client 3.0.1, we noticed quite a lot of new info-level logs generated by `org.apache.kafka.clients.Metadata`. The logs look like this: ``` [Producer clientId=producer-1] Resetting the last seen epoch of partition user-3 to 2044 since the associated topicId changed from null to O5w7zNcCTpKdylvkOUMesQ ``` They appear at seemingly random times, while keeping the same last seen epoch value. Proposed fix ------------ As suggested in apache#12378, I am updating the code condition to avoid generating a log when the last seen epoch reset operation is a no-op.
|
Thanks @fabien-chebel -- this was a solution I was thinking of when reviewing https://github.com/apache/kafka/pull/12378/files |
|
Another option I was thinking of was removing the log line from the case where the previous topic ID was null. |
jolshan
approved these changes
Aug 19, 2022
jolshan
reviewed
Nov 21, 2022
| log.info("Resetting the last seen epoch of partition {} to {} since the associated topicId changed from {} to {}", | ||
| tp, newEpoch, oldTopicId, topicId); | ||
| lastSeenLeaderEpochs.put(tp, newEpoch); | ||
| if (!Objects.equals(currentEpoch, newEpoch)) { |
There was a problem hiding this comment.
I'm wondering though if using a primitive and an Integer is ok here. Maybe we can just compare the value of currentEpoch?
3 tasks
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Context
Since updating to kafka client 3.0.1, we noticed quite a lot of new info-level logs generated by
org.apache.kafka.clients.Metadata.The logs look like this:
They appear at seemingly random times, while keeping the same last seen epoch value.
Proposed fix
As suggested in #12378, I am updating the code condition to avoid generating a log when the last seen epoch reset operation is a no-op.
Committer Checklist (excluded from commit message)