New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
KAFKA-14462; [6/N] Update Records #13536
Conversation
{ "name": "PreviousMemberEpoch", "versions": "0+", "type": "int32", | ||
"about": "The previous member epoch." }, |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We need this in the case where the epoch bump does not reach the member. In this case, it will retry with the old epoch and we want to accept it.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Could we move some of these comments into the "about" documentation?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Sure. Updated those when relevant.
{ "name": "TargetMemberEpoch", "versions": "0+", "type": "int32", | ||
"about": "The target member epoch." }, |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We need this in order to know what was the assignment epoch used to compute the assigned partitions, etc. When if diverges with the current assignment epoch, we know that we need to recompute the assignment.
{ "name": "AssignedPartitions", "versions": "0+", "type": "[]TopicPartitions", | ||
"about": "The partitions assigned to (owned by) this member." }, | ||
{ "name": "PartitionsPendingRevocation", "versions": "0+", "type": "[]TopicPartitions", | ||
"about": "The partitions being revoked by this member." }, | ||
{ "name": "PartitionsPendingAssignment", "versions": "0+", "type": "[]TopicPartitions", | ||
"about": "The partitions being assigned to this member." }, |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This is the biggest divergence from the KIP. Initially, I wanted to store only the full assignment and to bookkeep those in-memory. I found this extremely confusing during the implementation and error prone as well. It is much better to have three sets instead of ones.
{ "name": "GroupEpoch", "versions": "0+", "type": "int32", | ||
"about": "The group epoch." }, |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This is not necessary because we have a record for the group epoch.
{ "name": "RebalanceTimeoutMs", "type": "int32", "versions": "0+", "default": -1, | ||
"about": "The rebalance timeout" }, | ||
{ "name": "ServerAssignor", "versions": "0+", "nullableVersions": "0+", "type": "string", | ||
"about": "The server assignor to use; or null if not used." }, |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Missing fields.
{ "name": "TopicName", "versions": "0+", "type": "string", | ||
"about": "The topic name." }, |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Missing field.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
These changes make sense to me. Does it make sense to also update the KIP once this gets merged?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM
This patch updates the records introduced by KIP-848.
Committer Checklist (excluded from commit message)