New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
KAFKA-15078; KRaft leader replys with snapshot for offset 0 #13845
KAFKA-15078; KRaft leader replys with snapshot for offset 0 #13845
Conversation
If the follower has an empty log, fetches with offset 0, it is more efficient for the leader to reply with a snapshot id (redirect to FETCH_SNAPSHOT) than for the follower to continue fetching from the log segments.
} | ||
|
||
@Test | ||
public void testFetchRequestOffsetAtZero() throws Exception { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This test fails against trunk
.
|
||
Optional<OffsetAndEpoch> latestSnapshotId = log.latestSnapshotId(); | ||
final ValidOffsetAndEpoch validOffsetAndEpoch; | ||
if (fetchOffset == 0 && latestSnapshotId.isPresent()) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I have tried to figure out some other cases we prefer to read the snapshot, a common case is highWatermark
- fetchOffset
> count(snapshot record). Do you think it's possible to write count(snapshot record) in the snapshot header?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks @dengziming
We had a similar conversation in another PR: #13834 (comment)
In short, it is not clear to me that these improvements (implementation complexities) are a big win for the cluster metadata partition.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks @jsancio. LGTM as well
If the follower has an empty log, fetches with offset 0, it is more efficient for the leader to reply with a snapshot id (redirect to FETCH_SNAPSHOT) than for the follower to continue fetching from the log segments.
Committer Checklist (excluded from commit message)