KAFKA-7568; Return leader epoch in ListOffsets response #5855
Conversation
|
Is this needed for 2.1? |
|
@ijuma Nothing is depending on it yet, so I think it is not strictly needed. Nevertheless, I was considering bugging @lindong28 about it (late though it is) since the changes are straightforward and it is a little annoying to have an incomplete version bump. |
|
RC0 has already gone out, so unless it's a blocker, we should not include it. |
|
@ijuma I am fine with it going to trunk only. |
There was a problem hiding this comment.
@hachikuji : Thanks for the patch. Looks good overall. Just one comment below.
| @@ -1279,9 +1288,11 @@ class Log(@volatile var dir: File, | |||
| val segmentsCopy = logSegments.toBuffer | |||
| // For the earliest and latest, we do not need to return the timestamp. | |||
| if (targetTimestamp == ListOffsetRequest.EARLIEST_TIMESTAMP) | |||
| return Some(TimestampOffset(RecordBatch.NO_TIMESTAMP, logStartOffset)) | |||
| return Some(new TimestampAndOffset(RecordBatch.NO_TIMESTAMP, logStartOffset, | |||
| maybeLeaderEpoch(leaderEpochCache.earliestEpoch))) | |||
There was a problem hiding this comment.
The message format may have been changed after logStartOffset. In that case, leaderEpochCache.earliestEpoch may not match logStartOffset. So, we probably want to also make sure the offset corresponding to leaderEpochCache.earliestEpoch matches logStartOffset.
hachikuji
force-pushed
the
KAFKA-7568
branch
2 times, most recently
from
67a68c5
to
ab5552e
Compare
|
Thanks @junrao. Pushed an update to address your comment. |
|
Thanks @junrao. I will go ahead and merge. Note that we are investigating the jdk11 failures in https://issues.apache.org/jira/browse/KAFKA-7553 and elsewhere. |
As part of KIP-320, the ListOffsets API should return the leader epoch of any fetched offset. We either get this epoch from the log itself for a timestamp query or from the epoch cache if we are searching the earliest or latest offset in the log. When handling queries for the latest offset, we have elected to choose the current leader epoch, which is consistent with other handling (e.g. OffsetsForTimes). Reviewers: Jun Rao <junrao@gmail.com>
As part of KIP-320, the ListOffsets API should return the leader epoch of any fetched offset. We either get this epoch from the log itself for a timestamp query or from the epoch cache if we are searching the earliest or latest offset in the log. When handling queries for the latest offset, we have elected to choose the current leader epoch, which is consistent with other handling (e.g. OffsetsForTimes).
Committer Checklist (excluded from commit message)