KAFKA-19808: Handle batch alignment when share partition is at capacity #20738
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
github-actions
bot
added
triage
eduwercamacaro
pushed a commit
to littlehorse-enterprises/kafka
that referenced
this pull request
Nov 12, 2025
…ty (apache#20738) The PR fixes couple of issues which are related to the batch alignment: 1. Base offset not found for some acquisition lock timeouts. 2. Gaps reported by client when the topic cannot have any gaps. The root cause was the misaligned batches when cache is full but the fetch happens in middle of cached offsets i.e. some records are released because of acquisition lock time out or expilicitly. As the subsequent fetch can have lesser bytes of data but the method `lastOffsetAndMaxRecordsToAcquire` aligned the lastOffsetToAcquire to `endOffset` hence incorrect batches were created. The PR fixes that by adding a `min` comparison. Reviewers: Andrew Schofield <aschofield@confluent.io>, Abhinav Dixit <adixit@confluent.io>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
The PR fixes couple of issues which are related to the batch alignment:
The root cause was the misaligned batches when cache is full but the
fetch happens in middle of cached offsets i.e. some records are released
because of acquisition lock time out or expilicitly. As the subsequent
fetch can have lesser bytes of data but the method
lastOffsetAndMaxRecordsToAcquirealigned the lastOffsetToAcquire toendOffsethence incorrect batches were created. The PR fixes that byadding a
mincomparison.Reviewers: Andrew Schofield aschofield@confluent.io, Abhinav Dixit
adixit@confluent.io