KAFKA-19827: Call ack commit callback at end of waiting calls #20758
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
apoorvmittal10
approved these changes
Oct 23, 2025
Contributor
apoorvmittal10
left a comment
apoorvmittal10
left a comment
There was a problem hiding this comment.
LGTM, minor nit comment.
clients/src/main/java/org/apache/kafka/clients/consumer/internals/ShareConsumerImpl.java
Outdated
Show resolved
Hide resolved
ShivsundarR
approved these changes
Oct 24, 2025
Contributor
ShivsundarR
left a comment
ShivsundarR
left a comment
There was a problem hiding this comment.
Thanks for the PR. LGTM!
joshua2519
pushed a commit
to joshua2519/kafka
that referenced
this pull request
Oct 27, 2025
…#20758) The acknowledgement commit callback in the share consumer gets called on the application thread at the start of the poll, commitSync and commitAsync methods. Specifically in the peculiar case of using the callback together with commitSync, the acknowledgement callback for the committed records is called at the start of the next eligible call, even though the information is already known at the end of the commitSync's execution. The results are correct already, but the timing could be improved in some situations. Reviewers: Apoorv Mittal <apoorvmittal10@gmail.com>, Shivsundar R <shr@confluent.io>
TaiJuWu
reviewed
Oct 28, 2025
Comment on lines
+615
to
+619
| try { | ||
| handleCompletedAcknowledgements(false); | ||
| } catch (Throwable t) { | ||
| log.warn("Exception thrown in acknowledgement commit callback", t); | ||
| } |
Collaborator
There was a problem hiding this comment.
I find this change make testShareGroupMaxSizeConfigExceeded become flaky.
The following code will fix this issue but seem to violate the comments.
// Handle any acknowledgements which completed while we were waiting, but do not throw
// the exception because the fetched records would then not be returned to the caller
try {
handleCompletedAcknowledgements(false);
} catch (GroupMaxSizeReachedException e) {
log.warn("Exception thrown in acknowledgement commit callback", e);
throw e;
} catch (Throwable t) {
log.warn("Exception thrown in acknowledgement commit callback", t);
}
Member
There was a problem hiding this comment.
@TaiJuWu thanks for this find. The flaky behaviour and its root cause have already been logged in https://issues.apache.org/jira/browse/KAFKA-19840
eduwercamacaro
pushed a commit
to littlehorse-enterprises/kafka
that referenced
this pull request
Nov 12, 2025
…#20758) The acknowledgement commit callback in the share consumer gets called on the application thread at the start of the poll, commitSync and commitAsync methods. Specifically in the peculiar case of using the callback together with commitSync, the acknowledgement callback for the committed records is called at the start of the next eligible call, even though the information is already known at the end of the commitSync's execution. The results are correct already, but the timing could be improved in some situations. Reviewers: Apoorv Mittal <apoorvmittal10@gmail.com>, Shivsundar R <shr@confluent.io>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
5 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
The acknowledgement commit callback in the share consumer gets called on
the application thread at the start of the poll, commitSync and
commitAsync methods. Specifically in the peculiar case of using the
callback together with commitSync, the acknowledgement callback for the
committed records is called at the start of the next eligible call, even
though the information is already known at the end of the commitSync's
execution. The results are correct already, but the timing could be
improved in some situations.
Reviewers: Apoorv Mittal apoorvmittal10@gmail.com, Shivsundar R
shr@confluent.io