KAFKA-10292: Set min.insync.replicas to 1 of __consumer_offsets #9286
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
The test StreamsBrokerBounceTest.test_all_brokers_bounce() fails on 2.5 because in the last stage of the test there is only one broker left and the offset commit cannot succeed because the min.insync.replicas of __consumer_offsets is set to 2 and acks is set to all. This causes a time out and extends the closing of the Kafka Streams client to beyond the duration passed to the close method of the client. This affects especially the 2.5 branch since there Kafka Streams commits offsets for each task, i.e., close() needs to wait for the timeout for each task. In 2.6 and trunk the offset commit is done per thread, so close() does only need to wait for one time out per stream thread. I opened this PR on trunk, since the test could also become flaky on trunk and we want to avoid diverging system tests across branches. A more complete solution would be to improve the test by defining a better success criteria.
|
Call for review: @vvcephei @guozhangwang |
|
System test run for trunk: System test run for 2.6: System test run for 2.5: |
|
LGTM! |
|
@guozhangwang, Thank you for merging! Could you please cherry-pick this PR to 2.6 and 2.5? |
guozhangwang
pushed a commit
that referenced
this pull request
Sep 15, 2020
The test StreamsBrokerBounceTest.test_all_brokers_bounce() fails on 2.5 because in the last stage of the test there is only one broker left and the offset commit cannot succeed because the min.insync.replicas of __consumer_offsets is set to 2 and acks is set to all. This causes a time out and extends the closing of the Kafka Streams client to beyond the duration passed to the close method of the client. This affects especially the 2.5 branch since there Kafka Streams commits offsets for each task, i.e., close() needs to wait for the timeout for each task. In 2.6 and trunk the offset commit is done per thread, so close() does only need to wait for one time out per stream thread. I opened this PR on trunk, since the test could also become flaky on trunk and we want to avoid diverging system tests across branches. A more complete solution would be to improve the test by defining a better success criteria. Reviewers: Guozhang Wang <wangguoz@gmail.com>
guozhangwang
pushed a commit
that referenced
this pull request
Sep 15, 2020
The test StreamsBrokerBounceTest.test_all_brokers_bounce() fails on 2.5 because in the last stage of the test there is only one broker left and the offset commit cannot succeed because the min.insync.replicas of __consumer_offsets is set to 2 and acks is set to all. This causes a time out and extends the closing of the Kafka Streams client to beyond the duration passed to the close method of the client. This affects especially the 2.5 branch since there Kafka Streams commits offsets for each task, i.e., close() needs to wait for the timeout for each task. In 2.6 and trunk the offset commit is done per thread, so close() does only need to wait for one time out per stream thread. I opened this PR on trunk, since the test could also become flaky on trunk and we want to avoid diverging system tests across branches. A more complete solution would be to improve the test by defining a better success criteria. Reviewers: Guozhang Wang <wangguoz@gmail.com>
|
Yeah I've already cherry-picked to 2.6/2.5. |
|
Thanks a lot! |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
The test StreamsBrokerBounceTest.test_all_brokers_bounce() fails on
2.5 because in the last stage of the test there is only one broker
left and the offset commit cannot succeed because the
min.insync.replicas of __consumer_offsets is set to 2 and acks is
set to all. This causes a time out and extends the closing of the
Kafka Streams client to beyond the duration passed to the close
method of the client.
This affects especially the 2.5 branch since there Kafka Streams
commits offsets for each task, i.e., close() needs to wait for the
timeout for each task. In 2.6 and trunk the offset commit is done
per thread, so close() does only need to wait for one time out per
stream thread.
I opened this PR on trunk, since the test could also become
flaky on trunk and we want to avoid diverging system tests across
branches.
A more complete solution would be to improve the test by defining
a better success criteria.
Committer Checklist (excluded from commit message)