Refresh metadata after restart #404
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
When the consumer crash, the broker is restarted, but it relies on the lack of metadata to re-connect. This error happens deep into the consumer group flow, and it isn't clear
This was turned on when we used to use travis as the CI server
Nevon
approved these changes
Jun 25, 2019
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
We recently upgraded to Kafka 2 and are currently changing some broker configurations, which leads to a lot of broker restarts. During these scenarios, some consumers take too long to recover since they keep retrying on old brokers and fail to acknowledge the change.
This PR does three things:
1 and 2 improves the recovery time, instead of trashing the consumer and restarting it refreshes metadata and connect to the right broker
3 explicitly prepares the consumer group for the new run instead of relying on errors deep into the consumer group flow; this should improve the restart speed since consumers won't need to fail to execute these operations.