Application Client Backpressure: Pause/Resume via Kafka Consumer API#291
Merged
Conversation
…ew method to ensure the topic exists to eliminate the blocking
…d a bug for shared subs when the related consumers are not getting back on backpressure
…ent race conditions
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Pull Request description
#290
Previously, application client consumers were fully stopped and restarted on channel backpressure events. This had a bug where shared subscription consumers were not being resumed after the channel became writable again, causing message lag to grow continuously until the client reconnected.
This PR replaces the stop/start approach with Kafka's built-in pause()/resume() consumer API. Both main and shared subscription consumers are now paused when the channel becomes non-writable and seamlessly resumed when it recovers — without any consumer group lifecycle overhead. This leads to more efficient system resource usage, eliminates the lag accumulation bug in shared subscription processing, and improves overall throughput stability under backpressure conditions.
General checklist
Front-End feature checklist
Back-End feature checklist