Skip to content

Application Client Backpressure: Pause/Resume via Kafka Consumer API#291

Merged
dmytro-landiak merged 7 commits into
mainfrom
bug/app-shared-subs-backpressure
Mar 4, 2026
Merged

Application Client Backpressure: Pause/Resume via Kafka Consumer API#291
dmytro-landiak merged 7 commits into
mainfrom
bug/app-shared-subs-backpressure

Conversation

@dmytro-landiak
Copy link
Copy Markdown
Contributor

@dmytro-landiak dmytro-landiak commented Mar 4, 2026

Pull Request description

#290

Previously, application client consumers were fully stopped and restarted on channel backpressure events. This had a bug where shared subscription consumers were not being resumed after the channel became writable again, causing message lag to grow continuously until the client reconnected.

This PR replaces the stop/start approach with Kafka's built-in pause()/resume() consumer API. Both main and shared subscription consumers are now paused when the channel becomes non-writable and seamlessly resumed when it recovers — without any consumer group lifecycle overhead. This leads to more efficient system resource usage, eliminates the lag accumulation bug in shared subscription processing, and improves overall throughput stability under backpressure conditions.

General checklist

  • You have reviewed the guidelines document.
  • Labels that classify your pull request have been added.
  • The milestone is specified and corresponds to fix version.
  • Description references specific issue.
  • Description contains human-readable scope of changes.
  • Description contains brief notes about what needs to be added to the documentation.
  • No merge conflicts, commented blocks of code, code formatting issues.
  • Changes are backward compatible or upgrade script is provided.

Front-End feature checklist

  • Screenshots with affected component(s) are added. The best option is to provide 2 screens: before and after changes;
  • If you change the widget or other API, ensure it is backward-compatible or upgrade script is present.

Back-End feature checklist

  • Added corresponding unit and/or integration test(s). Provide written explanation in the PR description if you have failed to add tests.
  • If new dependency was added: the dependency tree is checked for conflicts.

@dmytro-landiak dmytro-landiak added this to the 2.3.0 milestone Mar 4, 2026
@dmytro-landiak dmytro-landiak added the Bug Something isn't working label Mar 4, 2026
@dmytro-landiak dmytro-landiak changed the title Bug/app shared subs backpressure Netty backpressure handling for Application clients Mar 4, 2026
@dmytro-landiak dmytro-landiak changed the title Netty backpressure handling for Application clients Application Client Backpressure: Pause/Resume via Kafka Consumer API Mar 4, 2026
@dmytro-landiak dmytro-landiak added the Core Minor improvement to Core services label Mar 4, 2026
@dmytro-landiak dmytro-landiak merged commit 7661b24 into main Mar 4, 2026
2 of 3 checks passed
@dmytro-landiak dmytro-landiak deleted the bug/app-shared-subs-backpressure branch March 4, 2026 17:04
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Bug Something isn't working Core Minor improvement to Core services

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant