Bounded in-flight dispatch requests in unordered delivery for back-pressure #1750
Conversation
Vert.x auto-subscribe and handling of records might grow unbounded, and it is particularly evident when the consumer is slow to consume messages. To apply backpressure, we need to bound the number of outbound in-flight requests, so we need to manually poll for new records as we dispatch them to the subscriber service. The maximum number of outbound in-flight requests is already configurable with the consumer parameter `max.poll.records`, and it's critical to control the memory consumption of the dispatcher. Signed-off-by: Pierangelo Di Pilato <pierdipi@redhat.com>
knative-prow-robot
added
do-not-merge/work-in-progress
Codecov Report
@@ Coverage Diff @@
## main #1750 +/- ##
============================================
+ Coverage 66.35% 66.42% +0.07%
- Complexity 611 617 +6
============================================
Files 129 129
Lines 5266 5283 +17
Branches 176 177 +1
============================================
+ Hits 3494 3509 +15
- Misses 1486 1488 +2
Partials 286 286
Flags with carried forward coverage won't be shown. Click here to find out more.
Continue to review full report at Codecov.
|
pierDipi
changed the title
knative-prow-robot
removed
the
do-not-merge/work-in-progress
|
[APPROVALNOTIFIER] This PR is APPROVED This pull-request has been approved by: matzew, pierDipi The full list of commands accepted by this bot can be found here. The pull request process is described here
Needs approval from an approver in each of these files:
Approvers can indicate their approval by writing |
|
/retest |
|
/cherry-pick release-1.1 |
|
/cherry-pick release-1.0 |
|
@pierDipi: new pull request created: #1838 In response to this:
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. |
|
@pierDipi: new pull request created: #1839 In response to this:
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. |
Vert.x auto-subscribe and handling of records might grow
unbounded, and it is particularly evident when the consumer
is slow to consume messages.
To apply backpressure, we need to bound the number of outbound
in-flight requests, so we need to manually poll for new records
as we dispatch them to the subscriber service.
The maximum number of outbound in-flight requests is already configurable
with the consumer parameter
max.poll.records, and it's critical tocontrol the memory consumption of the dispatcher.
Signed-off-by: Pierangelo Di Pilato pierdipi@redhat.com
Part of #1732
Release Note