fix(hybridcloud) Deliver payloads concurrently #66870
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
We're hitting the ceiling of how many messages we can deliver concurrently with single threaded workers. Because most of our time is spent in IO, we can get more throughput by sacrificing strong ordering and delivering messages in small concurrent batches. Should a batch contain a delivery that needs to be retried progress will be stopped. This means we could delivery n-1 messages out of order if we hit an intermittent error. Initially I only want to use parallel deliery for large mailboxes as they would naturally not have strong ordering due to the amount of concurrency they create. Both the usage of threaded io and the threadpool size can be controlled by options.
github-actions
bot
added
the
Scope: Backend
markstory
commented
Mar 13, 2024
Comment on lines
+241
to
+242
| if payload_record.attempts >= MAX_ATTEMPTS: | ||
| payload_record.delete() |
There was a problem hiding this comment.
This is a behavior difference from the single-threaded delivery. I was trying to avoid iterating records multiple times and having more complex logic. The downside is sending an 11th request.
Codecov ReportAll modified and coverable lines are covered by tests ✅
Additional details and impacted files@@ Coverage Diff @@
## master #66870 +/- ##
=======================================
Coverage 84.27% 84.27%
=======================================
Files 5308 5306 -2
Lines 237401 237334 -67
Branches 41066 41056 -10
=======================================
- Hits 200071 200025 -46
+ Misses 37112 37091 -21
Partials 218 218
|
RyanSkonnord
approved these changes
Mar 13, 2024
GabeVillalobos
approved these changes
Mar 13, 2024
JonasBa
pushed a commit
that referenced
this pull request
Mar 17, 2024
We're hitting the ceiling of how many messages we can deliver concurrently with single threaded workers. Because most of our time is spent in IO, we can get more throughput by sacrificing strong ordering and delivering messages in small concurrent batches. Should a batch contain a delivery that needs to be retried progress will be stopped. This means we could delivery n-1 messages out of order if we hit an intermittent error. Initially I only want to use parallel delivery for large mailboxes as they would naturally not have strong ordering due to the amount of concurrency they create. Both the usage of threaded io and the threadpool size can be controlled by options.
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
We're hitting the ceiling of how many messages we can deliver concurrently with single threaded workers. Because most of our time is spent in IO, we can get more throughput by sacrificing strong ordering and delivering messages in small concurrent batches.
Should a batch contain a delivery that needs to be retried progress will be stopped. This means we could delivery n-1 messages out of order if we hit an intermittent error.
Initially I only want to use parallel delivery for large mailboxes as they would naturally not have strong ordering due to the amount of concurrency they create. Both the usage of threaded io and the threadpool size can be controlled by options.