Add distribution strategy from partition key #491
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
joelfoliveira
force-pushed
the
feat/add-partitionkey-worker-strategy
branch
from
16e9961
to
df88731
Compare
1 task
joelfoliveira
force-pushed
the
feat/add-partitionkey-worker-strategy
branch
from
df88731
to
ba56385
Compare
joelfoliveira
changed the title
joelfoliveira
force-pushed
the
feat/add-partitionkey-worker-strategy
branch
from
ba56385
to
1087f5f
Compare
filipeesch
approved these changes
Dec 12, 2023
tests/KafkaFlow.UnitTests/Consumer/DistributionStrategies/BytesSumDistributionStrategyTests.cs
Outdated
Show resolved
Hide resolved
tests/KafkaFlow.UnitTests/Consumer/DistributionStrategies/BytesSumDistributionStrategyTests.cs
Outdated
Show resolved
Hide resolved
ruiqbarbosa
approved these changes
Dec 12, 2023
joelfoliveira
force-pushed
the
feat/add-partitionkey-worker-strategy
branch
from
1087f5f
to
31c3a02
Compare
joelfoliveira
force-pushed
the
feat/add-partitionkey-worker-strategy
branch
from
31c3a02
to
f024388
Compare
JoaoRodriguesGithub
approved these changes
Dec 13, 2023
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
The default worker distribution strategy uses the message key to distribute messages to consumers.
While the message key is the default partition strategy, it is not guaranteed the producer is using that strategy and can have implications in guaranteeing the message order when consuming messages.
This PR adds a new worker distribution strategy based on the partition key instead which better guarantees the message order.
Fixes #490 #440
How Has This Been Tested?
Unit tests were added
Checklist
Disclaimer
By sending us your contributions, you are agreeing that your contribution is made subject to the terms of our Contributor Ownership Statement