Improve performances: Remove the aggregateAsync
call
#749
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
To replicate the benchmark results:
sbt "zioKafkaBench/Jmh/run -wi 10 -i 10 -r 1 -w 1 -t 1 -f 5 -foe true zio.kafka.bench.ConsumersComparisonBenchmark"
Before (
master
branch):After:
IMO, these results are (I could be wrong, ofc) quite counter-intuitive/surprising.
This
aggregateAsync
call seems to be an optimisation: It reduces the number of operations for each message, some of these operations being done once per batch instead of once per message.Therefore, IMO, it's quite surprising that removing this call has such a positive impact on the performances. 🤔
@adamgfraser Would you expect such results? Is there a
runFoldChunkZio
that could allow us to keep the chunking?Cc @svroonland @erikvanoosten
Edit:
FYI, I made an experiment here to keep the chunking: #751
The results are surprising too. The peak performance is better but the variability in the results is quite high