Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve performances: Remove the aggregateAsync call #751

Merged
merged 4 commits into from Mar 27, 2023
Merged

Conversation

guizmaii
Copy link
Member

@guizmaii guizmaii commented Mar 26, 2023

To replicate the benchmark results:

sbt "zioKafkaBench/Jmh/run -wi 10 -i 10 -r 1 -w 1 -t 1 -f 5 -foe true zio.kafka.bench.ConsumersComparisonBenchmark"

The benchmark results BEFORE these changes (based on these changed: #749) are quite stable, and the average performance is quite good:

[info] # Fork: 5 of 5
[info] OpenJDK 64-Bit Server VM warning: -XX:ThreadPriorityPolicy=1 may require system level permission, e.g., being the root user. If the necessary permission is not possessed, changes to priority will be silently ignored.
[error] SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
[error] SLF4J: Defaulting to no-operation (NOP) logger implementation
[error] SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further details.
[error] SLF4J: Failed to load class "org.slf4j.impl.StaticMDCBinder".
[error] SLF4J: Defaulting to no-operation MDCAdapter implementation.
[error] SLF4J: See http://www.slf4j.org/codes.html#no_static_mdc_binder for further details.
[info] # Warmup Iteration   1: 2782.623 ms/op
[info] # Warmup Iteration   2: 2160.183 ms/op
[info] # Warmup Iteration   3: 2410.273 ms/op
[info] # Warmup Iteration   4: 2087.847 ms/op
[info] # Warmup Iteration   5: 1564.117 ms/op
[info] # Warmup Iteration   6: 1626.549 ms/op
[info] # Warmup Iteration   7: 2345.690 ms/op
[info] # Warmup Iteration   8: 2324.042 ms/op
[info] # Warmup Iteration   9: 2322.717 ms/op
[info] # Warmup Iteration  10: 1814.914 ms/op
[info] Iteration   1: 1800.236 ms/op
[info] Iteration   2: 1605.795 ms/op
[info] Iteration   3: 1887.400 ms/op
[info] Iteration   4: 1471.681 ms/op
[info] Iteration   5: 1265.121 ms/op
[info] Iteration   6: 1603.288 ms/op
[info] Iteration   7: 2387.570 ms/op
[info] Iteration   8: 1711.696 ms/op
[info] Iteration   9: 2411.515 ms/op
[info] Iteration  10: 2272.847 ms/op
[info] Result "zio.kafka.bench.ConsumersComparisonBenchmark.zioKafka":
[info]   1849.335 ±(99.9%) 185.495 ms/op [Average]
[info]   (min, avg, max) = (1170.867, 1849.335, 2703.178), stdev = 374.709
[info]   CI (99.9%): [1663.840, 2034.830] (assumes normal distribution)
[info] # Run complete. Total time: 00:06:39

The benchmark results AFTER these changes are quite unstable, but the peak performance is quite better:

[info] # Fork: 5 of 5
[info] OpenJDK 64-Bit Server VM warning: -XX:ThreadPriorityPolicy=1 may require system level permission, e.g., being the root user. If the necessary permission is not possessed, changes to priority will be silently ignored.
[error] SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
[error] SLF4J: Defaulting to no-operation (NOP) logger implementation
[error] SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further details.
[error] SLF4J: Failed to load class "org.slf4j.impl.StaticMDCBinder".
[error] SLF4J: Defaulting to no-operation MDCAdapter implementation.
[error] SLF4J: See http://www.slf4j.org/codes.html#no_static_mdc_binder for further details.
[info] # Warmup Iteration   1: 3443.871 ms/op
[info] # Warmup Iteration   2: 1770.033 ms/op
[info] # Warmup Iteration   3: 3526.743 ms/op
[info] # Warmup Iteration   4: 1888.485 ms/op
[info] # Warmup Iteration   5: 5033.633 ms/op
[info] # Warmup Iteration   6: 4413.383 ms/op
[info] # Warmup Iteration   7: 1503.804 ms/op
[info] # Warmup Iteration   8: 2806.786 ms/op
[info] # Warmup Iteration   9: 3364.834 ms/op
[info] # Warmup Iteration  10: 1269.350 ms/op
[info] Iteration   1: 686.590 ms/op
[info] Iteration   2: 3273.708 ms/op
[info] Iteration   3: 1413.702 ms/op
[info] Iteration   4: 1310.915 ms/op
[info] Iteration   5: 1307.911 ms/op
[info] Iteration   6: 2215.191 ms/op
[info] Iteration   7: 1435.898 ms/op
[info] Iteration   8: 1017.505 ms/op
[info] Iteration   9: 4297.892 ms/op
[info] Iteration  10: 4327.193 ms/op
[info] Result "zio.kafka.bench.ConsumersComparisonBenchmark.zioKafka":
[info]   1799.058 ±(99.9%) 515.830 ms/op [Average]
[info]   (min, avg, max) = (686.590, 1799.058, 5009.849), stdev = 1042.003
[info]   CI (99.9%): [1283.227, 2314.888] (assumes normal distribution)
[info] # Run complete. Total time: 00:07:31

Last run of the benchmarks with the latest version of this code:

[info] # Fork: 5 of 5
[info] OpenJDK 64-Bit Server VM warning: -XX:ThreadPriorityPolicy=1 may require system level permission, e.g., being the root user. If the necessary permission is not possessed, changes to priority will be silently ignored.
[error] SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
[error] SLF4J: Defaulting to no-operation (NOP) logger implementation
[error] SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further details.
[error] SLF4J: Failed to load class "org.slf4j.impl.StaticMDCBinder".
[error] SLF4J: Defaulting to no-operation MDCAdapter implementation.
[error] SLF4J: See http://www.slf4j.org/codes.html#no_static_mdc_binder for further details.
[info] # Warmup Iteration   1: 2049.754 ms/op
[info] # Warmup Iteration   2: 1481.757 ms/op
[info] # Warmup Iteration   3: 1578.479 ms/op
[info] # Warmup Iteration   4: 4836.508 ms/op
[info] # Warmup Iteration   5: 1980.513 ms/op
[info] # Warmup Iteration   6: 4928.249 ms/op
[info] # Warmup Iteration   7: 1472.769 ms/op
[info] # Warmup Iteration   8: 1263.501 ms/op
[info] # Warmup Iteration   9: 5014.016 ms/op
[info] # Warmup Iteration  10: 1258.164 ms/op
[info] Iteration   1: 1958.565 ms/op
[info] Iteration   2: 1735.329 ms/op
[info] Iteration   3: 1414.613 ms/op
[info] Iteration   4: 1224.270 ms/op
[info] Iteration   5: 1404.372 ms/op
[info] Iteration   6: 1236.202 ms/op
[info] Iteration   7: 3817.833 ms/op
[info] Iteration   8: 3835.710 ms/op
[info] Iteration   9: 1841.279 ms/op
[info] Iteration  10: 1332.895 ms/op
[info] Result "zio.kafka.bench.ConsumersComparisonBenchmark.zioKafka":
[info]   1636.533 ±(99.9%) 402.992 ms/op [Average]
[info]   (min, avg, max) = (783.816, 1636.533, 4721.287), stdev = 814.063
[info]   CI (99.9%): [1233.541, 2039.525] (assumes normal distribution)
[info] # Run complete. Total time: 00:07:09

@guizmaii guizmaii marked this pull request as ready for review March 27, 2023 00:12
@guizmaii guizmaii requested a review from iravid as a code owner March 27, 2023 00:12
@guizmaii guizmaii changed the base branch from fix_perfos to master March 27, 2023 00:27
@guizmaii guizmaii changed the title Keep chunking of commands Improve performances: Remove the aggregateAsync call Mar 27, 2023
@svroonland
Copy link
Collaborator

svroonland commented Mar 27, 2023

👍

@guizmaii guizmaii merged commit 04381bf into master Mar 27, 2023
46 checks passed
@guizmaii guizmaii deleted the fix_perfos_2 branch March 27, 2023 08:19
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants