New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
ARROW-12330: [Developer] Restore values at counters column of Archery benchmark #9985
Closed
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
@kiszk thanks for working on this! Could you please fix the archery unittests? |
Of course, I will fix it in a few days. |
kszucs
approved these changes
Apr 14, 2021
Dandandan
pushed a commit
to Dandandan/arrow
that referenced
this pull request
Apr 14, 2021
… benchmark This PR restores values at `counters` column of Archery benchmark. apache#9140 always suppressed values at `counters` column regardless `--no-counter`. In addition, this PR stores `counters` value into json file. Before ``` % archery benchmark diff --benchmark-filter="SetBitsTo" --output=head2.json HEAD HEAD~1 ... --------------------------------------------------------------------------- Benchmark Time CPU Iterations UserCounters... --------------------------------------------------------------------------- SetBitsTo/2 8.15 ns 8.15 ns 81991087 bytes_per_second=234.044M/s SetBitsTo/16 7.78 ns 7.78 ns 89928878 bytes_per_second=1.91429G/s SetBitsTo/1024 13.9 ns 13.9 ns 50372172 bytes_per_second=68.6182G/s SetBitsTo/131072 3508 ns 3508 ns 199335 bytes_per_second=34.7944G/s ---------------------------------------------------------------------- Non-regressions: (4) ---------------------------------------------------------------------- benchmark baseline contender change % counters SetBitsTo/16 1.877 GiB/sec 1.914 GiB/sec 1.975 {} SetBitsTo/2 230.566 MiB/sec 234.044 MiB/sec 1.509 {} SetBitsTo/131072 34.722 GiB/sec 34.794 GiB/sec 0.207 {} SetBitsTo/1024 68.593 GiB/sec 68.618 GiB/sec 0.037 {} ``` After ``` --------------------------------------------------------------------------- Benchmark Time CPU Iterations UserCounters... --------------------------------------------------------------------------- SetBitsTo/2 8.39 ns 8.39 ns 81980047 bytes_per_second=227.438M/s SetBitsTo/16 7.88 ns 7.88 ns 84936624 bytes_per_second=1.89105G/s SetBitsTo/1024 13.9 ns 13.9 ns 50376587 bytes_per_second=68.6064G/s SetBitsTo/131072 3513 ns 3513 ns 200598 bytes_per_second=34.7447G/s ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Non-regressions: (4) ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- benchmark baseline contender change % counters SetBitsTo/2 227.438 MiB/sec 235.984 MiB/sec 3.757 {'run_name': 'SetBitsTo/2', 'repetitions': 0, 'repetition_index': 0, 'threads': 1, 'iterations': 81980047} SetBitsTo/16 1.891 GiB/sec 1.913 GiB/sec 1.137 {'run_name': 'SetBitsTo/16', 'repetitions': 0, 'repetition_index': 0, 'threads': 1, 'iterations': 84936624} SetBitsTo/131072 34.745 GiB/sec 34.771 GiB/sec 0.075 {'run_name': 'SetBitsTo/131072', 'repetitions': 0, 'repetition_index': 0, 'threads': 1, 'iterations': 200598} SetBitsTo/1024 68.606 GiB/sec 68.624 GiB/sec 0.026 {'run_name': 'SetBitsTo/1024', 'repetitions': 0, 'repetition_index': 0, 'threads': 1, 'iterations': 50376587} ``` Closes apache#9985 from kiszk/ARROW-12330 Authored-by: Kazuaki Ishizaki <ishizaki@jp.ibm.com> Signed-off-by: Krisztián Szűcs <szucs.krisztian@gmail.com>
GeorgeAp
pushed a commit
to sirensolutions/arrow
that referenced
this pull request
Jun 7, 2021
… benchmark This PR restores values at `counters` column of Archery benchmark. apache#9140 always suppressed values at `counters` column regardless `--no-counter`. In addition, this PR stores `counters` value into json file. Before ``` % archery benchmark diff --benchmark-filter="SetBitsTo" --output=head2.json HEAD HEAD~1 ... --------------------------------------------------------------------------- Benchmark Time CPU Iterations UserCounters... --------------------------------------------------------------------------- SetBitsTo/2 8.15 ns 8.15 ns 81991087 bytes_per_second=234.044M/s SetBitsTo/16 7.78 ns 7.78 ns 89928878 bytes_per_second=1.91429G/s SetBitsTo/1024 13.9 ns 13.9 ns 50372172 bytes_per_second=68.6182G/s SetBitsTo/131072 3508 ns 3508 ns 199335 bytes_per_second=34.7944G/s ---------------------------------------------------------------------- Non-regressions: (4) ---------------------------------------------------------------------- benchmark baseline contender change % counters SetBitsTo/16 1.877 GiB/sec 1.914 GiB/sec 1.975 {} SetBitsTo/2 230.566 MiB/sec 234.044 MiB/sec 1.509 {} SetBitsTo/131072 34.722 GiB/sec 34.794 GiB/sec 0.207 {} SetBitsTo/1024 68.593 GiB/sec 68.618 GiB/sec 0.037 {} ``` After ``` --------------------------------------------------------------------------- Benchmark Time CPU Iterations UserCounters... --------------------------------------------------------------------------- SetBitsTo/2 8.39 ns 8.39 ns 81980047 bytes_per_second=227.438M/s SetBitsTo/16 7.88 ns 7.88 ns 84936624 bytes_per_second=1.89105G/s SetBitsTo/1024 13.9 ns 13.9 ns 50376587 bytes_per_second=68.6064G/s SetBitsTo/131072 3513 ns 3513 ns 200598 bytes_per_second=34.7447G/s ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Non-regressions: (4) ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- benchmark baseline contender change % counters SetBitsTo/2 227.438 MiB/sec 235.984 MiB/sec 3.757 {'run_name': 'SetBitsTo/2', 'repetitions': 0, 'repetition_index': 0, 'threads': 1, 'iterations': 81980047} SetBitsTo/16 1.891 GiB/sec 1.913 GiB/sec 1.137 {'run_name': 'SetBitsTo/16', 'repetitions': 0, 'repetition_index': 0, 'threads': 1, 'iterations': 84936624} SetBitsTo/131072 34.745 GiB/sec 34.771 GiB/sec 0.075 {'run_name': 'SetBitsTo/131072', 'repetitions': 0, 'repetition_index': 0, 'threads': 1, 'iterations': 200598} SetBitsTo/1024 68.606 GiB/sec 68.624 GiB/sec 0.026 {'run_name': 'SetBitsTo/1024', 'repetitions': 0, 'repetition_index': 0, 'threads': 1, 'iterations': 50376587} ``` Closes apache#9985 from kiszk/ARROW-12330 Authored-by: Kazuaki Ishizaki <ishizaki@jp.ibm.com> Signed-off-by: Krisztián Szűcs <szucs.krisztian@gmail.com>
michalursa
pushed a commit
to michalursa/arrow
that referenced
this pull request
Jun 10, 2021
… benchmark This PR restores values at `counters` column of Archery benchmark. apache#9140 always suppressed values at `counters` column regardless `--no-counter`. In addition, this PR stores `counters` value into json file. Before ``` % archery benchmark diff --benchmark-filter="SetBitsTo" --output=head2.json HEAD HEAD~1 ... --------------------------------------------------------------------------- Benchmark Time CPU Iterations UserCounters... --------------------------------------------------------------------------- SetBitsTo/2 8.15 ns 8.15 ns 81991087 bytes_per_second=234.044M/s SetBitsTo/16 7.78 ns 7.78 ns 89928878 bytes_per_second=1.91429G/s SetBitsTo/1024 13.9 ns 13.9 ns 50372172 bytes_per_second=68.6182G/s SetBitsTo/131072 3508 ns 3508 ns 199335 bytes_per_second=34.7944G/s ---------------------------------------------------------------------- Non-regressions: (4) ---------------------------------------------------------------------- benchmark baseline contender change % counters SetBitsTo/16 1.877 GiB/sec 1.914 GiB/sec 1.975 {} SetBitsTo/2 230.566 MiB/sec 234.044 MiB/sec 1.509 {} SetBitsTo/131072 34.722 GiB/sec 34.794 GiB/sec 0.207 {} SetBitsTo/1024 68.593 GiB/sec 68.618 GiB/sec 0.037 {} ``` After ``` --------------------------------------------------------------------------- Benchmark Time CPU Iterations UserCounters... --------------------------------------------------------------------------- SetBitsTo/2 8.39 ns 8.39 ns 81980047 bytes_per_second=227.438M/s SetBitsTo/16 7.88 ns 7.88 ns 84936624 bytes_per_second=1.89105G/s SetBitsTo/1024 13.9 ns 13.9 ns 50376587 bytes_per_second=68.6064G/s SetBitsTo/131072 3513 ns 3513 ns 200598 bytes_per_second=34.7447G/s ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Non-regressions: (4) ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- benchmark baseline contender change % counters SetBitsTo/2 227.438 MiB/sec 235.984 MiB/sec 3.757 {'run_name': 'SetBitsTo/2', 'repetitions': 0, 'repetition_index': 0, 'threads': 1, 'iterations': 81980047} SetBitsTo/16 1.891 GiB/sec 1.913 GiB/sec 1.137 {'run_name': 'SetBitsTo/16', 'repetitions': 0, 'repetition_index': 0, 'threads': 1, 'iterations': 84936624} SetBitsTo/131072 34.745 GiB/sec 34.771 GiB/sec 0.075 {'run_name': 'SetBitsTo/131072', 'repetitions': 0, 'repetition_index': 0, 'threads': 1, 'iterations': 200598} SetBitsTo/1024 68.606 GiB/sec 68.624 GiB/sec 0.026 {'run_name': 'SetBitsTo/1024', 'repetitions': 0, 'repetition_index': 0, 'threads': 1, 'iterations': 50376587} ``` Closes apache#9985 from kiszk/ARROW-12330 Authored-by: Kazuaki Ishizaki <ishizaki@jp.ibm.com> Signed-off-by: Krisztián Szűcs <szucs.krisztian@gmail.com>
michalursa
pushed a commit
to michalursa/arrow
that referenced
this pull request
Jun 13, 2021
… benchmark This PR restores values at `counters` column of Archery benchmark. apache#9140 always suppressed values at `counters` column regardless `--no-counter`. In addition, this PR stores `counters` value into json file. Before ``` % archery benchmark diff --benchmark-filter="SetBitsTo" --output=head2.json HEAD HEAD~1 ... --------------------------------------------------------------------------- Benchmark Time CPU Iterations UserCounters... --------------------------------------------------------------------------- SetBitsTo/2 8.15 ns 8.15 ns 81991087 bytes_per_second=234.044M/s SetBitsTo/16 7.78 ns 7.78 ns 89928878 bytes_per_second=1.91429G/s SetBitsTo/1024 13.9 ns 13.9 ns 50372172 bytes_per_second=68.6182G/s SetBitsTo/131072 3508 ns 3508 ns 199335 bytes_per_second=34.7944G/s ---------------------------------------------------------------------- Non-regressions: (4) ---------------------------------------------------------------------- benchmark baseline contender change % counters SetBitsTo/16 1.877 GiB/sec 1.914 GiB/sec 1.975 {} SetBitsTo/2 230.566 MiB/sec 234.044 MiB/sec 1.509 {} SetBitsTo/131072 34.722 GiB/sec 34.794 GiB/sec 0.207 {} SetBitsTo/1024 68.593 GiB/sec 68.618 GiB/sec 0.037 {} ``` After ``` --------------------------------------------------------------------------- Benchmark Time CPU Iterations UserCounters... --------------------------------------------------------------------------- SetBitsTo/2 8.39 ns 8.39 ns 81980047 bytes_per_second=227.438M/s SetBitsTo/16 7.88 ns 7.88 ns 84936624 bytes_per_second=1.89105G/s SetBitsTo/1024 13.9 ns 13.9 ns 50376587 bytes_per_second=68.6064G/s SetBitsTo/131072 3513 ns 3513 ns 200598 bytes_per_second=34.7447G/s ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Non-regressions: (4) ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- benchmark baseline contender change % counters SetBitsTo/2 227.438 MiB/sec 235.984 MiB/sec 3.757 {'run_name': 'SetBitsTo/2', 'repetitions': 0, 'repetition_index': 0, 'threads': 1, 'iterations': 81980047} SetBitsTo/16 1.891 GiB/sec 1.913 GiB/sec 1.137 {'run_name': 'SetBitsTo/16', 'repetitions': 0, 'repetition_index': 0, 'threads': 1, 'iterations': 84936624} SetBitsTo/131072 34.745 GiB/sec 34.771 GiB/sec 0.075 {'run_name': 'SetBitsTo/131072', 'repetitions': 0, 'repetition_index': 0, 'threads': 1, 'iterations': 200598} SetBitsTo/1024 68.606 GiB/sec 68.624 GiB/sec 0.026 {'run_name': 'SetBitsTo/1024', 'repetitions': 0, 'repetition_index': 0, 'threads': 1, 'iterations': 50376587} ``` Closes apache#9985 from kiszk/ARROW-12330 Authored-by: Kazuaki Ishizaki <ishizaki@jp.ibm.com> Signed-off-by: Krisztián Szűcs <szucs.krisztian@gmail.com>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR restores values at
counters
column of Archery benchmark. #9140 always suppressed values atcounters
column regardless--no-counter
. In addition, this PR storescounters
value into json file.Before
After