Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Archery] Allow passing options to Google Benchmark #39303

Closed
pitrou opened this issue Dec 19, 2023 · 0 comments · Fixed by #39324
Closed

[Archery] Allow passing options to Google Benchmark #39303

pitrou opened this issue Dec 19, 2023 · 0 comments · Fixed by #39324
Assignees
@pitrou
Labels
Component: Archery Component: Benchmarking Type: enhancement
Milestone
15.0.0

Comments

@pitrou
Copy link
Member

pitrou commented Dec 19, 2023

Describe the enhancement requested

archery benchmark run allows controlling the execution of micro-benchmarks with various CLI arguments, but for now it is not possible to pass extra parameters directly to the Google Benchmark executables.

One situation where we would like to do this is to increase the number of repetitions while decreasing the runtime of each single repetition. The former is currently done with --repetitions, but the latter would require passing --benchmark_min_time to Google Benchmark.

Component(s)

Archery, Benchmarking
@pitrou pitrou self-assigned this Dec 20, 2023
pitrou added a commit to pitrou/arrow that referenced this issue Dec 20, 2023
@pitrou
apacheGH-39303: [Archery][Benchmarking] Allow setting C++ repetition …
9a5374d 
…min time

We want to be able to increase the number of repetitions for each C++ micro-benchmark without increase the total runtime.
@github-actions github-actions bot mentioned this issue Dec 20, 2023
Merged
@kou kou closed this as completed in #39324 Jan 6, 2024
kou pushed a commit that referenced this issue Jan 6, 2024
@pitrou
GH-39303: [Archery][Benchmarking] Allow setting C++ repetition min ti…
1c1fa74 
…me (#39324)

### Rationale for this change

We want to be able to increase the number of repetitions for each C++ micro-benchmark without increasing the total runtime.

### What changes are included in this PR?

* Add a `--repetition-min-time` argument to set the repetition duration in seconds
* Add a `--cpp-benchmark-extras` argument to pass arbitrary arguments to Google Benchmark executables
* Add a couple tests with multiple benchmark repetitions

### Are these changes tested?

Not entirely. Command-line argument passing is not unit-tested.

### Are there any user-facing changes?

No.
* Closes: #39303

Authored-by: Antoine Pitrou <antoine@python.org>
Signed-off-by: Sutou Kouhei <kou@clear-code.com>
@kou kou added this to the 15.0.0 milestone Jan 6, 2024
@pitrou pitrou mentioned this issue Jan 22, 2024
Merged
pitrou added a commit that referenced this issue Jan 22, 2024
@pitrou
GH-39706: [Archery] Fix benchmark diff subcommand (#39733)
92c5afe 
### What changes are included in this PR?

This fixes a regression introduced in GH-39303.

### Are these changes tested?

Only manually.

### Are there any user-facing changes?

No.

* Closes: #39706

Authored-by: Antoine Pitrou <antoine@python.org>
Signed-off-by: Antoine Pitrou <antoine@python.org>
clayburn pushed a commit to clayburn/arrow that referenced this issue Jan 23, 2024
@pitrou @clayburn
apacheGH-39303: [Archery][Benchmarking] Allow setting C++ repetition …
2f63276 
…min time (apache#39324)

### Rationale for this change

We want to be able to increase the number of repetitions for each C++ micro-benchmark without increasing the total runtime.

### What changes are included in this PR?

* Add a `--repetition-min-time` argument to set the repetition duration in seconds
* Add a `--cpp-benchmark-extras` argument to pass arbitrary arguments to Google Benchmark executables
* Add a couple tests with multiple benchmark repetitions

### Are these changes tested?

Not entirely. Command-line argument passing is not unit-tested.

### Are there any user-facing changes?

No.
* Closes: apache#39303

Authored-by: Antoine Pitrou <antoine@python.org>
Signed-off-by: Sutou Kouhei <kou@clear-code.com>
dgreiss pushed a commit to dgreiss/arrow that referenced this issue Feb 19, 2024
@pitrou @dgreiss
apacheGH-39303: [Archery][Benchmarking] Allow setting C++ repetition …
91cd2ec 
…min time (apache#39324)

### Rationale for this change

We want to be able to increase the number of repetitions for each C++ micro-benchmark without increasing the total runtime.

### What changes are included in this PR?

* Add a `--repetition-min-time` argument to set the repetition duration in seconds
* Add a `--cpp-benchmark-extras` argument to pass arbitrary arguments to Google Benchmark executables
* Add a couple tests with multiple benchmark repetitions

### Are these changes tested?

Not entirely. Command-line argument passing is not unit-tested.

### Are there any user-facing changes?

No.
* Closes: apache#39303

Authored-by: Antoine Pitrou <antoine@python.org>
Signed-off-by: Sutou Kouhei <kou@clear-code.com>
dgreiss pushed a commit to dgreiss/arrow that referenced this issue Feb 19, 2024
@pitrou @dgreiss
apacheGH-39706: [Archery] Fix benchmark diff subcommand (apache#39733)
fc4bf4f 
### What changes are included in this PR?

This fixes a regression introduced in apacheGH-39303.

### Are these changes tested?

Only manually.

### Are there any user-facing changes?

No.

* Closes: apache#39706

Authored-by: Antoine Pitrou <antoine@python.org>
Signed-off-by: Antoine Pitrou <antoine@python.org>
zanmato1984 pushed a commit to zanmato1984/arrow that referenced this issue Feb 28, 2024
@pitrou @zanmato1984
apacheGH-39303: [Archery][Benchmarking] Allow setting C++ repetition …
4364fb6 
…min time (apache#39324)

### Rationale for this change

We want to be able to increase the number of repetitions for each C++ micro-benchmark without increasing the total runtime.

### What changes are included in this PR?

* Add a `--repetition-min-time` argument to set the repetition duration in seconds
* Add a `--cpp-benchmark-extras` argument to pass arbitrary arguments to Google Benchmark executables
* Add a couple tests with multiple benchmark repetitions

### Are these changes tested?

Not entirely. Command-line argument passing is not unit-tested.

### Are there any user-facing changes?

No.
* Closes: apache#39303

Authored-by: Antoine Pitrou <antoine@python.org>
Signed-off-by: Sutou Kouhei <kou@clear-code.com>
zanmato1984 pushed a commit to zanmato1984/arrow that referenced this issue Feb 28, 2024
@pitrou @zanmato1984
apacheGH-39706: [Archery] Fix benchmark diff subcommand (apache#39733)
b65ace7 
### What changes are included in this PR?

This fixes a regression introduced in apacheGH-39303.

### Are these changes tested?

Only manually.

### Are there any user-facing changes?

No.

* Closes: apache#39706

Authored-by: Antoine Pitrou <antoine@python.org>
Signed-off-by: Antoine Pitrou <antoine@python.org>
thisisnic pushed a commit to thisisnic/arrow that referenced this issue Mar 8, 2024
@pitrou @thisisnic
apacheGH-39706: [Archery] Fix benchmark diff subcommand (apache#39733)
eabc339 
### What changes are included in this PR?

This fixes a regression introduced in apacheGH-39303.

### Are these changes tested?

Only manually.

### Are there any user-facing changes?

No.

* Closes: apache#39706

Authored-by: Antoine Pitrou <antoine@python.org>
Signed-off-by: Antoine Pitrou <antoine@python.org>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@pitrou pitrou

Labels
Component: Archery Component: Benchmarking Type: enhancement
Projects
None yet
Milestone
15.0.0
Development

Successfully merging a pull request may close this issue.

GH-39303: [Archery][Benchmarking] Allow setting C++ repetition min time pitrou/arrow
2 participants
@kou @pitrou