Skip to content

Conversation

GuillaumeLagrange
Copy link
Contributor

This PR removes the confusion there was before between benchmark names and benchmark targets

Now, benchmark target filtering is only done through --workspace, --package and --exclude cli flags for both cargo codspeed build and cargo codspeed run.

In addition, cargo codspeed run now properly forwards the bench_name positionnal argument to the benchmark target, in order to filter benchmarks ran. This is a breaking change, but it makes cargo-codspeed more in line with cargo bench

The second commit restores the option to filter benchmark target but now follows the cargo build API using the --bench target that can be repeated

@GuillaumeLagrange GuillaumeLagrange force-pushed the cod-1351-allow-filtering-per-benchmark-with-cargo-codspeed branch 3 times, most recently from a95403c to adb6854 Compare September 25, 2025 13:25
Copy link

codspeed-hq bot commented Sep 25, 2025

CodSpeed Instrumentation Performance Report

Merging #122 will degrade performances by 6.78%

Comparing cod-1351-allow-filtering-per-benchmark-with-cargo-codspeed (43a2ff0) with main (0565866)

Summary

⚡ 2 improvements
❌ 1 regression
✅ 164 untouched

⚠️ Please fix the performance issues or acknowledge them on CodSpeed.

Benchmarks breakdown

Benchmark BASE HEAD Change
recursive[0] 240.3 ns 211.1 ns +13.82%
recursive[5] 320.3 ns 291.1 ns +10.02%
recursive_memoized[BTreeMap<u64, u64>, 0] 400.8 ns 430 ns -6.78%

Copy link

codspeed-hq bot commented Sep 25, 2025

CodSpeed WallTime Performance Report

Merging #122 will degrade performances by 20.91%

Comparing cod-1351-allow-filtering-per-benchmark-with-cargo-codspeed (43a2ff0) with main (0565866)

Summary

⚡ 9 improvements
❌ 9 regressions
✅ 136 untouched

⚠️ Please fix the performance issues or acknowledge them on CodSpeed.

Benchmarks breakdown

Benchmark BASE HEAD Change
build_vec 215 ns 205 ns +4.88%
large_setup 14 ns 13 ns +7.69%
from_elem[16384] 867 ns 911 ns -4.83%
bench_array1[10] 10 ns 9 ns +11.11%
bench_array2[10] 10 ns 9 ns +11.11%
iterative[20] 38 ns 40 ns -5%
generate_combinations[6] 1.3 µs 1.4 µs -3.54%
graph_coloring[4] 893 ns 935 ns -4.49%
hamiltonian_cycle[5] 626 ns 603 ns +3.81%
hamiltonian_cycle[6] 749 ns 705 ns +6.24%
hamiltonian_cycle[7] 803 ns 773 ns +3.88%
permutations[5] 11.5 µs 14.5 µs -20.91%
permutations[6] 76.9 µs 68.9 µs +11.6%
count_set_bits[0] 7 ns 8 ns -12.5%
count_set_bits[255] 7 ns 8 ns -12.5%
count_set_bits[42] 7 ns 8 ns -12.5%
count_set_bits[65535] 7 ns 8 ns -12.5%
sleep_1us 62.4 µs 20.4 µs ×3.1

Copy link
Member

@art049 art049 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm overall! @not-matthias I let you do the final review

…dspeed run`

There previously was a confusion between benchmark name, aka the name
the user gives to individual benchmarks, and benchmark targets.

Now, benchmark target filtering is only done through `--workspace`,
`--package` and `--exclude` cli flags for both `cargo codspeed build`
and `cargo codspeed run`.

In addition, `cargo codspeed run` now properly forwards the `benchname`
positional argument to the benchmark target, in order to filter
benchmarks ran. This is a breaking change, but it makes `cargo-codspeed`
more in line with `cargo bench`
@GuillaumeLagrange GuillaumeLagrange force-pushed the cod-1351-allow-filtering-per-benchmark-with-cargo-codspeed branch from adb6854 to 43a2ff0 Compare September 25, 2025 14:52
Copy link
Member

@not-matthias not-matthias left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

@GuillaumeLagrange GuillaumeLagrange merged commit 43a2ff0 into main Sep 26, 2025
13 of 14 checks passed
@GuillaumeLagrange GuillaumeLagrange deleted the cod-1351-allow-filtering-per-benchmark-with-cargo-codspeed branch September 26, 2025 08:43
qryxip added a commit to VOICEVOX/voicevox_core that referenced this pull request Oct 7, 2025
CodSpeedHQ/codspeed-rust#122 が入ったcargo-codspeed v4がリリースされてCI
が落ちたので、v4の方式に適応する。
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants