Skip to content

fix: parsing walltime results in cargo codspeed breaks backwards #101

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Conversation

not-matthias
Copy link
Member

@not-matthias not-matthias commented Jun 27, 2025

Builds upon #99

Changes in this PR:

  • Removed RawWalltimeData: We're now directly creating the WalltimeBenchmark struct
  • Move the WalltimeBenchmark/WalltimeResults logic to codspeed crate. Also moved the logic for clearing / collecting the results in case we want to change the path later on (so we dont have to change it in cargo-codspeed as well)

Copy link

codspeed-hq bot commented Jun 27, 2025

CodSpeed Instrumentation Performance Report

Merging #101 will degrade performances by 24.48%

Comparing cod-1058-parsing-walltime-results-in-cargo-codspeed-breaks-backwards (6cb8d30) with cod-1044-fix-divan-walltime-metric (f703a54)

Summary

⚡ 5 improvements
❌ 8 regressions
✅ 154 untouched benchmarks

⚠️ Please fix the performance issues or acknowledge them on CodSpeed.

Benchmarks breakdown

Benchmark BASE HEAD Change
sleep_50ms 595.3 ns 566.1 ns +5.15%
bench_array1[10] 225 ns 195.8 ns +14.89%
bench_array1[1] 90 ns 119.2 ns -24.48%
bench_array2[10] 225 ns 195.8 ns +14.89%
bench_array2[1] 90 ns 119.2 ns -24.48%
recursive_memoized[HashMap<u64, u64>, 0] 834.7 ns 805.6 ns +3.62%
find_highest_set_bit[0] 90.8 ns 120 ns -24.31%
find_highest_set_bit[1024] 90.8 ns 120 ns -24.31%
find_highest_set_bit[255] 90.8 ns 120 ns -24.31%
find_highest_set_bit[42] 90.8 ns 120 ns -24.31%
find_highest_set_bit[65535] 90.8 ns 120 ns -24.31%
sleep_100ns 536.9 ns 566.1 ns -5.15%
sleep_1ns 566.1 ns 536.9 ns +5.43%

Copy link

codspeed-hq bot commented Jun 27, 2025

CodSpeed WallTime Performance Report

Merging #101 will degrade performances by 50%

Comparing cod-1058-parsing-walltime-results-in-cargo-codspeed-breaks-backwards (6cb8d30) with cod-1044-fix-divan-walltime-metric (f703a54)

Summary

⚡ 21 improvements
❌ 16 regressions
✅ 117 untouched benchmarks

⚠️ Please fix the performance issues or acknowledge them on CodSpeed.

Benchmarks breakdown

Benchmark BASE HEAD Change
Iterative 20 ns 21 ns -4.76%
Recursive 45.6 µs 52.3 µs -12.89%
Iterative[20] 20 ns 21 ns -4.76%
Iterative[21] 21 ns 22 ns -4.55%
Recursive[20] 45.6 µs 52.3 µs -12.86%
Recursive[21] 73.8 µs 84.7 µs -12.89%
fibonacci_custom_measurement 369 ns 421 ns -12.35%
large_setup 13 ns 12 ns +8.33%
from_elem[2048] 253 ns 241 ns +4.98%
bench_array1[1] 1 ns 2 ns -50%
bench_array2[1] 1 ns 2 ns -50%
iterative[5] 8 ns 7 ns +14.29%
recursive[0] 2 ns 1 ns ×2
recursive_memoized[HashMap<u64, u64>, 0] 17 ns 15 ns +13.33%
sort 116.7 µs 137.2 µs -14.99%
generate_combinations[5] 827 ns 779 ns +6.16%
generate_combinations[8] 3 µs 4.1 µs -28.2%
generate_combinations[9] 4.4 µs 4.2 µs +6.02%
generate_parentheses[4] 7.3 µs 6.8 µs +6.67%
graph_coloring[4] 902 ns 1,046 ns -13.77%
... ... ... ... ...

ℹ️ Only the first 20 benchmarks are displayed. Go to the app to view all benchmarks.

@not-matthias not-matthias requested a review from art049 June 27, 2025 12:52
Base automatically changed from cod-1044-fix-divan-walltime-metric to main June 27, 2025 13:07
@not-matthias not-matthias merged commit 6cb8d30 into main Jun 27, 2025
12 of 13 checks passed
@not-matthias not-matthias deleted the cod-1058-parsing-walltime-results-in-cargo-codspeed-breaks-backwards branch June 27, 2025 13:21
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants