Skip to content

Add CodSpeed continuous performance benchmarks#12

Closed
codspeed-hq[bot] wants to merge 2 commits intomainfrom
codspeed-wizard-1772555661929
Closed

Add CodSpeed continuous performance benchmarks#12
codspeed-hq[bot] wants to merge 2 commits intomainfrom
codspeed-wizard-1772555661929

Conversation

@codspeed-hq
Copy link

@codspeed-hq codspeed-hq bot commented Mar 3, 2026

Summary

  • Adds CodSpeed integration for continuous performance tracking of the taskflow runtime benchmarks
  • Creates google_benchmark-based wrappers for 4 benchmarks: fib, skynet, nqueens, and matmul
  • Adds a GitHub Actions workflow (.github/workflows/codspeed.yml) that runs benchmarks on push to main, PRs, and manual dispatch
  • Adds the CodSpeed badge to the README

What's included

Benchmark files (cpp/codspeed/)

New directory containing CodSpeed-compatible benchmark wrappers using codspeed-cpp, which provides a google_benchmark compatibility layer:

  • bench_fib.cpp - Recursive fibonacci (fib(30))
  • bench_skynet.cpp - Skynet 100M task spawn benchmark
  • bench_nqueens.cpp - N-Queens solver (N=14)
  • bench_matmul.cpp - Recursive matrix multiplication (512x512)
  • CMakeLists.txt - Build configuration using FetchContent for codspeed-cpp and CPM for taskflow

The benchmark parameters are tuned to be smaller than the main comparison benchmarks to keep CI runtimes reasonable under CodSpeed's simulation mode.

CI workflow (.github/workflows/codspeed.yml)

  • Triggers on push to main, pull requests, and manual dispatch
  • Uses OIDC authentication (no token needed)
  • Builds and runs all 4 benchmarks in simulation mode

Next steps

Once the PR is merged:

  1. CodSpeed will start tracking performance on every push to main
  2. PRs will receive automatic performance regression reports
  3. Flame graphs and execution profiles will be available on the CodSpeed dashboard

Set up continuous performance tracking using CodSpeed with google_benchmark
wrappers for the taskflow runtime benchmarks (fib, skynet, nqueens, matmul).

- Add cpp/codspeed/ directory with CMake build and benchmark source files
- Add .github/workflows/codspeed.yml for CI integration
- Add CodSpeed badge to README
@tzcnt
Copy link
Owner

tzcnt commented Mar 3, 2026

Reimplement the benchmark wrapper for TooManyCooks, NOT Taskflow. Those files are in ./cpp/ToomanyCooks

@PyXiion
Copy link
Contributor

PyXiion commented Mar 3, 2026

Bruh

Reduce problem sizes to ensure reasonable CI runtimes under CodSpeed
simulation (valgrind) mode:
- fib: 30 -> 20
- skynet: depth 8 (100M tasks) -> depth 6 (1M tasks)
- nqueens: N=14 -> N=10
- matmul: 512x512 -> 256x256
@codspeed-hq codspeed-hq bot force-pushed the codspeed-wizard-1772555661929 branch from 9b459c6 to badc835 Compare March 3, 2026 17:33
@codspeed-hq
Copy link
Author

codspeed-hq bot commented Mar 3, 2026

Congrats! CodSpeed is installed 🎉

🆕 4 new benchmarks were detected.

You will start to see performance impacts in the reports once the benchmarks are run from your default branch.

Detected benchmarks


Open in CodSpeed

@codspeed-hq codspeed-hq bot marked this pull request as ready for review March 3, 2026 17:35
@tzcnt tzcnt closed this Mar 5, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants