Skip to content

Conversation

@abrookins
Copy link
Collaborator

Summary

Adds benchmark regression detection to fail CI when performance degrades by more than 20%.

Uses benchmark-action/github-action-benchmark to:

  • Store benchmark history in the gh-pages branch (under dev/bench)
  • Compare PR benchmarks against the baseline from main
  • Fail the build if any benchmark regresses by more than 20% (alert-threshold: '120%')
  • Comment on PRs when regression is detected
  • Only update the baseline when merging to main

This is a follow-up to #798 which added the benchmark test suite.

How it works

  1. First merge to main establishes the baseline (stored in gh-pages branch)
  2. Subsequent PRs are compared against this baseline
  3. If any benchmark is >20% slower, the build fails and a comment is added to the PR
  4. When PRs are merged to main, the baseline is updated

@abrookins abrookins force-pushed the benchmark-regression-detection branch from 54a1ace to 6b70362 Compare January 28, 2026 21:41
@abrookins abrookins force-pushed the benchmark-regression-detection branch from 6b70362 to 088df96 Compare January 28, 2026 21:48
@abrookins abrookins merged commit b2d9cea into main Jan 29, 2026
16 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants