ci: add capybara artifact comparisons to benchmark jobs#254
Merged
ci: add capybara artifact comparisons to benchmark jobs#254
Conversation
Contributor
There was a problem hiding this comment.
Pull request overview
Adds capybara artifact comparison and report aggregation to the benchmark CI jobs so PR outputs can be compared against the base branch and downloaded as a single merged report.
Changes:
- Removes the
pull_requesttrigger branch restriction so the workflow runs on PRs targeting any branch. - Adds “download reference artifact → run
capybara bara→ upload.capyreport artifact” steps to all four benchmark jobs. - Introduces a
merge-capybarajob that merges per-job.capyartifacts into a singlecapybara-reportartifact.
For each of the four benchmark jobs (npsim-gun, npsim-dis, eicrecon-gun, eicrecon-dis), add after the artifact upload: 1. Download the previous artifact from the base branch using dawidd6/action-download-artifact@v20 2. Run 'capybara bara' to compare current output against the reference 3. Upload a .capy artifact containing the comparison reports Add a merge-capybara job that combines all *.capy artifacts into a single capybara-report artifact for easy consumption. This PR should be opened after the npsim/eicrecon benchmarks PR merges to master, so that the base-branch artifacts exist as reference. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Remove the 'branches: [master]' filter from the pull_request trigger so that the benchmark jobs (including capybara comparisons) also fire on PRs targeting capybara or any other branch. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
wdconinc
force-pushed
the
capybara-compare
branch
from
03cd0c6 to
d738461
Compare
11 tasks
veprbl
approved these changes
Apr 24, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds capybara comparison steps to all four benchmark jobs introduced in #253, plus a
merge-capybarajob that combines the per-job reports into a single downloadable artifact.Depends on #253 — open against the
capybarabranch; will be retargeted tomasterautomatically when #253 merges.What this adds to each benchmark job
After the simulation/reconstruction artifact is uploaded, each job:
dawidd6/action-download-artifact@v20(warns and continues if not yet available)capybara barato compare the current output against the reference.capyartifact (comparison reports directory + sentinel file)A final
merge-capybarajob merges all four.capyartifacts into a singlecapybara-reportartifact.Workflow trigger change
The
pull_requesttrigger no longer restricts tobranches: [master]— it fires on PRs to any branch (includingcapybara), so this PR itself exercises the full pipeline.