Publish benchmark results to gh-benchmarks - #1
Merged
Conversation
Covers persisting cog/mosaic benchmark JSON to the existing gh-benchmarks branch and serving a static, client-side-rendered HTML dashboard from it. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
The committed benchmark.json is a bare array (commit_info/datetime were already dropped by the existing jq pipe), not the raw pytest-benchmark object as originally assumed. Wrap it to keep those fields for the report header, and collapse the double jq pipe into one call. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Picks up in-progress edits: drop the unused pull-requests write permission and the dead commented-out github-action-benchmark block, remove stray "Parse Results" echo lines, and fix "Merge Outputs" which was overwriting benchmark.json instead of writing siege_results.json. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Picks up in-progress edits: drop the unused pull-requests write permission and the dead commented-out github-action-benchmark block, add the benchmark.json jq extraction that was missing from "Run Benchmark", remove stray "Parse Results" echo lines, and fix "Merge Outputs" which was overwriting benchmark.json instead of writing siege_results.json. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Matches the actual invocation used in benchmark-cog.yml (run from repo root, not from within cog/). Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
The action does a single fetch+push with no retry/rebase-on-conflict, so concurrent pushes from the cog and mosaic workflows (both can trigger on the same pyproject.toml-touching commit) could race and fail outright. A hand-rolled fetch+rebase+retry loop actually delivers the safety this step needs.
Closes the residual push race the retry loop was built to handle: two runs of this same workflow (e.g. two merged PRs in quick succession) could otherwise still race on the same gh-benchmarks/cog/*.json files. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
git-auto-commit-action doesn't retry/rebase on push conflicts as the original plan assumed; document the retry-loop + concurrency-group replacement implemented during Task 6, and give Task 7 the corrected pattern up front. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
…om main Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
The architecture summary and Task 8's verification note still described the replaced action; update both to describe the retry-loop push script actually in use. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
commit_info/datetimeand splitserviceinto its own field in both workflows' JSON output (cog, mosaic), so results are self-describinggh-benchmarksbranch on every push tomain(gated topushevents only — PR runs never get write access), via a hand-rolled fetch+rebase+retry push (no third-party action) plus a per-workflowconcurrencygrouppull-requests: writepermission, dead commented-outgithub-action-benchmarkblock, stray debug echoes, fix "Merge Outputs" which was overwritingbenchmark.jsoninstead of writingsiege_results.json)index.htmlreport (already pushed directly togh-benchmarks, not part of this PR) fetches those JSON files client-side and renders latency bar charts + siege metric tables for both cog and mosaic, light/dark themedTest plan
prefers-color-scheme, including the "no data yet" fallback for missing filesgh-benchmarksreceives freshcog/*.jsonandmosaic/*.jsongh-benchmarks/ root) and confirm the report renders real data