chore(bench): accept first CI baseline (ci-hello-bench, 3/3) - #47
Merged
Conversation
Baseline produced by the bench.yml accept=true dispatch (run 28727174436) -- the workflow's first real batch run after the repository secrets were configured: 3/3 resolved on ubuntu-latest, which also empirically confirms the review-driven umask/uid and absolute-path fixes (a residual permissions defect would have scored 0/3). Config tuple matches the workflow pins (backend ci, MiniMax-M3, effort max, maxTurns 10). From this commit on, the PR bench gate compares for real: a resolve-rate drop below 3/3 fails the (non-required) bench check. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Commits the baseline produced by the first real
bench.ymlaccept=truedispatch (run 28727174436) after the repository secrets were configured, plus the regeneratedBENCHMARKS.md.ci, modelMiniMax-M3, effortmax, maxTurns10, scaffold1→ keyci-hello-bench-24238ad45fce7a94.--acceptin CI, committed by a human-reviewed PR — automation never writes to the repo.This PR's own
benchcheck is the end-to-end proof: it runs the gate against the baseline committed on this branch and should reportcompare: PASS. Thesmokecheck also runs its first real turn since the secrets were restored.🤖 Generated with Claude Code