perf: black-box performance harness + CI perf gates - #139
Merged
Conversation
Measurement core (PerfScenarioRunner + PerfSignals) reads framework-boundary signals only: allocated bytes, fresh visual instances via reference-identity set-diff, GC counts, retained floor. IRecipeGridDriver drives the real headless views over their public surface (transposed + canonical drivers), so the gates survive panel refactors. Gate hierarchy: exact invariants (FreshVisualInstances == 0 on scroll round-trips), same-process ratios (per-add scaling, transposed/canonical parity, selection time ratio), then soft byte baselines in Docs/perf/baselines.json with hand-set budget caps against compounding re-baseline drift. Probes are xunit v3 explicit tests: dotnet test and CI do not run them by default. Re-baselining is a file copy: PerfActualsFixture merges measured metrics over the loaded baselines and writes the proposed next baselines.json once per process. Docs/perf/README.md carries the commands, the gate hierarchy, and the re-baseline procedure. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
perf.yml runs the explicit suite on windows-latest for every master push (plus workflow_dispatch), separate from the regular ci workflow. Byte baselines are neutralized before the run: dev-testbed numbers are not comparable on a hosted runner, and an empty metrics document flips the byte tier to its record-only path while the invariant and ratio gates keep asserting. Measured values upload as the perf-actuals artifact for later promotion to a ci-hosted baseline. The canonical invocation becomes OS-neutral dotnet run (an xunit v3 test project is an executable): docs, probe headers, and the re-baseline guidance now name it; the built exe stays only for dotnet-trace capture, which must launch the test process directly. Adds the Rider run configuration and the Zed task for the same command, and strips comments that restated Docs/perf/README.md or the assertions beneath them (comment-hater audit). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Two squashed commits:
perf: add black-box performance harness with committed baselines — measurement core (
PerfScenarioRunner+PerfSignals) over framework-boundary signals (allocated bytes, fresh-visual identity diff, GC counts, retained floor);IRecipeGridDriverdrives the real headless views through their public surface, so gates survive panel refactors. Gate hierarchy: exact invariants (FreshVisualInstances == 0) → same-process ratios (per-add scaling, transposed/canonical parity, selection time ratio) → soft byte baselines with hand-set budget caps. Probes are xunit v3 explicit tests; re-baselining is a file copy of the fixture-merged actuals artifact. Docs inDocs/perf/README.md.ci: run perf gates on master push and wire IDE entry points —
perf.ymlruns the explicit suite on every master push (workflow_dispatchtoo), with byte baselines neutralized to their record-only path (dev-testbed numbers are not comparable on a hosted runner); invariant/ratio gates assert at full strength; measured values upload as theperf-actualsartifact. Canonical invocation is OS-neutraldotnet run ... -- -explicit only; Rider run configuration and Zed task added; comment strip per comment-hater audit.Verification: full normal suite 1444/1444 green, harness units 33/33, full explicit suite 8/8 probes green,
dotnet format --verify-no-changesclean.🤖 Generated with Claude Code