🤖 Update Mock Benchmark Results#6018
Conversation
Up to standards ✅🟢 Issues
|
There was a problem hiding this comment.
Review: Automated Mock Benchmark Update
This is a clean, automated documentation-only PR with no code changes — just refreshed benchmark data and timestamps. The automation pipeline is working as expected.
Overall Assessment
The vast majority of benchmark delta falls within normal run-to-run variance (1–5%). The data format, chart axes, and file structure are all consistent.
One Item Worth Investigating
In the Callback (with args) benchmark, TUnit.Mocks regressed ~13.7%:
| Previous | Current | Delta | |
|---|---|---|---|
| TUnit.Mocks (Callback with args) | 755.9 ns | 859.4 ns | +13.7% |
Compare that to the other libraries in the same run, which moved at most 2–3%:
- Imposter: 544.2 → 533.2 ns (-2%)
- NSubstitute: 4,686.3 → 4,704.6 ns (+0.4%)
A 13.7% swing for TUnit.Mocks while competitors are flat is larger than benchmark noise typically produces. It may be:
- A genuine performance regression introduced recently
- A one-off outlier due to CI machine load
Suggestion: If a second benchmark run shows the same pattern, it's worth investigating the callback-with-args path in TUnit.Mocks source generation for unintended overhead introduced since the last run.
Everything Else
All other metrics look within expected variance. The automation, file paths, chart regeneration, and JSON schema updates are correct. No structural concerns with the PR itself — safe to merge once CI passes.
Automated Mock Benchmark Update
This PR updates the mock benchmark documentation with the latest results from the Mock Benchmarks workflow.
Benchmarks Produced
Individual benchmark artifacts are available for download:
mock-benchmark-MockCreationmock-benchmark-Setupmock-benchmark-Invocationmock-benchmark-Verificationmock-benchmark-Callbackmock-benchmark-CombinedWorkflowmock-benchmark-summary(aggregated overview)Libraries Compared
Libraries are discovered dynamically from the benchmark output. See the generated
summary.jsonfor the full list.Changes
docs/static/benchmarks/mocks/latest.jsondocs/docs/benchmarks/mocks/docs/static/benchmarks/mocks/summary.jsonWorkflow Run
🤖 This PR was automatically created and will be merged automatically once CI checks pass.