demo(benchmark-react): report ops/s instead of ms#3808
Conversation
The Node benchmark (benchmark.yml) reports ops/sec via Benchmark.js, while the React benchmark reported milliseconds. Switch to ops/s (1000/ms) with customBiggerIsBetter so both benchmark graphs use the same units. Non-duration metrics (ref-stability counts, heap bytes) are unchanged. Made-with: Cursor
|
The latest updates on your projects. Learn more about Vercel for GitHub. |
|
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## master #3808 +/- ##
=======================================
Coverage 98.06% 98.06%
=======================================
Files 151 151
Lines 2843 2843
Branches 556 556
=======================================
Hits 2788 2788
Misses 11 11
Partials 44 44 ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
Rename list-detail-switch → list-detail-switch-10 to include the navigation count in the scenario name. Refactor listDetailSwitch() to accept (navigations, seedCount) instead of a single n that was used for both seeding and a hardcoded loop. Add machine specs after the results table. Made-with: Cursor
![cursor[bot]](https://avatars.githubusercontent.com/in/1210556?s=60&v=4){kind=small}
Ref-stability scenarios emit `count` (lower is better) which is incompatible with the customBiggerIsBetter CI tool, causing silent missed regressions. Exclude deterministic scenarios from CI; they remain available for local comparison runs. Made-with: Cursor
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes and found 1 potential issue.
Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.
1 participant
Motivation
The Node benchmark (
benchmark.yml) reportsops/secvia Benchmark.js (tool: 'benchmarkjs'), while the React benchmark reported milliseconds viacustomSmallerIsBetter. This made the two benchmark graphs use different units and scales, making cross-benchmark comparison confusing.Solution
Convert all duration-based React benchmark metrics from ms to ops/s (
1000 / ms), switching the CI action tool tocustomBiggerIsBetter. Non-duration metrics (ref-stability counts, heap bytes) are unchanged.Changed files:
bench/runner.ts—scenarioUnit()returns'ops/s'; samples converted viamsToOps()before stats computationbench/report.ts— JSDoc updatedbenchmark-react.yml—tool: 'customBiggerIsBetter'in both PR and push stepsbench/report-viewer.html— color logic and unit labels handle ops/s as bigger-is-betterREADME.md— scenario descriptions, results table, and interpreting-results updated.cursor/rules/benchmarking.mdc— reference updatedAlso reset
react-bench/data.jsongh-pages-bench(pushed separately) since the old ms-based history is incompatible with the new ops/s scale.alert-threshold: '150%'is equivalent for both directions — a 50% time regression = 1.5× ratio either way.Open questions
N/A
Made with Cursor
Note
Low Risk
Low risk: changes are confined to the React benchmark harness/CI reporting and docs, with no impact on runtime product code. Main risk is confusing or misinterpreting benchmark history due to the unit/direction flip and updated scenario parameters.
Overview
React benchmark metrics are now reported as throughput (
ops/s) instead of duration (ms). The runner converts duration samples via1000/ms, emitscustomBiggerIsBetterJSON, and reports React commit/trace metrics inops/sas well.CI is updated to use
rhysd/github-action-benchmarkwithcustomBiggerIsBetter, and the local HTML report viewer’s unit labels and fast/slow highlighting now treatops/sas higher-is-better.The
listDetailSwitchscenario is renamed/parameterized tolist-detail-switch-10(separating navigation count from seed size), and docs/guidelines are updated to reflect the new units and scenario naming.Written by Cursor Bugbot for commit 3337235. This will update automatically on new commits. Configure here.