Skip to content

Commit c41ea5f

Browse files
authored
chore(bench): regenerate metrics on the reference host (#753)
* chore(bench): regenerate metrics on the reference host First regeneration since #749 moved measurement off CI. Every number here was produced on the reference workstation; the series was previously measured on GitHub runners, so this is a machine change, not a code change. prev Intel Xeon 6973P-C (GHA, ci: true) next Intel i9-7980XE (reference host, idle 0.08%) The whole suite reads 39.56% slower at the median. Nothing regressed -- the Xeon is a current-generation server part and the reference box is a 2017 desktop. This is exactly the case the env fingerprint exists to make visible, and the case no scale factor can bridge. Because of that, the per-version history was re-measured with --force rather than topped up: leaving 1.0.0 and 1.0.1 on Xeon numbers while 1.0.2 came from the reference host would have put a 40% step in the middle of every sparkline. All three points now carry the same fingerprint. 1.0.2 also had no snapshot before this. Four of sixteen feature badges demote on identical code, purely because tiers are absolute thresholds and the reference machine is slower: createRegistry blazing -> fast useProxyRegistry blazing -> fast createDataGrid fast -> good createFilter fast -> good metrics.json now carries _env instead of _apparatus. * chore(bench): re-measure with the scaling governor set to performance The first pass recorded governor: powersave in every artifact, which host-guard.ts warns about — the published numbers were taken in a state the apparatus itself flags as untrustworthy. Fixed by setting the governor for the run. Two of the four tier demotions were governor artefacts and are now gone: createRegistry and useProxyRegistry are back to blazing, both having sat within a few percent of their edges. 14 of 16 badges are unchanged from the GHA-measured series. The two that remain are real. createDataGrid (Compute spans, 10,000 items: 39.5ms) and createFilter (Update query 10 times, 10,000 items: 39.3ms per op) both exceed 33.4ms — two frames at 60fps — on this host. That threshold is anchored to something a user can feel, unlike the per-item budgets, so these are the demotions worth keeping. * docs: describe the tier system that actually exists The Performance Tiers section documented a scheme replaced in #539 on 7 July: four ops/s buckets split by O(1)/O(n)/O(n2), a complexity-detection table keyed on benchmark names, and a claim that group tiers are the average of their benchmarks. None of that is true. getTier grades cost, not throughput: collection operations on per-item microseconds AND single-operation latency against frame budgets, worst axis wins; one-shot calls on call latency alone against their own budgets. getGroupTier takes the worst, never the average -- averaging is precisely what it exists to avoid, since one flattering microbench would mask a slow path. Replaces the four tier cards and the complexity table with the two budget tables the code implements, and says which thresholds are anchored to something real: 16.7ms is a frame at 60fps and 100ms is the perceptibility threshold, while the per-item and per-call budgets are conventions. Also drops 'pnpm metrics' from Running Benchmarks. It regenerates published artifacts, which now happens only on the reference machine; a reader following that instruction produces numbers CI will reject.
1 parent 4e0dd52 commit c41ea5f

6 files changed

Lines changed: 34820 additions & 25272 deletions

File tree

0 commit comments

Comments
 (0)