Commit c41ea5f
authored
chore(bench): regenerate metrics on the reference host (#753)
* chore(bench): regenerate metrics on the reference host
First regeneration since #749 moved measurement off CI. Every number here was
produced on the reference workstation; the series was previously measured on
GitHub runners, so this is a machine change, not a code change.
prev Intel Xeon 6973P-C (GHA, ci: true)
next Intel i9-7980XE (reference host, idle 0.08%)
The whole suite reads 39.56% slower at the median. Nothing regressed -- the
Xeon is a current-generation server part and the reference box is a 2017
desktop. This is exactly the case the env fingerprint exists to make visible,
and the case no scale factor can bridge.
Because of that, the per-version history was re-measured with --force rather
than topped up: leaving 1.0.0 and 1.0.1 on Xeon numbers while 1.0.2 came from
the reference host would have put a 40% step in the middle of every sparkline.
All three points now carry the same fingerprint. 1.0.2 also had no snapshot
before this.
Four of sixteen feature badges demote on identical code, purely because tiers
are absolute thresholds and the reference machine is slower:
createRegistry blazing -> fast
useProxyRegistry blazing -> fast
createDataGrid fast -> good
createFilter fast -> good
metrics.json now carries _env instead of _apparatus.
* chore(bench): re-measure with the scaling governor set to performance
The first pass recorded governor: powersave in every artifact, which
host-guard.ts warns about — the published numbers were taken in a state the
apparatus itself flags as untrustworthy. Fixed by setting the governor for
the run.
Two of the four tier demotions were governor artefacts and are now gone:
createRegistry and useProxyRegistry are back to blazing, both having sat
within a few percent of their edges. 14 of 16 badges are unchanged from the
GHA-measured series.
The two that remain are real. createDataGrid (Compute spans, 10,000 items:
39.5ms) and createFilter (Update query 10 times, 10,000 items: 39.3ms per
op) both exceed 33.4ms — two frames at 60fps — on this host. That threshold
is anchored to something a user can feel, unlike the per-item budgets, so
these are the demotions worth keeping.
* docs: describe the tier system that actually exists
The Performance Tiers section documented a scheme replaced in #539 on 7 July:
four ops/s buckets split by O(1)/O(n)/O(n2), a complexity-detection table
keyed on benchmark names, and a claim that group tiers are the average of
their benchmarks.
None of that is true. getTier grades cost, not throughput: collection
operations on per-item microseconds AND single-operation latency against
frame budgets, worst axis wins; one-shot calls on call latency alone against
their own budgets. getGroupTier takes the worst, never the average --
averaging is precisely what it exists to avoid, since one flattering
microbench would mask a slow path.
Replaces the four tier cards and the complexity table with the two budget
tables the code implements, and says which thresholds are anchored to
something real: 16.7ms is a frame at 60fps and 100ms is the perceptibility
threshold, while the per-item and per-call budgets are conventions.
Also drops 'pnpm metrics' from Running Benchmarks. It regenerates published
artifacts, which now happens only on the reference machine; a reader
following that instruction produces numbers CI will reject.1 parent 4e0dd52 commit c41ea5f
6 files changed
Lines changed: 34820 additions & 25272 deletions
File tree
- apps/docs
- public
- src
- data
- metrics
- pages/guide/fundamentals
0 commit comments