perf(bench): min-of-N interleaved with spread, and a Vulcanus block (#19)

[wormeyman]

Closes #19.

pnpm perf reported a median of 3, taken with every iteration of one view run back-to-back before the next started. It could not resolve the size of change it was used to gate - worse than no benchmark, because it produced confident numbers that were noise. Two perf decisions (placement-roll-NOTES.md, vulcanus-cliffs-NOTES.md) are already parked on hand-rolled loops because of it.

What changed

Four fixes, all from the issue:

1. Minimum of N, not median - timing noise is additive and positive, so the min is the least-biased estimator; the median sits in the middle of the noise distribution and moves with load.
2. Interleaved round-robin, so load drift hits every arm alike instead of landing entirely on whichever view was running.
3. Spread printed (max/min) beside every figure.
4. Within-process derived figures, with the output preamble saying explicitly not to diff absolutes across runs.

Plus the Vulcanus block the gate was always quoted against but the tool never measured: 512x512 at origin (0,0), terrain/resources/rocks/cliffs/all, with marginals and the ratio.

Knobs

Defaults satisfy the issue as written (min of 7, all blocks, ~19 min). But paying 19 minutes for one number is the friction that got timing loops hand-rolled three times, so:

pnpm perf                                        # everything, ~19 min
FMW_PERF_BLOCK=vulcanus pnpm perf                # ~3.7 min
FMW_PERF_N=3 FMW_PERF_BLOCK=vulcanus pnpm perf   # ~1.6 min

One correction to the issue's own prescription

#19 item 2 proposed all/terrain as the stable statistic. Measuring the rebuilt tool over two back-to-back runs says that is only half right:

figure	run 1	run 2	drift
terrain (absolute)	3402	3566	+4.8%
all (absolute)	8163	8225	+0.8%
ratio all/terrain	2.399	2.307	-3.8%
resources marginal	1753	1710	-2.5%
rocks marginal	1079	1079	0.0%
cliffs marginal	1894	1854	-2.1%

The marginals are the most repeatable figure, not the ratio. The ratio divides two absolutes that drift independently, so it amplifies their disagreement rather than cancelling it. It is still printed - it is the form the "under 2x terrain" gate is written in - but the header comment, the output preamble and both notes files now say to compare marginals, and that a few percent in a ratio is noise.

Reproduction against the hand-measured figures: partial, and the notes say why

DoD asked for a re-run reproducing terrain 3394 / resources 5406 / rocks 4756 / cliffs 5526 / all 8458, ratio 2.492.

• terrain reproduces: 3394 -> 3402 and 3566, inside the baseline's own 4.8% run-to-run drift.
• every overlay marginal is well outside that drift: resources 2013 -> ~1730, rocks 1362 -> 1079, cliffs 2133 -> ~1875.

Since the marginals are stable to ~2.5% run-to-run but differ from the recorded figures by 11-21%, those are real changes, not noise - and three PRs landed on exactly those paths after the 2026-07-27 measurement: #25 (rocks rendering, "14x too faint"), #28 (cliff_smoothing -> 1, which the notes measure as ~10% cheaper on the cliff pass), #32 (fixImpossibleCells, ~10% dearer). So the recorded figures are stale, not the instrument wrong. Both notes files are updated to say so.

The gate conclusion is unchanged: all measures 2.31-2.40x terrain, still past 2x.

Verification

All three blocks run end-to-end (exit 0, correct emit ordering, well-formed output), and the default suite still skips them: pnpm run verify = 1184 passed, 3 skipped (the third skip is the new Vulcanus block).

🤖 Generated with Claude Code

https://claude.ai/code/session_01Mcc72rnThnmTuBXZg63D4Y