From b821f7e7a94637cf71c3772ca7f82b7bfaaff53b Mon Sep 17 00:00:00 2001 From: Ronald Tse Date: Mon, 7 Sep 2026 01:51:55 +0200 Subject: [PATCH] site: the instrument section; neural demo pins index-v5 P3: /ml gains the measurement-discipline section - the five rules that exist because the naive version produced a wrong number we nearly shipped (full-set-only with the five subset-inflation instances; decode protocol as part of the number, beam-4 4.2x over greedy; label-provenance hashing; paired bootstrap CIs; checksummed index and the margins flip gate). P4: the neural demo moves its asset index pin from v3 to v5 (23-model catalog; proxy verified serving) and links its model select to the leaderboard for tier numbers. --- src/pages/ml.astro | 44 ++++++++++++++++++++++++++++++++++++++++++ src/pages/neural.astro | 14 +++++++++++++- 2 files changed, 57 insertions(+), 1 deletion(-) diff --git a/src/pages/ml.astro b/src/pages/ml.astro index 4561f58..0cd5688 100644 --- a/src/pages/ml.astro +++ b/src/pages/ml.astro @@ -352,6 +352,50 @@ const clientModels = [ + +
+
+
+

The instrument

+

Every trap we fell into, now a rule.

+

+ Measurement discipline isn't ceremony — each of these rules exists because the naive + version produced a wrong number we nearly shipped. They are why a number on this page + means the same thing next month, on another machine. +

+
+
    +
  • + Full evaluation sets, only. Five measured instances where evaluating + on a domain-neighborhood subset overstated student quality — worst case 3.2× (a claimed + 0.72pp gap that was 2.28pp on the full set). +
    • +
  • + The decode protocol is part of the number. Beam-4 with length + normalization inflated a student's PER 4.2× over greedy on the same artifact (12.06 + vs 2.85) — every published row names its decode. +
    • +
  • + Identical numbers must mean identical data. Label provenance is + content-hashed into every run record; a reproduced figure is only evidence of + reproducibility when the input pipeline is versioned. +
    • +
  • + CIs, not point estimates. Paired sentence-level bootstrap on every + verdict, deterministic seed — deltas come with their intervals. +
    • +
  • + Checksummed end to end. The index ships with a sha256 sidecar, every + zip member is verified on load, and a margins gate watches what the CER gate cannot + see — silent quantization flips (keeping the output head in fp32 cut them 36× at + +0.4% size). +
    • +
+
+
+ diff --git a/src/pages/neural.astro b/src/pages/neural.astro index b041702..1048a5c 100644 --- a/src/pages/neural.astro +++ b/src/pages/neural.astro @@ -22,6 +22,10 @@ import Base from "../layouts/Base.astro" +

+ Full-set numbers for every tier — the leaderboard and the client ladder — on + the neural layer page. +

@@ -41,6 +45,14 @@ import Base from "../layouts/Base.astro"