2.0.0

Two Opus 5 submissions join the benchmark, and grading them broke the method badly enough that the whole board was re-scored under a new rubric version.

Leaderboard (rubric v2)

#	Model	Effort	Bench	Depth	Code · Arch · UX · Robust
🥇	Opus 5	ultracode	83.8	74/90	8.5 · 9 · 9 · 8
🥈	Fable 5	ultracode	78.0	66/90	8 · 8.5 · 9 · 8.5
🥉	Opus 5	low	62.0	54/90	7 · 8 · 6 · 5
4	Opus 4.8	low	56.8	41/90	7.5 · 7.5 · 7 · 7.5
5	Fable 5	low	55.8	44/90	6.5 · 6.5 · 7 · 6.5
6	GLM 5.2	max	48.7	43/90	4.5 · 5 · 6 · 4.5
7	Grok	default	41.7	37/90	4 · 4 · 5.5 · 3.5
8	DeepSeek V4 Flash	default	41.3	29/90	5.5 · 4.5 · 6.5 · 5.5
9	Laguna S-2.1	default	33.0	33/90	2.5 · 2.5 · 4 · 2

v1 scores are not comparable to these. Every entry moved down and the order changed. That is what the major version bump means.

Why v2

v1 graded in two passes: grade, then adversarial calibration. The calibration pass turned out to rubber-stamp — on both new submissions it returned the first pass unchanged, once leaving 22 of 30 features at grade 3. It was approving "correct and complete" as exceptional.

v2 makes an independent audit mandatory: ten agents, blind to each other, try to refute every grade against the source and the live site; the judge then adjudicates each dispute. Grading only the new entries this way would have made the leaderboard a comparison of grading rigor, so rubricVersion was bumped and all nine were re-graded together by one judge under one method. Calibration anchors are now stated as criteria rather than pointers at other entries' scores, so they survive a re-grade.

New submissions

Both are one-shot autonomous runs of claude-opus-5 against the verbatim brief, self-provisioning their GitHub repo and Cloudflare deployment.

• ultracode — 288 turns, 33,239 LOC across 149 files, the full PokéAPI CSV dataset compiled into 6,032 static shards, all 1025 species reachable at runtime. It ran its own adversarial review before shipping.
• low — 100 turns, 4,491 LOC. Zero console errors, and its browse page still hands the user only 60 of 1025 species: the infinite-scroll observer attaches before its sentinel mounts, with no pagination fallback.

Integrity fixes

• A recorded deployment had drifted off its submission. None of the commits deployed to Grok's Pages project exist in its repo — its runtime signal was measuring a different app. scripts/verify-live.mjs now fingerprints every live URL against the vendored source and fails on divergence. Grok's runtime is measured against a local build of its own source, and it has no demo link.
• dexReach — the smoke test now scrolls the browse page to exhaustion and counts distinct species, across anchor, hash-route and click-handler UIs, flagging whether a pager exists. A dex that throws no errors and still stops at 60 of 1025 is now visible; pagination (DeepSeek's 30/page) is not mistaken for breakage.
• validate.mjs fails any entry on a stale rubric version or missing a note when its deployment is gone; duplicate feature grades are rejected; the pinned score test recomputes the formula instead of hardcoding a number.

Caveat

The v2 judge is Opus 5, which also built two of the entries. The audit exists precisely to counter self-preference and cut those two as hard as anything in the field; an Opus 4.8 cross-check landed within one point pre-audit. A fully independent judge would be stronger still — see methodology.