feat(data): ingest Composite-Bench, the certified long-horizon browser-use benchmark

[punitarani]

Ingests Composite Research's Composite-Bench release (published 2026-07-24, research.composite.com/composite-bench): a long-horizon, browser-based computer-use benchmark with optimality-certified gold answers, a fail-closed announced compute contract (300k-output-token episode budget, 110-step cap, pinned maximum reasoning effort per provider), anytime scoring with zero row exclusions, and grounding in Composite's corpus of real professional browser-use trajectories.

Rebased onto main after the GPT-5.6 Sol/Terra/Luna tier split + Frontier Elo v2 (D26) landed (#6). All ratings below are recomputed under D26's category-attenuated (1/√n_c) battle weighting, and the GPT-5.6 rows now map to the renamed gpt-5-6-sol slug.

What's added

Two new benchmark slugs (category agents) and 16 independent result rows across the 8 evaluated models, entered in corpus/ (research trail) and regenerated into data/ via bun run generate-dataset:

composite-bench (%) — the headline macro success: pass@1 exact match to certified-optimal solutions over 100 hidden test tasks in five slices (steward_7d, rulebook_7d, dedup, orienteering, gauntlet_7g). Curated bounds 0–85 (top score 74, Wilson CI upper 82).

Model	Macro	95% CI	Mean reward	Eval cost
Claude Fable 5	74%	[65, 82]	0.965	$1,166
Claude Opus 4.8	73%	[64, 81]	0.967	$605
GLM-5.2 (Max)	63%	[53, 72]	0.936	$107
Kimi K3	31%	[23, 41]	0.799	$187
Gemini 3.5 Flash	27%	[19, 36]	0.704	$286
MiniMax M3	26%	[18, 35]	0.713	$40
GPT-5.6 Sol	18%	[12, 27]	0.745	$671
Grok 4.5	9%	[5, 16]	0.728	$196

Per-slice exact successes, CIs, mean shaped rewards, eval costs, budget-exhaustion disclosures, and the release's serving/appendix findings are carried in each row's notes.

composite-bench-npp (/1) — the separately-preregistered sixth slice: number partitioning at the phase transition, certified unsolvable within budget (exact success 0/160 across all models, as predicted); the ladder score grades anytime answer quality on a log scale from 0 (arbitrary split) to 1 (perfect, never reached). Curated bounds 0–0.6. Fable 0.471 > Kimi 0.412 > Opus 0.367 ≈ Sol 0.366 ≈ Grok 0.352 > GLM 0.196 > Gemini Flash 0.085 > MiniMax 0.062.

Slug mapping decisions

• GPT-5.6 Sol → gpt-5-6-sol: the release labels the evaluated model "GPT-5.6 Sol", which is the flagship tier renamed from gpt-5-6 in feat(data): GPT-5.6 Sol/Terra/Luna tier split + Frontier Elo v2 (D26) #6. The Terra and Luna sibling tiers were not evaluated on Composite-Bench and correctly get no rows (no-variant-conflation rule).
• GLM-5.2 → glm-5-2-max, not the default glm-5-2-high: the protocol pins maximum reasoning effort per provider, matching the Max-tier entry.
• Source independent: Composite's own harness over vendor APIs — third-party to every model vendor, with preregistration, per-row serving artifacts, and published compute contracts.

Ranking impact (the data/derived/scores.json diff)

D26's category-attenuated fit rewards the models that win the new agents battles without letting one domain swing the rating linearly. Recomputed over 543 models · 130 benchmarks · 5,527 results:

• Kimi K3 holds feat(data): add the DeepMind Gemini + Poolside Laguna releases since Kimi K3 #1 (3054.4 → 3040.7) despite a weak 31% macro — one domain can't dominate the cross-benchmark rating.
• Claude Fable 5's board-topping 74% lifts it 2936.8 → 2946.8, past GPT-5.6 Sol into feat(data): ingest SealedBench sealed capability index #2 (was feat(data): add FrontierBench with its full 8-entry launch leaderboard #3).
• GPT-5.6 Sol drops to feat(data): add FrontierBench with its full 8-entry launch leaderboard #3 (2942 → 2918.4) on its weak 18%.
• Claude Opus 4.8 feat(data): add EQ-Bench 4 and Creative Writing v3 benchmarks (eqbench.com) #4 (up 2797.1 → 2801.4 on its 73%); GLM-5.2 (Max) #25 → #24; DeepSeek-V4-Pro / MiniMax-M3 swap #18/#19.
• Largest moves are all evaluated models: Fable +10.0, GLM +8.0, Opus 4.8 +4.3, MiniMax −3.7, Kimi −13.7, GPT-5.6 Sol −23.6.

Goldens and e2e literals re-pinned for the joint refit: frontier top-5, codex floor (1 → 0.9), movers, rankings/dashboard Elo (3054.4 → 3040.7), compare Elos (GPT-5.2 2517.2 → 2517.1, DeepSeek-V3.1 1938.9 → 1938.8), the dashboard closed-frontier leader (GPT-5.6 → Claude Fable 5, a real consequence of Fable overtaking Sol), and the Opus 4.6 succession delta (+208 → +207.9). The Llama-3.1-405B anchor and the snapshot golden are unchanged — neither has Composite-Bench data and D26 attenuation limits ripple.

Gates

• bun run generate-dataset — data/ regenerates byte-identically from corpus/; derive is deterministic
• bun run ci (build-catalog · typecheck · lint · 257 unit tests · build · budgets) — all green
• Playwright e2e — the two real re-pins above pass; the only remaining failure (compare.spec.ts save/restore) is a hydration-timing flake that also fails intermittently on clean main in this sandbox's shimmed Chromium, unrelated to this change

🤖 Generated with Claude Code

https://claude.ai/code/session_01MKrJ24r8at3fVs93EtxaR2