Skip to content

Releases: eanderson4/llm-bench-tower-building

TowerBench main-1 — first official benchmark run

Choose a tag to compare

@eanderson4 eanderson4 released this 06 Aug 18:21
68ba812

Which LLM builds the tallest tower? First official TowerBench run: 10 models × 5 seeds × 3 attempts on pillarsxl (30 blocks: 14 pillars, 10 lintels, 6 cubes; uncertainty contract sigmaX·sigmaV = 0.003), episodic mode with self-written notebooks between attempts. Zero failed runs. Score = mean over seeds of best-of-3 final standing height. Scripted naive baseline: 1.75 m.

# model height (m) ±σ tallest attempt 1→2→3 output tokens m / 100k tok
1 Claude Opus 5 8.52 2.4 11.07 6.50 → 6.85 → 8.10 390k 2.2
2 Claude Sonnet 5 8.46 2.2 11.94 6.30 → 5.54 → 4.84 396k 2.1
3 Claude Fable 5 7.81 1.0 9.10 6.45 → 4.01 → 5.98 284k 2.8
4 GPT-5.5 7.79 0.3 7.92 5.59 → 7.06 → 6.90 99k 7.8
5 DeepSeek V4 Flash 7.10 1.1 8.18 2.08 → 6.24 → 5.80 467k 1.5
6 GPT-5.6 Sol 6.16 1.4 6.87 3.17 → 3.43 → 4.76 76k 8.1
7 GLM-5.2 5.91 2.6 8.83 2.98 → 5.08 → 5.91 376k 1.6
8 Kimi K3 4.77 1.3 6.88 1.64 → 4.32 → 4.25 256k 1.9
9 Claude Haiku 4.5 3.91 3.4 9.82 1.99 → 3.40 → 1.26 64k 6.1
10 GPT-5.4 mini 1.79 0.4 2.40 1.38 → 1.11 → 1.68 31k 5.7

leaderboard

Highlights

  • Claude Opus 5 won partly by banking: ending an attempt early to protect an 11.07m tower (15 of 30 blocks used). The docs steer against this (#3) — it read the scoring rule anyway.
  • Claude Sonnet 5 built the run's tallest tower (11.94m) on a first attempt — and declined with notes (#5).
  • GPT-5.6 Sol reached 7.9m in 4 of 5 seeds and toppled its own tower late almost every time — yet is the most token-efficient builder (8.1 m per 100k output tokens).
  • Kimi K3 misread the orientation poses, burned 10 blocks, and wrote the wrong lesson into its notebook (#2).

compare

peak vs final

learning

Assets (below): the four infographics, 5 single-build videos + one 4-way race (720×1280 / 1600×900, h264), and the full replay bundle (per-attempt replays, run summaries, complete transcripts with per-turn token usage, aggregate JSON). Every replay re-simulates deterministically: npm run dev → viewer — see README.

External benchmark sources (comparison table, retrieved 2026-08-06, mixed reasoning configs): Artificial Analysis Intelligence Index v4.1 (artificialanalysis.ai); SWE-bench Verified — vendor launch posts + vals.ai leaderboard; ARC-AGI-2 — arcprize.org results pages (semi-private where available, ¹-marked otherwise).

Reproduce / extend: docs/PLAYBOOK.md. Protocol is frozen for main-1; new models can join the group with identical settings.