v1.8.0 — 2026 benchmarks, weekly staleness check

See CHANGELOG.

v1.7.0 — collapsible year groups

See CHANGELOG.

v1.6.0 — lifespan timeline

See CHANGELOG. Live: https://alloevil.github.io/llm-benchmarks-tracker/#timeline

v1.5.0 — timeline redesign

See CHANGELOG.

v1.4.0 — interactive saturation chart

See CHANGELOG. Live: https://alloevil.github.io/llm-benchmarks-tracker/#timeline

v1.3.0 — grouped tables, compact retired rows

See CHANGELOG.

v1.2.0 — benchmark pages, filters, sparklines, mobile

See CHANGELOG. Example: https://alloevil.github.io/llm-benchmarks-tracker/b/swe-bench-verified/

v1.1.0 — 中文 / zh-CN, newest-first tables

Added

• Simplified Chinese throughout: description_zh (required) on all 41 benchmarks and 18 evaluators, README.zh-CN.md, and a /zh/ site page with hreflang alternates and a language switch. Both READMEs are generated and drift-checked in CI.

Changed

• Model and agent benchmark tables (README + site) are sorted newest-first by released.
• Site CSS extracted to static/style.css, shared by both language pages.

v1.0.0 — sourced, schema-validated rewrite

Complete rewrite. Site: https://alloevil.github.io/llm-benchmarks-tracker/ · API: https://alloevil.github.io/llm-benchmarks-tracker/api/v1/index.json

Data

• 41 benchmarks (24 model-layer, 17 agent-layer), 18 evaluators, 179 sourced results.
• Every result row records source.url, source.kind (official-leaderboard / paper / independent-evaluation / developer-report / aggregator), accessed date and conditions (tools, reasoning effort, scaffold, pass@k, split, cost).
• Source mix: 74 official leaderboard · 46 paper · 13 independent · 36 self-reported · 10 aggregator.
• All 38 arXiv ids verified against arXiv metadata. Six wrong ids from the previous version fixed (ARC-AGI-2, MMMU-Pro, RULER, BFCL, τ-bench, BrowseComp); HLE task count, MMLU year and BrowseComp release date corrected.

Data model

• data/benchmarks/<id>.json, data/results/<id>.json (append-only ledger), data/evaluators/<id>.json; JSON Schema 2020-12 in schema/.
• Closed vocabularies for status, contamination_risk, domains, source.kind, evaluator.kind.
• Bidirectional supersedes / superseded_by chains (SWE-bench → Verified → Pro, ARC-AGI-1 → 2 → 3, …).

Tooling

• scripts/validate.py: schema + cross-file invariants (filename/id, dangling refs, ascending ledgers, percent range, future dates, undeclared splits, duplicate rows).
• scripts/build.py: generates README tables, static site, api/v1/*.json, sitemap, robots; --check guards README drift in CI.
• 21 contract tests; ruff; GitHub Actions for CI, Pages deploy, weekly link check.

Removed

• docs/*.md, research/*.json, data/timeline.json, hand-written index.html — duplicated, mutually inconsistent copies of the data.