Project name: Age of LLM™ — Benchmark (formerly "Modern War").
A strategic 1v1 benchmark between two LLMs. Two language models face off in a turn-based duel: the goal is to destroy the enemy base, either by nuclear bomb or by military conquest (tanks). The game measures reasoning under uncertainty (fog of war), diplomacy (bluff, deterrence, betrayal) and strategic timing. No strategic advice is given to the models — they must deduce everything on their own.
The ranking is point-based (win = 3, draw = 1, loss or mutual destruction = 0) and also aggregates average thinking time, average tokens per turn and the illegal-action rate.
This repository is the V2: rules heavily simplified compared to V1 (an old Ursina/3D game) and an isometric web viewer in Canvas 2D instead of 3D rendering.
Three pillars kept: fog of war + memory, full diplomacy, nuclear deterrence. Three layers removed vs V1: steel/trucks, unit HP, factories/research centers.
| What it means | When it moves | |
|---|---|---|
| game engine | The rules. Stamped into every replay; the only thing that decides whether two matches are comparable. | Only when a change could make a match go differently: rules, system prompt, observation, action validation. |
| site | Pages, runners, stats, ladder format, docs. | Freely. It never enters a replay. |
Every What's new section below states which of the two it moved. Releases up to
0.16.2 predate the split, when one number covered both — and 0.16.1/0.16.2 bumped
the engine number with no rules behind it, which is what prompted separating them.
The 12 opening matches were played on game engine 0.17.0; every challenge from now on runs on 0.18.0. The frozen V1 archive spans 0.9.2 → 0.15.0, which is why it is kept whole rather than continued.
If the rules ever change while a champion is standing: every challenge is still played live, both sides on the same engine, so no single tie mixes versions — but the champion's crown was won under the engine in force when it took it. When a change could plausibly have gone the other way, the opening is replayed; when it could not, the crown carries over and the release notes say so.
This repository publishes what changes the benchmark: every game-engine release, plus the rare site change that alters what is measured or how it is ranked. Ordinary site and tooling changes are tracked in the private repository — they cannot change how a match was played.
- Two version numbers — which number moves when, and why it matters
- Releases — 12 entries, newest first
- 1. Overview & architecture
- 2. Quick start
- 3. Game rules (complete)
- 3b. The Throne — format and scoring — how the crown is won, and what each result is worth
- 4. The Python engine (
engine/) - 5. The replay JSON schema
- 5b. The leaderboard (
data/leaderboard.json) - 6. The web viewer
- 7. Scripts & tools
- 8. Important conventions & invariants
- 9. Two-repo architecture
- License & Copyright
- Links
These are the benchmark's releases, not the site's. Every game-engine release is here — the rules, stamped into each replay — because only a game-engine change can make two matches non-comparable. Site and tooling releases are not: they live in the private repository.
Two site entries appear anyway, and both earn their place by changing what the benchmark claims, not how it looks: the move from a leaderboard to a ladder (site 0.17.0), and the side-bias audit that had to establish the engine favours neither slot (0.16.2). A redesign, a faster page, a fixed tooltip or a missing badge on a card will never appear here.
Scope: RULES (prompt text) + replay schema. No game behaviour changed. Game engine
0.17.0→0.18.0. Site stays0.17.1.
Since engine 0.15.0, a player with an enemy silo in view is told when that enemy
launches, and can fire back the same turn to force mutual destruction instead of
defeat. The system prompt said the opposite — that the opponent "cannot
consciously react" and that "there is no manual retaliation". A model that
believed it had no reason to look for the signal.
The prompt now documents it: when it can fire, that only the player moving second in a turn can receive it, and that it still says nothing about whether the enemy can launch. The mechanic is unchanged — only its description was wrong.
Replays now record it. players_state[].knowledge.enemy_launch_detected
stores what the player was told before deciding, and the viewer marks it in the
reasoning panel — separating "it knew and did not fire back" from "it never
knew".
The bomb never costs 13U. A systematic pass over the prompt — every claim run against the engine rather than compared to a constant — found one more slip, with no incidence on any match played. The pressure starts on turn 40, not after, and the 13U floor is only reached on turn 90, so a standard 80-turn match bottoms at 15U. The prompt now lists the schedule outright.
A ceasefire lasts 4 turns, not 3. Accepted on turn T, it is active on T, T+1, T+2 and T+3 — while the prompt said "3 turns" in three places. The wording was corrected rather than the engine: changing the value would be a rules change, and 6 of the 12 opening matches ran under a ceasefire. The discrepancy has cost nobody anything so far — zero ceasefire rejections across all 66 replays.
Diplomatic state left the chat. A ceasefire in force, and an offer still
awaiting an answer, now show as badges under the model names instead of being
readable only inside the diplomacy panel. Replays carry ceasefire_until so the
viewer reads the engine's own window rather than re-deriving it; older replays
are inferred from the accepted offer. It is not a rare case: 6 of the 12
opening matches spent time under a ceasefire, one of them for 22 half-turns.
The 12 opening matches keep their 0.17.0 stamp and are not replayed. Across all
66 replays the warning fired 3 times and was never actionable: the detecting
player always lacked the uranium or the silo to answer.
Scope: RULES. Game engine
0.16.0→0.17.0, shipped alongside site0.17.1. The ladder starts from zero on this version.
The 54 archived replays were re-read before the first ladder match: of the 300 actions the engine rejected over 2 036 half-turns, how many could the model actually have avoided? Three answers came back, all three the engine's fault.
- Units now deploy. The prompt promised a new unit could act the same turn,
but the engine assigns its id and spawn cell only after the model has
submitted its actions. All 60
Unit not foundrejections in the corpus, across 19 models, came from that — a unit now acts from its owner's next turn, and its id is read from the next observation instead of guessed. - Fog is tested first when building. "Cell already has a building" used to be checked before fog, so aiming into the dark confirmed an undiscovered enemy building — a failed build bought a free map probe. It now returns only "scout it first".
- Rejections caused by the unseen are no longer counted as illegal. Bumping
into a unit you cannot see is how a fog game reveals its board. The engine tags
those
cause="fog"and they are reported apart from illegal actions.
Left alone on purpose: building blind into fog, the largest category, is the model's mistake — computing your own field of view is a skill worth measuring.
The system prompt was also audited: seven pieces of strategic advice had crept into a prompt that claims to give none. Every payoff and mechanic they carried is still stated; only the imperative was cut.
Scope: SITE only. Site
0.16.2→0.17.0. Game engine untouched — andENGINE_VERSIONcorrected here from a phantom0.16.2back to0.16.0.
The point-table ranking is closed and replaced by a standing top-4 that models challenge into. A newly released model does not join the ladder: it meets #4 first and climbs until it loses. Every tie is played home and away with the sides swapped. If the two legs are level, the model that won its own leg in fewer turns takes the place — the same speed rule that settles the opening table, compared inside that tie rather than over careers.
index.html— the ladder is now the home page (the frozen V1 leaderboard moved tov1.html; the domain used to land on the archive): the standing, the throne, the challenge log and the reigns timeline, with each model's measured latency, real provider-billed cost, pinned endpoint and quantization.- The V1 leaderboard is frozen, not deleted. Its 54 matches span game engine 0.9.2 → 0.15.0, none of them the current rules, so it is kept whole and read only — every match, every replay, every figure — behind an ARCHIVE banner.
- Nothing is placed by decree: the first four models play a home-and-away round robin, and that table seeds the ladder.
- Real costs, not estimates. Prices × tokens cannot see prompt caching and read high; the provider-reported figure is what gets published.
- A cost is a price epoch, not a constant. Providers cut prices, and a model
that played before a cut would carry the old rate in its average for ever —
GPT-5.6 Sol's own replays show its billed rate halving mid-season. So beside
what a model's matches cost, the page shows what the same matches would cost
at today's rates, keeping the prompt-cache discount each one actually
received:
💲 $1.05 → $0.41 today. The second figure appears only when it moves the first by more than 5%. What was paid stays the primary claim; the re-price exists so two models benchmarked months apart can be compared at all. - A promotion is a date, not a price. A model can arrive on a launch
discount, and reading that as what it costs would flatter it for as long as
the offer lasts. Where the pinned endpoint is on sale the board says so —
PROMO −50%— with the list price and the at-list cost per match on hover. Gemini 3.7 Flash is why the rule is stated rather than a number trusted: it arrived at −75% and sits at −50% eight weeks later, having not changed. The cost figures published here are therefore built on list prices, so a model is never credited with an offer that expires. - Latency is a published metric, which it could not honestly be before every model was pinned to a single endpoint. It measures that one deployment — the lab's own where the lab serves one, a third party's hardware and queue where it does not. Each model's endpoint is published next to its latency, so which case applies is visible rather than assumed.
Older releases — 9 more, game engine 0.9.4 → 0.16.0
Scope: SITE only. Game engine unchanged at
0.16.0. This release bumpedENGINE_VERSIONto 0.16.2 with no rules behind it; corrected in site 0.17.0.Released before the two version numbers were separated, when a single version covered both.
The side-bias question is settled: the engine favours neither slot.
A 57-match mirror campaign (nvidia/nemotron-3.5-lightning, the same model
on both sides, so model strength is constant by construction) gives p2 51.9 %,
Wilson 95 % CI [38.9 – 64.6], p = 0.89 — indistinguishable from a fair coin.
The main corpus's 71.7 % falls outside that interval, and the two sets differ
significantly (Fisher exact, p = 0.047).
The imbalance therefore comes from slot assignment in the queue files — stronger models were written into the p2 field — and not from the game itself.
- The ranking needs no correction. Every A-vs-B match was a fair fight; the recorded outcomes stand. The 71.7 % only reflects which models sat in p2.
- Side-swapping stays recommended as hygiene, not as a fix: the audit covers a single model, and balanced pairings make neutrality verifiable from the corpus rather than something a reader must assume.
data/leaderboard.jsonnow publishes both figures underslot_biasandslot_bias.mirror, each with its Wilson interval, so no consumer reads a rate without its uncertainty.
Scope: RULES. Game engine
0.15.0→0.16.0.Released before the two version numbers were separated, when a single version covered both.
This release makes the system prompt fully rule-only. Engine bumped 0.15.0 → 0.16.0.
- Removed the last two tactical seed phrases from the system prompt. Earlier versions still contained two residual hints — "Push tanks + a scout into enemy ground to contest their economy" and "Scout with a drone early" — described as the "two tactical seed phrases present during data collection" in the companion paper (Section 2.7). They are now removed, so the prompt describes only the rules, win conditions and the JSON schema, with no strategic or build-order advice. Replays produced from v0.16.0 onward are a clean, advice-free baseline; the v0.9.2–v0.15.0 corpus (the 54 matches analysed in the papers) is unaffected and remains as collected, with the seed phrases present.
Scope: RULES. Game engine
0.14.0→0.15.0.Released before the two version numbers were separated, when a single version covered both.
This release adds a nuclear early-warning signal. Engine bumped 0.14.0 → 0.15.0.
- Nuclear early-warning (detected enemy launch). When the opponent launches a bomb, the player is now informed — but only if the player can currently see an enemy silo in their field of view. Fog of war is respected: a launch from a hidden silo stays secret. This gives a player with a visible enemy silo a one-turn window to retaliate (launch their own bomb the same turn → mutual destruction instead of a clean nuclear defeat).
Scope: RULES. Game engine
0.13.0→0.14.0.Released before the two version numbers were separated, when a single version covered both.
- Fog of war — enemy-side deposits are now remembered. A resource deposit
(gisement) scouted on the enemy side is exposed to the model and kept in
memory after discovery (
remembered_enemy_deposits:kind,pos,reserve,last_seen_tour,currently_visible) — just like remembered enemy buildings. Previously enemy-side deposits were filtered by territory and never revealed, even when a unit had eyes on them. Building a mine there still requires the cell to be in your current field of view (the enemy may have built on it since), and an exhausted deposit is dropped from memory.
Scope: RULES. Game engine
0.12.0→0.13.0.Released before the two version numbers were separated, when a single version covered both.
This release adds ChampionAgent, a strong fully deterministic scripted opponent meant to serve as a fixed reference opponent: in a "LLM vs Champion" match all the variance comes from the LLM alone, which makes the result far more reliable than two non-deterministic models facing each other. Engine and schema bumped 0.12.0 → 0.13.0.
ChampionAgent strategy (engine/agents.py)
- Efficient nuclear rush — builds uranium economy + an early silo + a scout drone, and launches the instant silo/fuel/enemy-base-discovered are all satisfied.
- Fights for the central mine instead of camping — pushes Tanks to destroy the enemy central mine (denying its fuel) and reclaim the deposit, and defends its own. This turns the match into a real military contest over the middle of the map rather than a passive economic race.
- Air defense (SAM) — when the opponent fields Fighters/Drones (a Tank cannot hit a Fighter; the SAM is the only counter), it builds and maneuvers SAMs to escort the Tank push and cover the center and its scout drone.
- Unified attack priority: (1) finish the enemy base if a Tank hit ends the game, (2) defense — destroy an enemy Tank threatening our base/silo, (3) kill the most valuable in-range unit (tactical triangle, SAM included), (4) deny center / chip buildings: central mine > silo > base > economy.
- Rational diplomacy with bluffing (uranium is secret): genuine ultimatum when crushing; ceasefire/ultimatum bluff when slightly behind; sues for peace when losing badly; always refuses ultimatums; one proposal per kind.
- Deterministic contextual key phrases on the message channel each turn.
Engine / runner
- Prompt clarification (
engine/prompts.py) — documents the action resolution order: the (up to 3) actions resolve one by one in the order listed, each on the board left by the previous one. Amovecan revoke the vision a laterbuildneeds (→ "not in your field of view"), and a unit cannot end itsmoveon a cell where a building is queued the same turn. This is a pure rules clarification — still no strategic advice. run_game.py—championis now selectable as--p0/--p1alongsidegreedy/random.run_queue.py— scripted (non-LLM) agents are selectable fromqueue.jsonvia"provider": "scripted"and"model": "champion"(orgreedy/random), with an optional fixed per-slot"seed". They are drop-in replacements for anLLMAgent.
Validation. 400+ Champion-vs-Champion matches terminate cleanly (~0.2 benign rejected actions/game), determinism verified (identical replay hash on equal seed), the Champion dominates Greedy/Random and survives a dedicated air-rush exploit bot (>90%). The residual slot skew in the deterministic mirror is an artifact of two identical strategies and is neutralized by side-swapping (play each LLM in both slots).
# LLM (player slot from .config.ini) vs the Champion
.venv/bin/python run_game.py --p0 llm --p1 champion # random map seed
.venv/bin/python run_game.py --p0 llm --p1 champion --seed 7 # fixed map seedTo pit a named model against the Champion (e.g. GPT-5.5) without editing
.config.ini, declare both players in a queue file and run it:
.venv/bin/python archive/measurement/run_queue.py --queue queue_gpt_vs_champion.jsonReliability. A single match is not enough for a meaningful result. Use ~10 minimum, 20-30 recommended, played with the LLM in both slots to cancel the mirror slot bias.
Scope: RULES. Game engine
0.11.0→0.12.0.Released before the two version numbers were separated, when a single version covered both.
This release improves central-mine contention and map fairness across seeds, with four targeted balance changes:
- SAM movement raised 1 → 2: the SAM can now keep pace with a Tank. A Tank + SAM pair can advance together to contest the central mine — the SAM protects the Tank from enemy Fighters while the Tank destroys the mine or holds the position. The SAM still cannot attack ground targets, so it creates no new win condition; it just makes escorted ground pushes viable.
- Central mine cost raised 3 → 4 C: rebuilding the central mine immediately after it is destroyed now costs as much as producing a Tank or a Fighter. The instant-rebuild loop is significantly more expensive, so holding the central deposit for several turns becomes a meaningful economic advantage rather than a quick reflex action.
- Uranium deposit reserve reduced 16 → 12: uranium deposits (normal and central) deplete faster, forcing more frequent redeployment and making the uranium income race tighter across the whole match.
- Extra mountains excluded from column 5 (side A) / column 7 (side B): seed-driven extra mountains are now confined to columns 2-4 (and their mirror 8-10). The column immediately adjacent to the central barrier (col.5 / col.7) is always kept clear. This prevents seeds where a randomly placed mountain blocks the entrance to a col.6 passage, which would asymmetrically favour the side whose approach was open.
Note — no strategic advice: as always, the prompt describes ONLY the rules and the action schema. The models receive no hints on how to play.
Scope: RULES. Game engine
0.10.0→0.11.0.Released before the two version numbers were separated, when a single version covered both.
This release rebalances the two win paths so military conquest competes head-to-head with the nuclear rush, and clarifies the rules shown to the models:
- Base HP lowered 8 → 4: the enemy base now falls in exactly two tank hits (tank damage is unchanged at 2 HP per hit). A single tank that reaches the enemy base finishes it in two turns, so a military push resolves in a comparable number of turns to a nuclear rush instead of being far too slow. Match length is unchanged (still ~16-22 turns) — only the relative viability of the military path moves. (Older replays were produced with 8 HP bases; they still play back correctly — the viewer clamps the HP bar, so a pre-v0.11 base simply shows a full bar until its HP drops below 4.)
- Line-of-sight rule made explicit in the prompt: the system prompt now states
the exact engine error (
"Line of sight blocked by a mountain or building") and spells out that an in-range target is not enough — the straight line to it must also be clear of any mountain or building. This was the most common cause of rejected ground attacks. No strategic advice is added; only the mechanic and its failure message are described. - Per-call API timeout raised 300 → 600 s to accommodate models run at
reasoning_effort = high, which can think for several minutes per turn. - Auto-save after every half-turn:
run_game.pyandrun_tournament.pynow write a partial replay to disk after each player's turn so match progress is never lost if the process is killed mid-game. - Pause + retry instead of crashing on API failure: if a model exhausts all 3
API retries on a single half-turn (timeout / API error / malformed JSON), the
runner pauses 10 minutes then retries the same turn from scratch — the failed
turn is never captured in the replay. After 6 consecutive failed pauses (1 hour
total) it raises
StallDetectedand stops cleanly. - Match resume after
StallDetected: a pickle snapshot (.snapshot.pkl) is saved alongside the partial replay after every half-turn. If a match is stopped byStallDetectedor a crash, resume it with:The API keys are re-read from.venv/bin/python run_game.py --resume replays/<match_id>.snapshot.pkl
.config.ini; everything else (turn, engine state, agents, token counts) is restored from the snapshot. The snapshot and partial JSON are deleted automatically when the match finishes normally. - Timestamp in terminal output: every
thinking…and result line now shows the local time[HH:MM:SS]so long matches are easy to monitor.
Note — no strategic advice: as always, the prompt describes ONLY the rules and the action schema. The models receive no hints on how to play.
Scope: RULES. Game engine
0.9.4→0.10.0.Released before the two version numbers were separated, when a single version covered both.
This release reworks the map and the central deadlock so matches stop looping on destroy/rebuild at the center, while keeping games short:
- Line of sight (LOS): ground attacks (tank, SAM) are now blocked by mountains AND buildings. A tank can no longer fire through the barrier or through a building — any mountain/building on the straight line between shooter and target makes the attack fail (the target's own cell never blocks, so you can still hit a building you directly aim at). Air attackers (fighter) ignore all obstacles. Mountains and buildings are real cover now.
- Mine enemy-side deposits / contest the economy: mines are no longer restricted to your own territory. You can build your own mine on any matching deposit that is free and in your field of view, including on the opponent's half. There is no instant capture of an intact enemy mine — you must first destroy it (tank hit) or wait for its deposit to exhaust, which frees the cell, then claim the deposit by building there while you hold vision. The silo stays restricted to your own territory.
- Dynamic, seed-driven terrain: for
seed > 0the central barrier on column 6 is regenerated every match (the central uranium deposit sits on a seed-driven row, the other column-6 cells are split into mountains and 2-3 passages, always with a passage above and below the deposit so the board stays connected). A few extra mountains are also scattered inside each territory (mirrored for balance) for varied cover/LOS.seed = 0still reproduces the canonical fixed layout. - Mines cost more (not tougher):
credit_mine/uranium_minenow cost 2 C (was 1) and the central mine 3 C (was 2). HP is unchanged (still 1-2 tank hits) so matches stay short — but the destroy/rebuild loop is no longer nearly free. - Resource depletion + respawn: every deposit holds a finite reserve
(
credits18,uranium16). A mine extracts its production from that reserve each turn; when it hits 0 the deposit is exhausted — the mine on it is removed and a fresh deposit of the same kind respawns on a free cell on the same side it ran out (the central one respawns on column 6). Forces redeployment instead of camping one spot. New replay events:deposit_exhausted/deposit_respawned. Deposit positions are now serialized per turn (turns[].deposits) since they move. building_damagedevent: a tank hit that damages but does not destroy a building now emits an event so the viewer can play the intermediate damaged sprite in sync with the shot (previously only the HP field changed).- Prompt / legend / observation updated: the LLM prompt no longer hardcodes
the
(6,3)center or the fixed mountain cells, documents LOS, the new costs and depletion; the player observation now includesterrain.deposit_reserves(units left per visible deposit). The viewer legend reflects all of the above.
Scope: RULES. Game engine
0.9.3→0.9.4.Released before the two version numbers were separated, when a single version covered both.
- Ultimatum consolation (rule change): the player who accepts an
ultimatum now scores 0.5 points instead of 0 — surrendering a lost
position beats fighting on for nothing. Reflected in the engine scoring
(
generate_stats.py,run_tournament.py), the LLM prompt and the viewer legend. - Fog memory now drops destroyed buildings: a building destroyed is removed
from each player's
remembered_buildings, so the AI no longer "remembers" a building that no longer exists (and the viewer stops showing a ghost of it). - Viewer — tank vs building animation sync: a mine/silo destroyed by a tank is re-injected as a ghost so it stays intact until the killing slice, then plays its destruction in sync (no more tank "firing at an empty cell").
- Viewer — remembered buildings shown greyed: in Player view, an enemy building discovered earlier but now out of sight is drawn dimmed on a greyed tile (memory) instead of vanishing into black fog.
- Mobile fixes: full-screen panels with a clearly tappable close button, long text now wraps inside panels (no right-edge clipping), the transport bar respects the dynamic viewport / safe-area (no scrolling to reach it).
- Viewer header logo replaced by the Rymentz logo.
The project has two decoupled halves, linked by a single contract: the JSON replay format.
┌─────────────────────┐ replays/*.json ┌──────────────────────┐
│ ENGINE (Python) │ ───────────────────────► │ VIEWER (Web) │
│ engine/ + run_game │ (frozen schema, §5) │ Canvas 2D iso │
│ plays the matches │ │ replays + ranking │
└─────────────────────┘ └──────────────────────┘
- Engine: simulates a complete match (rules, combat, economy, bomb, diplomacy, fog) and serializes each half-turn to JSON.
- Viewer: a 100% static site (vanilla HTML/CSS/JS, no backend) that reads these JSON files and replays them with animations, AI reasoning panels, diplomacy, replayable fog, and a multi-match leaderboard.
Guiding principle of the front end: nothing is hard-coded about dimensions or terrain — everything is read from the JSON. Changing the map does not break the front end.
modern-war-v2/
├── README.md # this file
├── config.example.ini # LLM config template (copy to .config.ini)
├── requirements.txt # venv deps (openai, fastapi-poe, Pillow)
├── .gitignore # ignores .config.ini, .venv, logs, __pycache__…
│ # + tournament_models.json, tournament_state.json,
│ # queue.json, models_validated.json (local runtime)
│
├── engine/ # ── GAME ENGINE (Python) ──
│ ├── __init__.py # package exports
│ ├── definitions.py # enums, dataclasses, ALL constants/stats
│ ├── game_map.py # 13×7 map, mountains/passages, seed-driven deposits
│ ├── state.py # game state, fog, knowledge A/B, bomb cost
│ ├── engine.py # action resolution, combat, bomb, diplomacy
│ ├── observation.py # serializes a player's state (fog) for the LLM
│ ├── prompts.py # SYSTEM_PROMPT V2 (English, no strategic advice)
│ ├── llm_agent.py # multi-provider LLMAgent + JSON parser + Config .ini
│ ├── agents.py # GreedyAgent (heuristic) + RandomAgent
│ └── replay_writer.py # serialization -> viewer JSON schema
├── run_game.py # runner: plays a single match (greedy/random/llm)
├── run_queue.py # match queue: runs specific 1v1 matches from queue.json
├── run_tournament.py # 3-round tournament: random → split → split (winners vs winners)
├── tournament_models.example.json # model list template (copy to tournament_models.json)
│
├── viewer.html # ── VIEWER (Web) ──
├── index.html # leaderboard page
├── generate_stats.py # generates data/leaderboard.json + replays/index.json
├── update_site.sh # generate_stats.py + git add/commit/push
├── scripts/
│ └── convert_webp.py # converts PNG sprites -> WebP (Pillow)
├── assets/
│ ├── css/style.css # "Into the Breach / Frostpunk" theme
│ ├── js/
│ │ ├── iso.js # isometric grid<->screen projection, z-order
│ │ ├── sprites.js # WebP preloading + type/state/variant resolution
│ │ ├── renderer.js # Canvas drawing: terrain, units, buildings, fog, camera
│ │ ├── player.js # playback machine + interpolation + sprite states
│ │ ├── stats.js # Canvas charts (eco/military/bomb/prod/losses)
│ │ ├── viewer.js # orchestration: transport, camera, docks, panels
│ │ └── leaderboard.js # leaderboard table construction
│ └── sprites/ # WebP sprites (units/buildings/effects/resources)
├── replays/
│ ├── index.json # lightweight list of replays (generated)
│ └── *.json # match replays (dummy + generated)
└── data/
└── leaderboard.json # per-model aggregates (generated)
The engine core has no external dependency (standard Python 3.10+). The
runner defaults to LLM vs LLM, so requirements.txt installs:
openai— the OpenAI-compatible SDK used for LLM agents (also drives Poe and Venice via theirbase_url).Pillow— only for the WebP conversion tool.
python3 -m venv .venv
.venv/bin/python -m pip install -r requirements.txt # openai + PillowHeuristic-only runs (
--p0 greedy --p1 greedy) do not importopenai, but it is installed by default since the runner plays LLM vs LLM out of the box.
Matches are launched only from the command line (
run_game.py). The web site is a read-only viewer: it does not play matches, it replays the JSON replays produced by the engine.
By default both players are LLM agents loaded from .config.ini — no
--llm flag is required:
cp config.example.ini .config.ini # first time: fill in the API keys
.venv/bin/python run_game.py # LLM vs LLM, RANDOM seed
.venv/bin/python run_game.py --seed 7 # LLM vs LLM, fixed map seed
.venv/bin/python run_game.py --out replays/my_match.json --match-id my_match
.config.ini(API keys) is git-ignored and must NEVER be pushed. Onlyconfig.example.ini(English template) is versioned.
Live progress: whenever an LLM agent plays, the runner prints the current turn, which model is thinking, its think time, the running average per half-turn and a projected match ETA (re-adjusted every turn) — no flag needed. Pass
-q/--quietto silence it,-vto also see rejected (illegal) actions.
Swap in a greedy/random agent on either side. Heuristic-only runs are silent
unless you pass -v (which also shows FAILs):
python3 run_game.py --p0 greedy --p1 greedy -v # both heuristic, with progress
python3 run_game.py --p0 random --p1 greedy
python3 run_game.py --p0 llm --p1 greedy # LLM vs heuristicThe runner writes a JSON replay into replays/.
Seed: if
--seedis omitted the seed is randomized at launch (so the deposit layout differs every match) and the resolved seed is saved in the replay (meta.seed) for reproducibility. Pass--seed Nfor a fixed layout. For LLM runs a non-zero[game] seedin.config.iniforces a fixed seed;seed = 0(or omitting--seed) randomizes.Turn limit: similarly,
--max-turns Noverrides everything; if omitted, LLM runs use[game] max_turnsfrom.config.iniand otherwise the default (80).
The site is 100% static, but fetch() requires an HTTP server (browsers block
fetch over file://):
python3 -m http.server 8765- Leaderboard: http://localhost:8765/
- Replay: http://localhost:8765/viewer.html?match=2026-05-30_001
python3 scripts/generate_stats.py # re-reads replays/*.json -> data/ + replays/index.jsonColumns 0-5 : Player 1 territory
Column 6 : Mountain/passage barrier + central uranium deposit
Columns 7-12 : Player 2 territory (mirror of 1)
Rows 0 to 6 (center row = 3)
Coordinates are always [column, row], origin at the top-left.
Internal balancing vs fog of war: the engine generates the map symmetrically so both sides are perfectly balanced — the mirror of
(c, r)is(12-c, 6-r). However, this symmetry is NOT disclosed to the LLM: the prompt never states that the map is symmetric, and a player's observation only contains the deposits in its OWN territory plus the shared central deposit. Enemy-side deposits are hidden by fog and cannot be deduced from symmetry — they must be scouted.
Seed-driven terrain (mountains, passages, deposits — all vary per match):
For seed > 0 the whole barrier on column 6 is regenerated and extra
mountains are scattered inside each half:
- The central uranium deposit sits on a seed-driven row of column 6 (not
always
(6,3)); it is itself a ground passage. - The other column-6 cells are split into mountains and 2-3 ground passages, with the guarantee of at least one passage above and one below the central deposit, so neither half is ever sealed off.
- A few extra mountains are placed inside side A and mirrored to B (never on a deposit, never on/adjacent to a base, never on column 6), giving ground combat real cover and line-of-sight play.
seed = 0 reproduces the canonical fixed layout: barrier mountains
(6,1) (6,2) (6,4) (6,5), passages (6,0) (6,3) (6,6), central deposit (6,3),
no extra mountains — kept deterministic for tests/replays.
Mountains block ground movement, construction, and line of sight for ground attacks (see §3.4). Air units (Drone, Fighter) fly over everything freely. Ground units cross column 6 only through its passages.
Bases: Player 1 at (1,3), Player 2 at (11,3). Bases are placed on the
2nd column (not on the edge) so that the 8 neighboring cells always
exist: the unit spawn can never be blocked by the edge.
No defensive bonus: attacking from the defender's side (or from any direction) gives no bonus/malus. Combat depends only on the tactical triangle, the mirror rule (attacker survives) and line of sight.
Deposits (seed-driven, internally mirrored for balance):
Player 1's deposits (columns 1-5) are drawn at random then mirrored to B internally → perfect balance (same count, same distance from the base). The central uranium deposit is on column 6 (seed-driven row, see above).
seed=0→ fixed canonical layout: Credits(2,1) (2,5) (10,1) (10,5), Uranium(3,3) (9,3), Central(6,3).seed>0→ 2 credit deposits + 1 uranium deposit per side, placed randomly but symmetrically (minimum spacing, away from base/mountains).
This mirroring happens internally for balance only. A player's terrain block
in its observation lists only its own deposits plus the central one (and a
deposit_reserves map of units left per visible deposit); the full set of
positions is present in the replay (omniscient spectator) but never revealed to
the LLM. Because deposits move when exhausted (§3.2), the replay also stores the
current deposit layout per turn in turns[].deposits.
At start: 5 Credits, 0 Uranium. Two resources only.
| Source | Production |
|---|---|
| Passive income | +1 C / turn |
| Credit Mine | +3 C / turn |
| Uranium Mine | +1 U / turn |
| Central Uranium Mine | +1 U / turn |
Credits = everything (units, mines, silo). Uranium = the bomb only. Uranium is secret to the opponent (but visible to the spectator).
Resource depletion + respawn (v0.10.0): each deposit holds a finite reserve —
credits18,uranium/uranium_central16 resource units. An operational mine extracts its production from that reserve each income turn; when the reserve reaches 0 the deposit is exhausted: the mine standing on it is removed (nothing left to extract) and a fresh deposit of the same kind respawns on a free cell on the same side it ran out (the central deposit respawns on column 6, still neutral). This is surfaced via thedeposit_exhausted/deposit_respawnedreplay events, the per-turnturns[].depositssnapshot, andterrain.deposit_reservesin each player's observation. The intent is to force redeployment over a match instead of camping a single deposit.
Flavor (spectator only): "credit" deposits/mines represent oil — the legend labels them "Credit (oil)". This is purely cosmetic and does not change any rule or value.
| Unit | Cost | Move* | Detection | Range | Can attack |
|---|---|---|---|---|---|
| Drone | 2 C | 3 | 3 | — | nothing (recon) |
| SAM | 3 C | 2 | 2 | 2 | aerial targets (Drone, Fighter) |
| Tank | 4 C | 2 | 1 | 2 | Tank, SAM, buildings |
| Fighter | 4 C | 3 | 2 | 2 | Tank, Drone, Fighter — NOT buildings, NOT SAM |
* Move = max cells per Move action, in Chebyshev distance
max(|dx|, |dy|).
Key rules:
- Each unit does at most 1 Move + 1 Attack per turn (2 actions max on the same unit). It can Move then Attack the same turn.
- Deployment delay (game engine 0.17.0): a unit produced this turn cannot Move or Attack until its owner's next turn — the same one-turn delay that governs buildings. Its id and spawn cell are chosen by the engine and appear in the next observation, so they are read rather than guessed.
- Only the Tank destroys buildings (2 damage/attack). The pivot of the counter-play.
- Ground units cross column 6 only through its passages.
- Line of sight: a ground attacker (Tank, SAM) cannot fire through a mountain or a building — flank around cover to land the shot. The Fighter (air) ignores obstacles for LOS.
A unit is intact or destroyed, no intermediate state.
Tactical triangle: Fighter → Tank → SAM → Fighter
| Attack | Result |
|---|---|
| Fighter → Tank | Tank destroyed, Fighter survives |
| Tank → SAM | SAM destroyed, Tank survives |
| SAM → Fighter | Fighter destroyed, SAM survives |
| SAM → Drone | Drone destroyed |
| Fighter → Drone | Drone destroyed |
| Tank → building | building −2 HP (destroyed at 0) |
| Fighter → building / SAM | invalid action |
| SAM → ground target | invalid action |
| Tank/SAM → target behind a mountain or building | invalid action (LOS blocked) |
Mirror rule (Tank vs Tank, Fighter vs Fighter): the attacker survives, the defender is destroyed. Initiative and positioning are rewarded.
Line of sight (v0.10.0): a ground attack is blocked if a mountain or a building lies on the straight line between the shooter and the target — you cannot fire through the barrier, scattered mountains, or a building. The target's own cell never blocks, so a tank can still hit a building it directly aims at. The Fighter (air) is not blocked. Mountains and buildings are genuine cover; flank to get a clean shot.
Defensive minigame: to destroy the enemy Silo/Base you must bring a Tank into contact; the defender protects with Fighters (which kill Tanks); you clear the Fighters with SAMs; the opponent destroys the SAMs with their Tanks. A recursive counter-play, readable on screen.
| Building | Cost | HP | Effect |
|---|---|---|---|
| Base | — | 4 | HQ, produces all units. Destroyed = military defeat |
| Credit Mine | 2 C | 2 | +3 C/turn, on any free [C] deposit you can see |
| Uranium Mine | 2 C | 2 | +1 U/turn, on any free [U] deposit you can see |
| Central Uranium Mine | 4 C | 3 | +1 U/turn, on the column-6 central deposit |
| Silo | 5 C | 3 | Required to launch the bomb (own territory only) |
Mine cost: mines were made pricier but not tougher so the centre stops looping on near-free destroy/rebuild while matches stay short. Ordinary mines went 1 → 2 C in v0.10.0; the central mine went 2 → 3 C in v0.10.0, then 3 → 4 C in v0.12.0. HP is unchanged throughout (still 1-2 tank hits).
Mine enemy-side deposits (v0.10.0): mines are no longer restricted to your own territory. You may build your own mine on any matching deposit that is free and currently in your field of view, including on the opponent's half. There is no instant capture of an intact enemy mine — destroy it (or wait for its deposit to exhaust) to free the cell, then build there while you hold vision. The silo remains own-territory only.
Construction: the Build action, no unit required. Mines on any free matching deposit you can see (yours or the enemy's side); Silo on a free cell in your own territory (not on a deposit nor a mountain). The central mine (column-6 deposit) can be built by either player. At most one building per cell. Remember deposits deplete and move (§3.2): a mine is not a permanent investment.
Base-adjacent cells are reserved: you cannot build (mine or silo) on any of the 8 cells adjacent to a base (yours or the enemy's). They are kept free so unit production — which auto-spawns on a free adjacent cell — can never be self-blocked by your own buildings. Build at least 2 cells away from your base.
Cell occupancy (build & move) — at most ONE unit per layer per cell:
- Build fails if the cell already has a building, contains a ground unit (allied or enemy), is a mountain, is adjacent to a base, or (for a mine) is the wrong deposit. Air units (Drone, Fighter) do not block construction.
- Fog: you can only build on a cell currently in your field of view
(detection range). A cell hidden by fog is forbidden (a hidden enemy unit could
be there). The central mine
(6,3)usually requires scouting it first. - A ground unit cannot move onto a cell occupied by another ground unit or a building. Air units (Drone, Fighter) fly over mountains, ground units and buildings, BUT two air units (even allied) cannot share the same cell — the engine rejects such a move with the reason "Destination occupied by an air unit".
Construction delay (⏳): a placed building is under construction until its owner's next turn. While under construction:
- it produces nothing — its effect only starts the turn after it finishes;
- a single attack destroys it instantly (regardless of its HP), whereas a finished building loses 2 HP per Tank hit. Defend your construction sites.
Each turn: 3 actions among:
| Action | type |
Description |
|---|---|---|
| Produce | produce |
Creates a unit on a free cell adjacent to the Base |
| Move | move |
Moves a unit (max cells, Chebyshev) |
| Attack | attack |
Attacks a target in range. Immediate resolution |
| Build | build |
Builds a Mine or a Silo |
| Launch | launch |
Launches the bomb (conditions required) |
| Wait | wait |
Skips an action |
- Everything is hidden at start, except the cells within the base's detection radius 2: the base (radius 2) reveals from turn 1 a block of 20 cells around it (i.e. 19 discovered cells around the base cell). The rest of the map stays dark until a unit/building approaches.
- Enemy units within the detection radius of at least one of your units/buildings are visible; out of range, they disappear from your state.
- Discovered buildings stay remembered with a
last_seen_tour. - The enemy base must have been discovered at least once to launch the bomb
(
enemy_base_discovered). Send a Drone early. - The replay serializes
players_state[i].visible_cells(list of[c,r]) to replay the true per-player fog in the viewer (Player 1/2 view).
| Type | Available | Effect |
|---|---|---|
| Free message | always | 1/turn, ~500 chars, archived |
| Ceasefire | turn 10+ | no attacks for 3 turns (+6 U bomb penalty) |
| Peace | turn 15+ | immediate draw if accepted |
| Ultimatum | turn 10+ | "surrender before turn X" (X ∈ [turn+1, turn+3]) — voluntary surrender: if accepted the proposer wins and the accepter gets 0.5 pt (vs 0 for a normal loss); refusing/ignoring it has no automatic consequence |
Launch conditions: intact Silo + Uranium ≥ current cost + enemy base discovered.
- Base cost: 25 U. Uranium is consumed on launch.
- Anti-stalemate pressure (post-T40): −2 U at turn 40, then −2 U every 10 turns (T50: −4, T60: −6…), floor 13 U. This pressure only affects the nuclear path; the military victory (tank on the base) stays available and immediate throughout.
- Ceasefire penalty: +6 U if launching during an active ceasefire.
- Simultaneous launches (resolved by the engine at the end of the turn): a
launch is secret (uranium and the shot are not visible to the opponent) and
"in flight". Both players play their half-turn, then the engine resolves it
via
resolve_launches()inadvance_turn: if only one launched → they win (nuclear); if both launched the same turn → mutual destruction (both lose). The bomb is in flight the turn it is launched and hits the enemy base at the end of that same turn. No manual retaliation: the play order of the turn changes nothing (the opponent may already have a ready bomb). Launching is a calculated bet, not a guaranteed win. - Military kill vs an in-flight bomb: if you destroy the enemy base with a Tank the same turn the opponent had already launched (their bomb is in flight), their bomb still detonates on your base at end of turn → this is a mutual destruction (both lose), not a clean military win.
| Condition | Result (victory_type) |
Points |
|---|---|---|
| You launch, the opponent does not (same turn) | VICTORY (nuclear) |
3 |
| Both launch the same turn | MUTUAL DESTRUCTION (mutual_destruction) |
0 / 0 |
| Enemy base at 0 HP | Military VICTORY (military) |
3 |
| Your ultimatum accepted | VICTORY (ultimatum) |
3 / 0.5 |
| Peace accepted | DRAW (peace) |
1 / 1 |
| Turn limit reached (default 80) with no winner | DRAW (timeout) |
1 / 1 |
Ultimatum consolation: the player who accepts an ultimatum loses the match but is awarded 0.5 points (instead of 0). Without it, accepting would be strictly equivalent to a 0-point defeat and there would be no incentive to ever surrender. This is the only loss worth more than 0 points.
Ranking: models are ranked by points per match (points_per_match =
average of win = 3 / draw = 1 / loss = mutual destruction = 0), so a model that
simply plays more matches cannot climb by accumulating points. Ties are broken
by win_rate, then total matches (more games = more reliable), then total
points. To avoid a single lucky game topping a long proven record, models with
fewer than the "min matches" threshold (default 1, adjustable on the page)
are listed after the ranked models and flagged provisional. Every column
is sortable (click a header). The leaderboard also shows total points,
average time/turn, average tokens/turn, $/match and the illegal-action rate.
Play order: who starts is drawn at random at the start of the match, then
alternates each turn (no permanent first-player bias). you_play_first
indicates the order of the current turn.
The home page is a champion and the line behind it, not a rating and not a ranking. This section is the reference for how the crown is won — the release notes above record when a rule changed, this records what it is.
- There is one seat to win. A newly released model plays the reigning champion and nobody else: one tie, two matches, and either it takes the crown or nothing on the board moves.
- Every tie is two legs, sides swapped. The challenger is player 1 in leg 1 and player 2 in leg 2. No pairing is ever played in one direction only.
- A new champion is inserted at the top, everyone below shifts down one, and the last name falls off the end of the line.
- The opening. An empty board cannot be filled by decree, so the first four models played a home-and-away round-robin — 12 matches — and that table produced the first champion and the first three seats behind it.
Until August 2026 the format was a climb: a challenger entered at
#4and moved up one place per tie won, keeping the highest place it took. It was replaced because its cost fell hardest on its best results — eight matches to reach#1against two to be eliminated, so the challenges worth watching were the ones the benchmark could least afford. Challenges played under the climb are still shown as climbs, anddata/ladder.jsonrecords the format each one was played under.
A tie is the sum of its two legs, on the same scale used everywhere else in the project:
| Leg result | victory_type |
Points |
|---|---|---|
| Win | nuclear, military, ultimatum |
3 |
| Draw | peace, timeout |
1 each |
| Loss by accepted ultimatum | ultimatum (loser's side) |
0.5 |
| Loss | — | 0 |
| Mutual destruction | mutual_destruction |
0 each — both lose |
Two lines are deliberate rather than obvious, and they pull in opposite directions for the same reason: incentives.
An accepted ultimatum pays the loser 0.5. Without it, surrendering a lost position would be worth exactly as much as fighting on to a 0-point defeat, and no model would ever accept.
Mutual destruction pays 0, not 1. It is not a draw: both bases are gone.
Scored as a draw — which the ladder mistakenly did until site 0.17.5 — a model
that is losing converts a certain 0 into a certain 1 by launching, so
launching-when-behind becomes strictly dominant and nuclear deterrence, one
of the three pillars this benchmark exists to measure, stops meaning anything.
A peaceful draw keeps its point, because nobody destroyed anybody.
- Level on points, and each side won a leg → the crown goes to the model that won its own leg in fewer turns. The comparison stays inside the tie on purpose: career averages would let matches won earlier, against entirely different opponents, decide a fight for the throne. "I beat you in 19, you beat me in 25" is a statement about these two games; "my average is lower" is not.
- Level on points and no leg was closed out — two draws (2-2), two mutual destructions (0-0), or one of each (1-1) → the champion keeps the crown. There is genuinely nothing to compare: who drew faster measures who sued for peace earlier, and who was annihilated faster measures nothing at all.
- Identical times → the champion keeps the crown.
Every step of a challenge records decided_by, so a tie shown next to a
challenger that took the crown says why rather than leaving the reader to
guess.
The line below the champion is not a ranking, and not a top 4. This is the claim the format gave up when the climb was retired, and it is worth being blunt about it: only the first seat is ever played for. A challenger that loses to the champion earns no seat at all, however close it came — and no model in the line has ever played any other model in the line for its position. Ordering those names by strength would be a claim no match on this site supports.
What the line is: the champion, then whoever came before it, most recent first — the succession. A seat says either this model held the crown, or this model won a seat under the climb that ran until August 2026, and the page prints which, per row.
It is therefore not a measurement of how far apart two models are either. A challenger that loses the throne match has played two matches, and the only thing they establish is that it did not take the crown. Precision accumulates in exactly one place — the throne, where the champion defends repeatedly and builds a record. Nothing else on the board is being measured at all.
Points decided the first champion and have not been compared across the board since. The distinction is easy to get wrong in both directions, so to be exact:
- The opening round-robin was a points table, and it is what produced the first champion and the first line — 12 / 12 / 6 / 6. Saying the page "is not a points table" would be false about its own origin.
- Nothing has been scored across the board since. The crown changes hands only when a challenger wins the two-leg tie played for it. No total is ever compared between two models that have not just played each other.
A seat is therefore not a record. The crown says one thing — nobody has beaten this model for it. The W–L record printed next to each model is aggregated from its matches for the reader and is never an input.
So a model can sit in the line on a losing record, and one does: Grok 4.6 held the throne for a week on 4W–4L. That is not a contradiction to be fixed, it is the format working. Reading the page as a live league table is the one mistake it invites, which is why the record is shown at all: hiding it would make the line look like a ranking that is still being computed.
Mutual destruction can never cost a champion its crown. It scores 0 for both sides, so it moves no gap: a tie of two mutual destructions is 0-0 and the champion keeps the crown, and a mutual destruction next to a leg the champion won is 0-3, same outcome. Only a challenger winning more legs than the champion takes the crown.
That is the exact reverse of the opening table, where a mutual destruction costs both participants a point against the models that were not involved — and can therefore drop one of them below the cut. Penalising in the opening, neutral in a defence, from one rule: a round-robin ranks models against a field, while the throne only ever records a duel.
The climb that ran until August 2026 also assumed transitivity: a model
placed #3 had never played #1. The throne format makes no such assumption
because it makes no such claim — after the opening, the only thing anyone plays
for is the crown. The opening is still a fully connected round-robin rather than
a seeding by decree, for the one moment that needs it: picking the first champion
out of a field nobody has beaten head-to-head yet.
Strict decoupling. The core (map/state/engine/agents) has no external
dependency; only llm_agent.py imports openai (and lazily, only when an
LLM agent is actually instantiated).
Single source of truth: enums (UnitType, BuildingType, ActionType,
DiplomacyKind, VictoryType), dataclasses (Unit, Building,
PlayerKnowledge, DiploRecord) and all constants (map, costs, stats, bomb,
diplomacy, seed-driven deposit generation). The enum values == the JSON schema
tokens, so the serializer is a 1:1 mapping. Combat tables: ATTACK_MATRIX,
TANK_BUILDING_DAMAGE. Note the UnitType enum value for the fighter is
fighter.
13×7 map: seed-driven terrain — the column-6 barrier (mountains/passages +
the central deposit row), extra in-territory mountains, and the resource deposits
are all generated from the seed and mirrored to side B for balance (see §3.1).
seed=0 reproduces the canonical fixed layout. Helpers: in_bounds,
is_mountain, is_passage, is_central, central_pos, deposit_at,
ground_passable, is_in_own_territory, chebyshev, adjacent_cells,
line_of_sight_blocked(x1,y1,x2,y2) (mountain between two cells, for ground
attacks), and the depletion API reserve_at / draw_from_deposit /
is_exhausted / remove_deposit / respawn_deposit, plus terrain_dict()
(full export for the replay).
Holds units, buildings, resources (credits/uranium per player), knowledge
A/B, diplomacy, ceasefire_until, combat stats. Key methods:
- ID factories:
new_unit_id,new_building_id(formatA_tank_3…), - access:
unit_at,units_at,building_at,get_unit,base_of,player_units,player_buildings, - fog:
visible_cells,update_knowledge(remembers enemy buildings + base discovery),visible_enemy_units, - bomb:
current_bomb_cost()(handles post-T40 pressure + ceasefire malus),ceasefire_active().
Core of the rules. Half-turn loop:
begin_half_turn(p) → execute_action(p, action) × N → end_half_turn(), then
advance_turn() once both players have played (income + turn++).
execute_actiondispatches to_do_produce / _do_move / _do_attack / _do_build / _do_launchand returns anActionResult(ok, detail, action)whereactionis normalized (from/to/result/unit/… fields added) so it can be written directly into the replay. A move onto an air-occupied cell is rejected with "Destination occupied by an air unit"; a ground-occupied cell with "Destination occupied by a ground unit".- Combat:
_do_attackappliesATTACK_MATRIX(fatal combat); the Tank damages buildings. Ground attacks are subject to line of sight (map.line_of_sight_blocked): a mountain between attacker and target makes the shot fail (air attackers are exempt). A non-fatal hit emits abuilding_damagedevent. Destroying the Base ends the match (MILITARY) and keeps the base as ahp=0wreck for the final frame. - Economy / depletion:
_grant_incomedraws each mine's production from its deposit's finite reserve;_exhaust_depositremoves a drained deposit and the mine on it, then respawns a fresh deposit of the same kind on the same side (eventsdeposit_exhausted/deposit_respawned). - Bomb:
_do_launchrecords an "in flight" launch (the bomb does NOT resolve immediately);resolve_launches(), called at the end of the turn after both half-turns, applies the effects: if only one launched → they win (NUCLEAR), if both launched the same turn → mutual destruction._apply_launchsets the enemy base tohp=0. A silo still under construction cannot launch. - Diplomacy:
submit_diplomacy(validates turn windows + ultimatum constraint[turn+1, turn+3]),respond_diplomacy(ceasefire/peace/ultimatum). GameOutcome(over, winner, victory_type)carries the result;winner∈{0, 1, -1}.
Serializes to the exact schema of the viewer (§5). One half-turn = one
turns entry (the active player alternates each entry). capture_half_turn(...)
is called after the actions are resolved; the units/buildings snapshots
are absolute (end-of-half-turn state). finalize(winner, victory_type)
builds meta + terrain, save(path) writes the file.
GreedyAgent: deterministic heuristic (builds up the economy, scouts with a drone, takes the central mine, builds a silo, rushes the bomb). Enough to exercise all engine paths and produce a realistic replay without an LLM.RandomAgent: near-random actions (debug).
Serializes a player's state respecting fog of war: it sees its own
units/buildings/resources, but only the enemy units currently in vision and the
enemy buildings already discovered (with last_seen_tour). The enemy uranium is
never exposed. The terrain field contains only the deposits in the player's
own territory plus the shared central deposit ((6,3)) — enemy-side deposits
stay hidden by fog so the player cannot deduce them from symmetry. Includes
bomb_cost, enemy_base_discovered/position, combat_stats,
events_against_you, last_turn_results/errors, diplomacy_pending,
opponent_last_message, diplomacy_history. This is exactly what the LLM
receives.
V2 system prompt in English. Describes only the V2 rules (13×7 map, 2 resources, 4 HP-less units, fatal combat, tactical triangle, bomb, diplomacy) and the JSON action schema. Defines Chebyshev. Gives no strategic advice — the model reasons on its own. It does not tell the model the map is symmetric and does not reveal enemy deposit positions: each player must scout the enemy half.
LLMAgent: OpenAI-compatible multi-provider client (Venice / Poe / generic). Optional extended reasoning (thinking,reasoning_effort, Poe modesdefault/deep; native Responses API for Claude/Gemini on Poe). JSON parsing robust across 10 strategies (from strictest to most permissive:<json>, markdown fence, object scan, truncated-JSON repair…), retries with backoff, token counting. On total failure →waitfallback (the match never crashes). Exposes the samedecidecontract as the heuristic agents pluslast_reasoning,last_content,pending_responses,total_tokens,effective_reasoning_effort()(reports the effort label used), andlast_retries(per-turn retry counts by cause:timeout/api_error/malformed).- Request reliability: each LLM call uses a 300 s client timeout. A turn
is attempted up to
max_retries(default 3) times, sleeping[10, 20, 30]s between attempts (capped at the last value). Each failed attempt is bucketed by cause — request timeout, other api_error, or malformed JSON — and the per-turn counts are written to the replay underperf.retries(NOT shown in the viewer UI; aggregated bygenerate_stats.pyintoretries_timeout/retries_api_error/retries_malformed/retries_total). No response content is stored, so retries carry no secrets. Config: loads.config.ini. Sections[venice]/[poe](credentials),[api](global fallback),[player1]/[player2](model, provider, thinking, reasoning_effort, reasoning),[game](seed, max_turns). Per-player credential resolution: explicitapi_key> named provider section >[api].make_agent(idx)builds the ready-to-useLLMAgent.
Common agent contract:
decide(engine, player, feedback=None) -> (actions: list[dict], reasoning: str, diplo: list[dict])
where diplo = [{"kind", "text", "target_turn"?}] and feedback =
{"results", "errors", "events"} carries the player's previous-turn action
results/errors plus the events suffered since (e.g. a unit destroyed by the
opponent's intervening half-turn), so the model can fix illegal actions and
react to combat. Diplomatic responses (accept/refuse) are exposed via
agent.pending_responses and applied by the runner.
Plugging in other providers: any OpenAI-compatible endpoint works via
[api]or a provider section. Keys live in.config.ini(git-ignored and never pushed); onlyconfig.example.ini(English) is versioned.
This is the contract between engine and viewer. A complete example lives in
replays/ (any generated match).
{
"schema_version": "1.0",
"meta": {
"match_id": "2026-05-30_001",
"date": "2026-05-30T14:30:00Z",
"seed": 1734920011, // resolved map seed (random unless --seed given)
"grid_width": 13, "grid_height": 7,
"max_turns": 80,
"bomb_base_cost": 25,
"players": [
{ "slot": 0, "model": "claude-opus-4.8", "name": "Player 1", "base": [1, 3], "reasoning_effort": "high" },
{ "slot": 1, "model": "gpt-x", "name": "Player 2", "base": [11, 3], "reasoning_effort": "off" }
],
"winner": 0, // 0 | 1 | -1 (draw / mutual destruction)
"victory_type": "nuclear", // nuclear|military|ultimatum|peace|mutual_destruction|timeout
"total_turns": 9
},
"terrain": { // mountains/passages are FIXED for the match;
"mountains": [[6,1],[6,2],[6,4],[6,5]], // deposits here = the turn-1 layout
"passages": [[6,0],[6,3],[6,6]], // (see turns[].deposits for current)
"deposits": {
"credits": [[2,1],[2,5],[10,5],[10,1]],
"uranium": [[3,3],[9,3]],
"uranium_central": [[6,3]]
}
},
"turns": [
{
"turn": 1,
"active_player": 0,
"players_state": [
{ "slot": 0, "credits": 5, "uranium": 0,
"bomb_cost": 25, "has_silo": false,
"reasoning": "…free reasoning text…",
"diplomacy_sent": null,
"visible_cells": [[0,1],[0,2],[1,3]],
"knowledge": {
"enemy_base_discovered": false,
"visible_enemy_units": [],
"remembered_enemy_buildings": []
} },
{ "slot": 1, "...": "..." }
],
"units": [
{ "id": "A_drone_1", "owner": 0, "type": "drone", "pos": [4,3], "altitude": "air" }
],
"buildings": [
{ "id": "A_base", "owner": 0, "type": "base", "pos": [1,3], "hp": 4 }
],
"deposits": { // CURRENT deposit layout this turn (they move
"credits": [[2,1],[2,5],[10,5],[10,1]], // when exhausted — see events)
"uranium": [[3,3],[9,3]],
"uranium_central": [[6,3]]
},
"actions": [
{ "player": 0, "type": "move", "unit": "A_drone_1", "from": [1,3], "to": [4,3] }
],
"diplomacy": [
{ "from": 1, "to": 0, "kind": "ceasefire", "text": "…", "turn": 2, "response": "refused" }
],
"events": [
{ "type": "unit_destroyed", "unit": "B_tank_1", "unit_type": "tank", "owner": 1, "by": "A_fighter_1" },
{ "type": "building_damaged", "unit": "B_silo_2", "unit_type": "silo", "owner": 1, "hp": 1, "max_hp": 3, "by": "A_tank_1" },
{ "type": "deposit_exhausted", "pos": [2,1], "deposit": "credits", "owner": 0, "mine": "A_credit_mine_3" },
{ "type": "deposit_respawned", "pos": [4,2], "deposit": "credits" }
],
"perf": { // optional — LLM agents only
"think_ms": 1840, // active player's thinking time this half-turn
"tokens": { "prompt": 1200, "completion": 350, "total": 1550 },
"retries": { "timeout": 0, "api_error": 1, "malformed": 0 } // optional, only if >0
}
}
]
}- Coordinates
[column, row], top-left origin. units/buildings= absolute end-of-half-turn snapshot (not a diff). The viewer reconstructs the moves viaactions[].from/to.altitude∈"ground" | "air". At most ONE unit per layer per cell: an air unit can share a cell with a ground unit/building, but two air units cannot share a cell.uraniumis secret in-game but present in the replay (omniscient spectator); masked by the front end in player view.reasoning= free text (can be long), shown in the side panel.bomb_cost/has_silo= current bomb cost and silo presence, for the HUD bomb gauge (otherwise the front falls back tometa.bomb_base_costand deduces the silo frombuildings).visible_cells= list of[c,r]currently visible to the player (for replayable per-player fog).knowledge= what each player knows this turn (replayable A/B fog).deposits(per turn): the current deposit layout ({credits, uranium, uranium_central}, each a list of[c,r]). Authoritative for rendering since deposits deplete and respawn during a match; the viewer reads this and falls back toterrain.depositsfor older replays.events(optional):unit_destroyed·building_destroyed·building_damaged(a tank hit that did NOT destroy: carrieshp/max_hp, lets the viewer play the intermediate damage sprite in sync with the shot) ·deposit_exhausted(a deposit drained to 0: carriespos,deposit, and themine/ownerremoved) ·deposit_respawned(a fresh deposit of the same kind spawned: carriespos,deposit).buildings[].under_construction(optional,true): present while the building is being built (placed but not yet operational). The front shows it with theconstructsprite. Absent once the building is finished.actions[].under_construction(optional) on abuildaction: marks the placement of a construction site.perf(optional, LLM agents):think_ms(duration ofdecide()) +tokensconsumed by the active player this half-turn. The HUD shows the token total and the last thinking time per player.invalid_actions(optional): list of actions rejected by the engine this half-turn, each{type, detail, …original fields}. Used by the benchmark to measure the illegal-action rate proposed by a model (reasoning quality). The leaderboard aggregatesinvalid_actions+invalid_action_rate.
Types: units drone | sam | tank | fighter · buildings
base | credit_mine | uranium_mine | uranium_mine_central | silo · actions
produce | move | attack | build | launch | wait · diplomacy (kind)
message | ceasefire | peace | ultimatum (+ response accepted | refused).
Optional meta fields:
players[].tokens({prompt, completion, total}) is added by the runner for LLM agents, to display token consumption in the leaderboard / stats.players[].reasoning_effort— the reasoning effort label for each player, one of"high" | "medium" | "low" | "off" | "NA"(NA= the provider returned no reasoning / it is unsupported; the list may be extended later). It comes from the config.ini(heuristic agents reportoff).
generate_stats.py aggregates every replay into data/leaderboard.json. A key
schema point:
Each competitor is keyed on the (model, reasoning_effort) PAIR. A model run at a different reasoning effort counts as a distinct leaderboard entry.
Each model entry therefore carries a reasoning_effort field ("high" | "medium" | "low" | "off" | "NA") alongside model, plus wins, losses, draws,
total, points (3 / 1 / 0.5 / 0 cumulative — mutual destruction scores 0
and counts as a loss for both, it is not a draw), points_per_match (average
points per match — the primary ranking key), win_rate, the outcome buckets
(nuclear_wins, military_wins, diplomatic_wins, and mutual_destructions,
which is not a win bucket) and the benchmark
performance fields (avg_think_ms, avg_tokens_per_turn, invalid_actions,
invalid_action_rate, the retry counters retries_timeout / retries_api_error
/ retries_malformed / retries_total, and the estimated cost
total_cost_usd / avg_cost_per_match). Models are ranked primarily by
points_per_match, then win_rate, then total, then points.
Likewise, replays/index.json items carry p1_reasoning_effort /
p2_reasoning_effort next to p1_model / p2_model.
The leaderboard table shows one row per (model, reasoning_effort) pair — the
effort badge (HIGH / MED / LOW / OFF / NA) sits next to the model name — and
includes a Pts/match column (the ranking key) plus the $/match column
(estimated average USD cost). Every column header is clickable to sort.
Column order: Pts/match | Total | W | L | D | Win rate | Pts | … — volume
first (Total), then detail (W/L/D), then derived stats (Win rate, Pts).
Models with fewer than the "min matches to rank" threshold (default 1) are moved below a separator and flagged provisional so a single lucky game cannot top a model with a long, proven record. Below the table, the Recent matches list shows every match in a single list sorted newest first (by date and time): there is no time-based archive, nothing disappears after N days. When there are more than 20 matches, the list is paginated (20 per page); changing page re-renders only the match block, and the search box filters across all matches (every page), resetting to page 1 of the results.
A "📷 Save as image" button (next to the filters) captures the visible leaderboard as a PNG — includes the logo, title, and a branded footer with the date and a link to ageofllm.org.
Imposed stack: vanilla HTML + CSS + JavaScript, isometric Canvas 2D
rendering, no build step, hostable on GitHub Pages. The whole UI (index.html,
viewer.html) is now in English.
Decoupling (so a future Three.js 3D backend is substitutable):
iso.js— pure grid↔screen projection math + z-order. Knows neither the DOM nor the data.sprites.js— preloads the WebP sprites and resolvestype/facing/state/variant→ image (see states below).player.js(Player.Playback) — playback state machine (turn, speed, play/pause) + building an interpolatedframe, sprite-state computation (construct/normal/damage/destroy/launch, stable variant, ghosts of destroyed units). Does not know the Canvas.renderer.js— draws aframeon the Canvas (terrain, units, buildings, shadows, fog, attack/launch overlays) + zoom/pan camera. No playback state.stats.js— computes per-turn series and draws vanilla Canvas charts (no library).viewer.js— the only module linking DOM + Player + Renderer (transport, camera, panel docks, keyboard).leaderboard.js— leaderboard table + recent-match list.
- WebP format (converted from PNG via
scripts/convert_webp.py, ~30% space saved, lossless). - Units:
idle/attack(4 directions ne/nw/se/sw; the drone has noattack) /destroy. A unit destroyed this turn is shown as a ghost with its destruction sprite. - Buildings:
construct(at the construction turn) →normal→damage(HP ≤ 50%) →destroy. The silo has dedicatedlaunchanddestroysprites; destroyed base/credit/uranium use the genericbuilding_destroy. - Variant
var1/var2: chosen stably (hash of the id) and kept for the whole match regardless of state. Same for resources (resource_credit.webp,resource_uranium.webp). - Per-player base: the base uses no random variant but a dedicated sprite
per side:
building_base_player1/building_base_player2(andbuilding_base_damage_player1/2). The SAM has dedicatedattacksprites (unit_sam_attack_{dir}). - Nuclear effects (slowed for drama, ≥ ~2 s):
decor_nuclear-missile_launch— at the launch turn, the missile slowly rises above the launcher's silo (launchActionMs ≈ 2200 ms). The silo switches to thelaunchsprite from the start of the slice and stays there until the end of the turn (it does not revert to the normal sprite).decor_atomic_explosion— at the resolution turn (end of turn), an atomic mushroom appears with a fade-in (nukeMs ≈ 2600 ms) on each base destroyed by the bomb (both in case of mutual destruction), foot placed under the cell to cover the base, radial flash on top. During this resolution, all buildings and units of the nuked player(s) switch todestroy. Appears only for a nuclear death.
The 3 actions of a half-turn are played one after another, never
simultaneously. The duration of a half-turn is actionMs × (number of non-wait actions) (actionMs ≈ 900 ms), except for the slowed nuclear moments: a turn
that launches the bomb uses launchActionMs ≈ 2200 ms per action, and the turn
that resolves the explosion lasts nukeMs ≈ 2600 ms. The normalized clock
animT 0→1 is split into N slices (one per visible action). Within each slice,
a local clock localT 0→1:
0.00 … 0.55— MOVE phase:from→toglide (smoothstep easing) of the unit involved + movement arrow (dashes + tip, player's color); a produced unit appears at the start of its slice. The moving sprite orients toward its destination (4 dirs).0.55 … 1.00— ACTION phase: attack line / launch flash / target'sdestroysprite (the victim only disappears at the slice of the attack that killed it). The attacking sprite orients toward its target (4 dirs).
Facing (
Iso.facingFromDelta) is derived from the dominant grid axis, not the screen-space diagonal:+col → se,-col → nw,+row → sw,-row → ne. This matches the idle facing (a base looksse/nwtoward the enemy half) so a unit moving toward an adjacent cell always turns the right way.
Buildings under construction (under_construction) stay in construct. The
end-of-half-turn snapshot is authoritative for the final positions. The owner
marker above each unit is a small dot in the player's color (not a bar, to
avoid confusing it with a health gauge).
- Playback: Play/Pause (
Space), speeds 0.5× / 1× / 2× / 4×. - Navigation:
←/→turn by turn, or the scrubber. - Camera: wheel to zoom (cursor-centered), drag to pan, buttons
+/−/⊙, keyboard+/−/0. - View: Spectator (omniscient) / Player 1 / Player 2. In Player view, the
true fog is replayed from
visible_cells(unseen cells are dark, hidden units/buildings invisible) and the enemy uranium is masked. - Back to leaderboard: "⮜ Leaderboard" link at the top-left of the viewer.
- Panels (stacked side docks, non-overlapping): 🧠 Reasoning · 💬 Diplomacy ·
📊 Statistics (5 tabs: economy / military / bomb / production / losses) ·
📖 Legend & rules (goal of the game, victory conditions, unit/building tables,
combat triangle, nuclear bomb, diplomacy options — everything a newcomer
needs to follow a replay).
Hhides/shows. Each panel is height-capped (min(dock height, 55vh)) and its body scrolls internally rather than growing unbounded. - Diplomacy log: each entry is prefixed with its turn number (
T1,T2…) for quick orientation. The panel auto-scrolls to the latest message while playing; scrolling is free when paused. - Reasoning panel: also auto-scrolls to the bottom each turn while playing; free when paused.
- Diplomacy notification: when a diplomatic message/proposal/response lands on the turn being viewed and the Diplomacy panel is closed, the 💬 rail button pulses and a transient toast appears (click it to open the panel); the new entries are highlighted in the log. It clears on the next turn or when the panel is opened.
- Player HUD: two top-centered bricks, below the header. The model name is emphasized (large, high-contrast), with the slot (Player 1/2), a color dot, and credits / uranium / units / buildings counters.
- Mobile / responsive: wrapping header, two-column HUD, panels as a scrollable bottom-sheet, pinch zoom, horizontally scrollable leaderboard table.
- Bomb gauge: each player card shows a progress bar (uranium / current
cost, → "READY" at 100%) and the 3 launch prerequisites: 🏗 Silo · ☢
Sufficient uranium · 📡 Enemy base discovered (green = met). In Player view, the
opponent's uranium progress is masked (
?). - LLM HUD: each player card shows the turn tokens + cumulative (▦) and the
turn's thinking time (⏱) with its average, read from
turns[].perf. Empty for heuristic matches.
| Script | Role |
|---|---|
run_game.py |
Single match (default: LLM vs LLM). Starting point for everything. |
run_queue.py |
Match queue: run a list of specific 1v1 matches from queue.json, one after another. Supports --resume and --reset. |
run_tournament.py |
3-round tournament: random draw → winners vs winners → split again. Each model plays exactly 3 matches. |
tournament_models.example.json |
Model list template (copy to tournament_models.json). |
generate_stats.py |
Re-reads replays/*.json → data/leaderboard.json + replays/index.json. |
update_site.sh |
generate_stats.py then git add data/ replays/ && commit && push (private repo). |
update_viewer.sh |
Regenerates stats then copies replays/ + data/ to the public viewer repo and pushes it. |
scripts/convert_webp.py |
Converts PNG sprites → WebP (Pillow, venv). |
.venv/bin/python run_game.py # single LLM vs LLM match
python3 run_game.py --p0 greedy --p1 greedy --seed 42 -v # single heuristic match
.venv/bin/python archive/measurement/run_queue.py # run all matches in queue.json
.venv/bin/python archive/measurement/run_queue.py --resume # continue interrupted queue
.venv/bin/python archive/measurement/run_queue.py --reset # restart queue from match 1
.venv/bin/python archive/tournament/run_tournament.py # 3-round tournament
.venv/bin/python archive/tournament/run_tournament.py --resume # continue interrupted tournament
.venv/bin/python archive/tournament/run_tournament.py --reset # restart from scratch
python3 scripts/generate_stats.py
./scripts/update_site.sh "New replays" # update private repo
./scripts/update_viewer.sh "New replays" # update public viewer repo
.venv/bin/python scripts/convert_webp.py --delete # PNG -> WebPUse this when you want to run specific 1v1 matches in sequence — for example, to evaluate a new model against 3 known opponents.
1. Create queue.json (git-ignored, never pushed):
[
{
"p1": { "name": "Kimi K2.6", "model": "kimi-k2-6", "provider": "venice",
"thinking": true, "reasoning_effort": "high",
"cost_input": 0.85, "cost_output": 4.66 },
"p2": { "name": "DeepSeek V4", "model": "deepseek-v4-pro-e", "provider": "poe",
"thinking": true, "reasoning_effort": "high",
"cost_input": 2.42, "cost_output": 4.85 }
},
{
"p1": { "name": "Kimi K2.6", "model": "kimi-k2-6", "provider": "venice",
"thinking": true, "reasoning_effort": "high",
"cost_input": 0.85, "cost_output": 4.66 },
"p2": { "name": "Gemini Flash 3.5", "model": "gemini-3.5-flash", "provider": "poe",
"thinking": true, "reasoning_effort": "high",
"cost_input": 1.52, "cost_output": 9.09 }
}
]Each player slot supports: name, model, provider, thinking,
reasoning_effort, reasoning, thinking_budget, cost_input, cost_output.
2. Run the queue:
.venv/bin/python archive/measurement/run_queue.py # run all matches in queue.json
.venv/bin/python archive/measurement/run_queue.py --resume # continue after an interruption
.venv/bin/python archive/measurement/run_queue.py --reset # wipe state and restart from match 1
.venv/bin/python archive/measurement/run_queue.py --stats # regen leaderboard after each match
.venv/bin/python archive/measurement/run_queue.py --queue my.json # use a different queue fileKey behaviours:
- Progress saved to
queue_state.json(git-ignored) after each match. --resumeskips already-completed matches (identified by index)._commentfields inqueue.jsonare ignored.- A failed match is logged and skipped; the queue continues.
- Final leaderboard regenerated at the end (or after each match with
--stats).
The tournament uses a 3-round fixed format optimised for 8 models (each plays exactly 3 matches):
| Round | Pairings |
|---|---|
| Round 1 | Random draw — completely random, regardless of score |
| Round 2 | Split — top-4 (by points) play each other; bottom-4 play each other |
| Round 3 | Split again — same logic with updated standings from rounds 1+2 |
With 8 models: 4 matches per round, no byes, every model plays exactly 3 real matches. For other numbers of models a Swiss fallback kicks in (pair by adjacent score, odd count gets a bye).
cp tournament_models.example.json tournament_models.json # edit with your models
.venv/bin/python archive/tournament/run_tournament.py
.venv/bin/python archive/tournament/run_tournament.py --resume # continue if interrupted
.venv/bin/python archive/tournament/run_tournament.py --reset # restart from scratchKey behaviours:
- State saved to
tournament_state.jsonafter each match →--resumecontinues mid-tournament. - A mid-match interruption replays the full match from the start on resume.
--statsregenerates the leaderboard after each match; omit for a single regen at the end.- Final standings printed after round 3.
Most models on Poe use the standard OpenAI-compatible endpoint (provider: "poe").
Claude models require a different API path (fastapi_poe native SDK) to enable
extended thinking (thinking_budget):
{
"name": "Claude",
"model": "claude-opus-4.8",
"provider": "poe_native",
"thinking": true,
"thinking_budget": 8000,
"cost_input": 5.00,
"cost_output": 25.00
}thinking_budget is the number of reasoning tokens Claude may use per turn (default 8000).
The same key as [poe] in .config.ini is used automatically — no separate credential needed.
Token counts for
poe_nativeare estimated (≈ chars / 4) since the native SDK does not expose exact counts. All other providers report exact counts from the API.
.venv/bin/python scripts/test_thinking.py # the slot in [player1]
.venv/bin/python scripts/test_thinking.py --player 2 --effort highOne real decide() call against the configured provider/model: the reply parses
as JSON with an actions list, thinking=true returns non-empty reasoning, and
thinking=false costs fewer tokens. Run it before a multi-hour campaign to catch
a broken key or a wrong model id while it still costs one call.
Gone:
test_models_poe.pyandtest_model_venice.py, which swept every reasoning mode of a provider into amodels_validated.json. The providers are still supported — only the sweep helpers went.
- Player slots:
0= Player 1 (base on the left),1= Player 2 (mirror). - Internal symmetry: mirror of
(c, r)=(12-c, 6-r). Used only for balanced map generation — never disclosed to the LLM and not exploitable to deduce the enemy layout. - Stable IDs for the whole match:
A_tank_3,B_silo_1,A_base… - Half-turn = one
turnsentry;active_playeralternates 0,1,0,1… - Absolute snapshots: never emit a diff.
- Destroyed base: kept as
hp=0in the final frame (visible wreck), not removed from thebuildingsarray. - Secret uranium: present in the JSON, masked on the front in player view.
- One unit per layer per cell: a ground unit cannot share a cell with another ground unit or building; two air units cannot share a cell either.
- Dependencies: engine core = standard Python;
openai(lazy import) for LLM agents (all providers exceptpoe_native);fastapi-poe(lazy import) only forpoe_native(Claude thinking_budget);Pillowonly for WebP conversion. All three are inrequirements.txt. Front end = 100% vanilla JS. - Runner default:
run_game.pyplays LLM vs LLM out of the box (no--llmneeded); pass--p0/--p1greedy/randomfor heuristic agents. Live progress prints automatically when an LLM plays (-qto silence). - No versioned secrets:
.config.ini(API keys),.venv/and theRessources/folder (Figma sources, etc.) are git-ignored and must never be pushed; onlyconfig.example.iniis committed. - Local runtime files (git-ignored, never pushed):
tournament_models.json(copy oftournament_models.example.json, edit with your models),tournament_state.json(auto-saved after each match, deleted by--reset),queue.json(your match list forrun_queue.py),queue_state.json(auto-saved after each match, deleted by--reset),models_validated.json.
This (private) repository contains both the engine and the viewer. A separate public repository (ageofllm-benchmark-viewer) hosts only the static site needed by GitHub Pages.
🎥 Presentation video: https://youtu.be/Ec-CV1uzyVY ⭐ Star the public repo: https://github.com/Macmachi/ageofllm-benchmark-viewer 🐦 Follow on X to know when new models are tested: https://x.com/ageofllm
When you publish the next update (./scripts/update_viewer.sh), the viewer reflects these
rule changes — the legend and replays now show dynamic seed-driven terrain
(central barrier + scattered mountains vary per match), line-of-sight blocked
by mountains and buildings, mine-able enemy-side deposits (no instant
capture), resource depletion + respawn (deposits move when exhausted), the new
2 C / 3 C mine costs, and the damaged-building animation synced to the
shot. Old replays still render correctly (the viewer falls back to the static
terrain.deposits and ignores absent events).
index.html
viewer.html
assets/ (css/, js/, images/, sprites/)
data/ (leaderboard.json — generated)
replays/ (*.json match replays + index.json — generated)
README.md
| File / folder | Reason |
|---|---|
engine/ |
Game engine source code |
run_game.py, run_ladder.py, match_runner.py |
Backend runners (run_queue.py / run_tournament.py retired to archive/) |
generate_stats.py, update_site.sh, update_viewer.sh |
Build/deploy scripts |
.config.ini |
API keys — git-ignored, never committed anywhere |
config.example.ini |
Config template |
requirements.txt |
Python deps |
tournament_models.json, tournament_state.json |
Runtime state |
Ressources/ |
Figma sources and design files |
scripts/ |
Dev tooling |
After new matches are completed, run from this directory:
./scripts/update_viewer.sh "Round 3 results"This single command: regenerates stats → copies replays + leaderboard to the viewer repo → commits and pushes. GitHub Pages updates automatically within ~30 s.
© Rymentz 2026 — All rights reserved.
The code, assets, sprites, and all content in this repository are the exclusive property of Rymentz. Reuse, redistribution, or sale of any part of this repository is strictly prohibited without prior written authorization from Rymentz.
"Age of LLM" is a trademark of Rymentz™.
- 🌐 Website: ageofllm.org
- 🐦 Follow on X: @ageofllm