civ6-logger

Capture everything that happens in a Civilization VI game — every action (production, builds, districts, unit orders, combat, diplomacy, research) and the full game board, turn by turn — so any turn's complete state is reconstructible, and the game is analyzable from the outside.

Built and verified on macOS / Apple Silicon (M4 Max, Steam/Aspyr port), single-player, base game through Gathering Storm.

Status: working end-to-end. It captures the full hex board (terrain, features, resources, improvements, districts, routes, owners, per-tile yields, worked tiles), every unit (type, health, promotions, XP, movement, charges), cities with per-district strength/HP/walls and buildings, city-states (suzerain + envoys), barbarians, natural wonders, government + policy cards, the tech & civic trees (incl. eurekas / inspirations), and all diplomacy (deals, war declarations + targets, captures, peace, denouncements) — for every civ, not just yours.

See docs/CAPABILITIES.md for the exact field-level matrix.

---

The one constraint that shapes everything: no Lua.log on Apple Silicon

Every Windows guide to "logging from a Civ6 mod" relies on Lua print() landing in Lua.log. That file is not written on ARM Macs (a regression since ~2020; the community patch "is not viable" on ARM). Civ's Lua is also sandboxed (no file I/O). So the hard problem was never reading game state — it was getting it out.

We solve it with two channels (details in docs/ARCHITECTURE.md):

1. Built-in report logs — the C++ gamecore writes a directory of turn-stamped CSVs (City_BuildQueue, Player_Stats, CombatLog, UnitOperations, DiplomacySummary, DiplomacyDeals, …). These do work on ARM and cover the "actions + per-turn stats + diplomacy" half for free.
2. The FireTuner debug socket (127.0.0.1:4318) — it opens on Apple Silicon even though Lua.log doesn't. We inject Lua into the running game over it and read results back over the socket, which gives the live hex board and anything else the gameplay API exposes.

---

Setup

1. Enable the tuner socket. In …/Firaxis Games/Sid Meier's Civilization VI/AppOptions.txt, under [Debug], set EnableTuner 1, then relaunch Civ6. ⚠️ This opens a localhost console that executes arbitrary Lua in your game — a real (if local-only) RCE surface. Set it back to 0 when you're done.
2. Python 3.9+ (stdlib only — no dependencies). Tools auto-resolve the macOS Civ6 paths; override with the CIV6_SUPPORT env var if your install differs.
3. Run a tool while in a loaded game (the gameplay Lua contexts only exist in-game). python3 tools/netcheck.py verifies the socket is up.

Running under Claude Code? The auto-approval classifier may gate python3 tools/*. A .claude/settings.local.json allow-rule for Bash(python3 tools/*) clears it. In a normal terminal this isn't needed.

---

Quickstart

# one-time: build the type/name decoder from the game's own DBs
python3 tools/decode.py dump

# parse the always-on report logs into a queryable store (works even offline)
python3 tools/parse_logs.py build
python3 tools/parse_logs.py summary --turn 160

# --- with Civ6 running, in a loaded game, EnableTuner 1 ---
python3 tools/export_board.py        # full board snapshot -> data/board/turn_NNNN.json
python3 tools/board_report.py        # readable dump of that snapshot
python3 tools/analyze.py             # strategic standings: you vs the field
python3 tools/diplo.py               # war declarations / targets / peace / denounce
python3 tools/deals.py               # every trade/peace deal, incl. AI-to-AI

# continuous per-turn capture (run in a terminal that stays open as you play):
python3 tools/capture.py watch
python3 tools/capture.py reconstruct --turn 150   # rebuild any captured turn

---

Tool catalog

Tool	What it does
tools/tuner.py	FireTuner socket client — run Lua in the live game, capture output (states / exec / smoke)
tools/export_board.py	One full board snapshot → data/board/turn_NNNN.json
tools/capture.py	Per-turn daemon: keyframe + delta capture (watch), plus reconstruct --turn N
tools/board_report.py	Human-readable rendering of a board snapshot
tools/analyze.py	Comparative strategic analysis (standings, rankings, city-states, threats)
tools/target.py	Scout an attack target: city defenses, garrison, terrain, forces in range
tools/diplo.py	Parse DiplomacySummary.csv → wars (+target/type), captures, peace, denouncements
tools/deals.py	Parse DiplomacyDeals.log → every deal item-by-item, incl. AI-to-AI
tools/parse_logs.py	Ingest all turn-stamped report CSVs → data/civ6log.sqlite + per-turn summary
tools/timeline.py	Stitch action-log + board captures into one per-turn record
tools/decode.py	Type ⇄ hash ⇄ human-name decoder (from DebugGameplay/DebugLocalization)
tools/civ6_paths.py	Resolve macOS Civ6 paths (run it to print them)
tools/netcheck.py	Diagnose the tuner socket (connect test + Civ pids + listeners)
tools/inspect_*.py	Dev probes that reflect the Lua API (how every field above was found)
scripts/probe.sh	setup/check/teardown for the diagnostic probe mod

Full reference: docs/TOOLS.md.

---

Data layout

data/
  types.json              decoded type table            (decode.py dump)
  civ6log.sqlite          Layer-0 report-log store       (parse_logs.py build)
  board/turn_NNNN.json    full board snapshots           (export_board.py)
  capture/turn_NNNN.json  keyframe + delta per turn      (capture.py watch)
  timeline/turn_NNNN.json merged action + board per turn (timeline.py build)
tools/                    the toolkit (stdlib Python)
mods/Civ6Logger_Probe/    diagnostic mod used to crack the exfil channel
scripts/probe.sh          probe orchestrator
docs/                     architecture, capabilities, tools, gotchas

data/ is gitignored. Snapshots are ~1 MB/turn (per-tile yields dominate); deltas are a few KB.

---

Caveats & deferred

• Report logs reset when a new game starts, and autosaves keep only ~10. For a full-game record, run capture.py watch from turn 1 (and bump AutoSaveKeepCount if you want a save trail).
• Live capture needs the game in a loaded game with the tuner up. A crash can leave the socket on the empty :4319 fallback — see docs/GOTCHAS.md.
• Deferred by choice: city-state category labels (Scientific/Militaristic/…), and back-dating tech/civic completion turns for techs finished before capture began (per-turn diffs cover everything going forward).

Docs

• docs/ARCHITECTURE.md — the two channels, the FireTuner wire protocol, keyframe/delta capture & reconstruction, the timeline stitch.
• docs/CAPABILITIES.md — exactly what's captured, the board-snapshot JSON schema, and the @@ wire-record format.
• docs/TOOLS.md — per-tool reference and examples.
• docs/GOTCHAS.md — the hard-won lessons & troubleshooting.