Trace Harness

Trace Harness

How Sightline keeps its behaviour honest while the engine underneath it is replaced. This page is for readers who want to understand how correctness is kept; the operating manual is docs/recording.md and tools/trace/README.md.

Two artifacts

• The matching build — the original decomp compiled with the IDO compiler to a byte-identical N64 ROM (make matching ends with MATCH!). It is the correctness oracle and stays buildable permanently.
• The trace harness (tools/trace/) — recorded input streams plus per-tick state hashes. Any change touching game logic must replay the recorded corpus unchanged.

What a trace is

Recording a level produces three files, and all three are needed to reproduce the run:

File	What it is
tools/trace/inputs/<level>.input	the controller input, 4 bytes per frame
tools/trace/inputs/<level>.eeprom	the cartridge save the session started from
tools/trace/traces/<level>.sltrace	per-tick state hashes — the baseline

The hashed state covers what a player could notice: entity positions and rotations, health, weapon and ammo state, the RNG cursor, alarm and objective flags, animation frame indices — and, in the current schema, guard perception and world props as well, so the corpus exercises far more of the AI than the earliest schemas could see.

What make trace-verify does

make trace-verify                    # every recorded level
make trace-verify LEVEL=facility     # one level, faster iteration

For each recorded level it replays the input stream against the matching ROM under the emulator, from the same cartridge save, captures a fresh per-tick state hash, and diffs it against the committed baseline. A pass prints <level>: OK. A failure is not a bare pass/fail: trace-diff reports the first divergent tick and which state fields differ, which is what turns "something feels wrong in Silo" into "tick 4412, guard 7's pathfinding state diverged".

The harness runs the ROM under the parallel_n64 libretro core, driven by Sightline's own frontend, because that core reproduces byte-identically with no tuning (mupen64plus did not — roughly one replay in four). The ROM is never modified: state is read from emulator RAM at addresses resolved from the matching build's linker map, so what runs is the byte-identical matching ROM.

Why

Parallel work on a codebase nobody holds a full mental model of multiplies the surface area for subtle behavioural regressions. Speed of authoring is not the bottleneck; speed of diagnosis is. The harness is what makes that diagnosis take minutes instead of weeks — and it is what let Phase 1 cut higher than earlier decomp ports could afford to, because changing game code became safe rather than merely careful.

A divergence means exactly one of two things. Unintentional: it is a bug, and the code is fixed, not the baseline. Intentional: it is documented in docs/divergences.md first, and re-baselined in a separate commit that touches nothing else. See Original Behaviour and Toggles.

The gate

trace-verify replays a level once. The Phase 0 gate replays every campaign level ten times, because both real failures the harness has caught were invisible to a single comparison:

make trace-gate-status        # what is passing, what is not
make trace-gate               # gate everything that needs it

All 20 campaign levels currently pass, and a recording that reaches the repository without a current gate result fails CI.

Recording

make trace-record LEVEL=<name>     # play it; close the window to stop
make trace-verify LEVEL=<name>     # replay the recording, diff state hashes
make trace-roundtrip               # prove record -> replay is faithful

trace-record builds a ROM that boots straight into the mission on Agent (cached under build/u/direct/), so there are no menus to walk, and always restores the matching ROM afterwards. trace-roundtrip proves the recorder writes what replay plays back — replay determinism alone would not catch a stream applied one frame early. The level names and their internal stage ids are tabulated in docs/recording.md.

What it needs

• Your own NTSC-U ROM (ROM Setup); make verify-rom runs first.
• The matching build (make matching) and its map, build/u/ge007.u.map.
• The parallel_n64 libretro core, downloaded separately and never committed. docs/toolchain.md gives the download commands for Linux and Windows and the SL_LIBRETRO_CORE override; the core's architecture must match your Python's.
• A display, or xvfb-run on a headless Linux host.

The harness is driven through make on Linux. The Windows scripts do not run it; tools\windows\test.ps1 is a headless liveness check of the native build, not a trace comparison.