LAWFUL: Law-Aligned Witness for Faithful Use of Latents Reproduction (figure-only + optional sanity recompute)

Companion artifacts for the paper. The repository has two stages:

• Stage 1 — Figure / table reproduction (lightweight, no model required). Renders the τ-physical-consistency-circuit table and the radial / pure-tangential α-sweep figures from precomputed JSON measurements. Runs in seconds on any machine with numpy, pandas, and matplotlib.
• Stage 2 — Sanity recompute (optional, requires PyTorch + GPU). Re-runs one row of the paper table (τ=0.3) end-to-end on a small RandomWalk subset using the released checkpoint. Verifies that the upstream measurement pipeline runs and returns numbers in the right ballpark; does not reproduce the full paper run.

---

Scope of this repository (read first)

This repository does not, by default, recompute any of the numbers reported in the paper. Stage 1 only re-formats precomputed measurements into figures and a table. Stage 2 recomputes a single τ row on a reduced-N subset and is intended as a smoke test of the pipeline, not as numerical reproduction of the paper figures.

Concretely (Stage 1):

• scripts/make_tau_table.py reads the per-τ Delta_suf, Delta_complement (= Δ_nec), and Delta_rand_{suf,nec}_{mean,std} values out of the JSON files in data/ and renders them as a LaTeX and Markdown table. No model is loaded, no activation patching is run.
• scripts/plot_radial_main.py and scripts/plot_tangential_pure_main.py read precomputed α-sweep curves out of data/e11_tangential.json and the physics references out of data/phys_*.json, then call matplotlib. No model is loaded.

The JSON files in data/ are the verbatim outputs of the upstream measurement pipeline:

artifact	produced by
circuit_discovery_results.json	scripts/cpt/circuit_discovery/run_search.py
circuit_discovery_complement.json	scripts/cpt/circuit_discovery/compute_complement.py
circuit_discovery_source_randomized.json	scripts/cpt/circuit_discovery/compute_paper_source_randomized.py
e11_tangential.json	merged α-sweep (alpha_sweep_cdagger9_radial_tanpure, parts pos/neg)
phys_*.json	kinematic-only physics references (no model)

The full upstream pipeline requires the trained checkpoint, the entire preprocessed mocap/radar dataset, an activation cache (~470 GB built by build_cache.py), and a GPU. Only the checkpoint and a subset of the mocap data are shipped here, for two reasons: (i) cache size, and (ii) the rest of the dataset cannot be redistributed.

Implication for the reviewer. The numerical values in figures/tau_table.tex and the α-sweep figures carry the same epistemic status as the values in the paper PDF — they are taken on trust unless the upstream pipeline is re-run against the released checkpoint. What this repository specifically demonstrates is (a) that the table layout and α-sweep plots follow deterministically from the released JSON measurements (Stage 1), and (b) that a single row of the table can be reproduced end-to-end on a 50-window subset using the released checkpoint and the upstream patching machinery (Stage 2).

---

Contents

dcs_repro/
├── make_figures.py                     # Stage-1 entry point
├── scripts/
│   ├── make_tau_table.py               # τ-circuit table (LaTeX + Markdown)
│   ├── plot_radial_main.py             # radial α-sweep
│   ├── plot_tangential_pure_main.py    # pure-tangential α-sweep
│   ├── build_mini_cache.py             # Stage-2: build 50-window activation cache
│   └── sanity_recompute_tau03.py       # Stage-2: recompute τ=0.3 row
├── src/                                # Stage-2: minimal model + DCS infra
│   ├── constants.py, data_loader.py, models.py, models_ablation.py,
│   ├── paths.py, preproc.py, utils.py
│   └── cpt/
│       ├── decisions.py, hooks.py
│       └── circuit_discovery/
│           └── cache.py, components.py, metrics.py, patched_M.py
├── configs/
│   └── config.yaml                     # STFT params (nperseg=256, noverlap=224)
├── data/
│   ├── circuit_discovery_results.json
│   ├── circuit_discovery_complement.json
│   ├── circuit_discovery_source_randomized.json
│   ├── e11_tangential.json
│   ├── phys_neg2to2_radtan.json
│   ├── phys_pure_modes_test_9alpha.json
│   ├── ckpt/best-88-0.4425.ckpt        # 20 MB, baseline_mean_pool_agg run
│   └── processed/
│       ├── eval_raw_full_sequences.pkl # full RandomWalk mocap, float32 (47 MB)
│       ├── scalers.pkl                 # fitted StandardScalers
│       └── meta.pkl                    # STFT freq axis etc.
├── figures/                            # outputs (created on first run)
├── requirements.txt                    # Stage-1 deps
├── requirements_recompute.txt          # Stage-2 deps (adds torch)
├── LICENSE
└── .gitignore

---

Stage 1 — Figure / table reproduction

Install

python -m venv .venv             # or: virtualenv .venv
source .venv/bin/activate
pip install -r requirements.txt

Tested with Python 3.10–3.12.

Reproduce all figures

python make_figures.py

Produces, in figures/:

• tau_table.tex / tau_table.md — τ-circuit table matching the paper (M_0, |C|, circuit C, Δ_suf, Δ_nec, Δ^rand_suf, Δ^rand_nec)
• radial_main.pdf / .png — radial-mode α-sweep
• tangential_pure_main.pdf / .png — pure-tangential α-sweep

Or run any script alone (scripts/make_tau_table.py, scripts/plot_radial_main.py, scripts/plot_tangential_pure_main.py).

Metric notes

• M_0 is the unpatched-circuit baseline, evaluated by the centroid-power metric in cpt.circuit_discovery.metrics. For this checkpoint, M_0(mag) = 0.9878.
• Δ_suf(C) = M_0 − M_suf(C); how much physical consistency is lost when only C is patched. Smaller is better for sufficiency.
• Δ_nec(C) = M_0 − M_suf(V\C); both nec quantities are evaluated on the unperturbed (base) input, with the complement V\C patched using the matched-α cache. C itself is left at its clean activation, so Δ_nec measures how much physical consistency is lost when only the rest of the network sees the α-update. Larger Δ_nec ⇒ C is more necessary.
• Δ^rand_suf / Δ^rand_nec are size-matched controls that patch the same components (C for suf, V\C for nec) on the unperturbed input but draw source activations from a random evaluation window (cache_alpha[j, α] with j ≠ i); reported as mean ± std over 10 seeds. The real circuit's Δ should lie far below the rand control.

Reproduction parameters used by the upstream search

parameter	paper value
family	mag
τ-grid	{0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9}
windows	2416 (all RandomWalk eval-split windows)
random-source seeds	10 (0..9)
n_prune_trials	8
prune_seed	0
n_random_subset_trials	100
random_subset_seed	0
ckpt	baseline_mean_pool_agg_20260202-210609/best-88-0.4425.ckpt

These are the defaults of scripts/cpt/circuit_discovery/run_search.py, compute_complement.py, and compute_paper_source_randomized.py in the upstream repository.

---

Stage 2 — Sanity recompute (optional)

Stage 2 runs one row of the paper table (τ=0.3) end-to-end against the released checkpoint, on a deliberately small subset of the data. The aim is to demonstrate that the patching pipeline is reproducible — not to match the paper's numerical values exactly.

What is reduced relative to the paper

dimension	paper run	Stage-2 default	reduction
windows	2416 (all RandomWalk eval)	50 (stride-uniform)	48× fewer
random-source seeds	10	5	2× fewer
τ values recomputed	7 (0.3..0.9)	1 (only τ=0.3)	7× fewer
circuits recomputed	5 distinct	1 (only T:{MLP, h6, h7})	—
family	mag	mag	same
dataset source	RandomWalk eval	RandomWalk eval	same
ckpt	best-88-0.4425.ckpt	best-88-0.4425.ckpt	same

The 50 windows are sampled uniformly across the full 2416-window RandomWalk sequence (np.linspace(0, 2415, 50).astype(int)) so the subset captures the full dataset's gait variation rather than a contiguous low-velocity segment.

What this implies for the recomputed numbers

With ~48× fewer windows and 2× fewer random seeds, the recomputed quantities have larger sampling error than the paper values:

• M_0 — the regression-based clean-model DCS over a smaller window set has wider variance. Empirically: |Δ| ≲ 0.05.
• Δ_suf(C) — most sensitive to which 50 windows are picked, because a few high-velocity windows can shift the regression slope. We allow |Δ| ≲ 0.20 from the paper value.
• Δ^rand_suf mean — averages over both windows and 5 random seeds. Mean is fairly stable, std is ~3-4× wider. |Δ| ≲ 0.30 on the mean.

These tolerances are what the verdict in sanity_recompute_tau03.py checks; they are deliberately loose. They will catch a broken pipeline (sign errors, wrong patch direction, hooks not firing, mis-loaded checkpoint) but tolerate the sampling variance from the small subset.

Install

pip install -r requirements_recompute.txt

If torch.cuda.is_available() returns False on a machine with a GPU, re-install torch matching your CUDA driver — see the comment block at the top of requirements_recompute.txt.

Run

Two commands. Step 1 builds the activation cache (heavy: ~20 GB on disk, 20 s on a single GPU, ~40 s on CPU). Step 2 runs the recompute (~30 s).

python scripts/build_mini_cache.py            # default: 50 windows, mag only
python scripts/sanity_recompute_tau03.py      # default: 5 random-source seeds

The cache lands in cache/mag/ (gitignored). Re-running build_mini_cache.py overwrites it.

Expected output

Numerical values from a reference run on the L40S GPU (deterministic given the cache):

quantity                      this run        paper          abs Δ
----------------------------------------------------------------
M_0_mag                         0.9666        0.9878         0.0212
Δ_suf(C)                        0.6367        0.4896         0.1471
Δ^rand_suf mean (n=5)           1.6555        1.8223         0.1668
Δ^rand_suf std                  0.1350        0.0360 (n=10)    —

Verdict:
  |M_0 − paper|         < 0.05  : PASS
  |Δ_suf − paper|       < 0.20  : PASS
  |Δ^rand mean − paper| < 0.30  : PASS
  overall                       : PASS

The full machine-readable output is written to figures/sanity_tau03.json.

What Stage 2 does not do

• Recompute τ ∈ {0.4, 0.5, 0.6, 0.7, 0.8, 0.9}.
• Recompute the necessity term Δ_nec(C) (would require computing patched_M(V\C, "suf") on the same cache; not currently in sanity_recompute_tau03.py).
• Recompute the α-sweep curves used by radial_main and tangential_pure_main (those use a different α-grid, α ∈ [−2, 2], which is not in the cache built by build_mini_cache.py).
• Run the greedy circuit search itself (we evaluate the fixed circuit C(τ=0.3) = T:{MLP, h6, h7}; the search procedure that selected this C is in the upstream run_search.py and is not included here).

Extending Stage 2 to cover any of the above is a matter of duplicating the upstream scripts; the data/ckpt subset shipped here is already sufficient for it.