Skip to content

MNIST champion checkpoints — 94.64% (3-seed mean, task 29)

Choose a tag to compare

@Mming-Lab Mming-Lab released this 16 Jul 05:18

Frozen circuits from the depth-exploration batch (issue #14) — the runs behind the current method headline, MNIST 94.64% (3-seed mean), single 500-gate net, no backprop across layers, no ensemble, no width scaling.

Each run took 7.6–11.8 h on a 6 GB RTX 3060. These assets let you skip that: a checkpoint stores every frozen layer's blueprint (wiring ia/ib + the 16-way gate logits), so the trained circuit rebuilds exactly.

asset seed committed layers selected depth test acc
task29_mnist_seed1.pt 1 51 41 94.36%
task29_mnist_seed2.pt 2 105 95 95.24%
task29_mnist_seed3.pt 3 53 43 94.32%

Config for all three: --dataset mnist --group-residual --skip-input --gates 500 --patience 10. All three stop on patience, not on the depth cap, so the protocol is uniform. Every run verified bit-exact through simplification (outputs identical = True).

Note the depth spread: seed 2 stays productive to depth 95 while seeds 1 and 3 are done by ~42. Seeds are not interchangeable here.

What you can do with them

# vote them as an ensemble (inference only, no retraining)
python tools/vote_checkpoints.py --members \
  task29_mnist_seed1.pt:41 task29_mnist_seed2.pt:95 task29_mnist_seed3.pt:43

# extend a seed further (resumes from the saved layers, trains onward)
python experiment.py --dataset mnist --device cuda --group-residual --skip-input \
  --seed 1 --skip-e2e --max-layers 120 --patience 20 --checkpoint task29_mnist_seed1.pt

Resuming replays the saved layers and restores the accumulator, best-accuracy and patience state bit-exactly, then trains onward — a fingerprint check refuses a resume if any setting that affects layer contents differs. Note --patience 10 alone won't extend seeds 1 or 3: they already exhausted it, so a resume stops immediately unless you also raise --patience. Loading uses weights_only=True (no pickle code execution).

Caveats

  • Accuracies above are the CUDA training probes. A CPU replay of the same circuit reads ~0.05 pt lower — counts/τ is computed as counts×(1/τ) on CUDA (one ULP off a true divide) and the residual readout accumulates that over 41–95 layers, reordering exactly-tied classes. The circuit is identical either way; see the numerical footnote in RESULTS.md.
  • The depth probe is the test set, so a patience 10 search also picks the best of more noisy draws — roughly 0.1–0.2 pt of each peak is likely optimism (seed 2's depths 90–105 sit in a 94.99–95.24% band). A validation split would settle it; not done here.

(Updated 2026-07-16: task29_mnist_seed2.pt was replaced — the original asset held 80 layers, truncated by a --max-layers 80 cap it was still climbing at. It now holds 105 layers and stops on patience like the others, which moved the headline 94.53% → 94.64%.)