Skip to content

Vol.2 checkpoints — MNIST 97.04% (3 seeds, 8k gates, no cross-layer backprop)

Latest

Choose a tag to compare

@Mming-Lab Mming-Lab released this 17 Jul 07:42

The three frozen circuits behind the Vol. 2 headline: MNIST 97.04% (3-seed mean), single network, no gradient ever crosses a layer boundary.

A checkpoint stores every frozen layer's blueprint (wiring ia/ib + the 16-way gate logits), so the trained circuit rebuilds exactly — no need to repeat the ~40 min/seed.

asset seed layers saved headline depth test acc gates (raw → simplified)
vol2_8000_seed1.pt 1 20 10 97.00% 80,000 → 66,225
vol2_8000_seed2.pt 2 12 10 97.08% 80,000 → 66,307
vol2_8000_seed3.pt 3 12 10 97.05% 80,000 → 66,100

Config: --dataset mnist --group-residual --skip-input --gates 8000 --batch 1024 --epochs 60. Every run verified bit-exact through simplification (outputs identical = True).

The headline depth is 10 for all three seeds, though each file holds more layers. Deeper is not better here: at depth 10 the network has already memorized the training set (train = 100%) and test freezes at ~97.0% — seed 1's extra 8 layers buy +0.04 pt. Truncating to 10 is free (layers[:10] is a complete circuit — a frozen layer is never retrained).

Verify or extend without training

# rebuild and re-check any depth
python tools/vote_checkpoints.py --members vol2_8000_seed1.pt:10

# vote the three seeds as an ensemble (inference only)
python tools/vote_checkpoints.py --members \
  vol2_8000_seed1.pt:10 vol2_8000_seed2.pt:10 vol2_8000_seed3.pt:10

# train onward from the saved layers
python experiment.py --dataset mnist --device cuda --group-residual --skip-input \
  --gates 8000 --batch 1024 --epochs 60 --seed 1 --skip-e2e \
  --max-layers 24 --patience 12 --checkpoint vol2_8000_seed1.pt

Resuming replays the saved layers and restores the accumulator, best-accuracy and patience state bit-exactly, then continues; a fingerprint check refuses the resume if any setting that affects layer contents differs. Loading uses weights_only=True (no pickle code execution).

Caveats

  • Gate efficiency is a clear loss. e2e-trained LGNs are far more compact: difflogic reaches 97.69% with 48k gates, LILogic Net 98.45% with 8k — vs ~66k here for 97.04%. The point of this repo is the constraint (layer-local training only), not the gate count.
  • Accuracies above are the CUDA training probes; a CPU replay of the same circuit reads ~0.05 pt lower (counts/τ is computed as counts×(1/τ) on CUDA, one ULP off a true divide, accumulated over 10 layers). The circuit is identical either way — see the numerical footnote in RESULTS.md.
  • The depth probe is the test set. A uniform depth 10 limits the selection bias but doesn't eliminate it; a validation split would settle it.

Raw logs: issue #15.