Skip to content

SUSS v0.1.1 (preprint, all runs complete)

Choose a tag to compare

@nonberw nonberw released this 07 Nov 08:47
· 13 commits to main since this release

All planned runs are complete; the paper's main text fits the L4DC 10-page limit.

New since v0.1.0

  • Scale study complete: 3x training, 2x depth or 4x width bring the excess-risk ratio to Bayes from 1.59x to 1.22-1.30x; a 0.1M-parameter model sits at 5.3x. On a 4-state/2-input system (3 seeds) the transformer is 4.4-4.9x from Bayes, yet its query Jacobian still matches the ridge estimate (R^2 0.96-0.97).
  • Ten-seed replication (seed = unit of analysis): KV-only patching of blocks 3-4 transfers 0.93 +- 0.01 of another episode's behaviour; fast weights read the Bayes estimate with 0.91 +- 0.01 vs 0.24 +- 0.11 for the residual stream at the query.
  • Construction and MuJoCo figure moved to the appendix to fit 10 pages.

Assets: main.pdf (paper), supplement.zip (pre-registration, deviations, code, tests, result summaries, pilot logs), arxiv_source.zip (LaTeX source with .bbl).