Repository navigation
v0.1.0
First release of a decoder-only transformer and reverse-mode autodiff engine in pure Python.
Highlights
- Forward pass matches PyTorch to within 1e-9 (opt-in validation group)
- Character-level training on public-domain prose: 3.64 bits per character versus a 4.30 unigram baseline
- Checkpointing with exact save and restore
- A pre-registered research study with 8 seeds: copying is carried by first-layer heads looking back a fixed distance, not an induction circuit; one of three hypotheses is not supported, two are mixed. See docs/RESEARCH.md
Install: pip install the attached wheel.