Repository navigation
Releases
Releases · adeoluo/transformer-from-scratch
v0.2.1
v0.2.0
[0.2.0] 2026-11-02
Performance
- Numpy autodiff engine with the same hand-written gradients (#3)
Documentation
- Standard README structure and language framing (#12)
- Horizontal layered architecture diagram
- Compact horizontal architecture diagram
- Remove authorship statements from docs and scripts
- Recorded demos of more commands, with their vhs tapes (#1)
- Recorded demos of the HTTP service and more commands (#2)
Build and CI
- Run CodeQL on main and on a weekly schedule only
Maintenance
- Add release automation, AGENTS.md, llms.txt and contributor docs
- Classify squash merges in the changelog by the commits they contain
v0.1.0
First release of a decoder-only transformer and reverse-mode autodiff engine in pure Python.
Highlights
- Forward pass matches PyTorch to within 1e-9 (opt-in validation group)
- Character-level training on public-domain prose: 3.64 bits per character versus a 4.30 unigram baseline
- Checkpointing with exact save and restore
- A pre-registered research study with 8 seeds: copying is carried by first-layer heads looking back a fixed distance, not an induction circuit; one of three hypotheses is not supported, two are mixed. See docs/RESEARCH.md
Install: pip install the attached wheel.