M1-128M — native 128M hybrid, trained from scratch
What it is: 128.6M-parameter hybrid — window attention + liquid core (selective decay, exp parameterization) in every layer. 12 layers, d_model 832, 512-token training window. English text continuation — a research backbone, not a chat model.
Serve it:
\\�ash
CKPT_PATH=hybrid_125m_serve.pt TOKENIZER=gpt2 python -m uvicorn serve.server:app
\\
Measured (see RESULTS.md / ABLATIONS.md):
- O(1) carried state: 0.381 MB flat from 512 to 1M tokens (8,063× smaller than a matched KV-cache)
- Selective-decay exp parameterization: parity length extrapolation 0.953–1.000 vs tanh 0.023
- Language-modeling quality is NOT the selling point (honest: PPL trails a matched Transformer at 125M — the case is memory form-factor and efficiency)
License: MIT.