GlubLM v0.1.0 - the goldfish remembers nothing
GlubLM v0.1.0
First release of GlubLM - an 18M-parameter transformer that plays a goldfish with a 10-second memory.
What's in the box
- 18.4M-parameter model with RoPE + SwiGLU + RMSNorm
- 60K-sample LLM-generated training dataset (multi-agent Claude team)
- Hard 48-token context window (physical 10-second memory)
- Browser demo via ONNX Runtime Web (~21 MB)
- pip install glublm
- HuggingFace Hub: model + dataset + Space
Results
- Test perplexity: 12.14
- Forward passes/sec: 94.2 (RTX 3060, batch 1, seq 48)
- 77 tests green, ruff clean
Inspired by
- GuppyLM by Arman BD
- Ted Lasso - "be a goldfish"