Releases: Den-Sec/glublm
Releases · Den-Sec/glublm
Release list
GlubLM v0.3.0 - 35M architecture upgrade
GlubLM v0.3.0
Architecture upgrade: 18M -> 35M parameters, 48 -> 96 token context.
Architecture
- 36.1M params (d_model=640, n_heads=10, ffn_hidden=1280)
- 96-token context window (was 48)
- Same RoPE + SwiGLU + RMSNorm stack
- ONNX uint8: ~40MB (browser-friendly)
Results
- Final loss: 1.19 (was 1.70)
- Significantly cleaner grammar, better comprehension
- Same goldfish personality (persona is in the data, not the params)
What improved
- Less grammatical garbling (~5% vs ~15%)
- Better understanding of user messages
- More coherent responses
- Longer generation window without truncation
Links
GlubLM v0.2.0 - companion quality upgrade
GlubLM v0.2.0
Companion quality upgrade. Trained on 75K dataset (60K original + 15K targeted supplement).
What changed
- Identity fixed: "who are you" now correctly says "a goldfish! in a bowl!"
- Memory trait active: forgetting/confusion references appear naturally in responses
- No more empty responses: min_new_tokens prevents premature EOS
- Playful energy: model is now silly and excited, not just poetic
- Body awareness: mentions fins, tail, scales, gills
- Bittersweet goodbyes: "the water will know you were here"
- Asks questions back: "do you have flakes where you live?"
Numbers
- 75,805 training samples (68K train / 7.5K test)
- 15K new samples across 12 targeted categories
- 78 tests green, ruff clean
- Final loss: 1.74 (15 epochs)
Links
GlubLM v0.1.0 - the goldfish remembers nothing
GlubLM v0.1.0
First release of GlubLM - an 18M-parameter transformer that plays a goldfish with a 10-second memory.
What's in the box
- 18.4M-parameter model with RoPE + SwiGLU + RMSNorm
- 60K-sample LLM-generated training dataset (multi-agent Claude team)
- Hard 48-token context window (physical 10-second memory)
- Browser demo via ONNX Runtime Web (~21 MB)
- pip install glublm
- HuggingFace Hub: model + dataset + Space
Results
- Test perplexity: 12.14
- Forward passes/sec: 94.2 (RTX 3060, batch 1, seq 48)
- 77 tests green, ruff clean
Inspired by
- GuppyLM by Arman BD
- Ted Lasso - "be a goldfish"