Repository navigation
HIM-native leg-1 — 280K params (dev checkpoint)
Pre-release
Pre-release
·
3 commits
to main
since this release
HIM-native leg-1: 146,576 → 279,824 params (1.91x Net2Net widen, hidden 128→256).
- 800 steps on 2.26M chars of real text (Tiny Shakespeare, Alice, Sherlock, Frankenstein)
- Loss 5.5270 → 2.4063; preservation max|diff| = 8.3e-07
- Context-MLP causal LM (not a transformer); real NumPy manual-backprop/Adam trainer
- The v0 checkpoint was overfit on 4,109 tokens (train 0.08 / eval 3.54); this leg trains on real text to correct that
Real training on real text. Small model — not a capability claim.
Downloads:
https://github.com/ItsNotAILABS/pocket/raw/refs/tags/him-native-leg-1/checkpoints/him-native-v0/leg-1/weights.npz
https://github.com/ItsNotAILABS/pocket/raw/refs/tags/him-native-leg-1/checkpoints/him-native-v0/leg-1/receipt.json
https://github.com/ItsNotAILABS/pocket/raw/refs/tags/him-native-leg-1/checkpoints/him-native-v0/leg-1/config.json