Skip to content

HIM-native leg-1 — 280K params (dev checkpoint)

Pre-release
Pre-release

Choose a tag to compare

@FreddyCreates FreddyCreates released this 10 Sep 20:08
· 3 commits to main since this release

HIM-native leg-1: 146,576 → 279,824 params (1.91x Net2Net widen, hidden 128→256).

  • 800 steps on 2.26M chars of real text (Tiny Shakespeare, Alice, Sherlock, Frankenstein)
  • Loss 5.5270 → 2.4063; preservation max|diff| = 8.3e-07
  • Context-MLP causal LM (not a transformer); real NumPy manual-backprop/Adam trainer
  • The v0 checkpoint was overfit on 4,109 tokens (train 0.08 / eval 3.54); this leg trains on real text to correct that

Real training on real text. Small model — not a capability claim.

Downloads:
https://github.com/ItsNotAILABS/pocket/raw/refs/tags/him-native-leg-1/checkpoints/him-native-v0/leg-1/weights.npz
https://github.com/ItsNotAILABS/pocket/raw/refs/tags/him-native-leg-1/checkpoints/him-native-v0/leg-1/receipt.json
https://github.com/ItsNotAILABS/pocket/raw/refs/tags/him-native-leg-1/checkpoints/him-native-v0/leg-1/config.json