smol-kimi-k3 TinyStories checkpoint

smol-kimi-k3 — TinyStories checkpoint

The first trained release of smol-kimi-k3, a 49.02M-parameter text-only
miniature of the Kimi K3 language backbone.

Training result

• 10,000 optimizer steps on TinyStories
• selected checkpoint: step 9,750
• best validation loss: 1.6277
• final measured throughput: 2,829 tokens/s
• peak allocated VRAM: 4.39 GiB
• trained locally on an RTX 4060 Laptop GPU with 8GB VRAM

Bundle

• model.safetensors — BF16 inference weights without optimizer state
• config.json — exact 49,019,770-parameter architecture
• tokenizer.json — 8,192-token TinyStories BPE tokenizer
• training_metadata.json — run configuration and final metrics
• sha256sums.txt — integrity checksums

Generate

gh release download v0.1.0-tinystories \
  --repo cneuralnetwork/smol-kimi-k3 \
  --dir pretrained/smol-kimi-k3

python3 generate.py pretrained/smol-kimi-k3/model.safetensors \
  --prompt "Once upon a time, a little fox"

Example from the exported model:

Once upon a time, a little fox was playing in the forest. He heard something
very special out of the forest.