Ava 0.1: her first trained brains

Ava's first public release: a brain-like language model in pure Rust, with leaky integrate-and-fire spiking neurons trained by surrogate gradients, k-winners-take-all sparse codes, and recurrent membrane memory instead of attention.

Model	Size	Trained on	Held-out bits per byte
Ava-S (release/ava-s)	d 192, 6 layers, 3.0M weights	48M tokens, 8 CPU threads, 2 h 06 min	1.872
Ava-M (release/ava-m)	d 384, 8 layers, 13.4M weights	24M tokens, one RTX 4070, 21 min	1.888

Highlights

• A dependency-free CPU stack (tokenizer, token shards, autodiff, threaded kernels, AdamW), written from the Rust standard library up.
• A CUDA trainer on candle with a fused LIF kernel and a hand-written backward: 3.2× faster than the unrolled graph, and a memory pool that stays flat over a run.
• The pre-registered v32 comparison of the spiking design against a transformer at three sizes.
• Model cards with exact commands, seeds, checkpoint hashes, unselected samples, and every training attempt disclosed.

She is a tiny research model. She will often produce nonsense; do not rely on anything she says. See the README for what she is, how to run her, and the roadmap.

Weights are also in the repository; the SHA-256 sums are in each release/*/SHA256SUMS.

Licence: PolyForm Noncommercial 1.0.0 (source-available).