Ava 0.4: a chat app that learns while you talk

Ava 0.4

Ava is a small language model built like a brain. She has spiking neurons, a fast memory that she writes and reads
as she goes, and a library she looks things up in. She is written in pure Rust and runs on an ordinary Windows PC.

The brain in this release is Ava 0.4.

What's new since 0.3

A chat app you can download. Ava now comes with a free Windows app. Double-click it and a chat page opens in
your browser. Behind the chat, a 3D brain lights up from her real activity, word by word. Everything runs on your
own computer. Nothing you type leaves it, and it works without internet or a graphics card.

She looks things up before she answers. She has a library of about 297,000 short passages, mostly from the
Simple English Wikipedia. Before each answer she finds the two best passages and reads them. The app shows you
what she looked up.

She can say "I don't know." She was taught to answer from what she reads, to greet you back, and to say
"I don't know" when she is not sure. Part of this she learned from her own mistakes. No rule in the code pastes in
that answer.

She keeps learning while you chat. A small part that works like the cerebellum adjusts her next-word guesses
from what she reads and says. Each chat keeps its own changes, and they slowly fade. Her saved brain never changes.
Delete a chat and she forgets what she learned in it.

A new brain inside. Ava 0.4's blocks have 14 new options, several of them brain-like. Nine of them won small
tests, alone or in pairs. The other five were too slow to test alone, so all 14 were tested together, and together
they beat the old block. For example, her neurons send small signed spikes (from -2 to +2) and their firing
thresholds adapt, and a quarter of the units in her wide layer may be active at once (one in eight before).

She read about half a billion word pieces. 387 million came from a new mix: about half of it is educational web
pages, and the rest is science sentences, the Simple English Wikipedia, speech and dialogue, books and reasoning
lessons. Then she read 108 million more from Ava 0.3's mix, with whole books and number practice.

She is as wide and as deep as Ava 0.3, but has more weights. Her size: d = 384, 8 layers, 6 memory heads of 64; 26,840,893 weights. Ava 0.3's: d = 384, 8 layers,
12 memory heads of 32; 19,962,048 weights.

New in v52. On top of Ava 0.4's 14 block options, she has the brain parts v52 kept (each tested alone first, and kept by a rule written down before the runs): short-term plasticity: each spiking neuron's output tires and recovers with use (depression and facilitation); neuromodulators: a small nucleus reads the first block and sends three signals to the later blocks, like noradrenaline (input gain), acetylcholine (memory write strength) and dopamine (output scale); dendrites: each unit of the wide layer is scaled by a gain from its block's memory read, like the top dendrites of cortical neurons; letters: every word piece also carries a small vector made from its own letters and their places, so she can see spelling; dopamine: from the second block on, the wide layer's output is scaled by how surprised she was by the last word; mood: four slow units follow how surprised she has been lately and scale the input of every later block's neurons; rhythms: every block's memory writes and reads are scaled, per head, by slow and fast rhythms, like theta and gamma waves; a thalamus: a small relay of 16 units reads every block's output and sets graded gates on what each block passes on; a cerebellum: 12,288 granule units make a sparse code of her state and the current word, and a learned read-out corrects her next-word guess; it is the part that keeps learning in the chat app; the homeostasis nudge: the rule that moves each neuron's threshold toward its target firing rate stays on while she grows and is taught. Her training also changed: the spike reset now passes its gradient back (--lif-reset-grad), the fix for the training spikes found in v52.

Fast on a normal processor. She talks at about 425 word pieces a second on a CPU. She uses 8-bit weights
for this. The first build of Ava 0.4 managed 60 to 80.

Steadier training. We found what made the first Ava 0.4's training blow up: a rule that kept pushing some
neurons' firing thresholds higher. With that rule switched off while Ava 0.4b grew and was taught, the blow-ups
stopped.

How she scores

	Ava 0.3	Ava 0.4
Reading, bits per byte (lower is better)	1.511	1.513
Grammar, BLiMP	74.1%	75.9%
Science multiple choice (dev set)	46.8%	41.7%
Simple questions: right / "I don't know" / wrong, of 100	–	54 / 31 / 15
Fresh sealed test, run once	–	not run
Talking speed in the app (CPU)	–	425 tokens/s

Good to know

• She is small. Her answers are often wrong or odd, so do not rely on them.
• On science multiple-choice questions she scores 41.7%, against 46.8% for Ava 0.3.
• She speaks English only. The app is for Windows 10 and 11 (64-bit).

Download and start

1. Download Ava-0.4-windows.zip. It holds the app and her brain.
2. Unzip the folder.
3. Double-click Ava.exe. Your browser opens the chat page.

Windows may warn that the publisher is unknown, because the app is not signed. Choose "More info", then "Run anyway".
She needs about 1 GB of free memory. No graphics card and no internet are needed.

Only want the weights? ava-0.4.safetensors is her brain on its own. It is the same file as model.safetensors in
the zip. SHA256SUMS.txt lists the checksums of every file.

What comes next

Ava's development is coming to an end for now. She needs more GPUs than I have: my single RTX 4070 cannot handle
the scale she needs in order to grow. If you want to use Ava, or would like me to keep working on her and you have
GPU time available, I would be happy to continue her development. Open an issue on the repository to get in touch.

Licence and credits

• Ava (the app and her brain): PolyForm Noncommercial License 1.0.0, non-commercial use only.
• The 3D brain in the app: BodyParts3D, © The Database Center for Life Science (DBCLS), CC BY 4.0.
• Her library: text from the Simple English Wikipedia by Wikipedia's contributors, CC BY-SA 4.0, changed (cut into
  short passages). library.tsv is shared under CC BY-SA 4.0 as well.

Ava 0.3: numbers, long books and sleep

Ava is a brain-like language model in pure Rust: spiking neurons, fast synapses that rewrite themselves as she reads, and sparse codes. Her working memory of a text is a fixed 408 KB, however long the text is. She is a tiny research model, trained on one home GPU.

What's new

Ava 0.3: one model that reads, counts, reads whole books, answers questions and sleeps. 20.0M weights, grown from v39's number-reading Ava on 108M more tokens (books, number practice, answers, recall practice), then put to sleep and taught by reward. Each test was pre-registered and run once on fresh data:

• Sleep: 23.8% → 96.2%. She reads facts about 150 new people, sleeps on them for 25 seconds, and answers questions about them (chance 25%).
• Synaptic consolidation. Before she sleeps, each weight's importance to her reading is measured, and important weights are held in place. Sleep then cost her reading +0.016 bits per byte instead of +0.046.
• Numbers: +18 points (60.5% vs 42.4%) on a fresh number test. She sees how long a number is before reading its digits; comparing numbers of different lengths went from 46.5% to 90.5%.
• Long books: her memory helps across 16,384 tokens of unseen books (0.033 bits per byte better than a memory cut every 256 tokens).
• Questions: 40.7% vs 33.0% on 1,200 held-out science questions (SciQ 56.5%).

What failed, reported as such:

• An attention read added so she could look things up went unused.
• Reward learning added nothing measurable.
• Her reading is 1.511 bits per byte, just past the pre-registered limit of 1.497 and a little below v0.2's Ava (1.487).

Downloads

File	Model	Save as
ava-0.3.safetensors	Ava 0.3	release/ava-0.3/model.safetensors

Check it with sha256sum -c SHA256SUMS in release/ava-0.3. Run her with --glance (see the model card, which also shows how to teach her your own facts in her sleep). The v0.2 models are still on the v0.2.0 release page.

Every number comes from a test written down before it ran, and every earlier look at a test set is disclosed.

Licence: PolyForm Noncommercial 1.0.0. Free for research, study, teaching and other non-commercial use; commercial use needs a separate licence.

Ava 0.2: a stronger brain that can answer questions

Ava is a brain-like language model in pure Rust: spiking neurons, fast synapses that rewrite themselves as she reads, and sparse codes. She has no attention, and her whole memory of a text is a fixed 408 KB, however long the text is. She is a tiny research model, trained on one home GPU.

What's new

Ava, a much stronger brain. 19.4M weights, trained on 387M tokens in 44 minutes on one RTX 4070.

• 1.487 bits per byte on held-out text (v0.1: 1.872; lower is better)
• 76.1% on BLiMP grammar pairs (chance 50%)

Ava-QA, taught to answer questions. 95 seconds of teaching on top of Ava. While she learned to answer, she also re-read her ordinary text, as the brain replays old memories while it learns.

• SciQ science questions: 41.1% → 56.4% right (66.2% with the passage shown; chance 25%)
• She may say "I don't know"; then 64% of the answers she gives are right
• Replay cut her forgetting of plain reading by about three quarters

A faster trainer. Hand-written training steps and fused CUDA kernels, bf16 matrix multiplies, and a chunk form of her memory that trains long texts 1.8× faster.

Three pre-registered tests against a transformer (v34–v36). Her design learns more per weight at small sizes, but a transformer gains more from size: at 17.8M weights she is 0.038 BPB behind its fitted line. Making her curve steeper is the main open problem.

Downloads

File	Model	Save as
ava.safetensors	Ava	release/ava/model.safetensors
ava-qa.safetensors	Ava-QA	release/ava-qa/model.safetensors

Check them with sha256sum -c SHA256SUMS in each folder. The model cards (Ava, Ava-QA) have the exact commands, seeds, checksums and unselected samples. The README's Quick start shows how to build, run, ask and train.

Every number comes from a test written down before it ran, and every earlier look at a test set is disclosed.

Licence: PolyForm Noncommercial 1.0.0. Free for research, study, teaching and other non-commercial use; commercial use needs a separate licence.

Ava 0.1: her first trained brains

Ava's first public release: a brain-like language model in pure Rust, with leaky integrate-and-fire spiking neurons trained by surrogate gradients, k-winners-take-all sparse codes, and recurrent membrane memory instead of attention.

Model	Size	Trained on	Held-out bits per byte
Ava-S (release/ava-s)	d 192, 6 layers, 3.0M weights	48M tokens, 8 CPU threads, 2 h 06 min	1.872
Ava-M (release/ava-m)	d 384, 8 layers, 13.4M weights	24M tokens, one RTX 4070, 21 min	1.888

Highlights

• A dependency-free CPU stack (tokenizer, token shards, autodiff, threaded kernels, AdamW), written from the Rust standard library up.
• A CUDA trainer on candle with a fused LIF kernel and a hand-written backward: 3.2× faster than the unrolled graph, and a memory pool that stays flat over a run.
• The pre-registered v32 comparison of the spiking design against a transformer at three sizes.
• Model cards with exact commands, seeds, checkpoint hashes, unselected samples, and every training attempt disclosed.

She is a tiny research model. She will often produce nonsense; do not rely on anything she says. See the README for what she is, how to run her, and the roadmap.

Weights are also in the repository; the SHA-256 sums are in each release/*/SHA256SUMS.

Licence: PolyForm Noncommercial 1.0.0 (source-available).