Repository navigation
Releases
ngram-learn RC3
ngram-learn RC3
RC2 plus: learning that requests do not notice, and facts about the user that stay the user's. Same branch
(axonzeta/TensorFold ngram-learn, tag ngram-learn-rc3); start at docs/recipe.md.
What changed since RC2
- Learning is invisible to requests. A grad op's start is now a step per lesson (each lesson's capture prefill),
andserve.shserves two streams withLEARN=1, so a request runs beside learnd's background chats instead of
behind them. Under chat traffic every ~10 s while learning: time to first token p50 82 ms, p99 0.25 s, no request
past 0.5 s, the same as without learning (RC2: p99 1.24 s, 5% of requests past 0.5 s). - Facts about the user stay the user's. They train only rows whose n-gram holds "my"/"I" (so "your dog's name"
is untouched), from 4–6 key phrases the model lists (the plain question forms included), with no training question
that holds the answer. They are committed when every training question comes back with nothing damaged; held-out
questions written freely may hold none of the keys and are reported. Through the server, from "A few things about
me: my dog is called Biscuit, I live in Llandudno, and my favourite colour is teal.": all three facts learn; "Where
do I live?" → "You live in Llandudno."; five of nine new phrasings answer; "What is my cat called?" and the other
controls are unchanged.
Results (unchanged from RC2 unless noted)
| Recall | Held-out | Neighbours unchanged | s / fact | |
|---|---|---|---|---|
| 20 chat facts | 0.87 | 0.78 | 0.95 | 167 (RC1 build) |
| 17 named facts | 0.98 | 0.91 | 0.96 | — |
| 2 facts, current build | 1.00 | 1.00 | 1.00 | 71–77 |
| 3 facts about the user, through the server (new) | 1.00 | 0.67 | 1.00 | ~190 |
1,000 sequential edits keep earlier facts at 0.995; MMLU 86.0% with and without the 1,000-edit overlay
(experiments/07-scale).
Gates: zig build test; tools/batch_check.py (BATCH-OK); tools/rc_gate.py (GATE-OK); the recipe from a clean
clone.
Known limits
- A fact is found only by questions holding one of its keys ("Which colour do I like most?" misses a fact keyed on
"my favourite colour"). - Two named facts on one key compete.
- Under traffic heavier than a request every ~10 s, learning slows (it runs in the gaps).
- One rank; overlays live in the server's memory and learnd pushes them again after a restart.
ngram-learn RC2
ngram-learn RC2
RC1 plus: facts about the user, twice the speed, learning while serving without delaying requests, and the scale
evaluation. Same branch (axonzeta/TensorFold ngram-learn, tag ngram-learn-rc2); start at
docs/recipe.md.
What changed since RC1
- Facts about the user learn ("My dog is called Biscuit.", "I live in Llandudno.", "My favourite colour is
teal."). In RC1 none did. learnd now gives a fact in the first person its own lesson: the model lists the phrases
people ask about it by ("my dog", "my dog's name", "my puppy"), the training questions use them, every phrase is a
key, and the rows train without the frequency penalty (a personal tag's overlay reaches only that tag's
requests). Through the server: "What did I name my puppy?" → "You named your puppy Biscuit."
(experiments/06-user-facts). - A fact in ~75 s (RC1: 124–187 s). The delta rule's backward no longer sums across columns with atomics at every
row (70% of a training pass); the pass is 1.6 s instead of 4.1 s. Greedy checks run only once the loss is low
(experiments/08-speed). - Learning while serving. The training pass is resumable layer by layer: the server advances it one layer an
idle moment, so a request waits for a layer (~35 ms), not a pass. learnd's own chats go at background priority.
Under chat traffic every ~10 s while learning: time to first token p50 88 ms (79 ms without learning), p90 0.28 s
(RC1-style back-to-back learning: 0.70 s and 1.63 s). - Scale (experiments/07-scale): 1,000 sequential CounterFact edits in the engine keep earlier facts at 0.995
with the control facts unchanged (51,005 rows); MMLU 86.0% with and without the 1,000-edit overlay (14 of 570
answers change, 5 to right and 5 to wrong). - Fixes: the probe applied an overlay loaded at start to no scoring op (
TF_NGRAM_OVERLAYwithscoreand
worker); learnd chose keys from held-out questions and could pick the answer as a key;metadatareaches the
engine (RC1). - Tried and dropped, with the evidence: stored targets with a joint solve (EngramEdit-style; experiments/05-v2:
v1 keeps earlier facts better), packed-int4 expert products (08-speed: no faster once served capture removed the
training forward).
Results
| Recall | Held-out | Neighbours unchanged | s / fact | |
|---|---|---|---|---|
| 20 chat facts, RC1 | 0.87 | 0.78 | 0.95 | 167 |
| 17 named facts, RC1 | 0.98 | 0.91 | 0.96 | — |
| 3 user facts, multi-key (chat_edit) | 0.89 | 0.83 | 1.00 | — |
| 2 facts, RC2 speed | 1.00 | 1.00 | 1.00 | 71–77 |
Gates: zig build test; tools/batch_check.py (BATCH-OK); tools/rc_gate.py against the server (GATE-OK: tags
change only tagged replies, drafted = plain with and without a tag); train_op_check.sh (every training kernel
against fp64 PyTorch); the recipe from a clean clone.
Known limits
- A fact about the user is found only by questions that contain one of its key phrases ("Where do I live?" misses
a fact keyed on "my town" and "where I live"). - Two facts on one key compete (the walkthrough's second Kestrel fact).
- About 5% of requests still wait up to ~1.3 s while learning runs (the start of a grad op, a greedy check, or a
background chat on the one stream). - One rank; overlays live in the server's memory and learnd pushes them again after a restart.
ngram-learn RC1
ngram-learn RC1
Facts learned into Qwen3.8 Flash Next's n-gram memory while it serves, on one DGX Spark: the model is taught a fact
in about three minutes, with no second copy of its weights and no dense weight change, and requests read it back
when they ask for its tag.
Branch: axonzeta/TensorFold ngram-learn (tag ngram-learn-rc1), on TensorFold zig-flashnext db28187
with MiaAI-Lab's Flash Next CUDA patches. Start here: docs/recipe.md, then
docs/walkthrough.md.
What's in it
- An overlay of n-gram table rows in the engine's n-gram gather, per sequence: a fact's change is ~25–55 rows of
160 values, removable exactly. - A training pass inside the CUDA engine on the served prefill's own activations and MoE routes, back through
every layer to the n-gram rows (fp32 gradients; every kernel checked against fp64 PyTorch); several questions in
one pass. - Learning in the server (
LEARN=1): learn ops queued onPOST /v1/ngram/opand run between requests;
overlays by tag, which a request picks with"metadata": {"tag": "..."}. - learnd, the learner beside the server: facts from text, model-written lessons, training, checks through the
server's chat (recall, held-out questions, twins and keep prompts unchanged), rounds, commit, per-fact delete;
/v1/slide/learnas Living Weights streams it.
Results (experiments/04-chat-facts)
20 fictional facts taught as chat turns, all on one Spark, the RC build:
| Recall | Held-out phrasing | Neighbours unchanged | s / fact | |
|---|---|---|---|---|
| all 20 facts | 0.87 | 0.78 | 0.95 | 167 |
| the 17 about named things (projects, people, organisations, products, places, identifiers) | 0.98 | 0.91 | 0.96 | — |
Learned one after another into one overlay, all 20 still score 0.85 / 0.70 / 0.95 together at the end (named
things 0.96 / 0.82 / 0.96).
Through the server, tagged requests change and untagged ones do not; drafted replies equal undrafted ones, with and
without a tag (tools/rc_gate.py). zig build test passes.
Known limits
- Facts about the user keyed on common words ("my dog", "I live", "my favourite colour") do not generalise:
their rows are shared with every sentence about any dog. Named things work. - One-token keys ("Kestrel", "Zorn") leak into other contexts with that token, and two facts on the same key
compete (the walkthrough's second Kestrel fact). - One rank, and a learn op holds the engine for one op at a time (~5 s): a request arriving mid-op waits for it.
- Overlays live in the server's memory; learnd keeps them on disk and pushes them again at its start or on
POST /v1/slide/sync.
Fixed on the way
- Random GPU faults (Xid 31) while training: training uploads raced the non-blocking training stream (a pageable
cuMemcpyHtoDreturns before its DMA lands). Uploads are now stream-ordered, and upstream's2f8284eis
backported. ~1,500 passes on three Sparks after the fix without a fault. metadatareached the engine only after the chat route's field whitelist learned it.