Tracker round 10: TT-007 closed — steady-state decode after ANE prefill costs nothing
Every ANE decode figure here was a ~60-token window, because the pinned
qualification prompt hits end-of-turn and `--max-new` never bound; the window
contains the one-time handover cost, so those figures were upper bounds and the
steady-state cost had never been measured.
The prompt fix — the qualification body plus "continue, do not stop early" — makes
the cap bind. `benchmark/ane_steady_state_decode.py` then differences two generation
lengths so every one-time cost cancels. On AgentWorld 35B-A3B 4-bit, an 11,948-token
prompt, two runs per arm and length: ANE prefill is 3.32x faster and costs **+0.7%**
of steady-state decode (7.26 against 7.21 tok/s) — nothing, inside the spread — with
no penalty even in the 64-token windows.
Engineering Notes records the method, the numbers, the caveats (medians of 2; the GPU
arm's decode follows a 10-minute GPU prefill against ANE's 3-minute one; Ornith is not
installed, so AgentWorld-4 stands in; its row wires the cache, so the pageable case is
TT-005's question via the new `TINYTITAN_KEEP_WIRED=0`), and a new trap: the first ANE
prefill of a process pays a 2.1x compile warm-up, so a single cold-process figure
overstates the sidecar. Benchmarks carries the table.
Wiki audit: correct 12 pages against the code, and retire a command that cannot run
Every page was read against the 5.7 tree. Three systemic causes account for most of
what the audit found.
**Docs described tuning that is no longer shipped.** FAQ and Runtime-Controls quoted
the family fallback and pre-repair arms, while the defaults are the per-install
`ModelProfile.table`: the 35B rows are 10 GiB/160 slots at 4-bit and 12 GiB/96 at
8-bit with prefetch on, and Qwen3.8 is 12 GiB/96 at 4-bit and 9.5 GiB/40 at 8-bit
with prefetch off. Also corrected: Home's Swift floor (6.3 -> 6.4) and its 35B disk
range; Getting-Started's dense sizes, and its claim that a pipe sets up the browser
window (that prompt needs a terminal); Features' missing 8-bit Qwen3.8, its three
clients where there are five, and "binary thinking" where Qwen3.8 has four levels.
**Version labels had drifted.** Benchmarks' "v4.7" section is 5.0's work — there is
no v4.7 tag — and it claimed a v4.6 baseline for Qwen3.8, which did not exist then.
Its three percentages were withdrawn rather than re-guessed: their baselines cannot
be re-derived from anything in the repository.
**The LAN Manager's remote-access recipe was impossible.** The harness refuses
`--host 0.0.0.0`, and a specific address fails the webserver plugin's schema
(`expected "127.0.0.1" | "0.0.0.0"`), so the profile does not load. Verified by
trying both on a throwaway DSH_HOME. The page, the plugin README and the plugin's
own boot log all told operators to do it; all three now say what is true, and the
boot log no longer advertises addresses nothing is listening on.
Also: Cookbook's launcher keys, OpenAI-Compatible-Server's `-fast` listing and its
per-surface reasoning field, System-Design's ANE default, MTP numbers and interface
list, TinyTitan-Plugin's `--web` owner and its session-title overclaim, the LAN
fence's IPv6 ranges and its missing config keys, an unquoted receipt error, and a
duplicated sentence.
Verified after: links, anchors and the sidebar clean; every cited path still
resolves; both plugin suites pass; all six lint gates green.
The wiki is TinyTitan
Every page except the Changelog is renamed by the same rule the checkout used
(`NVMAI_` -> `TINYTITAN_`, `NVMAI` -> `TinyTitan`, `nvmai` -> `tinytitan`), and
the wiki's own `assets/wordmark.svg` is fixed the way the repository's was: the
name was split across two coloured spans, so it read `NVM` + `AI` and had to
become `Tiny` + `Titan`, with the canvas widened from 132 to 176 so the longer
name is not clipped.
The Changelog keeps its entries as they shipped -- they are the record of
releases made under the old name -- and gains one line at the top saying the
project was renamed on 2026-09-14, so the page is not a mystery. Its release
links still resolve: GitHub redirects a renamed repository.
Home now leads with the project's brand image
(`assets/tinytitan-hero.png`), the same artwork the README leads with, and the
tracker gains the closed row.
agent
committed
Sep 14, 2026
Add the "Capital of Paris" measurement page
The report moves here from the repository, where it was a markdown file rather
than a wiki page: one deliberately ambiguous prompt plus a plain control
question, every served model and both engines, three repeats each, with every
reply quoted verbatim. It records that one install thinks with the switch off,
what the runtime now does with that (C90), and what the repeats settled about
the earlier single-run numbers.
Linked from the sidebar under Reference and from Benchmarks.
Head every page with the NVMAI wordmark
The wiki carried no visual identity at all, so the README and the wiki did
not look like the same project. Every content page and the sidebar now open
with the same two-colour wordmark the README slogan uses -- NVM in cyan, AI
in orange.
One file serves both themes here. The slogan needs a light and a dark
variant because its body text is neutral, but the wordmark is entirely
brand colours and both carry enough contrast on #ffffff and #0d1117, which
was checked against each before committing.
Deliberately not applied to the hundreds of inline "NVMAI" mentions in
prose: GitHub cannot colour inline text, so each one would have to become
an image, which would break text selection, search and screen readers for
no gain.
Add the v4.7 benchmark section
Every shipped model re-measured at its shipped defaults on the same
workload as v4.1 and v4.6, now that the expert-cache budget and prefetch
are per family. Qwen3.8-Flash-Next 6.82 tok/s, Ornith 22.65 / 11.89, Qwen
3.6 23.28 / 12.72.
Records two things the numbers do not show on their own. The gains exceed
the interleaved A/B that motivated them, because interleaving keeps every
configuration page-cache-warm and that flatters the baseline more than the
tuned configuration; and the 4-bit 35B rows needed seven runs, because at
three Qwen 3.6 4-bit returned two transient fast readings that would have
published a 13% gain for a configuration that had not changed.
Document Qwen3.8-Flash-Next, per-family tuning, and the new model ids
The wiki was last updated before three shipped changes, so it described a
runtime that no longer exists.
- Qwen3.8-Flash-Next 125B-A6B appeared only in the changelog. It is now in
Home, Features and System Design, including the install size, which is
about 161 GB and dominated by a 95 GB hashed n-gram table, and the point
that it runs on 24 GiB because the experts are streamed.
- NVMAI was described throughout as a "Qwen3.5-MoE 35B-A3B" runtime. There
are three families now, and Qwen3.8-Flash-Next does not share the tensor
contract the other two do.
- Model ids gained the routed-expert width (`_4-Bit`, `_8-Bit`). Every
example in the server guide and FAQ still showed the bare id, which the
server no longer accepts.
- ANE prefill was documented as experimental and opt-in; it is the default,
and falls back to the GPU when a model has no sidecar.
- The expert-cache budget was documented as a flat 8 GiB deriving 128 or 64
slots. It is per family now, with a clamp to half of physical memory, and
speculative expert prefetch was not documented at all.
The dated v4.1 and v4.6 benchmark sections are left as they are: they record
what was measured at the time, and their 8 GiB budget note was accurate for
those runs. Only the claim that ANE "stays off by default", which a reader
would take as current, is corrected there. Current numbers follow in a
separate commit once the re-measurement finishes.
Audit the wiki: current v4.6 state, remove engineering history records
- Move the v4.2/v4.3 experiment qualification record out of the wiki
(now docs/v4.2-experiments.md in the source repository) and drop its
navigation entries; the wiki keeps user-facing documentation only.
- Trim Benchmarks to current, decision-relevant results: v4.6, its v4.1
baseline, the MTP and coding/tooling qualifications behind the shipped
defaults, and fast-alias latency. Remove superseded 6-bit-era tables,
the stale prompt-length record, and the historical M5 reference.
- Bring Features and System Design to v4.6 (tiled Top-K sampling, opt-in
ANE prefill) and remove release-tag phrasing from runtime descriptions.
- Remove the empty Unreleased changelog section, the legacy 6-bit row in
Getting Started, and a duplicated server snippet.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Document v4.6: sampler rewrite, ANE prefill, benchmarks, runtime controls
Document NVMAI 4.1 release
agent
committed
Aug 21, 2026
Document NVMAI 4.0 defaults and qualification
Record concise qualification commands
Document Ornith concise tool qualification
Document Ornith native MTP support and benchmarks
Add Ornith prompt-length benchmarks
docs: refocus wiki on core user workflows
docs: update launchers and release history through 3.9
launchers: rename to server_launcher.sh + cli_launcher.sh (two launchers, same five questions)
docs: add 'physical limit' clarification at the top of FAQ, Home, Performance, and Benchmarks — decode is SSD-configuration-bound, not chip-bound
3.3: marketing lead — headline speed-up, Paris table first
3.3: state fast-alias time savings as x-factors (10x/37x/65x, 90-98%)
3.3: <model>-fast alias strips coding-CLI boilerplate (docs + CLI matrix)
docs: fix dangling links after experiment-page removal
Refresh for Qwen-only NVMAI: fix binary names, defaults, and broken links
- Getting Started and the server guide: rename the old fork's binary names
(TurboFieldfareRepack/CLI/Server/Mac) to NVMAIRepack/NVMAICLI/NVMAIServer/
NVMAIMac and the defaults suite to NVMAI; pgrep patterns updated
- Runtime Controls: expert-cache default 32 (64 accepted), prefill chunk
default 4096, new 4-bit chunk-sizing measurement added alongside the older
8-bit data point
- Qwen performance notes: profile updated to 32 slots; slot cap note updated
- System Design: Qwen-only banner, broken Sources/TurboFieldfare links fixed
to sources/NVMAI, trusted-install load default documented
- Optimization Journey: 2026-08 Qwen acceleration measurements (chunk sizing,
trusted-install startup, slot neutrality, rejected prefetch/mmap)
Migrate project documentation into structured Wiki