Wiki audit: correct 12 pages against the code, and retire a command that cannot run
Every page was read against the 5.7 tree. Three systemic causes account for most of
what the audit found.
**Docs described tuning that is no longer shipped.** FAQ and Runtime-Controls quoted
the family fallback and pre-repair arms, while the defaults are the per-install
`ModelProfile.table`: the 35B rows are 10 GiB/160 slots at 4-bit and 12 GiB/96 at
8-bit with prefetch on, and Qwen3.8 is 12 GiB/96 at 4-bit and 9.5 GiB/40 at 8-bit
with prefetch off. Also corrected: Home's Swift floor (6.3 -> 6.4) and its 35B disk
range; Getting-Started's dense sizes, and its claim that a pipe sets up the browser
window (that prompt needs a terminal); Features' missing 8-bit Qwen3.8, its three
clients where there are five, and "binary thinking" where Qwen3.8 has four levels.
**Version labels had drifted.** Benchmarks' "v4.7" section is 5.0's work — there is
no v4.7 tag — and it claimed a v4.6 baseline for Qwen3.8, which did not exist then.
Its three percentages were withdrawn rather than re-guessed: their baselines cannot
be re-derived from anything in the repository.
**The LAN Manager's remote-access recipe was impossible.** The harness refuses
`--host 0.0.0.0`, and a specific address fails the webserver plugin's schema
(`expected "127.0.0.1" | "0.0.0.0"`), so the profile does not load. Verified by
trying both on a throwaway DSH_HOME. The page, the plugin README and the plugin's
own boot log all told operators to do it; all three now say what is true, and the
boot log no longer advertises addresses nothing is listening on.
Also: Cookbook's launcher keys, OpenAI-Compatible-Server's `-fast` listing and its
per-surface reasoning field, System-Design's ANE default, MTP numbers and interface
list, TinyTitan-Plugin's `--web` owner and its session-title overclaim, the LAN
fence's IPv6 ranges and its missing config keys, an unquoted receipt error, and a
duplicated sentence.
Verified after: links, anchors and the sidebar clean; every cited path still
resolves; both plugin suites pass; all six lint gates green.
Remove the Mac app: the product is the engine and its server
The app, the out-of-process decode service it drove, and their library and test
targets are gone from the repository. The wiki followed them: the Interfaces
rows, the app preference block, the app-only control rows and the iOS roadmap
item all described a front end TinyTitan no longer has and will not carry. A
second front end is a second surface to build, keep in step with every engine
feature, and support, so the Roadmap now says a GUI or mobile front end is out
of scope rather than deferred, and the tracker's issue-#5 items are closed
instead of left open for a future session to pick up.
Where a page told the reader to do something in the app it now gives the real
equivalent: the launcher's --model keys and --bits, the served id from
/v1/models, and the OpenAI (http://127.0.0.1:8080/v1) and Anthropic Messages
(http://127.0.0.1:8080) endpoints. Every process guard is now exactly
'TinyTitanServer|TinyTitanCLI|TinyTitanPackageTests|swiftpm-testing-helper|mlx_lm|mlx-lm'.
Getting Started also gained the one-command install it never mentioned
(tools/install_tinytitan.sh, which leaves ~/.local/bin/tinytitan), since the
installer and the server launcher are now the whole product surface.
Changelog.md is untouched: it is the chronological record of what shipped, and
entries describing the app were true when they were written.
Compaction endpoint: landed, and in the supported API table
/v1/responses/compact returns the compacted input window the spec describes —
the caller's instructions verbatim, then one `compaction` item carrying the note —
and a client sends that back as the base `input` of its next response instead of
the whole transcript, which is what keeps a long session affordable to continue.
The note is produced with thinking off, then verified with the server's own
tokenizer: over budget it is compressed by a second pass rather than truncated,
and lines echoed from the instruction or a repetition loop are recognised as
failed passes. `docs/server-api.md` has the contract, including what
`encrypted_content` actually holds. Verified on the 4B and 9B at 4-bit: an
eight-turn session compacted in 16.5 s and 39.7 s, both notes keeping every
load-bearing fact. The acceptance suite has not been re-run against it yet; those
two tests are what it was written for.
Server page: the plugin reads models/ and follows it
Say what the plugin actually does now, because it is what a person relies on
when the picker is empty or stale: it lists only the installs really present,
one row per width; it watches models/ while the harness runs, so an install or a
deletion reaches the picker without a restart; and it reads the folder directly
when no server has been built yet.
Server page: the generated route labels carry the model width
The picker renders the route's `name` field and nothing else, and a catalog
names both widths of one model identically -- "Qwen 3.5 2B" is neither the
4-bit nor the 8-bit install specifically. Show the field in the reference block,
with the reason: `Qwen 3.8 Flash Next 125B-A6B (4-bit)` is what tells the two
apart in the picker.
Docs: the expert-cache recommendation is 30% of physical memory
Cookbook and server page: the launcher asks for the port (default 8080)
Server page: switching is what --models-dir does, and /v1/models does not list -fast
The page listed 'model switching' as something the server does not provide, four
sections above the line that documents --models-dir serving every installed model
by name. It also described GET /v1/models as listing the base model and its -fast
alias, which is not what it lists: the served ids, plus <id>@cpu where a second
engine is a real choice.
Both were verified live on the dense installs, along with the per-request thinking
level in both directions on all three surfaces — the account is in the tracker.
The wiki is TinyTitan
Every page except the Changelog is renamed by the same rule the checkout used
(`NVMAI_` -> `TINYTITAN_`, `NVMAI` -> `TinyTitan`, `nvmai` -> `tinytitan`), and
the wiki's own `assets/wordmark.svg` is fixed the way the repository's was: the
name was split across two coloured spans, so it read `NVM` + `AI` and had to
become `Tiny` + `Titan`, with the canvas widened from 132 to 176 so the longer
name is not clipped.
The Changelog keeps its entries as they shipped -- they are the record of
releases made under the old name -- and gains one line at the top saying the
project was renamed on 2026-09-14, so the page is not a mystery. Its release
links still resolve: GitHub redirects a renamed repository.
Home now leads with the project's brand image
(`assets/tinytitan-hero.png`), the same artwork the README leads with, and the
tracker gains the closed row.
agent
committed
Sep 14, 2026
Structured output is enforced, not refused
The server page said the server does not provide structured-output
enforcement, which was true when it was written. It now compiles
`response_format` / `text.format` / `output_config.format` into a byte-level
grammar that masks the sampler, so the sentence becomes a feature description
with the two caveats the guarantee needs: the schema subset is explicit and
everything outside it is refused by name, and thinking is off for a constrained
request because the grammar constrains every token.
The tracker gains the closed row with the mechanism, the three defects the
testing found (the copy helpers dropping the schema, whitespace being legal
forever, and the two decode paths that cannot honour a grammar) and the
verification.
agent
committed
Sep 14, 2026
The Messages API's thinking is per request, and the endpoints are listed
Two wiki pages carried the load-time premise the server no longer implements.
`OpenAI-Compatible-Server` never listed the Anthropic endpoints: its Supported
API table had `GET /health`, `GET /v1/models`, the two OpenAI POSTs and
`/v1/models/unload`. `/v1/messages` and `/v1/messages/count_tokens` are on it
now, with the `anthropic-version` header named as the selector and the sentence
that thinking is per-request on all three surfaces — `reasoning_effort` on the
OpenAI ones, `thinking` on the Messages API — with the thoughts returning as
`reasoning_content` there and as a `thinking` block carrying the empty
signature here. The launcher's reasoning-level question said only that a client
"can still switch it per request with `reasoning_effort`"; it now names the
Messages spelling and the budget rungs, and says the switch is real rather than
a log line.
`Project-Tracker` gains the closed row with the commit, the two revert proofs
and the battery.
agent
committed
Sep 14, 2026
The route writer, the thin bundle, and the two upstream asks
The server guide now says not to hand-write the DeepSeek Harness route: this
checkout generates it with `tools/dsh_route.sh`, and `plugins/dsh-nvmai` keeps
it current at boot while adding a compaction backend that does not think. The
tracker records all three — the generated block and its 10 tests, the bundle
with its 20 and the end-to-end check from the installed copy, and the two asks
that would make the bundle unnecessary.
The launcher's expert-cache rule is 40%
The server guide's RAM step and the tracker row carry the tighter rule and why
it moved: on the 24 GB machine the install's own 12 GiB profile left 11% of
memory free and glitched audio while the model was merely loaded, so the
launcher recommends 40% of physical memory — 3/6/9/12/25 GB on 8/16/24/32/64 —
and still lets an explicit --ram through with a red warning. The default path
keeps the install's measured profile, and the note now says what the rule is
instead of implying the default is bounded by it.
The developer role is served, and the clients live in one list
Two closures from the same session:
- The server guide's Chat Completions section says a `developer` message is
served as the system turn it stands for, and the DeepSeek Harness recipe's
`supportsDeveloperRole: false` is marked optional rather than required.
- The benchmarking guide documents the three rounds and where the client list
comes from: `NVMAI_CLIENTS` in tools/nvmai_models.sh, read by both the launcher
and the coder harness, with four coder clients and Zed as the editor client
that can only be wired and checked.
The tracker records both, with the measurements: three tests shown failing
without the role mapping, the golden gate byte-identical on a dense and an MoE
install, and Claude Code's first measured coder case (exit 0, 92.3 s, quality
pass) after installing it.
The launcher's RAM warning, instead of a cap
The server guide's RAM step describes what happens above half of physical
memory now: the runtime holds the profile's own value to that line, and an
explicit --ram past it is allowed and warned about in red -- system instability,
heavy swapping, slower tokens -- with the choice kept.
The tracker row is rewritten from the cap it described earlier the same day,
with the dry-run and pty evidence.
The launcher's model list, held to what is installed
The server guide says the model step offers only installs that are really under
models/, that the fallback list is filtered the same way, and that a model or
width which is not installed is refused with the widths that are.
The tracker records the change with what it replaced: a fallback menu that
offered every supported family at both widths, defaulted to an Ornith install
this checkout does not have, and left the dense installs out entirely.
The launcher's expert-cache ceiling, and the wiring gap behind it
The server guide's RAM step now says what the new cap is and why: half of the
Mac's physical memory, on the profile's own value and on --ram alike, because
the slot cache is wired and everything else has to fit beside it.
The tracker records the change with the measurement that prompted it — 14.72 GB
RSS and 11% memory free on a 24 GB Mac with Qwen 3.8 4-bit at the shipped
default — and adds the finding this does not fix: NVMAI_KEEP_WIRED can only turn
the wiring on, so the models that wire their cache cannot be told not to.
The native DeepSeek Harness route, and the developer-role defect
Connect a client gains the verified `llm-pi-ai` provider section — no plugin —
with the four traps a hand-declared route hits: pi-ai refuses a keyless route,
the Qwen template rejects the `developer` role pi-ai sends for a reasoning
model, `thinkingFormat` must be `chat-template` because that is the only place
NVMAI reads `enable_thinking`, and pi-ai's five-minute idle-stream default is
shorter than a cold local prefill.
The tracker records the migration as closed (three verified headless turns,
thinking off and on per request, plugin removed, the generated preset's
compaction row returned to the stock engine) and adds the `developer` role's
500 on the Chat Completions surface as open work, with the reproduction.
Record the dense Qwen 3.5 GPU port: both engines, engine per request
The tracker's "planned work" entry for the dense GPU engine is closed and
moved under "Recently closed", with the account of what the port cost: the
role-versus-width finding (C89), the four MoE-shaped load assumptions (C88),
the two config-derived traps (C86/C87), the keep-mask placeholder (C85), and
the prefill command buffer that was never committed. The CPU-only entry that
the port supersedes now says so and points at the new section instead of
reading as current.
The server guide no longer calls the engine a property of the model: the
dense installs show GPU+CPU, --engine picks for them, and <id>@cpu / <id>@gpu
select an engine per request. Getting-Started and Runtime-Controls drop the
"(CPU)" and "CPU engine" labels the port made wrong.
Tracker and server guide: which engine serves the CPU models, and their levels
The three dense Qwen 3.5 models (2B/4B/9B, 4- and 8-bit) are CPU-only -- the
GPU runtime refuses their family `qwen3_5_dense` by name -- so the engine is a
property of the model, not a choice, and the launcher now states it, asserts it
with `--engine`, and refuses the other engine with the family as the reason.
Their thinking levels are exactly off|on, now shown in the model list and asked
with off first and default.
Two server bugs found while verifying that: C82 (every thinking switch on an
installed CPU model was an HTTP 500, because both re-render paths passed a model
directory where a tokenizer folder is required) and C83 (a single-model `--cpu`
server could not open an installed dense model at all, failing with a raw Cocoa
config.json error). Both fixed with teeth; the guide documents starting a dense
install directly.
One launcher, and three stale claims corrected
Documents the consolidation in bb261d8: one tools/server_launcher.sh that
starts the server alone or with Codex, Claude Code, Qwen Code, OpenCode or
Zed, and asks about the model, the thinking level and the RAM limit
(1/2/4/8/16/32 GB, defaulting to the install's measured profile).
The launcher section was documenting a flow that had been replaced: the
old per-model argument positions and a per-quantization port scheme (8081
for 4-bit, 8083 for 8-bit) that the one-port catalog launcher removed. It
now describes what the script actually asks and what it writes per client,
including the two JSONC merges.
Two other corrections, both found while writing the forum series and both
of the kind a reader would have believed:
- Runtime Controls said --expert-cache-slots accepts 8/16/24/32/64/96/128
because that is what --help prints. RuntimeConfiguration accepts
40/48/112/160/192/256 as well. The wiki now lists the accepted set and
flags the help text as narrower.
- The tracker said item 9 would remove a shipped refusal. The QSA
indexer shipped the same day as the gate (f5b1720 decode, 23eb4c4
prefill) and the gate is now unreachable, so nothing is refused today.
Exact block-wise merging is still worth having, but it is no longer
unblocking anything.
Also: the ANE prefill default. The wiki's `on` is right --
ANEPrefillAttention returns .on when the variable is unset -- but
docs/v4.5-ane-prefill.md and the enum's own doc comment both still say off
by default. Recorded in the tracker as stale design notes; no wiki change
needed.
The tracker also gains the CPU side-engine as planned work, with the
measurements that decide its design (1 thread = 3% cost, ~7 tok/s; 92%
one-clause accuracy against 38-of-47 on decomposition), and a duplicate
`## 6.` heading is renumbered to 7.
Head every page with the NVMAI wordmark
The wiki carried no visual identity at all, so the README and the wiki did
not look like the same project. Every content page and the sidebar now open
with the same two-colour wordmark the README slogan uses -- NVM in cyan, AI
in orange.
One file serves both themes here. The slogan needs a light and a dark
variant because its body text is neutral, but the wordmark is entirely
brand colours and both carry enough contrast on #ffffff and #0d1117, which
was checked against each before committing.
Deliberately not applied to the hundreds of inline "NVMAI" mentions in
prose: GitHub cannot colour inline text, so each one would have to become
an image, which would break text selection, search and screen readers for
no gain.
Document Qwen3.8-Flash-Next, per-family tuning, and the new model ids
The wiki was last updated before three shipped changes, so it described a
runtime that no longer exists.
- Qwen3.8-Flash-Next 125B-A6B appeared only in the changelog. It is now in
Home, Features and System Design, including the install size, which is
about 161 GB and dominated by a 95 GB hashed n-gram table, and the point
that it runs on 24 GiB because the experts are streamed.
- NVMAI was described throughout as a "Qwen3.5-MoE 35B-A3B" runtime. There
are three families now, and Qwen3.8-Flash-Next does not share the tensor
contract the other two do.
- Model ids gained the routed-expert width (`_4-Bit`, `_8-Bit`). Every
example in the server guide and FAQ still showed the bare id, which the
server no longer accepts.
- ANE prefill was documented as experimental and opt-in; it is the default,
and falls back to the GPU when a model has no sidecar.
- The expert-cache budget was documented as a flat 8 GiB deriving 128 or 64
slots. It is per family now, with a clamp to half of physical memory, and
speculative expert prefetch was not documented at all.
The dated v4.1 and v4.6 benchmark sections are left as they are: they record
what was measured at the time, and their 8 GiB budget note was accurate for
those runs. Only the claim that ANE "stays off by default", which a reader
would take as current, is corrected there. Current numbers follow in a
separate commit once the re-measurement finishes.
Audit the wiki: current v4.6 state, remove engineering history records
- Move the v4.2/v4.3 experiment qualification record out of the wiki
(now docs/v4.2-experiments.md in the source repository) and drop its
navigation entries; the wiki keeps user-facing documentation only.
- Trim Benchmarks to current, decision-relevant results: v4.6, its v4.1
baseline, the MTP and coding/tooling qualifications behind the shipped
defaults, and fast-alias latency. Remove superseded 6-bit-era tables,
the stale prompt-length record, and the historical M5 reference.
- Bring Features and System Design to v4.6 (tiled Top-K sampling, opt-in
ANE prefill) and remove release-tag phrasing from runtime descriptions.
- Remove the empty Unreleased changelog section, the legacy 6-bit row in
Getting Started, and a duplicated server snippet.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Document NVMAI 4.0 defaults and qualification
Document binary thinking controls
Document launcher production defaults
Document Ornith as the default baseline
Document Ornith native MTP support and benchmarks
Document compressed KV cache and YaRN contexts