Runtime-Controls: the losing decode knobs are gone, not just off
TINYTITAN_PARALLEL_IO and TINYTITAN_PREFETCH_TOP_M are removed from the controls
table with their engine code: the parallel fill measured a wash (+0.4%, 2/3) and
the ring depth lost at every value but 1, so the profile row's depth is now the
only source and a fill is always parallel. The prefetch row records that the ring
is one read a layer and that every alternative allocation of that read was
measured negative.
Also drops the early-expert-hits clause from the decode-execution row: that path
(and the pooled layout it needed) measured a wash and is deleted, so
gpu-residency no longer has a profile that selects it on its own.
Runtime-Controls: the RAM-budget curve, and the clamp is the measured optimum
Two corrections from the 1-16 GB `--ram` sweep (Qwen3.8-Flash-Next 4-bit, short
prompt, 256 tokens, 2026-09-21). The clamp is a third of physical memory, not a
half, so this 24 GiB Mac runs 8 GiB / 64 slots rather than the row's 12 GiB - and
64 slots is the measured optimum (4.55-4.60 tok/s) while 96 gives 4.00 and 128
gives 2.26, despite the hit rate rising 0.757 -> 0.866 and reads falling
80.8 -> 44.5 GiB. The old text also called 96 slots "the fastest", which the
curve contradicts. 1 GB fails outright rather than running slowly: top-10
routing needs ten experts and 8 slots cannot hold them; the floor is 2 GB /
16 slots (2.87 tok/s).
Runtime-Controls: the bounded-I/O opt-out is rejected, not an option
The RAM budget stays enforced; the +4-5% decode the opt-out measured is not
available at the cost of an undeclared footprint. Recorded on the row.
Runtime-Controls: bounded I/O opt-out is +4-5% decode, nothing for prefill
Long-prompt rounds added (3.977 -> 4.261, 4.064 -> 4.090) and the prefill result:
no gain in 2/2 rounds, with swap growing in the page-cache arm.
Runtime-Controls: bounded I/O opt-out is worth about +5% decode
Measured 2026-09-21 on Qwen3.8-Flash-Next 4-bit: three paired rounds of the
shipped bounded path against TINYTITAN_BOUNDED_IO=0 give 4.414/4.727,
4.481/4.765, 4.444/4.560 tok/s (+5.4% mean, 3/3 positive, output identical,
swap flat). The row now carries that number and the reason it is not the
default - the page cache then holds expert data the declared budget does not
count - plus the fact that read advice only earns its keep in that mode.
Runtime-Controls: read advice is family-dependent, so do not flip it globally
Measured 2026-09-21 with the same paired, interleaved design on both families:
turning the read advice off gains about 1% on Qwen3.8-Flash-Next 4-bit and loses
about 3.5% on Qwen3.6 35B-A3B 4-bit. Sampling `iostat -d disk0 1` during
generation shows the advice adds no device traffic at all (108.6-109.4 GiB for
~80.7 GiB of demand expert reads in every arm), so its cost is the syscall and
scheduling work of one call per missed expert, not I/O. The row now says the
default stays `default` and why, rather than inviting a global flip.
Runtime-Controls: Qwen3.8's prefetch rows are depth 1, re-measured 2026-09-21
The per-family table still carried 'off' for both Qwen3.8 widths, with the Qwen
3.6 series' +21.3% figure pasted into the 4-bit row's Measured column. The 4-bit
row is depth 1 and the 8-bit row inherits it: re-measured on the current engine
the ring gives 3.993 -> 4.621 tok/s (+15.7%) on a 7-token prompt and 3.627 ->
4.158 (+14.6%) on a ~500-token one, output byte-identical, after the 2026-09-05
session had found it losing on 512-token story runs. The prose no longer dates
the ring's repair to that lossy measurement, and says what the ring actually
does: reads land before demand and become hits rather than merely warming pages.
Runtime-Controls: the sampling table is per family and Qwen3.8 has two rows
Replaces the stale 'presence penalty 0.0, nonzero not implemented' line with the
real table: Qwen3.8 thinking 1.0/0.95/20/0/0/1.0 and instruct
0.7/0.80/20/0/1.5/1.0, chosen by the request's mode; min-p is 0.0 everywhere and
the filter is not implemented.
Wiki audit: correct 12 pages against the code, and retire a command that cannot run
Every page was read against the 5.7 tree. Three systemic causes account for most of
what the audit found.
**Docs described tuning that is no longer shipped.** FAQ and Runtime-Controls quoted
the family fallback and pre-repair arms, while the defaults are the per-install
`ModelProfile.table`: the 35B rows are 10 GiB/160 slots at 4-bit and 12 GiB/96 at
8-bit with prefetch on, and Qwen3.8 is 12 GiB/96 at 4-bit and 9.5 GiB/40 at 8-bit
with prefetch off. Also corrected: Home's Swift floor (6.3 -> 6.4) and its 35B disk
range; Getting-Started's dense sizes, and its claim that a pipe sets up the browser
window (that prompt needs a terminal); Features' missing 8-bit Qwen3.8, its three
clients where there are five, and "binary thinking" where Qwen3.8 has four levels.
**Version labels had drifted.** Benchmarks' "v4.7" section is 5.0's work — there is
no v4.7 tag — and it claimed a v4.6 baseline for Qwen3.8, which did not exist then.
Its three percentages were withdrawn rather than re-guessed: their baselines cannot
be re-derived from anything in the repository.
**The LAN Manager's remote-access recipe was impossible.** The harness refuses
`--host 0.0.0.0`, and a specific address fails the webserver plugin's schema
(`expected "127.0.0.1" | "0.0.0.0"`), so the profile does not load. Verified by
trying both on a throwaway DSH_HOME. The page, the plugin README and the plugin's
own boot log all told operators to do it; all three now say what is true, and the
boot log no longer advertises addresses nothing is listening on.
Also: Cookbook's launcher keys, OpenAI-Compatible-Server's `-fast` listing and its
per-surface reasoning field, System-Design's ANE default, MTP numbers and interface
list, TinyTitan-Plugin's `--web` owner and its session-title overclaim, the LAN
fence's IPv6 ranges and its missing config keys, an unquoted receipt error, and a
duplicated sentence.
Verified after: links, anchors and the sidebar clean; every cited path still
resolves; both plugin suites pass; all six lint gates green.
Remove the Mac app: the product is the engine and its server
The app, the out-of-process decode service it drove, and their library and test
targets are gone from the repository. The wiki followed them: the Interfaces
rows, the app preference block, the app-only control rows and the iOS roadmap
item all described a front end TinyTitan no longer has and will not carry. A
second front end is a second surface to build, keep in step with every engine
feature, and support, so the Roadmap now says a GUI or mobile front end is out
of scope rather than deferred, and the tracker's issue-#5 items are closed
instead of left open for a future session to pick up.
Where a page told the reader to do something in the app it now gives the real
equivalent: the launcher's --model keys and --bits, the served id from
/v1/models, and the OpenAI (http://127.0.0.1:8080/v1) and Anthropic Messages
(http://127.0.0.1:8080) endpoints. Every process guard is now exactly
'TinyTitanServer|TinyTitanCLI|TinyTitanPackageTests|swiftpm-testing-helper|mlx_lm|mlx-lm'.
Getting Started also gained the one-command install it never mentioned
(tools/install_tinytitan.sh, which leaves ~/.local/bin/tinytitan), since the
installer and the server launcher are now the whole product surface.
Changelog.md is untouched: it is the chronological record of what shipped, and
entries describing the app were true when they were written.
Runtime Controls: the concurrent width is a power of two up to 256
The server-controls table listed --queue-limit but not the width it queues
behind, and described the queue as sitting "while one generation runs" — true
when the width was fixed at one, wrong since it became a setting.
Document both: --max-concurrent-sequences defaults to 1 and takes a power of two
up to 256, each running sequence holds its own KV cache so RAM rises and one
shared GPU makes every answer slower, the prompt cache is off above 1, and the
width actually built is clamped to what memory allows and logged. The
--queue-limit row now says what it queues behind.
The wiki is TinyTitan
Every page except the Changelog is renamed by the same rule the checkout used
(`NVMAI_` -> `TINYTITAN_`, `NVMAI` -> `TinyTitan`, `nvmai` -> `tinytitan`), and
the wiki's own `assets/wordmark.svg` is fixed the way the repository's was: the
name was split across two coloured spans, so it read `NVM` + `AI` and had to
become `Tiny` + `Titan`, with the canvas widened from 132 to 176 so the longer
name is not clipped.
The Changelog keeps its entries as they shipped -- they are the record of
releases made under the old name -- and gains one line at the top saying the
project was renamed on 2026-09-14, so the page is not a mystery. Its release
links still resolve: GitHub redirects a renamed repository.
Home now leads with the project's brand image
(`assets/tinytitan-hero.png`), the same artwork the README leads with, and the
tracker gains the closed row.
agent
committed
Sep 14, 2026
KAT-Coder-V2.5-Dev: prep landed, conversion pending
The new family member is a Qwen3.6-35B-A3B fine-tune, so it is the same code
path this runtime already serves. The tracker records what was checked against
the checkpoint before any download (31,333 tensors, none outside the converter's
namespaces, no vision tower despite the declared vision_config, the plan's
tensor split and the ~20 GB 4-bit output), what has landed (converter, installer
and launcher rows, profile rows, display name), and what remains (conversion,
installs, the app's two fingerprints which need the converted snapshot to
exist, and verification).
Two deviations from the checkpoint's own recommendations are stated rather than
hidden: presence_penalty 1.5 is not implemented by this runtime (it supports
0.0 only), and preserve_thinking has no equivalent here. The roadmap entry
gains a status line and its own open questions are answered -- the mrope block
is present and identical to AgentWorld's, and the checkpoint carries no vision
tensors.
Add the "Capital of Paris" measurement page
The report moves here from the repository, where it was a markdown file rather
than a wiki page: one deliberately ambiguous prompt plus a plain control
question, every served model and both engines, three repeats each, with every
reply quoted verbatim. It records that one install thinks with the switch off,
what the runtime now does with that (C90), and what the repeats settled about
the earlier single-run numbers.
Linked from the sidebar under Reference and from Benchmarks.
Document what a client sees when a model thinks with the switch off
Record the dense Qwen 3.5 GPU port: both engines, engine per request
The tracker's "planned work" entry for the dense GPU engine is closed and
moved under "Recently closed", with the account of what the port cost: the
role-versus-width finding (C89), the four MoE-shaped load assumptions (C88),
the two config-derived traps (C86/C87), the keep-mask placeholder (C85), and
the prefill command buffer that was never committed. The CPU-only entry that
the port supersedes now says so and points at the new section instead of
reading as current.
The server guide no longer calls the engine a property of the model: the
dense installs show GPU+CPU, --engine picks for them, and <id>@cpu / <id>@gpu
select an engine per request. Getting-Started and Runtime-Controls drop the
"(CPU)" and "CPU engine" labels the port made wrong.
Mid-session reasoning switches are honoured
Runtime Controls said a switch could be mapped but not applied, because
thinking and effort are baked into the tokenizer at load. That is no longer
true: the level is carried on the request and the session resolves a
tokenizer for it, so a coding agent can turn thinking off inside a live
session and back on for the next hard question, with no restart.
The section now describes what happens, the mechanism in one paragraph, and
the two consequences worth knowing: a switch re-prefills because the prompt
renders differently (a cache miss, not corruption -- the cache compares
rendered token IDs), and an unrecognised level is not a switch.
The tracker's reasoning entry moves from "what is still load-time" to the
verified result, with the table of the four cases measured on the running
9B -- including the one that did not work before, `reasoning_effort: off`
against a server loaded with `--reasoning on`.
Temperature defaults are per family, not one number
Runtime Controls listed a single `0.6` for `--temperature` and then said it
applies "to Qwen and Ornith at every supported weight precision". That is
true of Top-P, Top-K and the presence penalty, and not true of temperature:
Qwen3.8-Flash-Next is specified at 1.0 on its model card and ships that way.
The row now names both values, and the paragraph separates the shared
sampling controls from the one that differs by family -- 0.6 for Qwen 3.5
and 3.6 (and Ornith 1.5, AgentWorld and the CPU Qwen 3.5 dense models), 1.0
for Qwen3.8-Flash-Next -- and records that these fill gaps rather than
override an explicit request, a request field, or the Mac app.
Runtime requests: best-effort, and Qwen 3.5 9B
Two changes from 195eb3c.
Reasoning levels are now accepted and mapped rather than refused. A coding
agent's `reasoning_effort` was a 400 whenever the served template did not
name that exact level, which ended the turn and often the session; OpenCode,
Zed and Qoder all send it. Runtime Controls gains the per-family level table
(the old text claimed no family defines effort levels, which stopped being
true when Qwen3.8 landed) and the mapping rule, with the log line shown.
The page also states plainly what is still load-time: thinking and effort are
baked into the session's tokenizer, so a mid-session switch — including
turning thinking off — does not yet change what the model renders, and the
log says so instead of claiming a level that is not in force.
Qwen 3.5 2B / 4B / 9B join the model table in Getting Started. They install
as affine snapshots rather than .gturbo installs, so they carry no
verification receipt — recorded in both pages rather than glossed.
The tracker gets the reasoning policy (what was wrong, what now happens, what
is left), the 9B's two tied-output assumptions — the converter refused to
plan it, and the CPU engine would have run it and produced fluent nonsense —
and the snapshot-versus-receipt trade with the honest fix named for later.
One launcher, and three stale claims corrected
Documents the consolidation in bb261d8: one tools/server_launcher.sh that
starts the server alone or with Codex, Claude Code, Qwen Code, OpenCode or
Zed, and asks about the model, the thinking level and the RAM limit
(1/2/4/8/16/32 GB, defaulting to the install's measured profile).
The launcher section was documenting a flow that had been replaced: the
old per-model argument positions and a per-quantization port scheme (8081
for 4-bit, 8083 for 8-bit) that the one-port catalog launcher removed. It
now describes what the script actually asks and what it writes per client,
including the two JSONC merges.
Two other corrections, both found while writing the forum series and both
of the kind a reader would have believed:
- Runtime Controls said --expert-cache-slots accepts 8/16/24/32/64/96/128
because that is what --help prints. RuntimeConfiguration accepts
40/48/112/160/192/256 as well. The wiki now lists the accepted set and
flags the help text as narrower.
- The tracker said item 9 would remove a shipped refusal. The QSA
indexer shipped the same day as the gate (f5b1720 decode, 23eb4c4
prefill) and the gate is now unreachable, so nothing is refused today.
Exact block-wise merging is still worth having, but it is no longer
unblocking anything.
Also: the ANE prefill default. The wiki's `on` is right --
ANEPrefillAttention returns .on when the variable is unset -- but
docs/v4.5-ane-prefill.md and the enum's own doc comment both still say off
by default. Recorded in the tracker as stale design notes; no wiki change
needed.
The tracker also gains the CPU side-engine as planned work, with the
measurements that decide its design (1 thread = 3% cost, ~7 tok/s; 92%
one-clause accuracy against 38-of-47 on decomposition), and a duplicate
`## 6.` heading is renumbered to 7.
Head every page with the NVMAI wordmark
The wiki carried no visual identity at all, so the README and the wiki did
not look like the same project. Every content page and the sidebar now open
with the same two-colour wordmark the README slogan uses -- NVM in cyan, AI
in orange.
One file serves both themes here. The slogan needs a light and a dark
variant because its body text is neutral, but the wordmark is entirely
brand colours and both carry enough contrast on #ffffff and #0d1117, which
was checked against each before committing.
Deliberately not applied to the hundreds of inline "NVMAI" mentions in
prose: GitHub cannot colour inline text, so each one would have to become
an image, which would break text selection, search and screen readers for
no gain.
Document Qwen3.8-Flash-Next, per-family tuning, and the new model ids
The wiki was last updated before three shipped changes, so it described a
runtime that no longer exists.
- Qwen3.8-Flash-Next 125B-A6B appeared only in the changelog. It is now in
Home, Features and System Design, including the install size, which is
about 161 GB and dominated by a 95 GB hashed n-gram table, and the point
that it runs on 24 GiB because the experts are streamed.
- NVMAI was described throughout as a "Qwen3.5-MoE 35B-A3B" runtime. There
are three families now, and Qwen3.8-Flash-Next does not share the tensor
contract the other two do.
- Model ids gained the routed-expert width (`_4-Bit`, `_8-Bit`). Every
example in the server guide and FAQ still showed the bare id, which the
server no longer accepts.
- ANE prefill was documented as experimental and opt-in; it is the default,
and falls back to the GPU when a model has no sidecar.
- The expert-cache budget was documented as a flat 8 GiB deriving 128 or 64
slots. It is per family now, with a clamp to half of physical memory, and
speculative expert prefetch was not documented at all.
The dated v4.1 and v4.6 benchmark sections are left as they are: they record
what was measured at the time, and their 8 GiB budget note was accurate for
those runs. Only the claim that ANE "stays off by default", which a reader
would take as current, is corrected there. Current numbers follow in a
separate commit once the re-measurement finishes.
Document v4.6: sampler rewrite, ANE prefill, benchmarks, runtime controls
Document NVMAI 4.1 release
agent
committed
Aug 21, 2026
Document NVMAI 4.0 defaults and qualification
Document binary thinking controls
Document launcher production defaults
Document unified sampling defaults
Document Ornith native MTP support and benchmarks
Document compressed KV cache and YaRN contexts