Changelog

Changelog

The project was called NVMAI until 2026-09-14, when it was renamed to TinyTitan. Entries below keep the name each release shipped under, and their links still resolve (GitHub redirects the old repository name).

Compact user-facing release history. Follow the linked GitHub releases for complete source changes and assets.

5.9 — the GDN pair loads at the slot's width, and memory's value is measured

Released 2026-09-20. The prebuilt binaries and their checksum are on the release page.

• An install whose manifest names the GDN pair explicitly serves again. A manifest that carries linear_attn.in_proj_a/in_proj_b at the attention slot's own width was refused on load as if it were corrupt, which broke a qwen38flash 4-bit install completely (issue #16). The validator now honours the slot's width and bf16, and names the width it accepts when it still refuses another.
• The ten master prompts are runnable end to end. Long-session worlds as data, with foundation and carryable key sets derived from the truth rather than authored beside it, a scorer over the stored answers, and a driver that runs all ten. The result that matters is the baseline: a client's own 200-word summary wins or ties on three of the first four scenarios, so memory is argued against that, not against memory-off. Where the fact set is dense, arbitrary and revised, memory carries 20 of 20 against the summary's 14.
• Which model judges the side-engine's tasks is a measurement. benchmark/side_engine_judges.py runs the same cases through the resident 4B on the CPU and the model already on the engine: the 35B is better at the reply check the 4B cannot do (100% against 62%) and worse at duplication (75% against 100%), and the pair beats either alone.
• A T7 retrieval hint is queued before the search returns. The background caller's registration was fire-and-forget, so a caller that observed the hint immediately after a search could arrive before the question was queued — the race the thread-sanitizer gate caught. Nothing on the request path waits on a judgement either way.
• The plugin is catalogued. dsh-tinytitan is listed in awesome-dsh-plugin; the npm publish remains the operator's step.

5.8 — a memory side-engine, and a LAN manager for a fleet

Released 2026-09-19. The prebuilt binaries and their checksum are on the release page.

• Agent memory has a resident side-engine. A 4B runs on the CPU and answers one closed question at a time — keep this fact, do these two say the same thing, do they disagree, which kind of change is this — while the main model keeps the GPU. Durability, duplication, contradiction and supersession are wired into consolidation under a budget of six questions; nil means no decision, so an absent engine leaves the old path alone. The 4B is the default, the 9B optional, the 2B unused.
• A stored rule can hold a write back. rules/eyes now stops characters/marcus/eyes from changing, found by key with no model call; only model-derived facts are checked, so the person always overrules a rule.
• Retrieval ranks by rarity. Weighting each term by inverse document frequency takes the authored paraphrase set from recall@1 1 of 4 to 3 of 4, and T7 — the one scorer that reaches the semantic miss — now works in the background during idle windows instead of on the request path.
• The 102 GB n-gram table is shared between builds, not copied per quantization, with a gate that refuses a table whose constants do not match.
• Per-tensor bit widths resolve in the resident index, the enabler for a ~10 MB precision promotion. Its quality case was measured twice and is absent (18/20 against 18/20; 0.0097 ± 0.0067 nats, inside the instrument's floor), so the converter policy is deliberately not built.
• A LAN manager for a DeepSeek Harness fleet. dsh-lan-manager adds a LAN-scoped API for workspaces, sessions and prompts, and ttlanmanager is a terminal dashboard over it; reaching it from another machine is blocked until the harness allows a non-loopback bind.
• Both DSH plugins pin exactly 0.1.6-alpha.2 and refuse to run on any other release rather than write into a harness home they cannot identify.

5.7 — one command installs a built engine, and the app is gone

Released 2026-09-17. The prebuilt binaries and their checksum are on the release page.

• The install builds nothing. tools/install_tinytitan.sh downloads the published arm64 executables, verifies their sha256, unpacks them under ~/.tinytitan and asks one question — which model. No Xcode, Homebrew, Python or Node is needed, and --version TAG pins a release while --from-source keeps the clone-and-build path. See Getting Started.
• The model is a menu. All 16 builds are listed with their installed size and what each is for; Enter takes the default, --model NAME skips the menu, and through a pipe the installer takes the default and says so rather than hanging.
• Every script runs on the shell a factory Mac actually has. /bin/bash there is 3.2.57, and these scripts had only ever run under Homebrew's 5.x. Three classes were closed: a heredoc apostrophe that stopped 3.2 parsing the launcher at all, ${v^^}/mapfile that died mid-menu, and "${a[@]}" on an empty array (unbound variable under set -u), which broke the default --web path. tools/lint.sh shell now fails on all three.
• An optional browser chat window, isolated from any DeepSeek Harness you run: its own home, npm prefix, pnpm store and port, under ~/.tinytitan. The launcher's --web opens it already pointed at your model, with a chat preset that cuts a nine-word question from 4222 to 124 prompt tokens and a warm expert cache, so the first answer lands in 8.3 s instead of 161 s on the 35B.
• The Mac app is removed. The engine and its loopback server are the product; the supported clients are the ones you already have, plus the browser window as a client of the loopback server. This also leaves ServerVersion.current as the tree's single version literal, which release.sh enforces.
• POST /v1/responses/compact returns one opaque state item standing in for a conversation, preserving the system prompt and the core reasoning. The note is metered with the server's own tokenizer and compressed rather than truncated when it is over budget. Verified on the 4B and 9B.
• The concurrent width is any power of two up to 256 (the default stays 1), with the per-load memory budget doing the clamping and saying so in the log.
• Repository cleanup: the forum article series is gone with the forum, and adding a model is eight wiring points now that the app descriptor is not one.

5.6 — the three reported bugs, and an ANE answer of "no"

Released 2026-09-17. The prebuilt binaries and their checksum are on the release page.

• The app can change models. Model → Change Model now lists every selectable build and records the choice; a banner says to reopen, because the model directory, its settings file and the decode-service process are bound at launch. Previously the app read a model preference it had no way to write.
• The model installer finds its own binary. tools/install_models.sh used a target-triple build path that exists only when SwiftPM's triple matches the toolchain, and can point at a stale tree when it does. It uses the stable product path now, and a lint gate fails any build path that hardcodes a triple.
• A sidecar the Neural Engine refused can no longer be written or run. The exporter checks the compile markers, loads every function it records, and asks the compute plan which device each operation is assigned to — the refused Qwen 3.8 h12288 measures 0 of 173 operations on the ANE. The runtime declines a sidecar that lacks aneCompileVerified, so an old sidecar costs a GPU prefill instead of a ~38× slower CPU one.
• Qwen 3.8's sparse attention is correct on the ANE, and still stays on the GPU. The runtime folds the indexer's key selection into the mask the sidecar already takes — 0.47% against an independent reference, where a causal-only mask is off by 7.6%. Measured, the ANE is 0.72× the GPU at 4-bit and 0.87× at 8-bit, so no sidecar is installed for it; the gather-graph variant that would have removed the extra arithmetic was sized and rejected.
• Building from source needs Swift 6.4 (Xcode 27). The manifest floor now matches the toolchain the project is built and tested on.
• The dense Qwen 3.5 installs reach the ANE. They were on the 128-token default prefill chunk, which never matches the sidecar's fixed 4,096-token program; at 4,096 the 2B prefill measured 1.30× faster at ~2,500 tokens.
• The server has per-session KV and GDN state and admits up to four sessions, but serves them one at a time: width > 1 produced degenerate output on the real 2B and 4B, so the batched width is held at 1 until multi-session output is verified correct. The Responses API now echoes the sampling the server used instead of null.
• The wiki is a user guide. New: a Cookbook of copy-paste recipes with the output to expect, and a Technical Articles section whose first entry is the ANE prefill research.
• Repository cleanup: the processed deep-audit register was removed, the project tracker no longer carries a closure log, and three oversized source files were split as pure code motion with the build and 1,566 tests verified.
• Internal speeds are a release gate with a committed baseline, so a kernel or bandwidth regression fails a release instead of being noticed later.

5.5 — the project is TinyTitan, and JSON is enforced by a grammar

Released 2026-09-14. The prebuilt binaries and their checksum are on the release page.

• The project is TinyTitan everywhere — package, targets, executables, environment variables, repository and app. The binaries are TinyTitanServer, TinyTitanMac, TinyTitanCLI, TinyTitanRepack, TinyTitanDecodeService and TinyTitanBench; the environment variables are TINYTITAN_*; the app installs as TinyTitan.app; and the archive is tinytitan-5.5-macos-arm64.tar.gz. The old repository URL redirects, and releases before this one keep the name they shipped under.
• JSON output is enforced, not requested. response_format on Chat Completions, text.format on the Responses API and output_config.format on Messages compile into a byte-level grammar that masks the sampler, so the model can only emit a document the schema allows. The supported subset is small and explicit, and everything outside it is refused by name.
• Thinking is the request's own on every surface. On /v1/messages, thinking.disabled really turns it off, adaptive still means "you decide", and an enabled budget maps onto the levels the served model renders. Chat Completions gained the same per-request control, and reasoning tokens are reported in usage.
• A developer message no longer fails with HTTP 500. It renders as the system turn it stands for — what a harness that switches to that role once a model reasons needs.
• The launcher lists only installed models, and warns in red above 40% of this Mac's physical memory before using a larger --ram anyway.
• One client list for the launcher and the coder harness — Codex, Claude Code, Qwen Code, OpenCode and the Zed editor — with --round clients checking every client's wiring without loading a model.
• DeepSeek Harness support. tools/dsh_route.sh generates the harness's llm-pi-ai route from the installed models, and plugins/dsh-tinytitan/ keeps it current at boot and mounts a compaction backend that does not think.
• The six dense Qwen 3.5 installs have golden baselines, so a release now re-checks 2B/4B/9B at both widths instead of declaring them exceptions.
• The README is one benchmark table with a reproducible GPU-versus-CPU column for the dense models and no per-release callout — this page is the announcement.

5.4 — KAT-Coder-V2.5-Dev, and verification that never fetches a model

Released 2026-09-14. The prebuilt binaries and their checksum are on the release page. This release first ships the 5.3 work to users (see the note below), together with the release-verification policy and a native app icon.

• KAT-Coder-V2.5-Dev 35B-A3B is supported at 4-bit and 8-bit — Kwaipilot's agentic-coding fine-tune of Qwen 3.6, with its own temperature (1.0) rather than the series' 0.6. 17.86 tok/s at 4-bit, 6.91 at 8-bit, with the three verified continuations and a golden baseline per width.
• Routed experts are filed by index, not by arrival order. KAT is the first converted checkpoint whose experts ship one tensor per expert; fusing them in arrival order silently paired each routing decision with a different expert's weights. The install loaded, matched the checkpoint byte for byte, passed every structural check, and answered nonsense. A tools/lint.sh gate now fails if the order regresses.
• Both widths install from one download: tools/install_models.sh <model> both converts 4-bit and 8-bit in a single pass over one ~70 GB checkpoint, and a later second width reuses the snapshot instead of re-fetching it.
• Three size caps that refused legitimate files are fixed — the snapshot index (4 MiB against KAT's 9.7 MB, copied into both install paths), the resident index (a 1 MB staging budget applied to a ~28 MB output), and the runtime's manifest cap (4 MiB against the 6.25 MB file --verify-install had just accepted).
• The downloader survives a truncating link: length-verified 64 MiB ranges, a small connection pool, HTTP/1.1, stall detection, and no resume that could append to a truncated prefix.
• The tools resolve their own Python by capability rather than a pinned version.
• Release verification uses only the models already installed. The golden gate reports every target it could not check instead of skipping it silently, an installed model that no gate covers is an error, and the gate fails if it changes models/ at all — so a release is never made to pass by downloading, converting or re-installing a model.
• A native NVMAI app icon, replacing the upstream fork's bird: the wordmark's own palette, and reproducible with tools/make_app_icon.py.
• The binary archive carries NOTICE alongside LICENSE and THIRD_PARTY_NOTICES.md.

5.3 — tagged, never published

5.3 was prepared and tagged (v5.3) but never published, because the golden gate could not complete on the machine it was cut from: a storage provider had left most installs online-only and reading them failed. Nothing was missing from the release itself, so its content first shipped to users in 5.4. The tag is kept as the history it records.

5.2 — one server for every model, dense Qwen 3.5 on both engines

Release v5.2

• One server serves every installed model. --models-dir serves the whole catalogue on one port with one model resident at a time; /v1/models lists them and a request naming another switches to it. One launcher replaces the eight per-model start scripts and the separate client launcher: it starts the API alone or with Codex, Claude Code, Qwen Code, OpenCode or Zed, and asks what to launch, the answer style, the model, the thinking level and — only where a model has an expert cache — the RAM limit.
• The dense Qwen 3.5 2B / 4B / 9B run on the GPU as well as the CPU, and the engine is a per-request choice for them: the bare id is the GPU spelling and <id>@cpu / <id>@gpu name an engine explicitly. The refusal was lifted only after the GPU's logits matched the CPU engine's on the real install, layer by layer — which is the gate that caught k_proj/v_proj being read at the attention slot's 4-bit width while the install stores them at 8, a plausible wrong answer rather than an error.
• Optional agent memory: NVMAI_MEMORY=1 gives a model durable facts scoped to the repository a client is working in, inside the server process, with no database to install. The extraction marks whether the person asserted a fact or the model inferred it, and the guard (on by default where memory is on) stops an inferred fact from silently superseding an asserted one: unguarded the store answered 61% of a recorded session's questions, guarded 98%.
• Thinking is not the answer. Reasoning reaches clients as reasoning_content, apart from content, on every surface, and each model offers only the levels its template renders. A thought the model opens while thinking is off is now split out too — Qwen-AgentWorld 8-bit does exactly that, and its scaffold used to arrive as the answer — and the server logs it.
• The APIs coding agents speak: the OpenAI Responses API in full and the Anthropic Messages API, both exercised against the real Codex and Claude Code CLIs rather than only against test doubles.
• A deep audit of the whole tree: 89 code findings and 8 documentation defects, 0 open. The worst were silent rather than loud — an out-of-bounds router write on Qwen3.8-Flash-Next, a prompt-cache restore that left the sparse indexer holding the previous conversation's keys, and a fused kernel that could never be built, so every measurement of that path had measured the unfused one.
• Watchdogs, off by default: NVMAI_WATCHDOGS=1 watches for a repetition loop, a stall, a stub answer and a ping-pong turn, and names which of them may stop a generation rather than only report it.
• Structure and language baseline: the 2,599-line HTTP server is eight files, the test tree mirrors the source tree, the compiler warning count is zero, and the three upcoming Swift features the tree was already clean under are now enforced (the three with a real migration cost are recorded as deliberate, with what they would cost).
• This release's own measurements — every model, both engines, three repeats, every reply quoted verbatim, including the install that thinks with the switch off — are on One Prompt, Every Model. The 512-token story table under Benchmarks was measured at 5.1 and has not been re-run.

5.1 — the 35B models decode 10-13% faster at 8-bit

Release v5.1

• The expert prefetch ring was clogged and is now repaired. A reclaim rule introduced in 5.0.2 freed a speculative slot only once its layer index had passed, so a prediction made for the last layer of a token was never freed and the ring ran with almost no free slots. Every prefetch measurement since compared variants of a mechanism that was not running.
• Prefetch is on again for the 35B models, one read deep, measured per install: Qwen 3.6 +1.8% at 4-bit and +11.3% at 8-bit, Ornith 1.5 +1.8% / +12.6%, Qwen-AgentWorld +1.4% / +11.4%. Qwen3.8-Flash-Next keeps it off, where the repaired ring loses at every setting: its experts are larger and arrive after the layer that would have used them.
• The expert cache stays wired through prefill. It used to be released at prefill, swapped out under memory pressure, and faulted back in by the first decode token, costing 1.6-4.7 s per request. Short answers gain 5-7%; long ones about 1%; prefill is unaffected.
• A start script per model and quantization, eight in all, each on its own port so two can run at once. The interactive launchers and the start scripts now share one catalogue of models, paths and ports, and both read the served model id from the running server rather than assuming it.
• The Mac app recognizes the models you have installed. Its catalogue listed only MLX repacks, so after 5.0.1 moved installs to bf16-sourced builds it treated all of them as foreign checkpoints. It now fingerprints each of the eight, and where it cannot download a build itself it prints the installer command instead of a Download button.
• New decode diagnostics, and three levers measured and left off by default: early expert hits, a probe-weight prefetch gate, and a decayed-frequency cache policy.
• Story benchmark, base M3 24 GB: AgentWorld 21.74 / 12.28 tok/s (4 / 8-bit), Ornith 21.65 / 11.93, Qwen 3.6 21.41 / 12.37, Qwen 3.8 5.46 / 2.10.

5.0.2 — per-model tuning profiles, bigger expert caches for the 35B family

Release v5.0.2

• Every installed model and width has its own tuning row (expert-cache budget, prefetch depth and disk I/O tier, prefill chunk, sampling defaults, kernel switches), resolved family default, then row, then environment. Editing one install never moves another; NVMAI_* variables still override.
• The 35B-A3B models get bigger expert caches. A per-role split showed a fifth of the 4-bit token and over a third of the 8-bit token waiting on expert reads at the old 128 / 64 slots per layer. Qwen 3.6, Ornith 1.5 and Qwen-AgentWorld now ship 160 slots at 4-bit (10 GiB) and 96 at 8-bit (12 GiB), measured on each install with swap sampled: +9% at 4-bit and +15% at 8-bit, output byte-identical. 192 slots at 4-bit measured +4% more but starts paging on a 24 GB machine; --ram-budget 12G selects it.
• Qwen3.8-Flash-Next: 4-bit prefetch runs two deep on the utility disk I/O tier (+3%); 8-bit takes 40 slots (+8%). Two-layer-ahead prefetch and the fused hyper-connection gates measured washes and stay opt-in.
• Ornith 1.5 is rebuilt from its bf16 release through the shared converter; no third-party quantization remains in the installer.
• Story benchmark, base M3 24 GB: AgentWorld 21.28 / 11.16 tok/s (4 / 8-bit), Ornith 20.99 / 10.89, Qwen 3.6 20.95 / 11.23, Qwen 3.8 5.40 / 2.06.

5.0.1 — 35B decode regression fix, Qwen-AgentWorld

Release v5.0.1

• Fixed a 5.0 decode regression on the 35B models: per-kernel environment reads (about 800 per token) cost most of a 45 ms token. Qwen 3.6 4-bit went from 6.67 back to 19.21 tok/s; output unchanged on every golden.
• Added Qwen-AgentWorld 35B-A3B (4-bit and 8-bit), installed straight from Qwen's bf16 release: one download quantized a shard at a time into both widths, router, shared-expert gate, DeltaNet gating projections and every norm kept at bf16.
• Every install now comes from its own bf16 release through one converter; the embedding and head stay 8-bit for quality (--head-bits 4 exists and is worth about 20% on 35B 4-bit builds).
• Top-8 routing uses the one-simdgroup selector (byte-identical, a few percent on the 35B family).

5.0 — Qwen3.8-Flash-Next, and per-family decode tuning

Release v5.0

• Added Qwen3.8-Flash-Next 125B-A6B in 4-bit: 48 layers of gated DeltaNet with sparse-indexed attention, 512 experts at top-10, hyper-connection residual streams, a QSA key indexer, and hashed n-gram embeddings at layer 1. Text-only; the checkpoint's vision tower is not repacked. An install is about 161 GB on SSD, of which the n-gram table alone is 95 GB, and it runs on a 24 GiB machine because only a bounded slice of its experts is resident.
• The expert-cache budget and expert prefetch are now tuned per family and quantization rather than being one number for the catalogue. Both are governed by how much of the token is expert I/O, and that varies by nearly 3x across the shipped models. Qwen3.8-Flash-Next takes a 12 GiB budget (96 slots) and depth-1 prefetch, worth +21.3% together (5.735 to 6.957 tok/s); Qwen 3.6 and Ornith take depth-1 prefetch at 8-bit (+5.5% / +5.9%) and no prefetch at 4-bit, where it measured -4.2% / -3.9%. The tuned budget is clamped to half of physical memory, so a 16 GiB Mac is not handed a budget measured on a 24 GiB one. --ram-budget and NVMAI_PREDICTIVE_PREFETCH override both, in either direction.
• Speculative prefetch now filters the predicted experts by residency before capping the ring rather than after, so the budget is aimed at experts that would otherwise miss instead of being spent on ones already cached. Ring depth is a bandwidth decision and is sharply peaked at one read in flight: depth 1 measured +12.2% where depth 4 measured -9.8%.
• ANE prefill is now the default (NVMAI_PREFILL_ANE=off opts out). Models without an exported sidecar fall back to the GPU automatically.
• Breaking: API model ids now always end in the routed-expert width, for example ornith-1.5-35b-a3b_8-Bit. The width comes from the manifest rather than the name, so two quantizations of the same weights are distinguishable in /v1/models. The bare id is no longer accepted.
• NVMAICLI now derives its expert-cache slots the way the server does instead of defaulting to a flat 64 for every model, so the two front ends agree.

4.6 — Sampler rewrite and ANE prefill

Release v4.6

• Production sampling (Top-K 1...64, which includes the default Top-K 20) now runs through the tiled three-stage GPU reduction instead of a single-threadgroup kernel that re-scanned the full vocabulary once per kept candidate. Sampled output at a fixed seed is unchanged token-for-token; the per-token sampling cost fell from 15.5 ms to 1.4 ms. NVMAI_SAMPLER_PATH=generic keeps the old kernel as a measured control.
• Added the experimental opt-in ANE prefill path (NVMAI_PREFILL_ANE=on): full-attention prefill blocks run on the Neural Engine from a Core ML sidecar exported once per model with tools/export_ane_prefill.py. A 6,103-token prompt measured 2.31x faster prefill (132.90 s to 57.52 s) with decode unchanged. Off by default because its fp16 attention is not byte-identical to the GPU path; the greedy golden baseline runs with it off. Short prompts, prompt-cache resumes, and MTP chunks stay on the GPU automatically.
• The MTP verify pass now plans both rows' experts as one union, overlaps one parallel miss fetch with the shared expert, and reads the lm_head once per pass. Faster than the old width-4096 tile scheduler at width 2 with byte-identical output; NVMAI_MTP_VERIFY=tile retains the old schedule. MTP itself remains off: re-qualification still failed promotion at both quantizations even at 84-86% draft acceptance.
• Sampling is now visible to NVMAI_KERNEL_STATS as its own role, and MTP runs report per-pass phase attribution under NVMAI_RUNNER_STATS.
• Retained the v4.2 control-plane and v4.3 predictive-prefetch experiments as documented, non-default controls; production decode scheduling is unchanged from v4.1.
• On the release M3/24 GB benchmark, median Ornith decode improved from 16.45 to 22.53 tok/s at 4-bit and from 8.75 to 9.40 tok/s at 8-bit.

4.1 — Expert hit/fixup streaming

Release v4.1

• Decode now executes phase 1 for already-cached routed experts while missing experts are read from SSD, then runs a bounded fixup for those misses. Shared expert work remains overlapped with routing and I/O.
• Cache-slot leases prevent in-flight expert buffers from being evicted or overwritten. The default pread path still bypasses the macOS page cache and preserves the configured RAM bound.
• Added per-run hit/miss, reload, read-latency, hidden-I/O, and hit/fixup-layer telemetry plus a deterministic barrier-versus-hit/fixup benchmark.
• Kept the Metal expert-I/O backend experimental; production continues to use bounded pread because Metal I/O is not yet a complete faster replacement.
• Fixed command-buffer completion tracking found during release benchmarking. Golden output remained identical after the scheduling change.
• On the release M3/24 GB benchmark, median Ornith decode improved from 15.31 to 16.45 tok/s at 4-bit and from 8.31 to 8.75 tok/s at 8-bit.

4.0 — Ornith and extended context

Release v4.0

• Added an eight-cell Ornith coding/tooling matrix for 4-bit and 8-bit weights, Concise on/off, and Thinking on/off. Four isolated Swift, Python, TensorFlow, and PyTorch repairs receive up to 25 turns and must pass both public execution and chained hidden validation.
• Exposed Ornith/Qwen thinking consistently as the models' official binary off|on switch in the server, CLI, Mac app, launchers, and benchmark profile. Low/Medium/High modes are not offered because the model templates define no effort levels or thinking-token budgets.
• Made Ornith 1.5 35B-A3B 8-bit with Concise off and Thinking off the default installer, Mac app, launcher, benchmark, release gate, and real-inference golden baseline. Qwen 3.6 and Ornith 4-bit remain explicit options.
• Added pinned, text-only Ornith 1.5 35B-A3B installers for 4-bit and 8-bit. Ornith reuses the existing bounded Qwen3.5-MoE runtime and SSD expert streaming; its tokenizer and chat template remain model-specific. Vision is deferred.
• Added reproducible preparation and local import of Ornith's official native one-layer MTP draft. It shares the matching target embedding and head while keeping draft experts SSD-streamed and bounded. Cross-model Qwen/Ornith pairings are rejected. MTP remains experimental and off by default because the first M3 benchmarks found no repeatable speedup.
• Standardized Mac app, CLI, Chat Completions, Responses API, decode-service, and benchmark defaults across Qwen and Ornith quantizations: temperature 0.6, Top-P 0.95, Top-K 20, and presence penalty 0.0. Nonzero presence penalties remain unsupported; greedy correctness and MTP measurements still override temperature to 0 explicitly.
• Added 16-, 8-, and 4-bit KV-cache storage with 8-bit as the default. The cache remains bounded and grows on demand independently of model weight precision.
• Added optional YaRN context scaling for 512K and 1M contexts. Native RoPE is still the default; YaRN defaults to 1M when enabled and currently excludes MTP.

3.9 — Streaming and memory

Release v3.9

• KV state now grows on demand, bounded expert I/O is the default, and the server accepts a RAM budget that derives the expert-cache size.
• Six-bit support was withdrawn; current supported installations are 4-bit and 8-bit.

3.8 — Runtime measurement

Release v3.8

• Reduced transition idle time and improved decode throughput while preserving deterministic output.
• Added clearer GPU occupancy, routing, and runtime timing instrumentation.

3.7 — Correctness and release safety

Release v3.7

• Fixed sampler, moved-install, unload, and concurrency defects found in a production audit.
• Added lint, ThreadSanitizer, and real-inference golden-output gates.

3.6 — Model residency

Release v3.6

• Added lazy loading, idle unload, and manual model unload.
• Improved receipt recovery and prompt-cache consistency for fast-alias turns.

3.5 — Robustness

Release v3.5

• Hardened streaming tool calls, cancellation, UTF-8 detokenization, installer durability, and server queue behavior.
• Aligned server sampling defaults with the Mac app and CLI.

3.4 — Launchers and reasoning

Release v3.4

• Added the unified server and coding-CLI launchers.
• Added optional reasoning mode and removed the artificial default response cap when clients omit one.

3.3 — Fast alias

Release v3.3

• Added qwen3.6-35b-a3b-fast for chat-only coding-CLI requests with much less prompt prefill.

3.2 — Runtime controls

Release v3.2

• Corrected expert-cache and prefill defaults and improved API compatibility.

3.1 — Responses API

Release v3.1

• Added native POST /v1/responses support and direct Codex connectivity.

3.0 — Concise mode

Release v3.0

• Added per-model concise prompts and standardized model-directory names.

2.0 — Performance controls

Release v2.0

• Added parallel expert fills, larger expert caches, read advice, larger prefill chunks, and the kernel benchmark suite.

0.1 — Initial version

• Released Qwen 3.6 inference with the installer, CLI, Mac app, bounded expert streaming, and local OpenAI-compatible server.