v0.4.9, the review-and-gates release

The review-and-gates release. Two outside reviews of the current tree found ten defects between them; each is fixed here with a gate that fails on the previous binary. The test build was rebuilt around a shared object layer, which took the CI gate from 18.8 minutes to 4.5 with an identical list of checks. The tray icon is now the Xyntetik ensö, and xyntetik.com carries the new brand marks.

Fixed, from the first review (slot state)

• A KV ring (RUNNER_KV_RING=1) or tied-V (RUNNER_TIEDV=1) refuses the shared prefix tier, and that refusal returned before the slot's own rewind. With the default cache_prompt, every request appended to the previous prompt's context: identical prompts answered differently and the third overflowed a 128-token context. The slot now rewinds first.
• The server restored its sampling defaults per request from a copy taken before engine_init installed the stop-token repeat-penalty exemptions, so over HTTP a repeat penalty could punish the model's own turn terminator. The exemptions survive the restore.
• adapter.lora.alpha was not part of the adapter identity, so a reload under a rewritten alpha reused the old prefix snapshots (8 to 800 reused the whole prompt, logprobs off by 0.018). Alpha is hashed into the id.
• TTL expiry and the VRAM-yield release unloaded the target but not the draft, which stayed mapped with its KV. Both idle paths release it as POST /unload does.

Fixed, from the second review (training, schema, Messages)

• LoRA training on NoPE layers: the backward applied rope and reversed it on every layer, while the serving forward skips the rotation and scales Q by the attention temperature on NoPE layers. Both now follow the layer test. A NoPE fixture joins the finite-difference gate: worst relative error 1.7 before, at the float floor after.
• The schema compiler no longer weakens what it is given. Boolean false is refused instead of compiling to "anything"; an enum value outside the declared type is dropped and a const beside an enum must be a member; a type beside oneOf/anyOf is refused rather than ignored; ["integer","null"] stays integer.
• Anthropic Messages: thinking.type:"enabled" without budget_tokens is 400, as the API requires, and any mcp_servers value other than absent, null or an empty list is refused.

Build and CI

• make test links the shared engine sources from an object layer keyed by a checksum of the exact compiler command, so a flag change is a different directory and never a stale object. Rules whose flags differ from $(CFLAGS) still compile what they link. CI runs the target with -j; site- and docs-only changes no longer run the engine's gates.
• RUNNER_TTL_POLL_S sets the idle reaper's poll (default 5 s unchanged). The test harness sets 0.25 s.

Also in this release

• Capabilities report drafting from resident engines across all serving slots; single-model MTP serving restores its head after unload, keep_alive: 0 and TTL expiry.
• Model signature policy is enforced on named registry loads, reloads and additional slots (HTTP 409 model_signature_refused).
• LoRA adapter path and scale are preserved on every target model load; a failed adapter load refuses the target.
• Speculative decoding stops cleanly at the context boundary for model, MTP and lookup drafts.
• The tray icon is the ensö: bare when idle, with the spark when a model is loaded, the full Runner mark while inference runs.

Evidence

• CUDA smoke on an RTX 3070: PASS, 13 checks (docs/compat-reports/cuda-smoke-0.4.9-2026-09-05-rtx3070.json).
• Blackwell matrix: granite-4.1-8b cpu_cuda 9/9, gemma-4-E4B complete with cpu_cuda pass, Qwen3-30B-A3B in its known shape (tool 2/8 pre-existing; cpu_cuda not executed while the MIG slice is held).
• Local: make test exit 0, conformance 437 passed and 17 skipped, release-check 0.

One finding from the release run is recorded in docs/compatibility-program.md: a matrix binary built with the conda environment's exported CFLAGS fails the CPU/CUDA identity check at one near-tie per model, and the same source built with the Makefile's own flags passes. The CPU side of that identity is the build, not only the code.

Binaries: runner-linux-x86_64, runner-macos-arm64, runner-windows-x86_64.exe, with SHA256SUMS. Container: ghcr.io/joakimpalm-zen/xyntetik-runner:v0.4.9.

v0.4.8

The E-series and contraction release: Gemma 4's current shared-KV exports
load, the CPU prefill dot gives one answer whatever the batch shape, and the
serving surface gains cached_tokens, GET /metrics and a fourth draft
source that needs no weights.

• Batched prefill dot: blocking-independence by construction, not by
  codegen luck. The CPU prefill kernel (vec_dot_f32_multi and the 4x4
  tile over it) promises that a column's bits do not depend on how many
  columns travel with it; that is what lets one binary give the same
  logits whatever batch shape the scheduler, the prefix cache or -b
  hands it. The promise held on clang and broke on GCC hosts: the scalar
  tail (the last n mod 8 elements of a row) was written s += w[i] * x[i],
  which GCC may contract into one fused multiply-add or round the product
  first, and it decided per loop shape (-ffp-contract=fast is its gnu11
  default, independent of -ffast-math, which the kernels are built
  without). Measured with GCC 15 at -march=x86-64-v3 and native on an
  AVX-512 host: the 4-column block's tail compiled to vmulps plus
  vaddss, the 1- and 8-column tails to a vmulps quad plus
  vfmadd231ss, and the same column differed by one ulp between blockings
  (14 pairs in the gate); the ubuntu-latest CI hosts (GCC at -march=native) split the 8-column block from
  the 1-column path the same way (make-test, 2026-09-02). Every f32 dot
  tail is now an explicit fmaf, which has exactly one admissible result,
  so all blockings sum identically whatever the compiler unrolls or
  vectorizes; the SIMD bodies (already FMA intrinsics) are unchanged and
  the assembly shows no rounded product left in either kernel on GCC 15 or
  clang 22 at native, v4 and v3. Exposure, precisely: the split is
  the compiler's, not the ISA's. GCC 15 fails the old kernel at
  -march=x86-64-v3 too, the level the Linux and Windows release binaries
  pin, so those builds carried the same tail codegen; clang builds (the
  macOS release, a local clang make) did not. What it could have moved:
  the row length here is a projection's input width, a multiple of 8 in
  every family in the roster, so the tail never ran for a shipped model and
  no token could have depended on batch shape; the hole was in the kernel
  contract the register blocking rests on, which is exactly what the gate
  pins so the blocking can widen without touching the token contract.
  make test now compares every
  column of every blocking against a fixed-order scalar reference (the
  kernel's summation order written out with explicit fmaf, so the bits
  come from a written order rather than from the kernel under another
  blocking; it caught 358 cases on GCC where every blocking had drifted
  together and the old relative check saw 14) and against an exactly
  representable integer anchor; CI adds a simd-isa-levels job that builds
  the gate at ARCH_FLAGS=-march=x86-64-v3 (the release pin) and at
  -march=x86-64-v4 on purpose, the latter skipping with a printed reason
  on a runner without AVX-512. Prefill throughput unchanged, as
  the shape says it must be (a shipped model never runs the tail):
  SmolLM2-135M Q8_0 on an M1 CPU, six interleaved runs each, median 463.8
  prompt tok/s before and 463.9 after; greedy tokens identical.

• Gemma 4 E-series: the current shared-KV exports load. A layer at or
  past block_count - attention.shared_kv_layers computes no K and no V - it
  attends over the cache an earlier layer filled - so its attn_k.weight,
  attn_v.weight and attn_k_norm.weight are unreachable, and every
  quantized E-series export published since the BF16 one leaves them out
  (E4B: 666 tensors where the BF16 export has 720; layers 24..41 carry 14
  tensors each instead of 17). The loader demanded all three on every
  attention layer, so 0.4.7 refused Google's own QAT Q4_0 release, the
  ggml-org Q4_0 and the community QAT F16 conversion at load with
  error: missing tensor blk.24.attn_k.weight. They are now optional on
  exactly the shared-KV tail and still required on every KV-owning layer,
  where a missing one is refused by name as before. The E4B file pinned in
  the compat matrix is an older full-form conversion, which is why the matrix
  never saw this.
  Three gates, evidence in
  docs/compat-reports/eseries-shared-kv-2026-09-04/. (1) The anchor:
  scripts/gguf-drop-shared-kv.py (new) rewrote the local 720-tensor
  Q4_K_M into the 666-tensor form, dropping exactly 54 tensors on layers
  24..41 with all 666 survivors SHA-256 identical to their source blobs; the
  two files then score byte-identically (--score, 36 positions solo and 159
  chunked over the house corpus) and generate byte-identical greedy output.
  Unreachable weights must move no bit. (2) Google's official QAT Q4_0
  (gemma-4-E4B_q4_0-it.gguf, 5,154,941,280 bytes, 666 tensors) loads,
  answers the chat smoke and scores nll_mean 3.53626308 / ppl 34.3383595
  over the same 159 positions; it joins the pinned manifest as
  gemma-4-e4b-it-qat-q4_0 with a macOS CPU ledger row. Cross-engine
  agreement was not measured for it and is not claimed. (3) The gate CI runs:
  scripts/make-test-model.py --drop-kv shared|<layers> builds the same
  shape at fixture scale, and tests/test_eseries.py pins that a compact
  fixture loads, that it is bit-identical to the full one, and that a
  KV-owning layer missing its K is still refused with the exact error text.

• --draft-lookup: prompt-lookup drafts, the fourth draft source. The
  verify walk that checks a draft model's proposals, the MTP head's and
  grammar fast-forward's now also takes proposals from the context itself:
  each round the last n tokens (n from 5 down to 3, longest match first) are
  searched for in the prompt plus everything generated so far, and the tokens
  that followed their most recent earlier occurrence are drafted, up to
  --draft-k, continuing the match's own period past the context end. No
  weights, no draft forward, and no match drafts nothing, so a round without
  one costs plain decoding plus an integer search. Output stays
  token-identical to plain decoding, greedy and seeded, because the target
  decides every token; make test pins that on a repeating prompt, a prompt
  with no repeats and a tool-echo chat, pins the search against hand-computed
  proposals, and pins the accounting: runner_telemetry.speculation (source,
  rounds, drafted, accepted, and the lookup's share), the transcript's
  speculation object, draft.source in GET /v1/capabilities, and the
  three runner_speculation_* counters on /metrics, which keep counting
  every source. One draft source per run: --draft-lookup beside --draft
  or --mtp is refused at startup. Measured 2026-09-04 on an M1 CPU
  (docs/context-drafts.md, unrelated-prompt rows included): 1.47x on a
  verbatim repeat and parity elsewhere on SmolLM2-135M Q8_0; a loss on every
  row but the repeat on the compute-bound TinyLlama-1.1B Q2_K, since a
  rejected draft is a wasted verify column. The bandwidth-bound 3B to 8B
  Q8_0 case the source is meant for did not fit the 8 GB box resident that
  day and is still to be measured.

• usage.prompt_tokens_details.cached_tokens: the prefix-cache figure in
  the field a standard client reads. Runner has always counted, per request,
  how many prompt rows it did not have to prefill, and reported it as
  runner_telemetry.prompt_cached_tokens, which no OpenAI-shaped client looks
  at. It is now also carried where OpenAI carries it: on Chat Completions and
  legacy Completions (buffered and in the stream_options.include_usage
  chunk), and as usage.input_tokens_details.cached_tokens on Responses. The
  three renderings come from one function, so a client that flips stream on
  and off sees one shape. prompt_tokens keeps its meaning, cached tokens
  included, as at OpenAI, so no caller's total moves. Anthropic Messages does
  not gain it: cache_read_input_tokens describes Anthropic's product with
  Anthropic's semantics, and that decision stays pinned by its own test.

• GET /metrics: Prometheus text exposition 0.0.4. The counters /health
  and /v1/runner/prefix-cache already answer, in the format a monitoring
  stack ingests without a translator: requests, prompt/generated/cached
  tokens, generation seconds, microbatch steps and sequences, the
  prefix-cache hit/miss/store/eviction/reuse counters and its byte and entry
  gauges, the speculation round/drafted/accepted counters, and the two RSS
  gauges. Names carry the runner_ prefix, each sample is preceded by its own
  # HELP and # TYPE, and a body that would not fit its buffer is refused
  rather than truncated - a half-written exposition reads to a scraper as
  metrics that reset. On whenever --serve is, no flag, answered from the
  accept thread with atomic loads only, and it does not count its own
  requests. Advertised as features.prometheus_metrics.

v0.4.7 - the sublayer-removal release

v0.4.7 is the sublayer-removal release: the first GGUF whose header says a block has no attention, the writer that produces it, and the loader that reads it. One feature, measured on a real 31B file, with the artifact published beside it.

Measured, not promised: every number below has a host, a date and a file behind it.

Sublayer removal

• --remove-sublayer attn:N[,mlp:M,...] with --quantize physically drops a block's attention (or dense-FFN) tensors and declares the absence with a 0 at that block in the per-block attention.head_count / head_count_kv (or feed_forward_length) array. That is llama.cpp's own convention for attention-free and FFN-free blocks (its Nemotron-51B "deci" graph), not a private key. The block's pre-norm stays.
• The loader omits the branch instead of failing on a missing tensor, reserves no KV rows for a removed attention, refuses a declaration whose tensors are still present, and keeps refusing an undeclared missing tensor. GPU offload, --lora, --train, the MTP head, hybrid families, E-series, MoE FFNs and fused-QKV exports are declined by name for now.
• The gate is an independent path through the full math: a removed file scores bit-identically to the parent with the block's output projection zeroed, and differs from the untouched parent. Byte accounting is exact and the KV cache halves on a two-block fixture.
• On the real Gemma 4 31B Q4_0 (docs/sublayer-removal.md): the attn:48 cut drops 74,319,872 bytes of tensor data, frees 64 MiB of KV cache at ctx 4,096 and 512 MiB at 32,768, scores bit-identically over 4,562 positions to the same cut as a zeroed-weights file, and passes the raw-protocol bar against its parent at mean KLD 0.0239 with margin-qualified top-1 99.25 percent. Stock llama.cpp b10076 refuses the file with missing tensor 'blk.48.attn_q.weight', the intended failure.
• The artifact is published: Joakimpalm-Zen/gemma-4-31B-it-attn48-removed-Q4_0-GGUF. It needs this release or later to load, CPU path.
• Gemma-4 E-series is a value, not a key: every gemma-4 export carries the two E-series keys, and the dense files publish them as 0. The writer now refuses on a non-zero value, as the loader always did.

Evidence in this release

• docs/compat-reports/cuda-smoke-0.4.7-2026-09-04-rtx3070.json: the mandatory CUDA smoke gate on real hardware.
• docs/compat-reports/0.4.7-2026-09-04-blackwell*.json: the compat matrix on the Blackwell, CPU path, executed checks on granite-4.1-8b, Qwen3-30B-A3B and gemma-4-E4B. The MoE row's CPU/CUDA identity did not execute this cycle: the MIG slice was shared with a running study and had 16.2 GB free against 19.3 GB requested, which the ledger now records as not_executed / insufficient_vram with the line rather than as a failure. Its tool scenario fails 6 of 8 exactly as in the 0.4.5 and 0.4.6 matrices, a known item.
• docs/sublayer-removal.md: design, prior art, gates, and the measurements on the real file.

Compat harness

• A CUDA side refused for a stated VRAM reason is not_executed with the reason, never a failed identity; cpu_cuda_check.py surfaces the backend log's error lines when the server dies during startup.
• A CPU-only matrix (--gpu off) runs the chat smoke on the CPU path even where the row pins auto, instead of waiting 300 seconds for VRAM and timing out without asking the question.

Runner is pre-1.0. APIs, model coverage and certification envelopes may change between releases.

v0.4.6 - the provenance release

v0.4.6 is the provenance release: signed, chained inference receipts with a one-exit-code verifier and OpenSSF Model Signing verification of the loaded GGUF. It also consumes NextN/MTP predictor heads for the first time, fixes a two-weight-passes-per-round inefficiency in every speculative source, makes batched CPU prefill 30 to 65 percent faster bit-identically, and fixes NVFP4 decoding.

Measured, not promised: every number below has a host, a date and a file behind it, and the negatives are recorded next to the wins.

Provenance

• Signed, chained receipts. --keygen makes an Ed25519 key, --sign-key signs every byte of a transcript before its ,"signature" key (chain hash included), --transcript-prev links receipts, and --verify checks signature, trust (--trust-key, --require-signed), link and replay under one exit code. Every forgery class is UNVERIFIABLE before the model is loaded.
• OpenSSF Model Signing at load. --model-sig, --model-pubkey and --require-signed-model verify an OMS bundle for the GGUF: ECDSA over the DSSE pre-authentication encoding (P-256/384/521), the in-toto statement, and the file digest against the manifest. Key method only; certificate and keyless bundles are refused as unsupported, never passed. The anchor is a bundle written by the reference signer, model_signing 1.1.1.
• The release gate now refuses a compat report in which nothing ran, and the 2026-09-02 Blackwell matrix ran every executable class on all 25 pinned files.

Speculation

• --mtp drafts from an MTP-preserved export's own predictor block (Qwen3.5/3.6 MTP GGUFs), through the existing target-exact verify walk, so output is token-identical to plain decoding. First-draft acceptance 75 to 94 percent on Qwen3.5-4B; 1.31x decode on code and 1.08x on prose at one draft per round on a 32-thread AVX-512 box with the int8 route. A 4-core AVX2 desktop decodes slower at every width: speculation only pays where decode is bandwidth-bound. CPU path only for now.
• The walk no longer pays a solo forward per round: the round-ending token rides as row 0 of the next verify batch. Applies to draft models and grammar drafts too.
• CPU batches of 2 to 7 rows take the solo step's native dot per column, so a verify row's logits are bit-identical to solo decoding and a 2-row verify costs one weight pass plus a dot.

Performance

• Batched CPU prefill: a 4x4 register tile over 16-column chunks, +64.6 percent on an M1, +31 to 37 percent on an AVX2 desktop, +43 to 57 percent on a Threadripper, bit-identical, decode unchanged. The 4x8 tile that was slower on ARM64 is recorded as a negative.
• --gpu auto charges KV at the price the allocation pays, ring-aware and shared-KV-aware, so a ringed model fully offloads where it used to stop two layers short.

Correctness and formats

• NVFP4 decodes correctly: the per-tensor scale companion is bound in the loader and applied at the dot seam. Before this the opening tokens of a real NVFP4 file were garbage.
• --prune-experts writes <arch>.expert_count_per_layer and the loader validates it against every router tensor, refusing a disagreeing header by name.
• --yarn-factor overrides a model's native YaRN factor at runtime, and --context-surgery compiles a context change into a byte-preserving GGUF.
• README claims re-checked one by one against the binary on three hosts; eight corrections landed.

Evidence in this release

• docs/compat-reports/cuda-smoke-0.4.6-2026-09-02-rtx3070.json: the mandatory CUDA smoke gate, 13 checks on real hardware.
• docs/compat-reports/0.4.6-2026-09-02-macos.json: the M1 compat report.
• docs/benchmarks.md: the 2026-09-02 head-to-head on three hosts against current llama.cpp builds. Prefill is the column.
• docs/performance.md: the prefill tile, the MTP round profile, and the negatives.

Runner is pre-1.0. APIs, model coverage and certification envelopes may change between releases.

v0.4.5

v0.4.5: the dispatch-budget release

A loaner 128 GB M5 Max spent two days in the lab, and this release ships what it paid for: the Metal decode path was profiled to the microsecond class, then cut from 686 kernel dispatches per token to about 330.

Metal decode, faster and still exact

• -52% dispatches per token, +5-6% decode on gpt-oss-120b and Qwen3-30B-A3B, byte-identical. Three fusion phases: budget-line fusions, an attention-front megakernel (norm + Q/K/V + qk-norm + rope + KV store as one dispatch), then a widened front with the post-FFN add folded into the MoE expert sum. Gated across a six-architecture roster in make test. RUNNER_METAL_FUSE=0 restores the unfused path.
• Grouped simdgroup-MMA MoE prefill is the new default: +31% prefill on Qwen3-30B-A3B, +21% on gpt-oss-120b, decode untouched. Top-k routing flips near-ties under any reassociation, so this path answers to the published fidelity bar rather than byte identity; RUNNER_METAL_MOE_MM=0 restores the matvec path and is what every byte-identity gate pins.
• --parallel slots decode as one microbatch on Metal: 1.45-1.47x aggregate decode at 4-8 slots, bit-identical to sequential by twin-kernel construction, mutation-proven in make test.
• The KV ring now runs on Metal (v0.4.4 shipped it with a Metal refusal): bit-identical teacher-forced logprobs across many ring wraps.
• Speculative decoding works on a fully offloaded Metal target now that unified memory keeps the verify walk host-readable. Measure before relying on it: the measured 70B+1B pair decoded slower speculative than plain, and the release notes say why.

Correctness stories this release forced

• The fusion byte-identity contract was resting on compiler luck, and the release gate on an M1 caught it. Under fast math, cos/sin inline as polynomial chains whose fma contraction the compiler picks per call site: the M5 contracted the rope kernels alike, the M1 did not, and fused decode diverged from unfused at ULP scale. Rope trig is now one shared precise:: helper with contraction pinned off, so identity holds by construction on every device. Only running the gates on a second device family made this visible.
• gemma-4-26B-A4B emitted only token id 0 on Metal: the routed-expert GELU kernel was missing the overflow clamp its dense twin already carried. Fixed; 7 of 8 realistic prompts are now CPU/Metal byte-identical, against 0 of 8 before. make test-metal-bigmodel BIGMODEL=<path.gguf> exists because no tiny fixture could have found this.

Big-model coverage

• Sharded GGUF sets take a full Metal offload (a real 2-part 86 GB set measured byte-identical to the merged single file), compact-metadata shards load, and Q2_K/Q3_K expert kernels round out the low-bit roster. Together that took Qwen3-235B-A22B to a full 94-layer Metal offload, validated with its caveat stated.
• Four big-model Metal validations with committed evidence: gpt-oss-120b fully resident (64.6 tok/s sustained decode, zero swap, anchored against same-host llama.cpp with the remaining prefill gap named), Llama-3.3-70B, Qwen3-30B-A3B, gemma-4-31B.
• RUNNER_TIEDV derives tied K rows from stored V on gemma-4 (31B at 32k context: K cache 14.76 GB to 13.42 GB), a compute-for-memory trade that is explicitly not byte-identical and refuses every configuration it cannot keep honest.

Negatives that ship as negatives

RUNNER_PREFETCH is off by default because hinting the OS made the cold 63 GB load 60% slower on macOS. The -b default now reads total RAM instead of free RAM so the same command produces the same tokens on a busy day. The Metal 4 tensor GEMM stays opt-in at measured parity. Expert-major MoE prefill measured slower and ships off by default, writeup included.

Gates for this release: full make test green on macOS and in CI (9 jobs), conformance 418 passed, CUDA smoke PASS on an RTX 3070 with the report committed, release-check clean.

Full details in CHANGELOG.md.

v0.4.4

• Sliding-window layers were allocated KV they can never read, and now there
  is a way to stop paying for it. Every layer gets n_ctx cache rows, but a
  sliding layer clamps its attention start to p - swa_window + 1, so rows
  older than the window are written once and never read again. On models whose
  sliding layers are both more numerous and wider in KV than their full ones
  that is most of the cache: gemma-3-4b at -c 32768 allocates 4563 MB and can
  reach 793 MB of it, and gemma-4-E4B 1879 MB against 558 MB. -v now reports
  the reachable figure beside the allocated one, and RUNNER_KV_RING=1 gives
  those layers only the rows they can read, indexed modulo that count: 4563 MB
  becomes 800 MB, within 1% of the floor. This is a ceiling rather than a
  correctness bug -- answers were always right, the cache was simply larger
  than the model could use -- and it is the difference between a 4k context and
  a 32k one on a 24 GB device.
  The gate is the flat allocation itself. A ring holds exactly the rows the
  flat layout would have been read from, so the shipped unringed engine is the
  reference implementation: verified bit-identical, max |Δlogprob| exactly 0,
  on a CPU fixture and on an RTX 3070 at both a partial split (20 of 34 layers)
  and a full offload, 2121 scored positions each.
  It is opt-in because it costs something. The prefix cache and partial
  rewind address KV as flat absolute rows, so both are refused while a ring is
  active and a server loses shared-prompt reuse. Metal's attention kernels
  address the cache by absolute position too and are not ring-aware, so a Metal
  build refuses the ring with a message rather than returning wrong numbers.
• The KV cache is addressed as flat absolute rows in more places than anyone
  had written down, and the count was wrong twice. An external research
  branch named three host sites -- pfx_save/pfx_load, engine_rewind,
  kv_upload -- after hitting each one as a separate crash, including a
  prefix-cache overrun that was a memory-safety bug no measurement in that
  branch could see, because a one-shot -p never touches the prefix cache.
  Implementing the ring found a fourth host site (kv_copyback, identical
  shape, identical blind spot) and then a whole category the list omitted: the
  CUDA attention kernels themselves, seven KV addresses plus both store
  kernels, all indexing by absolute position. A first cut that fixed only the
  host mirrors produced nan for every scored position on a partial GPU split
  while the same build was bit-identical on the CPU path. Every device KV
  address now resolves through one kv_slot() helper, attn_args carries the
  ring, and the embedded PTX is regenerated. The canonical comment lives at
  model_kv_byte_off and says the list is a starting point rather than a proof
  of completeness, which is what it turned out to be.
• --score can now check itself. It reported plausible-looking numbers for
  models it could not score, with a correct n_scored and exit 0: every
  validity assertion passed, because they test that the arm RAN, not that the
  number is right. The response now carries n_vocab and the absolute
  next-token top1/top1_rate, which are bounded by facts outside this
  implementation -- a token with probability above 0.5 must be the argmax, and
  an argmax token must carry at least 1/n_vocab -- so a harness can bracket
  the reported count from the reported logprobs and refuse a run that disagrees
  with itself. It immediately earned its place: an anomaly that looked absent
  on short factual English (top1_rate 0.707) reappeared at 0.136 on a mixed
  corpus where four control models held 0.32-0.38, which nll_mean alone could
  not have separated from corpus difficulty.
• A refused draft no longer exits 0 in the local CLI. A draft is dropped on
  a vocabulary mismatch, a fully offloaded target, an unsupported file or out
  of memory, and the run continues without it. That default is deliberate and
  unchanged, and serve mode already reports draft.active over
  /v1/capabilities. One-shot and interactive chat had no such channel: the
  drop was a stderr line beside a successful exit, so automation collecting
  stdout recorded the unaccelerated baseline and labelled it speculative
  decoding. --draft-required fails the run instead. It requires --draft,
  and it is refused in serve mode rather than accepted with no effect, because
  a guard against silent no-ops that was itself a silent no-op would be the
  failure it exists to prevent.
• Qwen3 could not reason before calling a tool. The constrained grammar
  admitted <tool_call> but not <think>, so a thinking model asked for a
  tool had its reasoning channel closed by the schema. The discriminator now
  covers both openings, and atem_seq_add enforces its own capacity rather
  than trusting its caller.
• The polish register lands: RI-2 through RI-6. Capabilities report the
  EFFECTIVE execution mode rather than the configured one, and qualify
  request_telemetry per surface instead of advertising it flatly; a declined
  type-plan rule names the tensor and type it declined rather than an aggregate
  count; every refused sampling parameter names itself; and --help is an
  answer on stdout rather than a diagnostic on stderr. Two of the five register
  items turned out to have false premises on inspection and were split rather
  than implemented as written -- one contradicted a recorded owner deferral,
  the other a tested contract.
• Measured Shade findings land in the engine. The default thread count
  gains a ceiling of 32 (machines above 64 logical CPUs were spawning more
  threads than the work could use), and the q4_0 half of the signed-weight dot
  is reverted while the q8_0 half stays -- the q8_0 form is a
  cross-microarchitecture reproducibility fix, the q4_0 form was not.
• Smaller correctness and hardening. The CLI rejects chat-only flags
  (--system, --think, --no-think) in modes that would silently discard
  them; the server advertises every public route at startup and hardens its
  prefix-cache management routes; the Windows tray refuses a spawn whose
  command line does not fit rather than truncating it; expert-dimension
  iteration in the quantizer is unsigned; the build preflights its Python
  test dependencies instead of failing halfway through a suite; ring row
  sizing saturates instead of overflowing signed arithmetic at the int
  boundary; the GPU backend identity survives a release build's
  command-line CFLAGS, which would otherwise have compiled the Metal
  ring refusal out of the shipped binary; and compat_matrix.py resolves
  the runner path before probing its version, since Path("./runner")
  stringifies to runner, is not on PATH, and silently recorded a null
  version in the committed evidence.
• NVFP4: the gate could not have caught the bug, and now says so. v0.4.2
  claimed a decode gate that was a transcription of the implementation, so it
  proved the implementation agreed with itself. The changelog claim is
  corrected, the limitation is recorded as a test, and scripts/nvfp4-probe.py
  supplies the external anchor the unit test lacked: it validates the format
  against properties the file must have, needing no reference decoder. The
  probe also corrected its own first reading -- a large decoded standard
  deviation is NOT a decode error, because the per-tensor scale is applied in
  the compute graph rather than by the block decode.
• AGENTS.md gains the rule these releases keep re-learning: every gate
  needs at least one assertion whose expected value comes from outside the
  system under test. A green gate with no external anchor is evidence the
  system is self-consistent, and nothing more.

v0.4.3

The hardware-gate release. One real bug, and the gate that would have
caught it.

Every display-attached NVIDIA GPU was reported as unified memory.
v0.4.2's integrated-device probe queried CUDA driver attribute 17. That
is CU_DEVICE_ATTRIBUTE_KERNEL_EXEC_TIMEOUT, which reads 1 on any GPU
driving a display; CU_DEVICE_ATTRIBUTE_INTEGRATED is 18. So an
ordinary discrete desktop card answered unified_memory: true, printed
the load-time unified-memory notice, and had its GPU offload budget
clamped to OS-available RAM. Where free system RAM sits below VRAM,
that silently cuts the budget on hardware that has its own dedicated
pool. If you ran v0.4.2 on a desktop NVIDIA card, take this release.

Measured on an RTX 3070 with a monitor attached, against a
known-correct anchor so the numbering could not be assumed: attribute
16 MULTIPROCESSOR_COUNT reads 46, which is a 3070's SM count;
attribute 17 reads 1; attribute 18 reads 0.

The off-by-one is quiet in the worst possible direction. A headless
datacenter card reads 17 as 0 and looks correct, so the machine you
would naturally reach for to test CUDA is precisely the machine that
passes it. It takes a desktop with a display attached to fail.

Releases now pass a CUDA smoke gate on real hardware. GitHub's
runners have no GPU, so src/cuda.c is compiled on three platforms and
executed by none. That is how the above shipped, and v0.4.2's own
changelog said as much: the unified-memory path was reasoned and
reviewed but never run. scripts/cuda-smoke.py now runs against a live
device before a release is tagged, and its report is committed
alongside the compatibility report.

Its assertions are invariants rather than expected constants, so one
script is valid on a discrete desktop card, a headless datacenter card,
and a unified-memory device like a DGX Spark. The central check is
coherence: unified_memory is a claim that VRAM and system RAM are one
pool, so it has to agree with the sizes actually reported, and a device
claiming unified memory while its VRAM is plainly a separate pool fails
no matter which attribute number the driver was asked for. It also
cross-checks the load-time notice against --caps, because those come
from different call sites and disagreeing is itself a defect. It was
verified red against the v0.4.2 binary before being trusted green.

Nothing else changed. Everything in v0.4.2 still applies, including the
Qwen native tool protocol, the request-surface validation wave, and the
NVFP4 CPU support. Unified memory on genuinely integrated hardware
remains unconfirmed on a real device: the fix is now known correct on a
discrete card, and a Spark or Jetson report is still what would close
it.

Known issue: NVFP4 decoding is UNVERIFIED

Added 2026-08-30, after this release was published.

NVFP4 CPU support shipped in v0.4.2 and --caps advertises it under quants.
A field report on a DGX Spark shows a Qwen3.8-27B NVFP4 model loading
cleanly and then decoding a single repeated token under greedy sampling.
That is the signature of correct shapes and wrong values, and NVFP4 is the
only new variable in that configuration.

Being direct about why our tests did not catch it: the reference decode this
format is checked against is a transcription of the implementation, so it
proves the implementation agrees with itself. It cannot detect a wrong
element order or an ignored per-tensor scale, because both sides encode the
same reading of the format. We shipped NVFP4 from a specification with no
model of that type available to test against.

If you have an NVFP4 model, do not trust runner's output for it yet.
Every other quantization type is anchored by models that demonstrably serve
correctly; NVFP4 is not. The root cause is being investigated with the
reporter, and this note comes out when an external anchor exists.

No other format is affected, and nothing about Spark or NVFP4 has been
promised as supported.

Full details in the CHANGELOG.

v0.4.2

The no-silent-defaults release. Two review passes went through the
three chat surfaces asking one question of each field: what happens
when a request is wrong? Too often the answer was "it is repaired and
answered 200" - a missing field defaulted, an unrenderable turn
dropped, an invalid ordering rewritten, a control validated and then
never read. A prompt that does not say what the caller submitted is
the failure mode nothing downstream can detect, because the response
looks like a success. More than twenty of those are now HTTP 400s that
name the field. Qwen2.5 and Qwen3 also gain native tool calling here,
and the first external hardware report (a DGX Spark) brought NVFP4
support on the CPU and honest unified-memory reporting.

Qwen speaks its own tool protocol. Declarations render into the
family's trained # Tools / <tools> block, a call comes back as its
<tool_call>{"name":...,"arguments":...}</tool_call> turn, results
replay as grouped <tool_response> blocks, and tool names and
argument schemas are constrained directly in that native grammar
instead of runner's generic JSON envelope. Buffered and streaming
output map back to the OpenAI shape, and the same conversation renders
byte-identically on /v1/chat/completions, /v1/responses and
/v1/messages. Qwen3 history also keeps the reference template's
empty <think> block before a trailing historical assistant answer,
which runner did not render at all.

Histories are validated before anything is rendered. Every chat
turn must be an object with an explicit role and usable content: a
turn with no role used to be rendered as user, and a turn whose
content was neither string nor array was dropped from the prompt
outright. Replayed tool_calls must carry a non-empty id, a
function type, a non-empty name, and arguments that parse as a JSON
object. Role sequences no template can represent (a system turn after
history has started on llama2/gemma/mistral/apertus/ornith, or broken
user/assistant alternation) are refused by one shared check on all
three surfaces rather than rendered into something the reference
template would never produce. Responses input items and Anthropic
tool_use/tool_result blocks get the same treatment, including a
required function_call_output.output: an absent member and an empty
string used to collapse to the same empty tool turn, so broken agent
history looked accepted while the model was handed an event that never
happened. A tool result that names no call is now refused on every
family, not just Harmony, instead of being rendered under the
template's 'unknown' fallback or named after its tool_call_id - a
function name invented from an identifier and declared nowhere.

Runner is text-only, and now says so. Image, file and other
non-text content parts are refused on Chat Completions, Responses and
Anthropic tool_result content, rather than being removed while the
adjacent text is answered successfully.

Eight more fields were accepted and then ignored. seed: 0 asked
for a reproducible run and did not get one, because the sampler's
xorshift64 has a fixed point at state 0 and only a seed above zero was
adopted. Anthropic thinking.type was validated and never read, so
{"type":"disabled"} rendered exactly like a request that said
nothing. The request timeout was handed only to the decode loop, so a
long enough prompt overran its own bound by the whole prefill and then
reported "context overflow" instead of a timeout; it is polled at
every prefill chunk now and expiry there answers 408, naming the
prompt. Replayed Anthropic reasoning was dropped unconditionally while
chat replayed Harmony reasoning from the same conversation, so one
model described two ways gave two different prompts. Wrong-typed
tools[].type was normalized to "function" on all three surfaces. A
tool_result after text in the same Anthropic message was silently
reordered, answering 200 for a conversation that had been rewritten.
And usage.output_tokens_details.reasoning_tokens was hard-coded to
zero; it is counted per token now, which is also what makes an
unenforced thinking budget visible to the caller.

Six latent defects, none reachable on this tree. Each is a hole
the next change falls into: the scheduler not paying back its wait
count on the stop path; Metal's two hand-kept pipeline lists asserting
11 of the 16 types the loader admits, while the encoders index those
tables with no nil check (the check now runs off the admission
predicates themselves); a NUL byte in a request header hiding the real
terminator from every parse below it; outside_reason adding a
would-have-written count to a length; a failed strdup publishing
instance records with NULL model names; and json_escape writing its
terminator under cap 0.

The accept thread no longer drains requests. /health,
/v1/models, /v1/capabilities and /unload are answered from the
accept loop, which used to consume the request there under a 0.5 s
budget so that closing would not reset the connection and discard its
own reply. It now hands any request it cannot prove bodyless to a
slot, whose bounded reader consumes the whole request before
replying, so the only thread calling accept() never waits on a
client. Allocation failures inside the tool envelope and the Harmony
prompt renderer are also 500s now rather than being reported to the
caller as a 400, and /health can no longer identify a resident model
by a truncated name that /v1/models spells in full.

The Python consumer boundary. ManagedRunner.start() accepts only
a /v1/capabilities whose new pid field matches the child it
spawned, so a pre-existing Runner already on the port cannot be
mistaken for a successful startup. Startup leases treat unreaped
zombie owners as stale claims. And cancel_event now interrupts a
silent blocking stream read instead of being observed only between SSE
events.

First external hardware report: DGX Spark GB10. The arm64 plus
CUDA combination shipped untested and works. Two things it found:
NVFP4 (ggml type 40, NVIDIA's block-scaled FP4) refused with
"unsupported type 40 (?)" and now dequantizes and serves on the CPU
path, gated against an independent double-precision reference decode,
with CUDA and Metal declining it by name so --caps advertises it
under quants and not gpu_quants. And --caps reported
unified_memory: false on a machine that is nothing but, because the
field was a compile-time constant; it is queried from the device now,
and on an integrated device the GPU offload budget is capped at
OS-available RAM, because "VRAM free" and "RAM free" are two views of
one pool and treating them as two budgets over-promises.

What that last item has and has not been verified against. The
NVFP4 decode is gated by an automated test and runs on every platform.
The CUDA-side changes are covered by no automated test, because CI has
no GPU. The integrated-device probe and the unified-pool budget cap in
particular have not executed on integrated hardware at all: the field
report that prompted them was made against v0.4.1, so the fix has not
been back-confirmed on the machine that found the bug. Reasoned and
reviewed, not yet run. If you are on a Spark, a Jetson or any other
unified-memory device, this release wants your report rather than
claiming it already has it.

Full details in the CHANGELOG.

v0.4.1

The parser-agreement release. A deep review of the constrained-output
validators found five ways generation could emit a document that
Runner's own parser - and therefore its tool-argument readback -
refuses. All five are fixed by making the validators and json_parse
share their definitions outright, and the review left behind three
standing gates that test the machinery differentially instead of
trusting it. If you use schema-constrained output, tool calling, or
JSON mode, this release is worth taking: each of these could turn an
HTTP 200 into arguments the caller cannot read.

• Numbers: the validators completed spellings like 9e999 or deep
  subnormals that json_parse refuses (strtod's ERANGE, both
  directions). The old overflow check used isfinite(), which the
  release build's -ffast-math folds away - dead code in every shipped
  binary. One shared predicate (json_number_text_ok) now decides
  number acceptance for parser and validators alike, and the refusal
  lands on the exponent digit that commits the spelling, so terminating
  the number always stays legal. Free-object subtrees get the same rule.

• Strings: JSON mode took any byte >= 0x20 as string content, so
  ill-formed UTF-8 (lone continuation bytes, overlong leads, 0xF5..)
  flowed into documents json_parse rejects - in plain --json output
  and in every free-object subtree of a tool schema. Both validators
  now share one sequence classifier, and the closer finishes a
  truncated scalar before writing its closing quote.

• Duplicate keys: json_parse refuses a repeated object key AFTER
  unescaping. JSON mode accepted duplicates outright; the schema map
  guard caught only same-spelling repeats ("a" versus "a"
  slipped through); and both force-closers could complete a truncated
  key INTO a duplicate. One guard now serves both validators, hashing
  decoded key content - raw bytes, decoded escapes, surrogate pairs as
  their scalar - and the closers extend a force-closed key until it is
  unique. The guard's capacity fails closed: instead of silently
  un-tracking keys past its limit (which let a 17th key duplicate an
  earlier one unchecked), the comma that would start an untrackable
  entry is refused while } stays legal - objects are bounded at 24
  tracked keys, never wedged, never unchecked.

• The closer's invented minimum: exclusiveMinimum: 0 compiles to
  a clamped edge of 4.94e-324 - the smallest subnormal, in bounds by
  construction, refused at read-back. Force-closing a bounded number or
  a minItems fill used to emit it verbatim; the fill now scans for the
  smallest spelling that both parses and satisfies the bounds.

• /health metrics honesty: on Linux, current RSS and peak RSS
  come from different kernel counters whose accounting can lag each
  other, so the endpoint could report a peak below the present - in the
  number a supervisor budgets machines with. The pair is now clamped
  self-consistent at the moment it is reported.

The instrumentation that found all of this ships with the release and
runs on every push: a seeded differential walker inside make test
that probes all 256 bytes at every step of walks through legal document
space (state-comparing, against a poisoned scratch), a differential
libFuzzer target in CI, and a new conformance-sanitized CI job that
drives the full conformance suite against the ASan+UBSan binary -
added because a plain build proved unable to surface undefined
behavior, and it caught the /health bug on its very first run.

One platform note, recorded honestly: strtod's ERANGE behavior in the
subnormal band is libc-specific (glibc and macOS refuse 1e-320,
Windows UCRT accepts it), and json_parse inherits the host's verdict.
Validator and parser always agree on any given host; cross-platform
acceptance of subnormal-edge numbers can differ, and the determinism
documentation carries that scope.

Full details in the CHANGELOG.

v0.4.0

The receipts release. A one-shot run can now be recorded as a
replay-verifiable transcript and checked later on any machine that has
the model: --transcript writes the receipt, --verify replays it and
diffs, and the demo that matters is that a forged token with a correctly
recomputed chain hash is still caught - you can forge the hash, not the
model. Around that headline sits an honesty wave: the API now refuses
prompt controls it cannot honor instead of returning 200 with the data
missing, quantize can no longer report success over a file it did not
change, constrained output always parses and matches its own schema, and
four more hostile-GGUF gaps closed on the load path. Metal picked up
routed-MoE prompt batching, gated byte-exact against the serial path.

• --transcript F (notarized inference): records model and binary
  sha256, verification profile, full config and exact seed, prompt and
  output token ids, exact output bytes, and a chain hash covering every
  byte of the file before the ,"chain" key - recomputable with a text
  editor and sha256sum. Same binary + same record replays bit-exact;
  cross-ISA replays token-exact. The record survives PATH invocation
  (the running image is hashed, not argv[0]), preserves every accepted
  RNG seed exactly (seed 2^53+1 no longer rounds through a double),
  records the run that actually happened (effective KV type, batch,
  workers, GPU layers), and always emits valid JSON, hostile device
  strings included.

• --verify F: three verdicts, three exit codes - VERIFIED (0),
  DIVERGED at token N (2), UNVERIFIABLE (3: altered record, wrong model
  sha, wrong adapter). The record's profile and sampling config drive
  the replay, overriding CLI flags, and every recorded value passes the
  CLI's own validation before it reaches inference.

• The API never ignores a prompt control: legacy Completions now
  rejects echo:true and non-null prompt_logprobs with a
  parameter-naming 400 until Runner can return the requested prompt-side
  output; neutral SDK forms (echo:false, prompt_logprobs:null) still
  pass. Previously a scoring client got HTTP 200 and silently aligned
  against a response missing the data it asked for.

• Quantize honesty: a --type-plan whose container is malformed no
  longer loads as "keep everything" and exits 0 over an unchanged file;
  a plan selecting nothing is refused by name; same-width declines are
  reported; --merge-lora refuses adapter tensor names that would land
  two tensors in one slot; quantizing from a split GGUF no longer emits
  a single file that declares itself multi-part (unreadable everywhere,
  ours and llama.cpp alike); outputs are forced to stable storage before
  the rename that publishes them.

• Constrained output agrees with its schema: integers can no longer
  carry leading zeros past the grammar, minimum/exclusiveMinimum
  intersect instead of replacing each other, closer fills stop short of
  eating the closing bracket, UTF-8 string lengths count consistently
  and never strand continuation bytes, duplicate keys in free-keyed
  objects are refused, and required whitespace is maskable no more.

• Hostile-GGUF load path: the special-token sort is no longer
  quadratic in a count the file controls; Gated DeltaNet dimensions get
  the range ceilings the sibling gates always had; an
  expert_shared_feed_forward_length of 0x80000000 no longer silently
  drops the shared expert; refusal messages print bounded printable
  bytes, never raw attacker strings; embedded NULs in token arrays are
  rejected.

• Recurrent models never reuse a truncated prefix snapshot: the
  prefix cache clamp paired shortened attention rows with a fold state
  from the full prompt, so qwen35 / granite-hybrid / nemotron-h servers
  under memory pressure could return wrong tokens with nothing on the
  wire saying so. The clamp now drops the entry; storing nothing costs a
  prefill.

• Metal MoE prompt batching: routed-MoE prefill batches
  normalization, routing and expert-slot work across the tile
  (MoE-class dispatches 12,528 -> 288 on a real routed MoE), output
  tokens byte-exact and expert routes bit-identical to the serial path,
  gated. RUNNER_METAL_MOE_BATCH=0 restores the serial path.

• Python client: unverifiable startup leases age out after a TTL
  instead of deadlocking or being stolen (and the lease probe pins
  locale and timezone, so a Stockholm shell no longer reads a launchd
  owner as a pid reuse); stream timeouts split first-byte from stall on
  content, so a big prompt's prefill silence no longer trips the stall
  window; plus an explicitly opt-in, notify-only update check.

• Published-claim scripts fail loudly: idle_coexistence now stamps
  time-to-first-token on the first content frame and measures the
  process tree; competitor-freshness exits nonzero when it compared
  nothing; the release gate's stale-version scan matches again after
  going vacuous at v0.2.0.

Full details in the CHANGELOG.