hipfire v0.4.2

hipfire v0.4.2

Released 2026-10-14. The previous release is v0.4.1.1 (a patch release, tagged 2026-10-08), which followed v0.4.1 (2026-10-07). The full change list is the v0.4.2 section of CHANGELOG.md.

Every number below comes from an individual change-level campaign or the NPU release gate, not from a cumulative release matrix; each row names its card, fixture and date. Synthetic pp8192 is not prompt-file prefill, and cold, first-after-ready and steady cache states differ. Multi-GPU numbers name the card count.

Terms used in these notes

• Flash-Next (FN): Qwen3.8-Flash-Next / Qwen4, normally shipped as gptq3 with q4 PLE where stated. Results for the MQ4-trunk candidate are reported separately.
• 27B: Qwen3.8-27B mq4-xts, with qwen3.8-27b-dflash.mq4v2 as the draft in batching campaigns unless specified.
• Halo is Strix Halo (gfx1151). R9700 is the Radeon AI PRO R9700 (gfx1201).
• VMM: virtual memory management, the memory mode that batching serves use (serve.vmm_batch) to reserve lane state, checkpoints and KV space.
• PLE: per-layer embedding, the large embedding table that Flash-Next stores as q8 or q4 sidecar files.
• C1, C4, C8: concurrency (C) of 1, 4 and 8 simultaneous requests in a benchmark. C1 is a lone request.
• Width (L), L8 and L16: the DFlash2 draft proposal width L, the number of tokens proposed per step. L8 is width 8 and L16 is width 16; these notes write them as "width 8" and "width 16".
• IU4: the DFlash verify route that uses 4-bit verify activations, produced by the A4 producer and consumed by the IU4 GEMM, in place of the F16 verify.
• AR: ordinary autoregressive decoding, with no speculation. MTP: the native multi-token-prediction speculative head. DFlash: speculative decoding with a draft model.
• TTFT: time to first token. PM: PeaceMaker, the native kernel path. EP: the multi-GPU mode (topology.ep) in which every rank holds the trunk and an expert shard.
• CS95: the reported uncertainty interval from predict ab for a measured change.

---

Headline numbers

Change	Hardware and fixture	Result	Source
Flash-Next 16K prefill with the NPU trunk on (v0.4.2 tag)	Strix Halo + XDNA2, NPU trunk on, bounded memlock (LimitMEMLOCK=1318060032), 16,355-token prompt, client TTFT, 3 fresh processes × 3 requests	2,386 tok/s (all-9 median TTFT 6,854 ms); steady requests 6,849.5 ms; greedy p16k byte-identical to the gate build; −3.4 % TTFT vs the gate build 1be80ed54, measured interleaved	Shipped-tip measurement (j327), 2026-10-14
NPU trunk auto vs off	Strix Halo + XDNA2, default serve config, p16k (16,355 tokens), 3 fresh processes per arm, median of per-process medians	TTFT 7,878 → 7,085 ms (−10.1 %); p56k 30,047 → 27,260 ms (−9.3 %)	CHANGELOG, NPU trunk entry, 2026-10-13
27B DFlash exact 16-row verify GEMMs (HIPFIRE_GFX1151_XLDS)	Strix Halo, 27B XTS + paired DFlash draft, merge_sort_thinking_off.txt / prose_river_short.txt, 3 fresh processes per arm	code 97.9 → 135.3 tok/s (+38.2 %); prose 19.0 → 26.3 tok/s (+38.4 %); AR unchanged	CHANGELOG, gfx1151 XLDS entry, 2026-10-13
Flash-Next EP2 owner-routed prefill + expert staging (opt-in EP)	2× R9700, pinned Flash-Next XTS, committed prompts, 3 fresh processes per arm	p8k / p16k: one card 3,033 / 3,754 → EP2 3,943 / 4,308 tok/s (+30.0 % / +14.7 %)	CHANGELOG, EP owner-routed prefill entry, 2026-10-13
27B paired DFlash by default	One R9700, mq4-xts, singleton serve, C1 sampled, effort xhigh, 4,096-token cap, 3 fresh processes per arm	44.00 → 74.49 tok/s (+69.3 %) vs the MTP auto kill-switch configuration	CHANGELOG, 27B DFlash default entry, 2026-10-12
DFlash width 8 on batching serves	One R9700, 27B mq4-xts + MQ4V2 draft, conc_steady 64 code prompts × 256, C8 steady greedy	254.6 → 324.6 tok/s (+27.5 %); C1 114.8 → 107.8 (−6.1 %)	CHANGELOG, speculation.dflash_block entry, 2026-10-11
IU4 verify at width 8	Same R9700 fixture, C8 steady greedy, 8 fresh processes per arm	340.1 → 374.5 tok/s (+9.9 %); C1 −7.7 %	CHANGELOG, DFlash IU4 verify entry, 2026-10-11
q4 PLE on Halo	Strix Halo, Flash-Next, Ciru 8K steady after the default prewarm, one run per cell	1,828.4 tok/s (q8 about 1,599)	CHANGELOG, PLE sidecar entry, 2026-10-11
Deep appends reuse the prompt	Strix Halo, Flash-Next, +512 tokens onto a 192K / 256K prompt	9.3 → 141.8 / 4.7 → 197.5 tok/s	CHANGELOG, deep-append entry, 2026-10-11
gfx1100 Qwen3.5-4B decode fusions (#845)	RX 7900 XTX, Qwen3.5-4B	decode +5.5 % (MQ4), +3.0 % (MQ3), bit-identical	CHANGELOG, #845 entry, 2026-10-13

Highlights

• Strix Halo runs Flash-Next trunk input projections on the XDNA2 NPU by default. fusions.npu_trunk = auto arms only with an open NPU and matching keyed calibration, otherwise logs one reason and runs GPU-only; other architectures are untouched. On the shipped tip, 16K prefill runs at 2,386 tok/s (all-9 median TTFT 6,854 ms; 16,355 tokens, client TTFT, 3 fresh processes × 3 requests; −3.4 % TTFT vs the gate build 1be80ed54, measured interleaved), and auto vs off cuts p16k TTFT by 10.1 %. Greedy on equals the bit-exact emulate twin. The trunk pins about 1.05 GB, so the host needs a bounded memlock limit (see Known issues). HIPFIRE_QWEN4_NPU_TRUNK=off is the kill switch.
• 27B DFlash on Strix Halo decodes about 38 % faster with exact 16-row verify GEMMs (HIPFIRE_GFX1151_XLDS, default on for exact gfx1151, 0 is the kill switch); decoded text is byte-identical to beta.
• Flash-Next zab trunk-A4 tier ships as a .trunk-overlay sidecar applied by default on gfx1151 (HIPFIRE_QWEN4_TRUNK_OVERLAY, 0 restores the stock weights). hipfire pull fetches and verifies it next to the stock base; quality is within the approved +0.015 KLD cap and tied on HumanEval, GSM8K and MMLU-Pro. Trunk A4 now also arms on ragged prefill chunks.
• Flash-Next can run past its native 262,144-token context with engine-level YaRN RoPE scaling (#856).
• Qwen3.8-27B uses its paired DFlash draft by default. When the registry declares a paired draft for the tier and the draft file is present, DFlash turns on with no configuration. On one R9700 (canonical mq4-xts, singleton serve, sampled, three fresh processes per arm, both arms on the same binary), C1 decode rose from 44.00 to 74.49 tok/s at reasoning effort xhigh (4,096-token cap) and from 52.91 to 116.65 tok/s at medium. Without the draft file the load keeps MTP auto. HIPFIRE_DFLASH_MODE=off or hipfire config qwen3.8:27b set dflash_mode off turns it off.
• Qwen3.8 and Ornith 1.5 reason at medium effort by default. When no effort is set, the registry cards now supply medium instead of xhigh, so default thinking is shorter. Explicit config, CLI or request values still win; set reasoning_effort = "xhigh" to get the previous behaviour back.
• 27B DFlash is faster under concurrency on batching serves. On one R9700, the new automatic per-load width of 8 raises steady greedy C8 throughput from 254.6 to 324.6 tok/s (+27.5%), at a lone-request cost of −6.1% at C1 (114.8 → 107.8 tok/s); width 16 is restored with HIPFIRE_DFLASH_BLOCK=16.
• IU4 verify adds a separate concurrency gain. In its own campaign on one R9700 at width 8, IU4 raised steady greedy C8 throughput from 340.1 to 374.5 tok/s (+9.9%) against the F16 verify, with an additional C1 cost of −7.7%; HIPFIRE_CB_VERIFY_IU4=0 turns it off. This campaign is separate from the width campaign, so the two gains are not chained. Sampled lanes keep their singleton speculation mode, and shared draft, verify, target-preparation, rollback and upload work across lanes preserves the admitted singleton arithmetic.
• Halo Flash-Next prefill is faster with q4 PLE. On Halo, in a one-run-per-cell campaign, the Ciru 8K steady rate after the default prewarm is 1,828.4 tok/s with q4 PLE, against about 1,599 tok/s with q8. The prewarm budget is bounded, and first-after-ready remains below steady under memory reclaim.
• Halo Flash-Next lanes run faster with exact PeaceMaker rows. In staged lanes, which are opt-in, the default Halo PM lane rows raise gptq3/q4 VMM C8 conc_steady throughput (64 prompts × 256 tokens, same-binary interleaved fresh processes) by +21.6% on Halo with identical text; TTFT p50 is unresolved for this change.
• Flash-Next expert parallelism (opt-in, experimental) gains native MTP and faster prefill. With EP on four R9700s (FN gptq3, three interleaved fresh processes per arm), AR tg128 reaches 61.2 tok/s against 36.1 on one card (1.70×), but 42-token TTFT is slower at 408 ms against 286 ms. On two R9700s, owner-routed prefill raises p8k / p16k prefill from 3,033 / 3,754 to 3,943 / 4,308 tok/s, and native MTP now runs under EP when requested.
• Serving, toolchain and developer features: Messages and Responses API adapters, repaired startup and CLI behavior, ROCm 10.1 kernel packs, embedded PM decode and closed-route kernel preload ship in serve and the toolchain. Qwen4 developer profiles provide synthetic dense execution and tiny-artifact production, not upstream Qwen4-27B compatibility. Railgun and the shared-weight host remain developer-only.

Feature modes and defaults

Exact kernel replacements do not make q4 PLE, a changed width or IU4/FP8 numerics identical to the previous release. The tables below are scoped: not every row is an inherited default, and a default that applies only inside an opt-in feature is listed with that feature.

Defaults changed in 0.4.2

Feature	Applies to	What changes	Kill switch
Reasoning effort	Qwen3.8 (all qwen3.8:27b* tiers) and Ornith 1.5 (ornith-1.5:35b-a3b*, including their general profiles) when no effort is set	Unset or auto effort resolves to medium; the registry cards supplied xhigh before. The rendered prompt is byte-identical to the official template at medium. Explicit user, per-model, CLI or request values still win, thinking off still drops effort, and no think-token cap is implied. The Ornith coding (xhigh) and instruct (none) profiles, Flash-Next and other models are unchanged.	reasoning_effort = "xhigh" in config, or "reasoning_effort": "xhigh" on the request.
27B DFlash	Qwen3.8-27B tiers whose registry entry declares a paired draft (qwen3.8:27b, -mq3, -mq3-xt, -mq3-pro, -mq4-xt, -mq4-xts, -mq4-pro), when the draft file is present	The registry layer sets speculation.dflash = "auto", so DFlash is on with no configuration. Without the draft file the load keeps MTP auto and does not fail. The mq5* and mq6* tiers and every other model keep dflash_mode = off. Explicit global, per-model and one-shot config outrank the registry.	hipfire config qwen3.8:27b set dflash_mode off, HIPFIRE_DFLASH_MODE=off, or NixOS services.hipfire.dflashMode = "off".
Flash-Next MTP AR floor	Flash-Next singleton native MTP	A request may retire to AR only after 16 observed windows instead of at the 3-window probe; greedy output is byte-identical. Continuous-batching lanes are unchanged. Its own silicon A/B is pending, so no speed claim is made for this form.	HIPFIRE_MTP_FLOOR_MIN_WINDOWS=3 restores retirement at the probe budget.
Flash-Next MTP draft cap	Native Qwen4 MTP on gfx1151 (Halo)	speculation.mtp_k / HIPFIRE_MTP_K defaults to auto, which is 4 for native Qwen4 MTP on gfx1151 and 3 everywhere else. Greedy output is byte-identical to cap 3 on the measured prompts. On Halo (pinned Flash-Next, three fresh serve processes per arm, thinking off) the code geomean went from 67.9 to 70.5 tok/s (+3.8%), a 1,200-token count-up from 69.4 to 77.0, 9K prose from 45.9 to 46.5, and short prose was unchanged at 52.4.	HIPFIRE_MTP_K=3 or speculation.mtp_k = 3.
Sampled DFlash acceptance	Sampled DFlash2 selector path (and PLD) on gfx1201, singleton and continuous-batching lanes	Accept, residual and bonus are decided on the GPU by a PeaceMaker kernel over the resident softmax rows, instead of downloading rows to the host. The law is the same rejection sampling; each window uses a fixed number of uniforms, so a seed reproduces its output, but seeded sampled text differs from the host loop. Greedy output is unchanged; other architectures decide on the host. No speed claim is made.	HIPFIRE_DFLASH_DEVICE_ACCEPT=0 (developer kill switch).
Pageable host copies	Every Linux process (#814)	Before the HIP runtime loads, an unset GPU_PINNED_MIN_XFER_SIZE is set to 100000, so every copy to or from pageable host memory goes through clr's pinned staging buffer instead of pinning its source in place, where host reclaim can stall the GPU queues. Greedy text was identical with and without the change on gfx1201 and gfx1151; no speed change is resolved. A pre-set GPU_PINNED_MIN_XFER_SIZE is respected.	HIPFIRE_PINNED_XFER_DEFAULT=0 leaves clr's own default exactly; GPU_PINNED_MIN_XFER_SIZE=0 is not an opt-out.
DFlash2 width	27B DFlash on serve.vmm_batch	speculation.dflash_block / HIPFIRE_DFLASH_BLOCK defaults to auto, which picks width 8 when continuous_batch_size is at least 4 and width 16 otherwise. The width is fixed per load and does not follow the live batch.	HIPFIRE_DFLASH_BLOCK=16 or speculation.dflash_block = 16 restores width 16.
IU4 verify	Exact gfx1201 with a symmetric dense MQ4G256V2 target, at width 8	IU4 verify is on by default, with IU4 prefill and both A4 fusions on. Singleton-only width-16 loads keep the F16 verify.	HIPFIRE_CB_VERIFY_IU4=0 disables it, and =1 forces it on.
Sampled VMM lanes	Serves with serve.vmm_batch on	serve.batch_sampled / HIPFIRE_SERVE_BATCH_SAMPLED is on by default. Eligible AR, MTP and FAST_SAMPLE selector-chain DFlash requests keep their singleton mode and wire-seed equivalence.	HIPFIRE_SERVE_BATCH_SAMPLED=0 keeps sampled requests singleton.
PLE format	Flash-Next	memory.ple_format / HIPFIRE_PLE_FORMAT defaults to auto, which uses q4 on gfx1151 and q8 elsewhere.	HIPFIRE_PLE_FORMAT=q8 or memory.ple_format = "q8" restores q8 numerics.
q4 warming	Flash-Next with q4 PLE	Idle prewarm (memory.ple_prewarm_bytes / HIPFIRE_PLE_PREWARM_BYTES) defaults to auto = 15 GiB, capped at MemAvailable − 8 GiB. Load fill (memory.ple_load_fill_bytes / HIPFIRE_PLE_LOAD_FILL_BYTES) defaults to auto = 14 GiB. Ready never waits for load fill.	HIPFIRE_PLE_PREWARM_BYTES=0 turns idle prewarm off, and HIPFIRE_PLE_LOAD_FILL_BYTES=0 turns load fill off.
PM decode	Exact gfx1201	kernel.pm_decode defaults to auto, which uses five embedded W1 modules. The change is exact, and no speedup is claimed.	HIPFIRE_PM_DECODE=0 restores hipcc.
Kernel readiness	Closed single-GPU gfx1201 27B MQ4 XTS with native fp8 K/V, Q8 DeltaNet and default AR/MTP	Load now includes kernel preparation. Other routes still load kernels lazily.	HIPFIRE_JIT_JOBS=1 disables compile concurrency but not planning.
Tool argument replay	Qwen4 and dense Qwen3.8	Replay is on by default. Reordered echoes preserve typed values, and only exact extensions of the committed record are accepted.	HIPFIRE_QWEN4_TOOL_ARG_REPLAY=0 opts out.
MQ4 Flash-Next few-row verify	MQ4 Flash-Next trunk on gfx1151 and gfx1201	Few-row verify is on by default. The shipped MQ6 model is unchanged.	HIPFIRE_QWEN4_MQ4_ROWS_PM=0 restores per-row GEMV.
MQ4 Flash-Next Halo prefill	MQ4 Flash-Next trunk on Halo	Halo prefill uses the MQ4 PeaceMaker twins by default. The shipped MQ6 model is unchanged.	HIPFIRE_QWEN4_MQ4_X4_PM=0 restores hipcc bt4 and unfused HC.
Halo shared-expert prefill	Halo	Prefill uses an exact fused gate/up, a BF16-recipe SwiGLU and an F16 conversion.	HIPFIRE_QWEN4_SHARED_GU_PM=0 restores four launches.
APU admission	Flash-Next on APUs	Memory accounting uses corrected GTT free memory. An omitted Flash-Next max_seq is chosen automatically, and snapshots yield to growth. Discrete Flash-Next keeps 32768. An explicit memory.max_seq / --max-seq fixes the cap but not the accounting.	No kill switch is stated.
Flash-Next NPU trunk	Strix Halo (gfx1151 + XDNA2) with matching keyed calibration	fusions.npu_trunk / HIPFIRE_QWEN4_NPU_TRUNK defaults to auto: trunk input projections of paired prefill banks run on the NPU; any refusal logs one reason and runs GPU-only. Needs about 1.05 GB of memlock. Other architectures do no NPU I/O.	HIPFIRE_QWEN4_NPU_TRUNK=off or fusions.npu_trunk = off.
27B DFlash 16-row verify GEMMs	Exact gfx1151	HIPFIRE_GFX1151_XLDS on by default; byte-identical.	HIPFIRE_GFX1151_XLDS=0.
zab trunk-overlay sidecar	Stock Flash-Next MQ4 XTS on gfx1151	HIPFIRE_QWEN4_TRUNK_OVERLAY defaults to auto: the pulled .trunk-overlay replaces the 108 GDN z/a/b projections and arms trunk A4 (numerics change within the +0.015 KLD cap).	HIPFIRE_QWEN4_TRUNK_OVERLAY=0 restores the stock weights and route.
EP2 prefill and ownership	Flash-Next under EP2 (opt-in EP)	Owner-routed prefill chunk MoE, long-prefill expert staging and balanced host-mapped expert ownership are on.	HIPFIRE_EP_OWNER_ROUTED_PREFILL=0, HIPFIRE_EP_EXPERT_STAGE=0, HIPFIRE_EP_BALANCED_OWNERS=0.

Opt-in and experimental features

Feature	Scope	Default	How to control it
Flash-Next lane staging	Flash-Next serves with serve.continuous_batch_size >= 2	Experimental and off	serve.vmm_batch / HIPFIRE_SERVE_VMM_BATCH=1 enables it, and HIPFIRE_SERVE_VMM_BATCH=0 disables it. A lone arrival stays singleton.
Flash-Next MTP lanes	Inside staged lanes with the MTP head loaded	On by default within staged lanes, with a per-lane AR floor	serve.batch_spec = false or HIPFIRE_SERVE_BATCH_SPEC=0 keeps AR.
Halo PM lane rows	Inside staged lanes, on exact gfx1151 lane shapes outside capture and recording	On by default within staged lanes. The change is byte-identical, and TTFT is unresolved.	HIPFIRE_FN_LANE_PM_ROWS=0 restores the earlier rows.
Wide verify chunks	Exact gfx1201 dense MQ4G256V2 targets	Developer feature, off	HIPFIRE_CB_VERIFY_CHUNK128=1; inside it PM is on by default, and HIPFIRE_CB_VERIFY_PM=0 uses exact hipcc K32 twins.
FP8 verify	Admitted gfx1201 symmetric targets	Developer feature, off, and it stays off	HIPFIRE_CB_VERIFY_FP8=1
PLE hot-row store	UMA	Off (default 0)	memory.ple_hot_store_bytes / HIPFIRE_PLE_HOT_STORE_BYTES
Compact forward scratch	gfx1201 with all-symmetric IU4 experts	Developer feature, off	HIPFIRE_QWEN4_COMPACT_SCRATCH=1
Prefill PM invariant	gfx1151	Developer feature, off	HIPFIRE_PREFILL_PM_INVARIANT=1
QSA F16 K/V arena	Flash-Next prefill attend on gfx1151 (F32 QSA K/V, head_dim 256)	Opt-in, off; exact, no speed gain resolved	HIPFIRE_QWEN4_QSA_F16_ARENA=1
Multi-GPU EP	Two or four R9700s (gfx1201)	topology.ep / HIPFIRE_TOPOLOGY_EP defaults to 1	Set topology.ep or HIPFIRE_TOPOLOGY_EP; HIPFIRE_QWEN4_EP=0 forces one card.
Save-free refusal reporting	gfx1201 save-free rollback	Opt-in, off	HIPFIRE_CB_DN_SAVE_FREE_DIAG=1
Qwen4 developer profiles	qwen4-dev-profiles builds only	Off	Build with qwen4-dev-profiles.
Railgun backend	Developer-only gfx1201	Redline remains the default	HIPFIRE_RAILGUN_BACKEND=railgun
Shared-weight host	Linux gfx1201, developer-only	Off	HIPFIRE_WEIGHT_HOST=1 HIPFIRE_WEIGHT_HOST_SOCKET=<path> plus fabric admission; HIPFIRE_WEIGHT_HOST=0 gives a private load.

Strix Halo NPU trunk

On Strix Halo (gfx1151 + XDNA2), two consecutive Flash-Next prefill banks (8,192-row chunks, or a rebalanced last pair) run their GDN and QSA trunk input projections on the NPU, layer-major, with the GPU consuming the NPU's row-major arenas directly. npu_trunk takes off|auto|on|emulate; emulate is the GPU-only bit-exact twin, and greedy on == emulate at p16k, 9.3K and 57K, including session-cache hits inside a pair. Quality was accepted at ΔKLD WT2 −0.0005 / code24 +0.0052, both within the +0.015 cap. Calibration schema 4 is BF16-direct only.

Prompt (default serve config, 3 fresh processes per arm)	off → auto TTFT
9,330 tokens	4,625 → 4,319 ms (−6.6 %)
p16k	7,878 → 7,085 ms (−10.1 %)
24,438 tokens	11,944 → 11,159 ms (−6.6 %)
p56k	30,047 → 27,260 ms (−9.3 %)

The release gate on beta 1be80ed54 reran 16K with the NPU trunk on under the bounded memlock limit: 3 fresh processes × 3 requests on 16,356 tokens, all-9 median TTFT 7,085 ms (2,309 tok/s), steady requests 7,079 ms (2,310 tok/s). The shipped tip 6ea3f5076 adds the two changes below. Measured on Strix Halo with the NPU trunk on, bounded memlock, 16,355 tokens, client TTFT, 3 fresh processes × 3 requests: the tip's all-9 median TTFT is 6,854 ms (2,386 tok/s) and steady requests 6,849.5 ms, −3.4 % TTFT vs the gate build 1be80ed54, measured interleaved. Greedy p16k output is byte-identical to the gate build.

Bank B's PLE rows are now fetched ahead and the trunk arenas ask for transparent huge pages (exact: greedy is byte-identical across on, emulate and off; no kill switch). Interleaved against that base, 3 fresh processes × 3 requests per arm, 16K TTFT measured 7,054 ms against 7,111 ms (median of per-process medians): no slower, but the gain is not resolved beyond noise. On a loaded host THP backed only about 44 % of the arenas.

The fused HC kernels now also run under the NPU pair route: an HC fusion cursor carries each bank's pairings across its step windows, so the attention HC gate hand-off, the hcw output-projection epilogue and the MoE HC row fold fire as they do on the GPU-only route (exact: greedy on, emulate and off bytes match). Measured on base 93720b4fb without the PLE-prefetch + THP stack, interleaved, 3 fresh processes × 3 requests per arm: 16K all-9 median TTFT 6,896 ms against 7,194 ms (2,372 against 2,274 tok/s), and every candidate TTFT was lower than every base TTFT.

Memlock. The trunk pins its userptr arenas, 1,053,843,456 B for the shipped layout. At open it raises the soft limit to exactly what it needs when the hard limit allows; otherwise auto logs WARN npu-trunk: off (...) with the bytes and the bounded fix and serves GPU-only, and on is a load error. Set LimitMEMLOCK=1318060032 (systemd) or ulimit -l 1287168; infinity is not needed. See docs/npu/npu-trunk.md, "Host memlock limit".

Strix Halo 27B DFlash

The 16-row DFlash verify projections now run a dedicated gfx1151 kernel (4 waves × 16 weight rows, the activation panel staged in LDS once per block). Measured on Strix Halo, 27B XTS with its paired DFlash draft, hipfire bench --backend noslots --workload stateless --max-tokens 256, 3 fresh processes per arm: code 97.9 → 135.3 tok/s (cycle 144.9 → 104.8 ms, τ 13.18 both), prose 19.0 → 26.3 tok/s, AR 15.0 → 15.0. Text is byte-identical to beta with the switches on and off. The same entry adds an exact DeltaNet snapshot bulk copy (HIPFIRE_DN_SNAPSHOT_BULK_OFF=1 opts out) and hidden-ring scatter fusion (HIPFIRE_HIDDEN_SCATTER_FUSE_OFF=1) on gfx1151.

PeaceMaker barriers on gfx1151 now retire in-flight LDS loads of the slots they free, so the 27B A4 gate/up epilogue and the Qwen4 QSA select kernel give the same bytes when another process shares the GPU.

27B DFlash / serving and batching

Paired DFlash by default

Qwen3.8-27B tiers whose registry entry declares a paired draft now load with speculation.dflash = "auto" from the registry layer, so a present draft file turns DFlash on with no configuration; an absent file keeps MTP auto. Both arms of each comparison ran the same candidate binary on one R9700 (card B) with the canonical mq4-xts target, singleton serve and three fresh processes per arm, and every arm's output bytes were identical across its three processes. "Before" is the kill-switch configuration (dflash_mode off, MTP auto), not the previous release binary.

Metric	MTP auto → paired DFlash
C1 sampled, effort xhigh, 4,096-token cap	44.00 → 74.49 tok/s (+69.3%)
C1 greedy, effort xhigh, 4,096-token cap	44.35 → 64.93 tok/s (+46.4%)
C1 sampled, explicit effort medium, 4,096-token cap	52.91 → 116.65 tok/s (2.20×)
HumanEval, thinking on, medium, sampled	163 → 162 of 164 (McNemar p = 1.0)
GSM8K first 300, thinking on, medium, sampled	300 → 298 of 300 (p = 0.5)

All 928 thinking-on quality requests closed their reasoning and ended on a stop, none on the length cap. The quality comparison shows no detected regression; it is not a proof of equivalence. The xhigh speed runs all ended on the 4,096-token cap while still thinking, with coherent text. Turn it off with hipfire config qwen3.8:27b set dflash_mode off, HIPFIRE_DFLASH_MODE=off, or NixOS services.hipfire.dflashMode = "off".

Width and IU4: gains and lone-request cost

The width is fixed by configuration, so a peer's arrival cannot change the generated text. Legacy drafts keep their declared width, and experimental multi-slot serving accepts only width 16. Changing the width can change the text: 15 of 64 code prompts differ between width 16 and width 8.

The campaigns below ran on one R9700 with the 27B mq4-xts target and the MQ4V2 draft, VMM batch size 8 and HIPFIRE_CB_VERIFY_CHUNK128=1. The greedy workload is conc_steady.py with 64 code prompts × 256 tokens. For the width comparison, each arm used three interleaved fresh processes after a discarded warm-up, and the BetterBench v1 T0.7 C8 comparison used five per arm. For the IU4 comparison, both arms ran the same binary, one of them with HIPFIRE_CB_VERIFY_IU4=0, with eight interleaved fresh processes per arm and HIPFIRE_CB_PHASES=1 in both; BetterBench used three per arm. These are separate campaigns, so their results are not compounded.

Change and metric	Before → after tok/s	Qualification
Width 16 → 8, C1 on a batching serve	114.8 → 107.8	−6.1%; singleton-only loads keep width 16
Width 16 → 8, C4 / C8 steady greedy	227.4 → 272.6 / 254.6 → 324.6	+19.9% / +27.5%
Width 16 → 8, sampled C8 aggregate	168.5 → 195.2	+17.3%
IU4 at width 8, C1 on a batching serve	107.6 → 99.3	−7.7%, an additional cost
IU4 at width 8, C8 steady greedy	340.1 → 374.5	+9.9%, CS95 +5.3..+14.7%
IU4 at width 8, sampled C8 aggregate	192.3 → 199.9	+3.9%, CS95 +2.8..+5.1%

C1 costs stack on a batching-configured serve. A lone request pays both the width cost and the IU4 cost. To get the earlier behavior back, set both HIPFIRE_DFLASH_BLOCK=16 (or speculation.dflash_block = 16) and HIPFIRE_CB_VERIFY_IU4=0. The two switches are independent, and neither is a universal speed improvement.

IU4 changes the verify activations to A4, while the head stays F16 and the GDN scan stays F32. Eager verify disables the graph and retained PM4 routes, and a missing producer raises an error instead of mixing routes. If the FP8 opt-in is enabled it wins over the default, and explicitly forcing both FP8 and IU4 fails. Within IU4, batched and singleton texts match, but identity with F16 is not promised.

Same-binary paired greedy singleton quality	Width 16 → 8	F16 → IU4
HumanEval	157 → 157 of 164	157 → 156 of 164
GSM8K 5-shot	1,282 → 1,280 of 1,319	1,282 → 1,282 of 1,319
MMLU-Pro subset	871 → 871 of 1,400	871 → 871 of 1,400

For the width change, the GSM8K McNemar p-value is 0.63 and the MMLU text is identical. For IU4, the mean verify KLD against F16 is 8.8e-3 with top-1 agreement of 97.2%. Tied quality does not mean identical items.

Exact batching defaults and kill switches

The developer defaults below preserve the admitted singleton arithmetic. Shapes that are not eligible keep their stated incumbent route, so the scope of each row matters.

What it does and where it applies	Exact kill switch
On gfx12 with MQ4V2 non-MoE targets, full-block lane drafts share one forward pass and head, while lone, tail and otherwise ineligible requests stay singleton.	HIPFIRE_CB_DFLASH_DRAFT_BATCH=0
On gfx1201, the F16 draft overwrite removes the pre-zero step, outside FP8, IU4, capture and recording.	HIPFIRE_DRAFT_COLLAPSE_OFF=1 (this also disables the gfx1100 collapse)
On gfx1201, the batched draft uses a 64-row scope that excludes the lane fc and context.	HIPFIRE_CB_DRAFT_ROWS64=0
On gfx1201, eligible sampled draft runs share one forward pass and head while preserving the selector, RNG and q, and the master draft batch must be on.	HIPFIRE_CB_SAMPLED_DRAFT_BATCH=0
On gfx1201, context and fc packs are grouped by their original arithmetic class, either overwrite WMMA or singleton R2.	HIPFIRE_CB_DRAFT_LANE_CTX=0
On gfx1201, the draft convolution runs lane-locally.	HIPFIRE_CB_DRAFT_LANE_CONV=0
On gfx1201, greedy lane candidate scoring runs in the selector.	HIPFIRE_DFLASH_SELECTOR_LANES=0
The noise embedding is batched, and gfx1100 keeps the collapse builder.	HIPFIRE_DRAFT_NOISE_EMBED_BATCH=0
Singleton and lane draft copies are stream-ordered, with lane positions hoisted.	HIPFIRE_DFLASH_ORDERED_DRAFT=0
Rollback-tape copies are batched, for multi-request and singleton verify respectively.	HIPFIRE_CB_TAPE_COPY_BATCH=0 / HIPFIRE_VERIFY_TAPE_COPY_BATCH=0
On gfx1201, the ChainVerify DeltaNet preamble and recurrence use the fast Q8 and tape path, outside capture and recording.	HIPFIRE_CB_LANE_GDN_BATCH=0
On gfx1201, ragged DeltaNet and save-free rollback follow the singleton frame order.	HIPFIRE_CB_LANE_GDN_VARROWS=0
On gfx1201, fp8 VerifyAttn lane FullAttention uses its own partial scratch.	HIPFIRE_CB_LANE_FA_BATCH=0
On gfx1201, owner-exchange rollback is admitted by capacity and alias checks.	HIPFIRE_CB_DN_SAVE_FREE=0
VMM uploads send only changed descriptors and coalesced positions.	HIPFIRE_CB_UPLOAD_COALESCE=0
On gfx1201, an exact F16 sampled shared head is used, but not with FP8, IU4, A8, capture, recording or G0.	HIPFIRE_CB_SAMPLED_PACKED_HEAD=0
On gfx1201, equal-config sampled runs share target softmax, compaction and download.	HIPFIRE_CB_SAMPLED_TARGET_BATCH=0
On gfx1201, sparse target and selector work is used, while identity, overflow and CACTUS requests stay dense.	HIPFIRE_DFLASH_SPARSE_TARGET=0
On gfx1201, MQ4V2 verify tiles are used for 16 < N < 64 rows, while capture and replay stay one-tile.	HIPFIRE_WMMA_BATCH_TILES=0

Save-free rollback admits an equal pooled EF capacity that covers the logical extent, with S exact-sized and owners distinct. Ragged-row entry extends the earlier equal-row admission. HIPFIRE_CB_DN_SAVE_FREE_DIAG=1 is an opt-in, default-off switch that reports refusals, and it does no diagnostic work when it is off.

The throughput A/B comparisons for the packed head and pooled EF remain unresolved at three runs per arm. The lane FullAttention and ragged DeltaNet phase gains have unresolved throughput under host slow mode. The convolution and noise-lookup phase gains do not establish end-to-end wins, upload coalescing is below the default bar, and singleton tape at C1 is within noise. The oracles and text checks support exactness, not numerical-mode equivalence.

Opt-in modes and long prompts

• Wide chunks: HIPFIRE_CB_VERIFY_CHUNK128=1 is a developer, default-off switch that allows up to 128 DFlash rows per verify chunk on exact gfx1201 dense MQ4G256V2 targets. Inside it, PM is on by default, and HIPFIRE_CB_VERIFY_PM=0 uses the exact hipcc K32 twins. Turning the outer flag off restores smaller chunks. In one R9700 HTTP and process on/off comparison, C8 code rose from 238.8 to 282.6 tok/s (+18.4%), and C1 and C2 were unchanged. This does not apply to MTP.
• FP8: HIPFIRE_CB_VERIFY_FP8=1 is a developer, default-off numerical alternative for admitted gfx1201 symmetric targets, covering DFlash and MTP singleton, continuous-batch, head and replay paths. Unsupported forwards fail instead of mixing in F16. In paired greedy DFlash evals, HumanEval went from 157 to 154 of 164, GSM8K from 1,282 to 1,280 of 1,319, and MMLU from 871 to 871 of 1,400. HumanEval changed by −1.8 points, with seven F16-only passes against two FP8-only passes. The mean verify KLD is 1.15e-3 with top-1 agreement of 99.12%. FP8 stays off.

VMM reserves lane state, checkpoints and the remaining KV space, and a request waits in the inbox instead of failing the whole batch. Smaller DFlash lanes borrow scratch, omit unread checkpoints and leave the promoted singleton vacant, and MTP state is charged as well. On one 32 GB R9700, 27B/DFlash VMM size 8 fits all eight ~4K and ~8K lanes with text equal to singleton. The long8 rate rose from 92.82 to 101.54 tok/s, and xl8 went from 70.70 to 73.15 / 69.21 in two processes, which is duration noise. TTFT p50 rose from 12.9 to 17.1 s because all prompts prefill up front. Fitting the lanes is not a latency improvement. HIPFIRE_SERVE_VMM_BATCH=0 restores serial staging, but not the old allocation algorithm.

Flash-Next prefill / decode / lanes / PLE / EP

PLE and exact prefill

Hash-bound .ple-q8 and .ple-q4 sidecars separate the PLE table from the base model, and a hash mismatch is refused. hipfire ple derive --format q4 <model> writes the q4 sidecar atomically, and a q4 load derives it if it is absent. hipfire ple split <model> produces the base and q8 files. The pull option --ple auto|q4|q8|all|none selects which files to fetch, and auto never derives or downloads anything and falls back with a note. The approved q4 numerics are WT2 KLD 0.104865 against 0.103252 for q8, matching the q4 emulation logits. HIPFIRE_PLE_FORMAT=q8 restores the q8 format.

Halo PLE campaign	Prompt and cache state	tok/s
q4, one run per cell	Ciru 8K first-after-ready, table cold	1,653.2, TTFT 4.95 s; Q8 ~1,553
q4 after default prewarm	Ciru 8K steady	1,828.4; Q8 ~1,599
Same campaign	Natural and agentic 8K steady	1,840.2 / 1,832.7
Load fill, same session as beta 662c7b21d, fresh process per cell	Ciru 8K first-after-ready	1,713.6; beta 1,622–1,633
Same load-fill campaign	Prose and agentic 8K first-after-ready	1,649.0 / 1,720.9; beta 1,619–1,645 / 1,693.9
Same load-fill campaign	Ciru 8K steady	1,883.3; beta 1,866.9–1,886.9

Prewarming the whole table over the available RAM pushes memory reclaim into the request: with a 26.8 GiB table and ~24 GiB available, Ciru 8K steady fell to 1,734.5. Bounded warming therefore matters. Load fill at ≤ 0.2 GB/s retries evicted pages, and ready does not wait for it. UMA upload drops weight-file pages. The first request still lags steady state under kswapd, the kernel's memory-reclaim daemon. Disable load fill with HIPFIRE_PLE_LOAD_FILL_BYTES=0, and restore the q8 rows with HIPFIRE_PLE_FORMAT=q8.

The hot-row store remains opt-in, UMA only, and default 0 (off), configured by memory.ple_hot_store_bytes or HIPFIRE_PLE_HOT_STORE_BYTES. Prepare it with hipfire-quantize ple-hot-rank and ple-hot-pack. Byte-identical page-cache rows replace the swapped anonymous store, which the CHANGELOG blames for the earlier Halo Ciru 8K outliers of 737–1,476 tok/s; page-cache rows are dropped under memory pressure, never swapped. Hits and misses decode in parallel, and an ascending reader cursor clusters SSD reads. Large requests decode into a lease on the reader threads, which removes the Halo Ciru 8K 118 ms serial chunk-0 publish wait. The ticket completes before cache insertion, quiesce still waits, and the rows and eviction order are unchanged.

Halo exact shared-expert fusion, paired MQ6 dequantization, HC gate reuse and a compact GDN grid preserve state and logits. The Ciru 8K campaign used three paired holds with the warm-up discarded, and the synthetic campaign used three fresh processes per arm.

Halo exact campaign	Before → after	Qualification
Ciru 8K release-matrix	1,944.9 → 1,964.8 tok/s	TTFT 4,209 → 4,166 ms; +1.0%, CS95 +0.4..+1.7%; cache state not stated
Synthetic pp8192	1,961.5 → 1,993.4	Synthetic, not prompt-file; +1.6%, CS95 +0.6..+2.6%
MTP fill, Ciru 8K	1,518.7 → 1,557.1	TTFT 5,390 → 5,257; cache state not stated; decode 56.66 → 56.63 is noise, three interleaved processes

MTP fill uploads the ids and maps the head once and stream-orders the hidden copy, using the same kernels in the same order. On R9700, exact MQ6 paired-code synthetic pp8192 went from 3,837 to 3,841 at the default chunk, and from 4,895 to 4,926 with the opt-in HIPFIRE_PREFILL_CHUNK_ROWS=8192; this is not a Ciru 8K result.

On R9700 with MTP on, the Flash-Next MTP prompt fill now runs batched and expert staging works under wide-hidden capture, both exactly. Steady 8K prompt time went from 3,394 to 2,222 ms on Ciru 8K and from 4,133 to 2,225 ms on code. HIPFIRE_QWEN4_MTP_BATCHED_FILL=0 and HIPFIRE_QWEN4_EXPERT_STAGE_CAPTURE=0 turn the two parts off.

QSA incremental grids are sized by rows rather than by maximum context, and they are byte-exact. Corrected GTT sizing and the automatic Flash-Next context make snapshots yield to growth, and they refuse impossible residency before allocation. Discrete and carve-out accounting is unchanged, and an explicit memory.max_seq sets the cap.

The MQ4 trunk's few-row exact verify fixes the MTP/AR divergence, and Halo prefill uses MQ4 twins. For the candidate qwen3.8-flash-next.mq4t-gptq.mq4, Halo medians of three bench runs, with a fresh process per arm, were: MTP 40.0 → 54.2, synthetic pp8192 1,034 → 1,885, and AR 37.2 → 37.1 (noise). These are not the shipped MQ6 pin and not Ciru 8K. The opt-outs are in the defaults table. HIPFIRE_QWEN4_COMPACT_SCRATCH=1 stays a developer, default-off switch on gfx1201 with all-symmetric IU4 experts; it shares storage in a fixed placement exactly, and no larger default chunk is promoted.

Lanes and EP

Flash-Next greedy lanes remain experimental and opt-in, and they refuse tools, images, logprobs and penalties. The gateway now admits histories by typed capability. On Halo with q4 PLE AR, with four conversations × three turns at 2048 depth and one run per arm, the text was equal and wall time went from 38.3 to 28.1 s. A singleton promotes at a committed boundary together with its trunk, head, hidden and draft state, and an armed ring or open snapshot refuses. Native-MTP lanes use the floor, and HIPFIRE_SERVE_BATCH_SPEC=0 keeps AR. The pooled floor prices row-group spill.

On Halo with gptq3/q4, VMM C8 conc_steady 64 × 256 and same-binary interleaved fresh processes, PM-off ran at 65.91 / 63.40 / 66.35 tok/s and PM-on at 78.98 / 79.42 / 79.42 / 79.30 tok/s: +21.6%, CS95 +6.5..+38.8%, with identical text. TTFT p50 is unresolved: it was 847 / 905 / 1,006 ms with PM off and 978 / 996 / 953 / 976 ms with PM on, a +6% point estimate with a CS95 of ±40%, and the PM-on values sit inside the PM-off range. Before the spill-floor revision, p50 went from 825 to 1,355 ms, which kept the flag off. HIPFIRE_FN_LANE_PM_ROWS=0 restores the earlier rows.

HIPFIRE_PREFILL_PM_INVARIANT=1 stays a developer, default-off switch on gfx1151: short prefill and appends take the long-route arithmetic, while decode, MTP and the head are unaffected. The lab setting HIPFIRE_QWEN4_GDN_PREFILL=wy|persistent|canon is separate, and unset routes are unchanged.

EP (topology.ep / HIPFIRE_TOPOLOGY_EP) defaults to 1, and two-card and four-card R9700 EP is opt-in and experimental; HIPFIRE_QWEN4_EP=0 forces one card. Every rank holds the trunk and an expert shard, and the owner exchange is copy-only. EP requires gfx1201 with all directed peers and pp = tp = 1. Tested EP2 and EP4 state and logits equal one card.

Inside EP, three additions are on by default under EP2: owner-routed prefill chunk MoE (HIPFIRE_EP_OWNER_ROUTED_PREFILL=0 disables), long-prefill expert staging (HIPFIRE_EP_EXPERT_STAGE=0) and balanced per-call expert ownership on host-mapped layers (HIPFIRE_EP_BALANCED_OWNERS=0). EP4 owner-routed prefill is refused at admission until its hardware cells run. On 2× R9700 the combined build passed 25/25 exactness compares against one card; p8k / p16k prefill went from 3,033 / 3,754 tok/s on one card to 3,943 / 4,308 on EP2. Balanced ownership raised EP2 decode by 3.8–4.3 % (MTP code 80.6 → 84.1, MTP prose 63.7 → 66.1, AR code 48.5 → 50.5 tok/s; short outputs).

Native MTP now runs under EP when asked for (--spec mtp / speculation.mtp = on); auto under EP stays AR. On R9700s with route mtp+ngram: code 44.4 tok/s on one card, 81.7 on EP2, 121.6 on EP4; prose 34.5 / 63.4 / 87.1 tok/s (short outputs, decode tok/s). Lanes stay refused under EP.

FN gptq3 R9700, one timing admission, three interleaved fresh processes per arm	Result
Four GPUs, 32K fit per rank	49 layers resident, +3.77 GiB headroom
Two GPUs, 32K fit per rank	30 of 49 layers resident, 19 host-mapped, +0.34 GiB headroom
Four GPUs, AR tg128 with speculation off	61.2 tok/s against 36.1 for one-card graph AR with host experts; 1.70×
Four GPUs, 42-token TTFT	408 ms against 286 ms on one card, slower because of replicated prefill and P2P

Per-process serialization of compile and publish fixes the object and source races that concurrent ranks hit on a cold cache.

Other additions

• Opt-in POST /v1/systemone decide endpoint (#790). Classification requests (choice, score, yes/no-or-unclear) answered from one prefill of label logits instead of generated text, with optional per-model temperature calibration. Chat completions and the default serve path are unchanged.
• Flash-Next session-cache SSD tier (opt-in, #840). memory.session_cache_disk_bytes gives device-evicted or unloaded prefill snapshots a disk budget in memory.session_cache_dir; snapshots restore across restarts of the same build.
• Multi-slot embeddings. The arch-17 encoder (pplx-embed-v2-late-0.6b) serves concurrent embed requests as one batched forward, bit-identical to serial; no throughput figure is claimed.
• gfx1100 and gfx1201 DFlash work. gfx1100 DFlash drafts share MQ4V2 input rotations (#838, default on for exact gfx1100, three independent switches); the exact-F16 projection producers, GDN pre/tape fusion and batched FA prep from #759 now also run on gfx1201, byte-identical; an exact gfx1201 MQ4V2 K5120 gate/up decode specialization is opt-in (kernel.gfx12_mq4v2_gateup_k5120); an experimental gfx1100 long-context DFlash FA2 split-KV verifier is opt-in (#817, kernel.gfx1100_fa2_split_verify, not greedy-identical to off).
• gfx1100 Qwen3.5-4B decode and MQ3-Lloyd prefill (#845). Bit-identical fusions, decode +5.5 % (MQ4) and +3.0 % (MQ3) on an RX 7900 XTX; each fusion has a 0 kill switch.
• Qwen3.5-family VMM KV admission (#833). Exact gfx1100 TP1 admits FWHT2/3/4 K with Q8 or Lloyd2/3/4 V.
• Engine-level YaRN RoPE scaling (#856). Flash-Next can run past its native 262,144-token context.
• Redline tapes now record the Lloyd2/Lloyd4 V-cache decode write, which had page-faulted the VMM arm on gfx1100. Serve queue config fallbacks use the registered defaults (serve.queue_timeout_ms 600000 ms). Detached hipfire serve keeps --vision and forwards --max-seq once.

Qwen4 day-0 developer profiles

The feature requires a qwen4-dev-profiles build and is off by default; production artifact and config admission excludes synthetic profiles. Dense FFNs execute the declared BF16 recipe and profile geometry with optional PLE and MTP, and decode uses source BF16 with no retained body. Unsupported rotary bases refuse before allocation, and Flash-Next lanes apply only to Flash-Next. The tiny producer hipfire-quantize --qwen4-dev-tiny source-exact|packed writes a complete synthetic HFQM file and reopens it and checks extents before publishing, and packed roles must tile or the producer refuses. Flash-Next production is unchanged, and there is no upstream Qwen4-27B compatibility or quality claim.

Kernels / PeaceMaker / packs / ROCm

ROCm 10.1 with HIP 7.16 and clang 24 brings code object v6 and the JIT ABI, a common path resolver, compiler-major SiLU goldens and the native-writer layout. A producer shared-scratch fix preserves scale broadcast. Halo pins cover the warm cache only, and the cold-JIT chunk-0 problem is not fixed. Packs are pinned to HIP 7.16.26385 and clang 24 within the range [10.1, 11.0). A different local hipcc compiles locally, while a compiler-free install admits the pack. The HIPFIRE_ROCM_IMAGE and range overrides remain. A bounded shared worker budget is set with HIPFIRE_PACK_JOBS or --jobs N. The gfx11 opt-out pack MMQ avoids first-use compilation for that module.

On the closed gfx1201 27B route, load includes kernels. HIPFIRE_JIT_JOBS=1 serializes compilation, an unplanned dispatch refuses, and other routes load lazily with an explanatory log. This affects startup and readiness only. PM decode images and sources are pinned and mismatches refuse, and HIPFIRE_PM_DECODE=0 restores hipcc. In a same-card comparison, AR medians were 37.17 against 37.08 and MTP was 71.4 / 55.1 against 71.5 / 55.2 tok/s, over three fresh processes per arm, with identical state and pins and no speedup.

The developer RADV/ACO import remains non-product, and captured-body parity does not establish speed. The first warm-cache gfx1201 GEMV diagnostic shows no hipcc win and no resolved PM advantage, so performance promotion has stopped.

PeaceMaker's ISA tables gain the remaining gfx11 and gfx12 opcode encodings (64 new gfx11 and 51 new gfx12 identities, each checked against Mesa's registration and ROCm assembly and disassembly). This is table and codec coverage only: no new numerical mode is enabled until its hardware comparison passes, and committed kernel bundles and pinned kernel sources are unchanged.

Serve / CLI / API / metrics

The Messages endpoint POST /v1/messages and the Responses endpoint POST /v1/responses, both streaming and non-streaming, adapt to the chat pipeline's admission, lease, tool, think, stop and worker behavior, and they share its cache and batch behavior. Messages supports content and tool round trips, thinking budgets, stops, sampling, usage, errors and events, and absent thinking means thinking is off. Responses supports instructions, input reasoning, function items, tools, effort, text format, budgets, incomplete status and events. Messages prefill, server and hosted tools, and previous_response_id are refused, and responses are not stored. API headers are allowed by CORS.

Detached readiness now checks the PID and health, and it reports early fatal reasons and conflicts. Stop and restart target only this install and home. ps and run use the saved port, and a streamed answer ends with a newline. serve --spec off|auto|ngram|dflash|mtp|dspark and --max-seq N on serve and run override the load config and are forwarded to the detached child. Auto-attached speculation explains how to opt out. Errors for tag suggestions, missing downloads, closed pipes and retired keys are repaired, and the empty-list suggestion is smaller. The draft registry no longer fetches the vision tower. Explicit symmetric draft conversion uses --mq4v2 --mq4v2-symmetric or --mq3v2 --mq3v2-symmetric; both flags of a pair are required, and the default is unchanged.

The concurrency bench now loads the actual serve VMM route, warms each K, and uses matched chat turns, neutral penalties and a nonce. It runs real multiturn conversations and records throughput, TTFT, TPOT, concurrency, tau, routes and cache receipts. --bench-shape steady|betterbench and --bb-corpus DIR choose the workload. This is a measurement repair and not itself a speedup. Sampled modes expose equivalence, and over-cap, think-budget and stop requests stay singleton before start and never substitute AR. Flash-Next history and done receipts are now typed.

/metrics changed, and one change breaks scrapers (#841). hipfire_model_loaded is replaced by hipfire_model_info{model="…"} 1, which has no sample while no model is resident. The decode and prefill tok/s histograms now use 1 … 5000 tok/s buckets instead of the 10 … 60000 ms latency buckets, so their bucket boundaries change. hipfire_ttft_milliseconds now stops at the first generated token, reasoning included; before, a thinking model recorded the time to the first answer token. New series: prompt, completion and cached-prompt token counters, a gateway-measured time-per-output-token histogram, a hipfire_spec_tau summary on speculative routes, retry counters, and the gauges hipfire_engine_up, hipfire_model_loading and hipfire_context_capacity_tokens. Successful /v1/images/* requests now count in the request totals and latency. The full list is in docs/SERVE.md. The TUI Dashboard gains a live metrics card fed by /metrics, which shows "metrics unavailable (older serve)" against an older serve (#842).

With serve.multi_slot on, serve --max-seq N now also sets the per-slot context, replacing serve.multi_slot_ctx for that load, so total KV grows with serve.multi_slot_slots (#828).

Railgun / developer tools

HIPFIRE_RAILGUN_BACKEND=railgun is developer-only on gfx1201 and works with a rebuilt compiler, PM-native receipts and sidecars. R9700 27B state and logits match Redline and the HIP graph at the tested contexts, and tg128@512 ran at 41.06 against Redline's 41.38 tok/s, −0.8%. Redline remains the default, and the per-host corpus must be rebuilt to match.

The shared-weight host is Linux gfx1201, default-off and developer-only: it needs HIPFIRE_WEIGHT_HOST=1 HIPFIRE_WEIGHT_HOST_SOCKET=<path> plus fabric admission, and HIPFIRE_WEIGHT_HOST=0 gives a private load. Weights are shared read-only while tenant state and scratch stay private, and an unsupported setup refuses with no fallback. A clean unload releases its lease, and shutdown drains. Its acceptance is pending, and no speed claim is made.

HIPFIRE_DEVICES now reaches the common GPU-init path and rejects an invalid selection before HIP. EP dump and bisection are opt-in, and selection is unchanged. The CPU Radiowave schema makes no runtime or parity claim.

Tooling: crates/hipfire-arch-qwen4/reference_oracle/kld_teacher.py is a source-precision Qwen4 KLD teacher for qwen4_kld import. It runs the pinned upstream BF16 operators over a local copy of the pinned checkpoint, so KLD also covers expert quantization (#815).

Route oracles: the single-route oracle gains a Qwen3.6 MoE cell (qwen3.6:35b-a3b-mq4r), and the device-mesh fixtures and route-oracle tests are repinned to the canonical qwen3.8:27b-mq4-xts file. These are test and harness changes only (#843).

Fixes / compatibility

• Sampled speculative top-k, min-p and nucleus sampling now use the AR candidate-pool law across DFlash, MTP and n-gram, on host and GPU. Greedy output is unchanged and sampled trajectories change, and there is no switch back to the earlier incorrect law. The kernel costs more. On a card A gfx1201 27B serve, with three interleaved fresh processes per arm against e9044be, five × 256 and the same sampling, DFlash went from 76.63 to 76.27 and MTP from 59.18 to 58.49 tok/s, within spread.
• Draft per-row and per-window uniforms replace coupled draws, and the host bonus uses the truncated acceptance law, including PLD, the selector and budget-bound FAST_SAMPLE. Sampling is request-deterministic, but it is not identical to the sampled output before this fix.
• On gfx1200 and gfx1201, DFlash eager and graph chain attention are now identical, and greedy output becomes the former graph text. Other verify crossovers are unchanged.
• On gfx1201, producers now honor HIPFIRE_GFX12_MQ4V2_FP8_{QKVZA,GATEUP,QKV,RESID}=0, which fixes stale F32 opt-outs, and the default bytes are unchanged.
• GDN replay is provisioned before execution, and a failure refuses instead of falling back numerically. Scratch is shared. HIPFIRE_GDN_REPLAY_ML_OFF=1 is a diagnostic switch and is not byte-identical. The expanded MoE contraction matches the unchanged single-device D4, which repairs EP.
• Developer Flash-Next PM4 drains staging and fails closed, a census mismatch discards the tape and keeps HIP, and SiLU capture owns its arguments. Pool reclaim and retries, and partial snapshot cleanup, repair allocation failures.
• Generic (llama-family) DFlash no longer panics the daemon on a 1–3-row verify block (#819). The panic path itself is untested: no generic DFlash target/draft pair was available, so the crash was not reproduced before or after the fix. What was verified is that the non-capture path is byte-unchanged and that a registry DFlash pair serves identical text.
• North-Mini-Code (Cohere2-MoE) over hipfire serve no longer fails every request; the route now emits gen_start (#846).
• Qwen4 long-context all-row logits beyond 2 GiB are now written in full. The gfx1151 MQ4/MQ6 PM head kernels address their output through a 2 GiB raw-buffer descriptor with 32-bit byte offsets, so a single 8,192-row head over the 248,320-token vocabulary silently left scored logits unwritten. The all-row head now projects contiguous descriptor-bounded row tiles, without changing prefill chunks, context or scoring windows. A launch guard refuses oversized PM outputs instead of dropping stores, and the symmetric PM MoE down launcher refuses BF16 grouped outputs that reach 4 GiB; default 8,192-row prefill stays admitted. PM kernel bytes are unchanged.
• Flash-Next singleton native MTP no longer retires to AR at the 3-window probe. A request may retire to AR only after 16 observed windows; inside that grace a request whose options lose to AR keeps drafting at its cheapest measured depth. On Strix Halo real-text serve, retirement at the probe budget had retired 5 of 9 sampled 32K requests within their first windows, and those decoded at 28–29 tok/s against 40–46 when drafting continued. Greedy output is byte-identical and continuous-batching lanes are unchanged. An earlier form (the whole probe budget raised to 16) raised sampled 32K from 35.8 to 42.9 tok/s but cost 1–5% on short requests; this form leaves short requests' routing unchanged, and its own silicon A/B is pending. HIPFIRE_MTP_FLOOR_MIN_WINDOWS=3 restores the earlier retirement.
• The Windows installer maps named Radeon iGPUs to their gfx architecture. This is name detection, not hardware validation (#836).
• Radiowave finds the Windows HIP SDK's llvm-objdump.exe and related tools (#835).
• Test portability and clippy fixes (#850, #851).

Upgrade notes

hipfire stop
hipfire update
hipfire serve qwen3.8:flash-next

Review the q4, width, IU4, FP8 and sampling changes separately, because one opt-out does not restore all of the prior bytes. To prefer lone-request (C1) speed on a batching serve, set width 16 and turn IU4 off together; singleton-only loads are unchanged. HIPFIRE_PLE_FORMAT=q8 restores the q8 PLE format; select sidecars explicitly when download behavior matters. Matching packs are required for compiler-free use, and preload is not universal. Build from source with cargo build --release -p hipfire-daemon, not with the removed example. Retired shipped keys warn and are dropped (kernel.mw16 until 0.5.0), while never-shipped keys and typos fail. max_tokens includes thinking, so for answer-only output set chat_template_kwargs.enable_thinking: false, which keeps reasoning from exhausting the visible budget.

Three changes affect existing setups without any config change. Monitoring: scrapers that read hipfire_model_loaded must switch to hipfire_model_info{model}, and dashboards built on the old tok/s histogram buckets must be rebuilt. Reasoning effort: Qwen3.8 and Ornith 1.5 now think at medium when no effort is set; set reasoning_effort = "xhigh" for the previous depth. 27B speculation: Qwen3.8-27B loads with a paired draft on disk now use DFlash instead of MTP; HIPFIRE_DFLASH_MODE=off or hipfire config qwen3.8:27b set dflash_mode off keeps MTP.

Known issues / tradeoffs

• A lone request on a batching-configured serve pays both the width cost and the IU4 cost, and HIPFIRE_DFLASH_BLOCK=16 together with HIPFIRE_CB_VERIFY_IU4=0 restores the earlier behavior.
• Halo lane TTFT is unresolved for the PM lane rows, and HIPFIRE_FN_LANE_PM_ROWS=0 restores the earlier rows.
• Long all-resident batching front-loads prefill and raises TTFT p50, and HIPFIRE_SERVE_VMM_BATCH=0 restores serial staging.
• The first PLE request can still lag steady state under memory reclaim, and over-warming costs throughput. HIPFIRE_PLE_PREWARM_BYTES=0 turns idle prewarm off, HIPFIRE_PLE_LOAD_FILL_BYTES=0 turns load fill off, and HIPFIRE_PLE_FORMAT=q8 restores the q8 format.
• The hot-row store stays off at its default of 0 unless memory.ple_hot_store_bytes or HIPFIRE_PLE_HOT_STORE_BYTES is set, and no load-time measurement is reported for its page-cache form.
• The draft-phase slow mode occurs with F16 verify too and was not fixed by the IU4 entry, and no fix or opt-out is given.
• A later overwrite campaign observed no draft slow mode in either arm but cannot attribute why it disappeared, so it must not be called fixed; no fix or opt-out is given.
• EP has slower short-prompt TTFT, excludes lanes, refuses EP4 owner-routed prefill, and is experimental, opt-in and forced back to one card with HIPFIRE_QWEN4_EP=0.
• The FP8 task-eval drop keeps FP8 verify off, and it stays off unless HIPFIRE_CB_VERIFY_FP8=1 is set.
• A Messages request that matches a stop sequence reports end_turn with stop_sequence: null, and no fix or opt-out is given.
• The diagnostic per-layer replay enabled by HIPFIRE_GDN_REPLAY_ML_OFF=1 is not byte-identical, so leaving that variable unset avoids it.
• Cold-JIT compilation on Halo's chunk 0 remains open, the Halo quality pins require a warm kernel cache, and no fix is claimed for that cold-cache path.
• The generic DFlash small-block panic fix (#819) was not exercised on a generic DFlash pair, so the panic path is untested.
• The Flash-Next MTP AR-floor grace (16 windows) has no silicon A/B of its final form; greedy output is byte-identical, and HIPFIRE_MTP_FLOOR_MIN_WINDOWS=3 restores the earlier retirement.
• Sampled Flash-Next MTP is not seed-stable across fresh processes, because the route chooser decides from measured wall time. Fix planned for 0.4.3.
• The Strix Halo NPU trunk needs about 1.05 GB of locked memory (1,053,843,456 B). Below that limit npu_trunk=auto warns and serves GPU-only, and npu_trunk=on is a load error; the bounded host limit (LimitMEMLOCK=1318060032 / ulimit -l 1287168) is in docs/npu/npu-trunk.md.
• The first 16K-shaped request in each process pays about 80–170 ms of one-time setup on Strix Halo (first requests 7.16–7.26 s TTFT against a steady 7,079 ms in the release gate).
• Uniform MQ6G256 V1 (qt15) Qwen3.5 MoE native MTP on gfx1200/gfx1201 with FP16 WMMA is refused when a head would attach; HIPFIRE_FP16=0 permits explicit MTP. This guards a WMMA defect; it does not fix it.
• Explicit CPU MoE expert execution (memory.offload_exec=cpu) is opt-in and not bit-identical to resident or PCIe execution; follow-up for 0.4.3.
• Gemma 4 Redline (retained PM4 replay) and its Redline lowering are unvalidated and off by default; the gfx1201 shadow fails for the E4B Q8.
• Gemma 4 defaults on gfx1151 and for the 26B-A4B change numerics, and no KLD was measured. On gfx1151, HIPFIRE_GEMMA4_PREFILL_BATCH=1 HIPFIRE_GFX1151_Q8_DECODE_ATTN_GQA_GEMMA=0 restores beta bytes; the 26B-A4B fusions have no kill switch.
• Gemma 4 EAGLE (opt-in) tau moves per prompt (mean 2.821 → 2.830 on gfx1100); text equals beta on all five battery prompts.

Internal recording inventory, crate maps, CPU identity records and test expectations were maintained without additional product-speed claims.

Not in this release

• #795 (Qwen3.5 layers as declarative Steps) moves to 0.4.3.
• Flash-Next VMM sampled lanes with takeovers on did not pass their gate.
• WMMA verify attention stays off by default; it was never a default, so nothing regresses.

Contributors

Thanks to the contributors whose PRs landed in 0.4.2:

• @nwoolmer: POST /v1/systemone decide endpoint (#790).
• @HUSRCF: gfx1100 DFlash shared MQ4V2 input rotations (#838); S3/S5/S6 of #759 on gfx1201; Qwen3.5-family VMM KV admission (#833); experimental gfx1100 FA2 split-KV verifier (#817); gfx1201 MQ4V2 K5120 gate/up decode specialization.
• @fivetide: Flash-Next session-cache SSD tier (#840); resident session-cache snapshot for deep Flash-Next appends past the pool budget, e.g. +512 tokens on a 256K prefix 4.7 → 197.5 tok/s on Strix Halo (#847, HIPF-46); engine-level YaRN RoPE scaling (#856).
• @ghazni101: gfx1100 decode campaign (#845).