Skip to content

hipfire 0.4.3

Choose a tag to compare

View all tags
@github-actions github-actions released this 24 Oct 05:24
· 341 commits to master since this release
v0.4.3
1f9baa9

hipfire v0.4.3

Released 2026-10-24. The previous release is v0.4.2.1 (patch release, 2026-10-14), which followed v0.4.2 (2026-10-14). The full change list is the v0.4.3 section of CHANGELOG.md.

This release leads with prefill speed on the Radeon AI PRO R9700 for A3B (Qwen3.5/3.6-class mixture-of-experts) models, and ships the first Flash-Next sub-4-bit tiers, a large wave of community contributions, LFM2 support and the new PeaceMaker .rip / pm_loop kernel-authoring tools.

Prefill perf comparison on Qwen3.6-35B-A3B (R9700, gfx1201)

Prefill tok/s 0.4.2 (mq4r) 0.4.3 (mq4r) 0.4.3 (mq4rs) Speedup
pp512 3,292 3,332 11,589 3.5x
pp1024 3,248 3,607 15,718 4.8x
pp4096 2,886 3,737 19,828 6.9x
pp8192 2,488 3,615 20,035 8.1x
pp16384 1,957 3,561 18,545 9.5x

The new qwen3.6:35b-a3b-mq4rs quant is where most of the gain comes from: 3.5x to 5.5x faster than the same 0.4.3 engine on the old mq4r file. The 0.4.3 engine alone is 1.0x to 1.8x on mq4r, and it removes 0.4.2's long-context falloff (0.4.2 loses 41% from pp512 to pp16384; 0.4.3 on mq4r does not fall off). Users on the old mq4r should pull mq4rs. The speedup column is 0.4.3 mq4rs over 0.4.2 mq4r.

Fast mq4rs prefill is gfx1201 only. On other GPUs the mq4rs file needs HIPFIRE_MOE_SYM_IU4=0 (about 5x slower prefill).

How it was measured: card B (R9700), hipfire bench --matrix --pp 512,1024,4096,8192,16384 --tg 1 --ctx 1 --runs 3 --warmups 1 --spec off, each cell the median of 9 samples from 3 fresh processes, arms interleaved in one timing hold (job j3197). "0.4.2" here means v0.4.2.1. Each version ran its default config, and the defaults differ: 0.4.2.1 uses KV q8 and a 512-row prefill chunk; 0.4.3 uses fp8 KV and an 8192-row chunk; and the symmetric IU4 MoE prefill route is only admitted on the mq4rs file. The 0.4.2 to 0.4.3 mq4r column is therefore "what a user gets from upgrading without changing the model file", not a single-variable engine A/B. The mq4rs arm used the published qwen3.6:35b-a3b-mq4rs file (sha256 8806a742d558ebe43256010096cd82cfac3a20be5218ad56a3f74267db001894, identical to the file on Hugging Face), and the mq4r arms used qwen3.6:35b-a3b-mq4r (sha256 4685c140…, matching its registry pin). Raw data: a3b-compare/raw/, report A3B-PREFILL-042-VS-043.md.

Detail, this cycle's final lever steps on qwen3.6-35b-a3b-d53.mq4rs only (not part of the table): pp16000 (two chunks, 8192 + 7808 rows), 3 fresh processes per arm, three separate same-session A/Bs: 16,084.5 → 16,694.8 (+3.8%), 16,670.6 → 18,021.6 (+8.1%), 18,341.9 → 18,587.9 (+1.3%), +15.6% in total as the sum of the three, not a single A/B (the card runs a few percent slower on some days than others; the plan recorded about 3% between 20 and 22 October). These routes are exact-gfx1201 defaults, byte-identical to the code they replace: KLD unchanged on qwen3.6-35b-a3b-d53.mq4rs (the four-corpus KLD sequence is byte-equal to the previous build at every step, served text identical). This does not hold for the asymmetric qwen3.6:35b-a3b-mq4r file, whose prefill numerics change slightly (see Known issues). The benchmarked file uses the symmetric expert recipe (qt44 gate/up with a qt53 down projection, built with hipfire-quantize --moe-expert-down mq4g128v2-symmetric); see Known issues for what that means for the published A3B files.

Every other number below comes from a named change-level measurement, not from a cumulative release matrix. Each row names its card, fixture and method. Numbers from different sessions are never silently chained: where a headline is the sum of several steps, the steps are listed. The A3B table above is a version-to-version comparison, not a change-level measurement, and its caveats are stated under it.

Headline numbers

Change Hardware and fixture Result Quality
Flash-Next MQ6 INT8 prefill, now default on gfx1201 One R9700, Flash-Next, pp16384 at the default 4096-row chunk, 3 fresh processes per arm 4,720.5 → 4,899.3 tok/s (+3.8%) against the previous F16 route Quality bar passed (paired KLD change within the +0.010 budget on both corpora); final route is byte-identical to the INT8 HIP route
Flash-Next compact forward scratch, now the default on gfx1201 (measured at the 4096-row chunk) One R9700 (card A), canonical split base qwen3.8-flash-next-base.mq4-xts, hipfire bench --matrix pp16384 and tg128 plus --spec auto, 3 fresh processes per arm, Full vs compact scratch, both arms at the 4096-row chunk. The gptq3 monolith takes the same default Forward scratch 4,388 → 1,270 MiB; placement reserve 7,695 → 4,576 MiB; experts resident in VRAM 6,948 → 8,234 (+1,286). pp16384 5,024 → 5,405 tok/s (+7.6%); tg128 33.56 → 34.84 tok/s (+3.8%); --spec auto prose 45.8 → 49.0 (+7.0%), code 68.2 → 73.9 (+8.4%). Opt out with HIPFIRE_QWEN4_COMPACT_SCRATCH=0. The CPU symmetry scan is also gone from load (8 s warm / 229–310 s cold → 0.1 ms) Byte-identical: kldseq WT2 and code24 equal Full vs compact, serve AR and MTP transcripts byte-identical, greedy_mtp_matches_ar 2/2 on the monolith
Flash-Next default prefill chunk is now 16,384 rows on gfx1201 (with compact scratch): faster prefill, slower decode One R9700 (card A), gptq3 monolith, hipfire bench --matrix pp16384, AR tg128 (synthetic: random token ids, no routing locality, so it understates real-text decode) and --spec auto (fiction_lighthouse prose, humaneval_0_has_close_elements code prompt files), 3 fresh processes per arm, 16,384-row chunk vs the 4096-row chunk, both with compact scratch pp16384 5,405 → 6,431 tok/s (+19.0%). Decode costs more than at 4096: AR tg128 (synthetic) 34.88 → 33.11 (−5.1%); --spec auto prose 49.1 → 44.3 (−9.8%), code 73.7 → 66.1 (−10.3%); resident experts 8,234 → 6,578. HIPFIRE_PREFILL_CHUNK_ROWS=4096 is the documented override for decode-heavy workloads. Plain AR on real text at the new default with the hot set on: 49.4 / 41.6 / 36.9 tok/s (glimmer coding prompt / fiction prose / code edit; card C, HOTSET-DEFAULT-CHECK.md). At the 16,384 chunk the compact forward scratch is 8,520 → 4,776 MiB (−3,744 MiB) against the old full-allocation compact plan Byte-identical: kldseq WT2 22e45981… and code24 88038c3b… unchanged and serve AR+MTP transcripts byte-identical at 16,384 + compact (scratch-diet landing and W1–W4 gates), greedy_mtp_matches_ar pass
Flash-Next prefill, four exact-kernel waves (W1–W4) One R9700 (card B), Flash-Next, pp16384 at the 16,384-row chunk with compact scratch (now the default; measured with HIPFIRE_PREFILL_CHUNK_ROWS=16384 HIPFIRE_QWEN4_COMPACT_SCRATCH=1), hipfire bench --matrix, 3 fresh processes per arm, each wave an interleaved same-session A/B W1 9791018512: 5,506.8 → 5,926.1 (+7.6%). W2 4be4d09b60: 5,903.9 → 6,028.8 (+2.1%). W3 e21244d890: 6,037.2 → 6,385.5 (+5.8%). W4 9d74311a53: 6,380.1 → 6,449.8 (+1.1%). The four steps were measured in separate sessions and are not chained into one total. With HIPFIRE_PREFILL_CHUNK_ROWS=4096 the pass is limited by the PCIe expert stream and these wins mostly hide: the W1 build measured about 5,015 tok/s there with the full scratch (card B, fn-scratch-diet/CENSUS.md). W2–W4 were not separately measured at 4096 Byte-identical: kldseq code24 and WT2 identical at every wave; Redline decode hash unchanged and shadow bit-exact; serve AR and MTP transcripts identical
Halo Flash-Next 16K prefill, NPU on Strix Halo + XDNA2, 16,355-token prompt, client TTFT, 3 fresh processes × 3 requests, all-9 median, method of job j1745 with the TTM-pool drain fixed 2,365.6 tok/s median, below the 2,386 gate (−20.4, −0.9%). Kaden accepted the miss for 0.4.3. The nine samples: 2,314.2 / 2,404.9 / 2,403.4, then 2,316.0 / 2,406.8 / 2,404.5, then 2,208.9 / 2,350.9 / 2,365.6. Four requests ran 2,403–2,407 tok/s, matching the 2,395.5 of the 10-17 pass on the earlier head 0d06c380b (v0.4.2 shipped 2,386); the miss comes from slow first requests after the drain. Measured on head 9d74311a53 (see the receipts note) n/a (exact kernels)

Highlights

  • A3B prefill is 3.5x to 9.5x faster on the R9700 with the new qwen3.6:35b-a3b-mq4rs quant, and 1.0x to 1.8x faster on the old mq4r from the engine alone (table at the top; 0.4.2.1 vs 0.4.3, each version's default config; fast mq4rs prefill is gfx1201 only). No quality change on qwen3.6-35b-a3b-d53.mq4rs from this cycle's lever steps. Four byte-identical PeaceMaker routes that were opt-in are now defaults on exact gfx1201 (+3.8%). A wave-per-row router and a stable multi-workgroup scatter replace the slow single-workgroup path, the shared-expert down projection folds its sigmoid gate and residual into one kernel, the RMSNorm and sigmoid gate that fed the activation producer are folded into it, and the flash-attention kernel is about 13% faster per layer (+8.1% together). The MoE down projection now reads a grouped-position activation plane (+1.3%). On qwen3.6-35b-a3b-d53.mq4rs every step is gated by byte-for-byte kernel oracles, a byte-equal KLD sequence, identical served text and Redline replay. The asymmetric qwen3.6:35b-a3b-mq4r file is different: its prefill numerics change slightly (see Known issues). HIPFIRE_MOE_ROTATE_I4_PM=0, HIPFIRE_PREFILL_IU4_C8_PM=0, HIPFIRE_GDN_CHUNK_PREP_PM=0 and HIPFIRE_MOE_SHARED_DOWN_SIGMOID_EPI=0 restore the earlier routes.
  • Flash-Next MQ6 prefill uses an exact INT8 GEMM by default on gfx1201. The 6-bit weight codes enter the INT8 matrix instruction exactly, a one-pass producer replaces a two-pass activation quantizer, and a PeaceMaker twin replaces the hipcc kernel. +3.8% at pp16384 on one R9700. HIPFIRE_MQ6_I8=0 restores the F16 route and HIPFIRE_MQ6_I8_PM=0 keeps INT8 but uses the hipcc GEMM. Strix Halo is unchanged: the NPU takes those projections at 16K, and the INT8 kernel measured slower than F16 on RDNA 3.5, so it stays opt-in there.
  • Flash-Next gets smaller sub-4-bit tiers: mq3-xts and mq2-xts. See "New Flash-Next model tiers". Model cards with measured KLD, including the 7900 XTX numbers, are on Hugging Face.
  • Strix Halo repetitive-prompt NaN is fixed, and it costs no prefill speed. A long prompt with many repeated tokens could turn the Flash-Next/Qwen GDN prefill NaN on Strix Halo and the 7900 XTX (the router guard then failed the request). The chunked kernel now inverts its matrix in F32. The first version of the fix cost 238 ms of a 16K prefill on Halo (GDN chunk kernel 372 → 611 ms); the final version runs at 364.6 ms, below v0.4.2.
  • LFM2.5 support (#885 by @ghazni101 on top of @fivetide's decode port): decode as typed dispatch steps for LFM2.5 dense and MoE models, correct think-channel and tool-call shapes, real stop tokens, a fix for the MoE down kernel that made LFM2.5-8B-A1B emit degenerate text, and bring-up of LFM2.5-24B-A2B with partial-GPU offload.
  • PeaceMaker .rip and pm_loop kernel tooling. .rip scripts can now compose the production iu4_gemm, mq6_i8_gemm and fp8_gemm generators and the grouped MoE kernels, run on a CPU emulator that is checked against silicon, and pm_loop closes the iteration loop in one command (emit, certify, check against a CPU reference and the shipped kernel, time with ABBA). Developer tooling only; no serving path changes except where listed elsewhere.
  • 55 community PRs from three contributors landed (see Contributors), including Flash-Next DFlash2 speculation, the generic device-mesh executor with the first tensor-parallel forwards, Qwen3.5 and DeepSeek4 decode and prefill moved onto typed dispatch steps, gfx1151 decode and DFlash speedups, and gfx1100 decode work.
  • Flash-Next on the R9700 keeps its hot experts resident by default (trade-off, below), and Flash-Next on a 7900 XTX gets a narrow-link prefill path (below).
  • Flash-Next on the R9700 now prefills 19% faster by default and decodes 5–10% slower in the measurements below; HIPFIRE_PREFILL_CHUNK_ROWS=4096 restores the old decode speed. On gfx1201 the default prefill chunk for Flash-Next is now 16,384 rows, with the compact forward scratch (on by default for symmetric packed Flash-Next: the canonical split base qwen3.8-flash-next-base.mq4-xts and the gptq3 monolith). Against the same build at the 4096-row chunk (card A, gptq3 monolith, 3 fresh processes per arm): pp16384 5,405 → 6,431 tok/s (+19.0%), but AR tg128 34.88 → 33.11 (−5.1%, synthetic tg128: random token ids, no routing locality, which understates real-text decode), --spec auto prose 49.1 → 44.3 (−9.8%), code 73.7 → 66.1 (−10.3%) (the fiction_lighthouse and humaneval_0_has_close_elements prompt files), and resident experts 8,234 → 6,578, because the larger chunk reserves more VRAM for its forward scratch. If your workload is short prompts with long generations, set HIPFIRE_PREFILL_CHUNK_ROWS=4096. Output is byte-identical (kldseq WT2 and code24, serve AR and MTP transcripts). For scale, plain AR decode on real text at the new default (hot set on) measures 49.4 / 41.6 / 36.9 tok/s (glimmer coding prompt / fiction prose / code edit; card C, split base, q4 PLE rows, 128 tokens, 3 fresh processes per arm, HOTSET-DEFAULT-CHECK.md).
  • The expert hot-set sidecar is pulled and applied by default on gfx1201 (hipfire pull qwen3.8:flash-next fetches the 213 KB .expert-hotset with no flag; the load applies it on gfx1201 only). On real text it is worth +14% to +26% MTP decode (fiction prose +14.4%, glimmer coding prompt +26.3%) and +7% to +40% plain AR, with no prefill loss (pp16384 +0.5%) and byte-identical output (decoded text equal with and without the sidecar). The copy-heavy code edit under MTP is the exception, −3.1%. Measured on card C against the same split base without the sidecar, at the 16,384 default chunk (HOTSET-DEFAULT-CHECK.md), on q4 PLE rows. The gfx1201 default PLE is q8; it was checked on the RC and is speed-neutral against q4 (pp16384 +0.2%, plain AR decode −0.6% to −1.6%; Q8-PLE-RC-CHECK.md). HIPFIRE_QWEN4_EXPERT_HOTSET=0 turns it off.
  • The compact forward scratch is the default on gfx1201. On the split base at the 4096-row chunk (card A) it cuts the forward scratch from 4,388 to 1,270 MiB and the placement reserve from 7,695 to 4,576 MiB, so the load keeps 8,234 experts resident instead of 6,948: pp16384 5,024 → 5,405 tok/s (+7.6%), tg128 33.56 → 34.84 (+3.8%), --spec auto prose 45.8 → 49.0 (+7.0%), code 68.2 → 73.9 (+8.4%). At the 16,384 chunk the compact scratch is 4,776 MiB (8,520 MiB with the old plan). HIPFIRE_QWEN4_COMPACT_SCRATCH=0 restores the full scratch. The CPU symmetry scan is also gone from load (8 s warm / 229–310 s cold → 0.1 ms; a GPU proof of all 48 routed layers at attach decides, and a refusal re-plans the full allocation).
  • Four more exact Flash-Next prefill waves (W1–W4) are default-on on gfx1201. They are byte-identical kernels: HC row folds, GDN gate → A8 fusion, MoE-front producer fusion and fused HC down (W1); the MQ6 INT8 128×256 tile default with a packed BF16 epilogue (W2); HC-up v2 retile and MoE row-fold/mixer retune (W3); the load-batched GDN producer v2 (W4); the BF16 qkv store stays opt-in. Their gains are measured at the 16,384-row chunk (+7.6%, +2.1%, +5.8%, +1.1%, separate sessions, not chained); at a 4096-row chunk the PCIe expert stream hides most of them. Kill switches: HIPFIRE_QWEN4_HC_ROW_FOLD=0, HIPFIRE_QWEN4_GDN_I8_FUSE=0, HIPFIRE_QWEN4_MOE_FRONT_FUSE=0, HIPFIRE_QWEN4_HC_DOWN_FUSED=0 (W1), HIPFIRE_QWEN4_HC_UP_V2=0 (W3), HIPFIRE_QWEN4_GDN_CHAIN_FUSE=0 (W4). Sources are MERGE-QUEUE-LOG.md entries 20cc3979e8 (16,384 default chunk), c55f00cc44, 62507eaad5, 02134ca613, 60f47e4022, 9791018512, 4be4d09b60, e21244d890, 9d74311a53.
  • First-class container path (compose.yml, ROCm 10.1 full and slim runtime images). The full image is published at ghcr.io/warpfront/hipfire:0.4.3 (also :latest), built and pushed by the tag's container-image workflow.
  • New commands: hipfire fit <model>, hipfire doctor (runtime and driver preflight), hipfire calibrate, hipfire doctor --npu, and hipfire config tuning presets.

New Flash-Next model tiers: mq3-xts and mq2-xts

Two new split-base Flash-Next files, smaller than the 125 GB mq4-xts. Both load their per-token embedding rows (PLE) from a shared sidecar pair, qwen3.8-flash-next.ple-q4 (29.1 GB) or .ple-q8 (54.4 GB), so one sidecar serves every tier.

Tag Base size Download with q4 / q8 PLE What it is GPUs
qwen3.8:flash-next-mq3-xts 59.1 GB ~88.2 GB / ~113.5 GB 3-bit experts on 16,839 of 24,576 routed experts; everything else is the MQ4 XTS bytes R9700 (gfx1201), Strix Halo (gfx1151)
qwen3.8:flash-next-mq2-xts 46.9 GB ~76.0 GB / ~101.3 GB 6,306 experts stay 4-bit; the other 18,270 are 2-bit (E8P) 7900 XTX (gfx1100), R9700 (gfx1201)
qwen3.8:flash-next-mq2-xts-4500 44.5 GB ~73.7 GB / ~98.9 GB 4,500 experts stay 4-bit; the other 20,076 are 2-bit (E8P) 7900 XTX (gfx1100), R9700 (gfx1201)
  • These tiers lose some quality against the default mq4-xts. The model cards on Hugging Face (hipfire-models/qwen3.8-flash-next) carry the measured KLD for each tier, including 7900 XTX (gfx1100) numbers for the MQ2 roots. For example, mq2-xts (6,306 hot experts) decode on a 7900 XTX measures KLD 0.1966 on WikiText-2. KLD numbers are never comparable across tiers or against MQ4 without reading the card's scope notes.
  • Strix Halo refuses the mq2 tiers cleanly in this release.
  • hipfire pull asks which tier and which PLE format you want. q4 PLE is preselected on Strix Halo; q8 is the exact reference and is never silently replaced when you ask for it.
  • Minimum builds: mq3-xts needs ca0aee9c9 or later and mq2-xts needs ea2abd111 or later. This release satisfies both.
  • These files use a per-expert container (the "slab" format), different from the older dense mq3 and from the earlier, refused MQ2 formats.
  • Not supported on these tiers: expert parallelism and CPU expert offload. Pipeline parallelism supports them (see below). They otherwise load single-device.

Runtime support for pruned ("coder") models. The engine can load a Flash-Next file that keeps only a subset of routed experts per layer; removed experts store no bytes and the router masks them out. Models without a keep list are unaffected, byte for byte. No pruned coder model is in the registry in 0.4.3. An experimental unregistered MQ3-XTS coder build (k243) is on the Hugging Face repo for testing only.

Flash-Next on a single R9700: expert hot set on by default (read the trade-off)

On an R9700, Flash-Next mq4-xts keeps the most-used experts resident in VRAM instead of whole layers (chosen from a decode-routing sidecar that hipfire pull fetches). It is exact. It speeds up long prefill and costs some short-context decode and short prefill:

Measured on one R9700 Change
vs hot set off, prefill 8K and 16K +18% to +19%
vs hot set off, short-context decode −9.5%
vs hot set off, pp512 −6%
vs v0.4.2.1, long prefill (2K / 8K / 16K) +15% to +23%
vs v0.4.2.1, decode (ctx 128 to 20K) −3% to −11% (worst at ctx 128)

HIPFIRE_QWEN4_EXPERT_HOTSET=0 restores whole-layer placement. If your workload is short prompts with long generations, set it. A phase-aware budget is planned for 0.4.4. Dense models and Strix Halo are not affected. The hot set is gfx1201 only by default.
[NOT RE-MEASURED at RC 1851b72ce7: the table above was measured before the forward-scratch diet, the W1–W4 waves, the compact-scratch default and the 16,384-row default chunk. The default chunk is now 16,384 rows and keeps 1,656 fewer experts resident than the compact scratch at 4096 (8,234 → 6,578 on the gptq3 monolith), which changes hot-set placement and the short-context decode trade-off (measured: AR tg128 −5.1% against compact at 4096, a chunk comparison, not the hot-set on/off comparison). Re-measure the hot-set on/off and v0.4.2.1 rows on the RC head, or keep the "measured before" caveat when publishing.]

Strix Halo (gfx1151)

  • Flash-Next keeps the GatedDeltaNet recurrent state in F32 by default (memory.state_quant; state_quant = "q8" restores the previous Q8 state). This is not bit-identical to the old default. Measured against BF16, Flash-Next KLD improves by 0.0068 (code) and 0.0118 (WikiText-2 subset), and cost stayed within ±2%. The F32 state is larger and admission charges for it; hipfire never lowers context or lanes on its own.
  • Decode and speculation, measured by the contributor (@fivetide, #908, #909, #930), not re-measured on Halo hardware by the release gates: 27B DFlash with a Q8_0 lm_head decodes 1.8× faster (the 16-row lm_head is 6× faster, #908); Flash-Next 8-request AR throughput 53.7 → 75 tok/s aggregate through lane promotion, two-launch GDN lanes and PeaceMaker row kernels (#909); Flash-Next native-MTP decode +6% from PeaceMaker multi-column verify kernels, and DFlash2 verifies only the draft's confident prefix (block 8, 42.5 → 52.0 tok/s greedy) (#930).
  • Muse Glimmer prefill 5× faster and DFlash decode ~3× faster, bitwise identical (@fivetide, #900).
  • Agent-turn appends under 512 rows use the faster ≥512-row kernels; per-token PLE fetch no longer stalls the GPU; split-K decode attention; exact QSA select and pooled mirror (256K decode −5% latency). q8 QSA K/V is the default on gfx1151 and gfx1100.
  • NPU trunk: auto learns its rates live; an NPU fault no longer fails later requests; the burst of amdxdna … Failed insert pages kernel lines on Linux 7.1+ is gone. New hipfire doctor --npu plus a power-mode check (a machine left on balanced power is slower with no error).

7900 XTX (gfx1100) and narrow PCIe links

  • Flash-Next prefill on a narrow link. On a measured sub-16 GB/s link (the XTX at x4, 7.09 GB/s) the default prefill chunk becomes 8192 rows and mq2 host experts stage through borrowed slab VRAM: pp8192 265.1 → 931.9 tok/s (3.52×), at a cost of pp512 −14.4% and decode −10.7% (one hold, exact beta d8865a16b). The layer-major S1 prefill path selects itself from 1,537 rows; its measured base at pp16384 is 1,738.9–1,750.2 tok/s (about 1.87× beta), an accepted-with-exception figure (two fresh processes, six samples per arm), not a release-gate statistic. A new hipfire calibrate record lets the engine stop probing the link at every load, and the CPU "D2" split is a measured per-load default only where the CPU provably beats the link.
  • Contributor work: default FA2 split-KV verifier on exact gfx1100 (+13.6% DFlash, @HUSRCF #894), dense Qwen3.5 on Exact VMM AR lanes (@HUSRCF #897), exact TP2 whole-token HIP graphs now default on gfx1100 and gfx1201 pairs (@HUSRCF #903).

Fixes

  • A repetitive prompt no longer makes Flash-Next/Qwen GDN prefill NaN on gfx1151 and gfx1100 (above).
  • Qwen3.5 MoE (A3B, Ornith) DFlash and native MTP produced corrupted greedy output after a regression: the batched MoE DeltaNet body stopped writing the GDN tape (on Strix Halo, A3B DFlash τ fell from 7.47 to 2.5). Restored by @fivetide (#935); A3B DFlash and MTP greedy output is byte-identical to the earlier build again.
  • Native MTP on R9700 expert-slab files crashed after the first token (an unset kernel argument); fixed with a regression test that checks every embedded entry.
  • A NaN or infinity in a Flash-Next MoE router row now fails the request closed with the layer and row named, instead of faulting the GPU. Expert consumers range-check ids.
  • Flash-Next on the R9700: auto expert placement now charges what a full-context request and the long-prefill stage really take (fixes an out-of-memory at a 763-token prompt on a 32 GB card and a 16K prompt on mq2).
  • Flash-Next load no longer page-faults every expert header (load phase 177.9 s → under 0.25 s on the affected path); discrete GPUs read weights in parallel from a cold page cache.
  • A client abort during a native-MTP or DSpark prefill is a cancel, not a validation error (#822).
  • Expert-parallel prefill no longer overflows a peer rank's stack on a ~500-token prompt; thinking-off requests speculate again; a turn the AR floor retired continues live in the next turn.
  • qwen3.5:27b-mq3 / qwen3.6:27b-mq3 decode (#789), the expanded MoE down kernel for moe_intermediate_size other than 512, gemm_q8_0_batched above 64 rows (#732), the Nix flake build (#645), ParoQuant group-size-64 checkpoints (#651), Gemma 4 EAGLE sampled/penalized/logprobs requests (#837).
  • hipfire pull/quantize check HF_TOKEN up front; config profiles merge instead of replacing the global config; CU masking is refused on RDNA after it hung an R9700's scheduler; GPU spin waits on peer cards are bounded at 500 ms.

Models, tools and platform

  • LFM2.5: see Highlights. The registry entries (lfm2.5:350m, :1.2b, :1.2b-thinking, :8b-a1b) are unchanged; 24B-A2B has no registry tag in this release.
  • EmbeddingGemma 2 (google/embeddinggemma-2): single-vector embeddings for text, images, audio and video through hipfire embed and POST /v1/embeddings (@ghazni101, #888). Registry tag embeddinggemma-2:q8f16.
  • hipfire pull asks which tier, PLE format and draft sidecars you want on a terminal, shows the download total and checks free disk; flags and non-terminal use keep the previous defaults. hipfire fit <model> [--ctx N] says what fits using the engine's own admission code. hipfire doctor is now a runtime/driver preflight (doctor --npu is unchanged). hipfire config list groups settings by intent, config tuning previews and applies speed-versus-quality presets, and the TUI Settings tab edits every key inline.
  • Flash-Next pipeline parallelism (opt-in, experimental, host-staged, no peer-to-peer needed): topology.pp 2 to 8. Greedy native MTP supported; mq2-xts/mq3-xts load under PP. Qualified on 3× R9700 byte-identical to one card; no 7900 XTX run and no performance claim.
  • Expert parallelism now admits 2 to 8 cards of one exact architecture, derived from one sealed mesh plan (@fivetide, #905). Opt-in and experimental.
  • Device-mesh executor (@fivetide, #931–#936): tensor-parallel collectives, a GatedMlp split rule, and the first generic tensor-parallel forwards for dense Qwen3.5/3.6/3.8, matching the hand-written route byte for byte under emulation. Not a serving default.
  • Typed dispatch: Qwen3.5/3.6/3.8, DeepSeek4, Muse Glimmer, LFM2, MiniMax-M2 and Gemma 4 decode/prefill/verify bodies now run as typed dispatch step programs (@fivetide, #881–#889, #915–#919, #929, #933). Behavior is bit-identical to the previous builds on the gates that ran.
  • Serve: optional bearer-key authentication (serve.api_key_file); multi-slot engine metrics on /metrics; Gemma 4 serve returns non-stream bodies, streams finish with usage, and tool calls work.
  • Windows: the CLI and daemon cross-build for x86_64-pc-windows-msvc; hipfire stop, ps and diag work natively. Windows GPU serving is not part of this release.
  • MI300X (gfx942): hipfire-native FP8 Flash-Next from the stock FP8 checkpoint (hipfire-quantize --recipe stock-fp8); no throughput claim.
  • Container: compose.yml, a ROCm 10.1 full image (default, carries hipcc) and a slim runtime image, baked gfx1100/gfx1151/gfx1201 kernel packs, non-root entrypoint, rootless-podman fixes. docs/CONTAINER.md is a quickstart.
  • Kernel packs for gfx1100, gfx1151, gfx1201, gfx906 and gfx942 are attached to this release, as in v0.4.2.

PeaceMaker .rip and pm_loop (developer tooling)

  • peacemaker-rip check | fmt | explain | features | run: located diagnostics with fix-its, an offline fact sheet (instruction count, registers, LDS) before any GPU time, and a CPU run of every kernel on a wave32 emulator.
  • The emulator is checked against silicon (hw_diff runs the same code object on a card and on the emulator and reports three verdicts), and its instruction table no longer refuses instructions it can execute.
  • .rip composes the production iu4_gemm (including the new 2:4-sparse SWMMAC K-loops), mq6_i8_gemm (with a tile axis) and fp8_gemm generators, and the grouped MoE gate/up/down kernels, each pinned to the shipped bytes by digest.
  • pm_loop emits a variant, certifies it, checks it against a CPU reference and the shipped object, ABBA-times it, and prints a JSON record, and says when a difference is immaterial. It covers the A3B prefill producers and the gfx1201 decode projections.
  • Certification results are cached by content, so re-checking unchanged bytes is free.

Upgrade notes

  • Halo Flash-Next: F32 GDN state is the default (state_quant = "q8" restores the previous numerics). Generation text can differ from 0.4.2.1 on gfx1151.
  • Flash-Next on gfx1201: the default prefill chunk is 16,384 rows (with the compact forward scratch). Prefill is faster and decode is 5–10% slower; set HIPFIRE_PREFILL_CHUNK_ROWS=4096 for decode-heavy workloads (see Highlights and Known issues).
  • Flash-Next on the R9700: hot-set default (above).
  • The new registry catalog (generic ple map, split-base MQ4 XTS aliases, retired tags) is published to master at tag time. v0.4.2 and v0.4.2.1 clients parse registry fields strictly, and the new catalog adds a ple field they do not know, so they may fail to parse it and fall back to their cached or bundled catalog (derived from the v0.4.2.1 source and the catalog diff; not exercised on an old binary). hipfire registry status prints the warning. Update to 0.4.3 to get the new tiers. Monolithic Hugging Face files stay published for existing clients.
  • qwen3.8:flash-next-rtn-asym and qwen3.8:flash-next-mq6q8-pleq8 are retired from new pulls (use qwen3.8:flash-next); already-installed verified files keep running.
  • Windows GPU serving, the optional bearer key and CU-mask refusal are described above.

Known issues / trade-offs

  • The A3B speed-up applies to files built with the symmetric expert recipe, and qwen3.6:35b-a3b-mq4rs is that file. The benchmarked qwen3.6-35b-a3b-d53.mq4rs is published at hipfire-models/qwen3.6-35b-a3b (Hugging Face commit 600165f69f, 18.70 GB / 17.42 GiB, sha256 8806a742…1894) as registry tag qwen3.6:35b-a3b-mq4rs. Its KLD on WikiText-2 is 0.129, against 0.206 for qwen3.6:35b-a3b-mq4r, measured with the same harness. Caveats: the fast prefill is gfx1201 only; on other GPUs set HIPFIRE_MOE_SYM_IU4=0 (prefill is about 5× slower); there is no MTP sidecar yet; and the router and lm_head are 4-bit (built with --no-q8-router). The routes are on by capability (qt44 gate/up with qt53 down, verified on every MoE layer); the older qwen3.6:35b-a3b-mq4r and ornith-1.5:35b-a3b-mq4r files are not claimed to use them, and no Ornith mq4rs file is announced here.
  • A3B qwen3.6:35b-a3b-mq4r (the asymmetric file): prefill numerics change slightly. The 0.4.3 A3B prefill levers that came in with the FP8 integration (c4c058a20f) are not byte-identical on this file. WikiText-2 prefill KLD goes 0.2082 → 0.2115 (Δ +0.0034, CI95 [+0.0009, +0.0058]), inside the quality bar; decode per-token KLD is unchanged. Greedy output can differ from 0.4.2 at near-ties. Evidence: a3b-mq4r-divergence/. This does not apply to qwen3.6:35b-a3b-mq4rs, whose levers were byte-identical across the A3B work.
  • R9700 hot-set trade-off: short-context decode and pp512 are slower than with the hot set off.
  • Flash-Next prefill is faster by default on gfx1201 and decode is slower: the default chunk is 16,384 rows. Against the compact scratch at 4096 rows (card A, gptq3 monolith), pp16384 is +19.0% (5,405 → 6,431 tok/s) but AR tg128 is −5.1% (34.88 → 33.11; synthetic tg128 with random token ids, no routing locality, so it understates real-text decode), --spec auto prose −9.8% (49.1 → 44.3) and code −10.3% (73.7 → 66.1), with resident experts 8,234 → 6,578. For decode-heavy workloads (short prompts, long generations) set HIPFIRE_PREFILL_CHUNK_ROWS=4096. The 16,384 default applies under the compact scratch; under the full scratch (HIPFIRE_QWEN4_COMPACT_SCRATCH=0) the default chunk stays 4096, and load admission can still drop to a smaller rung (8192, 4096, 2048, 1536) when memory is short. W1–W4 gains are largest at the 16,384 chunk; at 4096 the PCIe expert stream hides most of them. The compact forward scratch is the gfx1201 default for symmetric packed Flash-Next (split base and gptq3 monolith); other Flash-Next artifacts follow the engine's own eligibility rules and fall back to the full scratch. Receipts: fn-scratch-diet/runs/bench/c16.
  • A default hipfire pull qwen3.8:flash-next on gfx1201 also downloads the 0.306 GB trunk overlay, which only gfx1151 applies (the loader skips it elsewhere). It is wasted bandwidth and disk, not a correctness problem; the fix is in 0.4.4. The default pull also selects q8 PLE rows on gfx1201; the Flash-Next gates and the hot-set measurement used q4, and q8 was then checked on the RC and found speed-neutral (Q8-PLE-RC-CHECK.md).
  • 7900 XTX: pp512 and decode pay for the narrow-link prefill chunk (above); only one P1 hold of evidence.
  • Strix Halo: the 16K gate (≥ 2,386 tok/s) was missed on head 9d74311a53 at 2,365.6 tok/s because of slow first requests after the TTM-pool drain (steady state 2,403–2,407); accepted for 0.4.3. First requests in each process pay about 80–170 ms of one-time setup; a rare GPU memory fault during deep prefill remains open (docs/KNOWN-ISSUES.md).
  • Strix Halo: each process exit leaves about 80 GiB in the kernel TTM page pool, and the loader's RAM check does not count it. A back-to-back reload can therefore be refused until that memory frees. Fix planned for 0.4.4.
  • Host-placed expert CPU execution (memory.offload_exec=cpu) is opt-in and not bit-identical to GPU execution; the measured per-load D2 default only arms where the CPU wins.
  • PP is host-staged, experimental, and prompts under 4,096 tokens always prefill cold.
  • Several gfx1151 results above are the contributors' own measurements; the merge gates ran CPU tests, identity and serve checks on gfx1201 but had no Halo hardware for those PRs.
  • Native Windows GPU serving is not part of this release.

Not in this release

  • Flash-Next row-exact wide verify (caps stay 8) and the derived-rate MTP depth chooser: landed default-off; no speed claim.
  • A3B speculation (AR floor, n-gram, derived K): dropped for 0.4.3.
  • q8 error-feedback GDN state (state_quant = "q8_ef"): opt-in only, never selected by auto.
  • Pruned coder model in the registry, R9700 phase-aware hot-set budget, and the XTX slow-link redesign: 0.4.4.
  • Open community PRs that were not ready: #870 (DeepSeek4 expert paging, draft), #920–#922 (gfx1201 FA2 split-KV verifier drafts), #939 and #940 (gfx1100 drafts).

Contributors

Thanks to the contributors whose PRs landed in 0.4.3. Authorship was preserved on every landed commit.

  • @fivetide (the large majority of the community work this cycle): merged on GitHub: #881, #883, #886, #889 (Glimmer and DeepSeek4 on typed dispatch), #893 (GatedMlp design), #905 (Flash-Next expert parallelism from one sealed mesh plan), #907 (retire greedy DFlash to AR below break-even), #908 (gfx1151 Q8_0 lm_head, DFlash 1.8×), #909 and #930 (Flash-Next decode and speculative verify on gfx1151), #910 (Nix flake), #911 (Gemma 4 EAGLE sampled verify), #912 (legacy MQ3 decode), #913 (ParoQ4G64), #914 (batched Q8 lm_head hardening), #915 and #917–#919 (Qwen3.5 on typed dispatch), #916 (Redline refuses unrecorded launches), #924 (qwen2 reset gate), #928 (Flash-Next DFlash2 speculator), #929 and #933 (GatedMlp op), #931, #932, #934, #936 (device-mesh tensor parallelism), #935 (restore the GDN tape copy), #937 (one-row attend position). Landed on beta on 2026-10-17 from closed PRs: #871–#879 (FWHT K quantizer thresholds, test fixes, buffer-pool reuse, LFM2 typed dispatch and stop ids, MoE down K≠512), #884 (MiniMax typed dispatch), #891, #892, #895, #896, #898–#901 (Glimmer cleanup, DS4 DSpark/SWA fixes, /metrics, gfx1151 batched WMMA fix), #902 (gfx11 W4A8 prefill, opt-in), #904 (Qwen MoE context on unified memory).
  • @ghazni101: LFM2.5-24B-A2B bring-up with partial-GPU offload (#885); EmbeddingGemma 2 (#888).
  • @HUSRCF: gfx1100 Exact VMM AR lanes (#897); default FA2 split-KV verifier on gfx1100, +13.6% DFlash (#894); exact TP2 whole-token HIP graphs (#903).
  • Draft PRs that did not make this release stay open: #870 (@nwoolmer), #920–#922 (@paleluna82-del), #939 and #940 (@HUSRCF).

Receipts

Internal evidence for the numbers above (workspace release-0.4.3):

  • A3B 0.4.2.1 vs 0.4.3 prefill table: A3B-PREFILL-042-VS-043.md and a3b-compare/raw/ (9 processes, 3 per arm, card B, job j3197; binary md5s and sha256s in the report and a3b-compare/binary-sha256.txt).
  • A3B pp16000: a3b-20k/runs/L0-ab-001/L0-ab-summary.json (16,084.5 → 16,694.8), a3b-20k/runs/integ-ab-001/integ-ab-summary.json (16,670.6 → 18,021.6), a3b-20k/runs/dg-ab-001/dg-ab-summary.json (18,341.9 → 18,587.9). Beta commits b60e2e0df7, 50026e1052, 6ce736db31 (MERGE-QUEUE-LOG). KLD: a3b-20k/quality/ kldseq compare, 4/4 groups byte-equal at every step. Plan and per-stage breakdown: a3b-20k/PLAN.md.
  • Flash-Next MQ6 INT8: fp8-lean/RESULTS.md (c8 row: 4,720.5 → 4,899.3, +3.79%; KLD tables; Halo 0.00% and why). Beta commits e077891f1c, 88e9f0d6f9.
  • Halo 16K gate: halo-16k-prefill/RECEIPT.md (stale head 0d06c380b receipt) and halo-16k-prefill/rc-runs-9d74311a5381c22ede869bb600b3d8f3dc708c4d/ (summary.json, drain-log.txt, identity.txt, r1–r3). That run's identity.txt records head 9d74311a5381c22ede869bb600b3d8f3dc708c4d, not the final RC 1851b72ce7; the directory halo-16k-prefill/ also holds three failed earlier attempts (rc-runs-attempt1-pool-load-fail, -attempt2-r2-load-fail, -attempt3-drain-race). Q8 PLE check: Q8-PLE-RC-CHECK.md.
  • Flash-Next prefill and scratch diet: fn-10k/LAND-W1.md and fn-10k/runs/bench/{land2,land3ab,land4ab} (W1–W4 A/Bs at the 16,384 chunk), fn-scratch-diet/CENSUS.md and fn-scratch-diet/runs/bench/{exact,final2} (old default vs 16,384 chunk, scratch MiB, resident experts, kld hashes), for the compact default fn-scratch-diet/runs/{kld/pk,serve/pk-split,bench/pk} (card A, split base, Full vs compact), for the 16,384 default chunk fn-scratch-diet/runs/bench/c16 (card A, gptq3 monolith, today = compact at 4096 vs c16), MERGE-QUEUE-LOG.md (20cc3979e8, c55f00cc44, 9791018512, 4be4d09b60, e21244d890, 9d74311a53, 62507eaad5, 02134ca613, 60f47e4022).
  • Hugging Face cards: hf-cards-evidence/mq2-card-publication.receipt.json (HF commit fc845731a5).
  • XTX: CHANGELOG entries for G2 staging, S1 slow-link prefill and the narrow-link default.

Install

curl -fsSL https://hipfire.dev/install.sh | bash

To pin this release and its prebuilt kernel packs:

curl -fsSL https://hipfire.dev/install.sh | bash -s -- --tag v0.4.3

Full notes: CHANGELOG.md.

Contributors

fivetide, ghazni101, and 3 other contributors
Assets 17
Loading