Qwen3.8-Flash-Next on Strix Halo (gfx1151): 17 → 47 tok/s — working MTP on ROCm, measurements, reproducible stack

[drluoto]

Sharing a measured, reproducible stack for Qwen3.8-Flash-Next (qwen4exp) on Strix Halo (Ryzen AI Max+ 395 / Radeon 8060S, gfx1151, 128 GB unified, ROCm 7.1) — including what we believe is the first working native MTP speculative decoding for this model on ROCm.

Headline, single-stream decode on real coding workloads (UD-IQ4_XS, greedy):

tok/s	no speculation	full stack
file rewrite @ 8k ctx	16.8	47.1
new code @ 8k	16.8	31.7
file rewrite @ 24k	~15	28.6
new code @ 24k	~15	25.4

The stack is three composable levers:

1. GPU TOP_K — the QSA indexer runs ggml_top_k over the whole context 12×/token, and stock HIP falls back to CPU past ne=1024 (no CUB). This is why qwen4exp decode collapses with depth on ROCm — the knee is at exactly d1024. Fixed by CUDA: enable the CUB path on HIP via hipCUB #26592 (hipCUB) or ROCm: add radix TOP_K for long rows #27466 (native radix). +38–53% end-to-end at 24k.
2. Native MTP (qwen4exp : add NextN/MTP draft head (--spec-type draft-mtp) for Qwen3.8-Flash-Next #27836) + crusaderky's detached-head loader fix + a converter whitelist fix (rmonsurate/llama.cpp#1). Output verified byte-clean at long prompts — an earlier community MTP port produced genuine-looking tok/s while emitting multilingual noise above ~1k prompt tokens, so read the output, not just the counter.
3. ngram-mod on top: --spec-type draft-mtp,ngram-mod.

Everything packaged:

• Guide with build/run lines and the pitfalls (llama-bench's -ub 2048 trap, GGML_CUDA_DISABLE_GRAPHS=1 on ROCm < 7.13 and why, dio vs mmap load, why fleet serving isn't worth it on this MoE): https://github.com/drluoto/flash-next-strix-halo
• Pre-assembled branch: drluoto/llama.cpp → strix-halo-flash-next
• ROCm-verified MTP sidecar from the official checkpoint: drluoto/Qwen3.8-Flash-Next-MTP-GGUF
• The head was extracted with HTTP range reads — 5.2 GB fetched instead of the 360 GB checkpoint: shard-scalpel

Credits where due: rmonsurate (MTP), crusaderky (loader), Geramy (hipCUB + rocPRIM capture diagnosis), jadenmach2 (radix TOP_K), Unsloth (port + quants), JJJYmmm/Qwen (arch). Raw measurement data lives in the three PR threads. Happy to run follow-ups on this hardware.