v0.6.1 - DeepSeek-V4 wo_a no longer falls back to the CPU

One fix on top of v0.6, for the DeepSeek-V4 decode regression reported in #2 by @aic0d3r.

What was wrong

Upstream #26531 added TENSOR_ALLOW_RESHAPE, which recomputes a tensor's strides from the
shape the model code asks for:

nb[dim] = dim == 0 ? ggml_type_size(type) : ne[dim-1]*nb[dim-1]

nb[1] is a ROW stride, and a row of a block-quantised tensor is ne[0]/blck_size blocks,
not ne[0] elements. For q8_0 with ne[0]=4096 that produced 409634 = 139264 instead of
12834 = 4352. F16 and F32 weights were unaffected, because their block size is 1, which is
why it went unnoticed.

That malformed metadata is exactly what buft_supported() probes the backend with, so the
weight looked non-contiguous, the Vulkan supports_op refused the mul_mat, and the loader
quietly put the weight in a CPU buffer.

It became reachable when #26577 gave dflash and deepseek4 a 3d wo_a. Both landed in the
upstream merge that became v0.5, which is why builds before that are unaffected.

What it cost, measured

DeepSeek-V4-Flash UD-IQ3_XXS + DSpark drafter, gfx1151/RADV:

	v0.6	v0.6.1
attn_wo_a matmuls on the CPU backend	all 43 attention layers	none
Vulkan0 model buffers	10284.28 + 5162.99 MiB	10386.28 + 6624.99 MiB
graph splits at bs=1, main / draft	174 / 8	88 / 2

So one matmul per attention layer, plus 1462 MiB of weights, was living on the host and
being synchronised with every token. That is the reported signature: GPU utilisation down,
CPU up, decode hit hardest and prefill a little.

Generation is unchanged, this is purely about where the work runs.

Notes

The driver is unchanged from v0.6 (Mesa 26.3.0-devel RADV git-d18d598e, libdrm 2.4.133),
and the payload uses the same compiler as the v0.6 payload, so v0.6 and v0.6.1 are directly
comparable.

This is an upstream defect, not a Strix Halo one. Any Vulkan user running DeepSeek-V4 or
dflash with a quantised wo_a has been paying it since #26577. It is being reported
upstream separately.

Thanks to @aic0d3r for the report and for the clean isolation work, which is what made this
findable.

v0.6 - coopmat2 MoE crash, scheduler split cut, FA q8_0 accumulator

Three correctness fixes on top of v0.5, plus the 15 upstream and Strix-tuning commits that
landed since. Vulkan payload only this time; the HIP image is unchanged from v0.5 and was not
rebuilt or revalidated here.

What changed

• coopmat2 mul_mat_id pipelines were created with the wrong descriptor count (e6f10a7).
  A post-v0.5 commit raised mul_mat_id_param_count to 6 for the fused MUL epilogue and gave
  every mul_mat_id shader a binding = 5, including the coopmat2 one, but the coopmat2
  pipeline block kept passing a literal 5. On any device that selects coopmat2 this aborted
  at the first expert matmul, so every MoE model was a hard crash. It also silently
  specialized those pipelines as plain matmuls, since the same value doubles as the
  is-this-mul_mat_id flag. Reproduced and fixed-verified on an RTX 3070: unpatched aborts at
  ggml-vulkan.cpp:8419, patched gives 4826/4826 on MUL_MAT_ID and coherent MoE generation.
  Affects the dev-20260806-* prereleases, not v0.4 or v0.5.

• Backend splits stopped being cut at 30 inputs (76ad2ba). Upstream #22789 replaced the
  fixed split-input array with a growable one and changed the split-cutting heuristic from the
  constant to split->inputs_capacity, which doubles on demand and never resets. Once a split
  slot grows, the cut point ratchets up, splits get longer, and every cross-backend input copy
  in a split stays live to its last use inside it, n_copies times over under pipeline
  parallelism. Only multi-backend setups reach this. The growable array stays, the cut goes
  back on the constant.

• FA MMQ overflowed the f16 accumulator with q8_0 K (d83217c). acc is an int32 sum of
  dotPacked4x8EXT results, bounded by d_per_step*4*127*127 with q8_0 on both sides, which
  runs past f16 range at GGML_PREC_DEFAULT and turns the score into +inf. Scales are now
  applied in fp32 before narrowing. This is an upstream bug and it was fixed here once before;
  the rebase onto b10133 dropped it, and the upstream merge then added the kv_view=1 test
  variants that exposed it again. No model path reaches it, both ggml_flash_attn_ext call
  sites force GGML_PREC_F32, but it took the suite from 32695/32733 to clean.

Validation

Hand-cut from an on-box build on gfx1151, the same tree that was tested:

• test-backend-ops: 32733/32733 on Vulkan0, 0 failures, 2/2 backends. v0.5 reported
  16661/16661; the suite has roughly doubled since, mostly from upstream variants added in the
  merge.
• gpt-oss-20b MXFP4, pp2048/tg64, r=3, quiet box, same driver on both arms:
  parity with v0.5. The candidate's prefill (1712.3 t/s) sits inside v0.5's own
  run-to-run band (1709.9 to 1723.7), and decode is 79.06 against 79.03 and 79.15.
• Payload smoke: libraries resolve with no not found, binaries report 10349 (d83217c67),
  generation coherent at 80.8 t/s.

Driver

Unchanged from v0.4.1. Mesa 26.3.0-devel RADV (git-d18d598e), libdrm 2.4.134 built with
--prefix=/usr.

Upgrading

Replace the vulkan/ directory from the tarball, or re-pull
ghcr.io/nathanw1014/strix-halo-llamacpp:vulkan.

issue #2 diagnostic: v0.6 fork commits without the upstream rebase

Diagnostic build for #2, not part of the v0.x line and not benchmark-validated.

This is v0.6's fork content without the upstream rebase: b7b85da (the build reported
as fast) plus only the four Vulkan commits that came after it, cherry-picked clean, with
the upstream merge 7a57bed left out entirely.

• 3980a48 vulkan: four env-gated Strix Halo prefill fixes for delta-net MoE
• 30296fa vulkan: reject ne[3] > 1 in the mul_mat_id scale epilogue
• 0525bfc vulkan: create coopmat2 mul_mat_id pipelines with the real param count
• d33bd12 vulkan: scale the FA MMQ dot product in fp32 before narrowing

76ad2ba is intentionally absent. It patches upstream #22789's growable split inputs,
which do not exist at b7b85da, and that base already cuts on the constant, so its
behaviour is already present.

What it is for

If v0.6 is slower than b7b85da on your box, this build separates the two possible causes:

• no-rebase is fast, v0.6 is slow -> the regression arrived with the upstream merge,
  and I will bisect that range.
• both are slow -> it is one of my four commits.
• both are fast -> it is not the llama.cpp source at all.

Toolchain note, please read before comparing

This build uses gcc 13.3.0, the same compiler as the v0.6 payload, which was hand-cut
on-box. So no-rebase vs v0.6 is a clean same-toolchain comparison, and that is the
pairing that answers the question. The b7b85da payload was an automated CI build with
gcc 14.2.0, so that pairing carries a compiler difference. Measured here, that difference
is worth about 7.5% on CPU-backend prefill in gcc-13's favour and nothing on GPU work,
so it does not explain a v0.6 slowdown, but it is worth knowing it exists.

Driver is unchanged from v0.6: Mesa 26.3.0-devel RADV (git-d18d598e), libdrm 2.4.133.

Status of the investigation

I could not reproduce the reported regression on my 64 GB box across seven configurations,
including speculative decoding at 34k context and a forced CPU/GPU split that exercises
the scheduler code from #22789. One thing is already ruled out: on DeepSeek-V4-Flash with
the DSpark drafter, graphs reused = 0 on both builds, so lost graph reuse is not the
mechanism.

dev-20260809-d83217c — llama.cpp strix-halo-vulkan@d83217c

Automated dev build of Nathanw1014/llama.cpp@strix-halo-vulkan at d83217c — vulkan: scale the FA MMQ dot product in fp32 before narrowing

Compile-tested and container-smoke-tested only; CI has no gfx1151, so no benchmark or correctness validation ran. Benchmark-validated releases are the v0.x line.

Changes since dev-20260809-76ad2ba: Nathanw1014/llama.cpp@76ad2ba...d83217c

Built 2026-08-09T13:20:00Z (automated dev build)
source: d83217c672c72cbeeed3e6216ef4f7a8dbc74679 d83217c (vulkan: scale the FA MMQ dot product in fp32 before narrowing)
mesa: d18d598e2  libdrm: 2.4.133  glslc: shaderc v2026.3-dev 49a8724d561c13db22b52f99f2a0e2707a9a9e3c
vulkan bins: llama-bench llama-cli llama-server 
vulkan driver: libdrm.so.2 libdrm.so.2.133.0 libdrm_amdgpu.so.1 libdrm_amdgpu.so.1.133.0 libvulkan_radeon.so radeon_icd.x86_64.json 
vulkan dir size: 97M

Images: ghcr.io/nathanw1014/strix-halo-llamacpp:vulkan-dev (rolling) / :vulkan-dev-20260809-d83217c (pinned)

dev-20260809-76ad2ba — llama.cpp strix-halo-vulkan@76ad2ba

Automated dev build of Nathanw1014/llama.cpp@strix-halo-vulkan at 76ad2ba — ggml: cut backend splits on the input constant, not the grown capacity

Compile-tested and container-smoke-tested only; CI has no gfx1151, so no benchmark or correctness validation ran. Benchmark-validated releases are the v0.x line.

Changes since dev-20260808-e6f10a7: Nathanw1014/llama.cpp@e6f10a7...76ad2ba

Built 2026-08-09T10:18:14Z (automated dev build)
source: 76ad2ba9f959c6b209f730461295d11d21182044 76ad2ba (ggml: cut backend splits on the input constant, not the grown capacity)
mesa: d18d598e2  libdrm: 2.4.133  glslc: shaderc v2026.3-dev 49a8724d561c13db22b52f99f2a0e2707a9a9e3c
vulkan bins: llama-bench llama-cli llama-server 
vulkan driver: libdrm.so.2 libdrm.so.2.133.0 libdrm_amdgpu.so.1 libdrm_amdgpu.so.1.133.0 libvulkan_radeon.so radeon_icd.x86_64.json 
vulkan dir size: 97M

Images: ghcr.io/nathanw1014/strix-halo-llamacpp:vulkan-dev (rolling) / :vulkan-dev-20260809-76ad2ba (pinned)

dev-20260809-3be50cc — llama.cpp strix-halo-vulkan@3be50cc

Automated dev build of Nathanw1014/llama.cpp@strix-halo-vulkan at 3be50cc — llama : fix reshaped-tensor row stride for block-quantised types

Compile-tested and container-smoke-tested only; CI has no gfx1151, so no benchmark or correctness validation ran. Benchmark-validated releases are the v0.x line.

Changes since dev-20260809-d83217c: Nathanw1014/llama.cpp@d83217c...3be50cc

Built 2026-08-10T02:39:41Z (automated dev build)
source: 3be50ccc22633a72a1f93c0fa4db5177144b5c0c 3be50cc (llama : fix reshaped-tensor row stride for block-quantised types)
mesa: d18d598e2  libdrm: 2.4.133  glslc: shaderc v2026.3-dev 49a8724d561c13db22b52f99f2a0e2707a9a9e3c
vulkan bins: llama-bench llama-cli llama-server 
vulkan driver: libdrm.so.2 libdrm.so.2.133.0 libdrm_amdgpu.so.1 libdrm_amdgpu.so.1.133.0 libvulkan_radeon.so radeon_icd.x86_64.json 
vulkan dir size: 97M

Images: ghcr.io/nathanw1014/strix-halo-llamacpp:vulkan-dev (rolling) / :vulkan-dev-20260809-3be50cc (pinned)

dev-20260808-e6f10a7 — llama.cpp strix-halo-vulkan@e6f10a7

Automated dev build of Nathanw1014/llama.cpp@strix-halo-vulkan at e6f10a7 — vulkan: create coopmat2 mul_mat_id pipelines with the real param count

Compile-tested and container-smoke-tested only; CI has no gfx1151, so no benchmark or correctness validation ran. Benchmark-validated releases are the v0.x line.

Changes since dev-20260806-86e3f34: Nathanw1014/llama.cpp@86e3f34...e6f10a7

Built 2026-08-09T07:34:22Z (automated dev build)
source: e6f10a799fae282c87f37d6f42d7fd74361510b1 e6f10a7 (vulkan: create coopmat2 mul_mat_id pipelines with the real param count)
mesa: d18d598e2  libdrm: 2.4.133  glslc: shaderc v2026.3-dev 49a8724d561c13db22b52f99f2a0e2707a9a9e3c
vulkan bins: llama-bench llama-cli llama-server 
vulkan driver: libdrm.so.2 libdrm.so.2.133.0 libdrm_amdgpu.so.1 libdrm_amdgpu.so.1.133.0 libvulkan_radeon.so radeon_icd.x86_64.json 
vulkan dir size: 97M

Images: ghcr.io/nathanw1014/strix-halo-llamacpp:vulkan-dev (rolling) / :vulkan-dev-20260808-e6f10a7 (pinned)

dev-20260806-86e3f34 — llama.cpp strix-halo-vulkan@86e3f34

Automated dev build of Nathanw1014/llama.cpp@strix-halo-vulkan at 86e3f34 — ggml-cuda: partition MMQ waves over rows and columns

Compile-tested and container-smoke-tested only; CI has no gfx1151, so no benchmark or correctness validation ran. Benchmark-validated releases are the v0.x line.

Changes since dev-20260806-6e7b355: Nathanw1014/llama.cpp@6e7b355...86e3f34

No container images for this build — the portable tarball above is the only artifact. The :vulkan-dev tag was never published; the ghcr push step failed on every dev build (fixed in 87d2ed5).

Built 2026-08-07T03:24:12Z (automated dev build)
source: 86e3f34fc974a1e85511e5d35f1196011892bf5f 86e3f34 (ggml-cuda: partition MMQ waves over rows and columns)
mesa: d18d598e2  libdrm: 2.4.133  glslc: shaderc v2026.3-dev 49a8724d561c13db22b52f99f2a0e2707a9a9e3c
vulkan bins: llama-bench llama-cli llama-server 
vulkan driver: libdrm.so.2 libdrm.so.2.133.0 libdrm_amdgpu.so.1 libdrm_amdgpu.so.1.133.0 libvulkan_radeon.so radeon_icd.x86_64.json 
vulkan dir size: 97M

dev-20260806-6e7b355 — llama.cpp strix-halo-vulkan@6e7b355

Automated dev build of Nathanw1014/llama.cpp@strix-halo-vulkan at 6e7b355 — tests: cover MMQ tile boundaries in MUL_MAT and MUL_MAT_ID

Compile-tested and container-smoke-tested only; CI has no gfx1151, so no benchmark or correctness validation ran. Benchmark-validated releases are the v0.x line.

Changes since dev-20260806-624217d: Nathanw1014/llama.cpp@624217d...6e7b355

No container images for this build — the portable tarball above is the only artifact. The :vulkan-dev tag was never published; the ghcr push step failed on every dev build (fixed in 87d2ed5).

Built 2026-08-07T00:20:31Z (automated dev build)
source: 6e7b355cb50cca840b127c97d17f512da238a5c7 6e7b355 (tests: cover MMQ tile boundaries in MUL_MAT and MUL_MAT_ID)
mesa: d18d598e2  libdrm: 2.4.133  glslc: shaderc v2026.3-dev 49a8724d561c13db22b52f99f2a0e2707a9a9e3c
vulkan bins: llama-bench llama-cli llama-server 
vulkan driver: libdrm.so.2 libdrm.so.2.133.0 libdrm_amdgpu.so.1 libdrm_amdgpu.so.1.133.0 libvulkan_radeon.so radeon_icd.x86_64.json 
vulkan dir size: 97M

dev-20260806-624217d — llama.cpp strix-halo-vulkan@624217d

Automated dev build of Nathanw1014/llama.cpp@strix-halo-vulkan at 624217d — vulkan: reject ne[3] > 1 in the mul_mat_id scale epilogue

Compile-tested and container-smoke-tested only; CI has no gfx1151, so no benchmark or correctness validation ran. Benchmark-validated releases are the v0.x line.

Changes since dev-20260805-7a57bed: Nathanw1014/llama.cpp@7a57bed...624217d

No container images for this build — the portable tarball above is the only artifact. The :vulkan-dev tag was never published; the ghcr push step failed on every dev build (fixed in 87d2ed5).

Built 2026-08-06T10:55:54Z (automated dev build)
source: 624217d62f059950c2c2b80de6948d1e5f8c1047 624217d (vulkan: reject ne[3] > 1 in the mul_mat_id scale epilogue)
mesa: d18d598e2  libdrm: 2.4.133  glslc: shaderc v2026.3-dev 49a8724d561c13db22b52f99f2a0e2707a9a9e3c
vulkan bins: llama-bench llama-cli llama-server 
vulkan driver: libdrm.so.2 libdrm.so.2.133.0 libdrm_amdgpu.so.1 libdrm_amdgpu.so.1.133.0 libvulkan_radeon.so radeon_icd.x86_64.json 
vulkan dir size: 97M