Skip to content

v0.6 - coopmat2 MoE crash, scheduler split cut, FA q8_0 accumulator

Choose a tag to compare

@Nathanw1014 Nathanw1014 released this 09 Aug 11:12

Three correctness fixes on top of v0.5, plus the 15 upstream and Strix-tuning commits that
landed since. Vulkan payload only this time; the HIP image is unchanged from v0.5 and was not
rebuilt or revalidated here.

What changed

  • coopmat2 mul_mat_id pipelines were created with the wrong descriptor count (e6f10a7).
    A post-v0.5 commit raised mul_mat_id_param_count to 6 for the fused MUL epilogue and gave
    every mul_mat_id shader a binding = 5, including the coopmat2 one, but the coopmat2
    pipeline block kept passing a literal 5. On any device that selects coopmat2 this aborted
    at the first expert matmul, so every MoE model was a hard crash. It also silently
    specialized those pipelines as plain matmuls, since the same value doubles as the
    is-this-mul_mat_id flag. Reproduced and fixed-verified on an RTX 3070: unpatched aborts at
    ggml-vulkan.cpp:8419, patched gives 4826/4826 on MUL_MAT_ID and coherent MoE generation.
    Affects the dev-20260806-* prereleases, not v0.4 or v0.5.

  • Backend splits stopped being cut at 30 inputs (76ad2ba). Upstream #22789 replaced the
    fixed split-input array with a growable one and changed the split-cutting heuristic from the
    constant to split->inputs_capacity, which doubles on demand and never resets. Once a split
    slot grows, the cut point ratchets up, splits get longer, and every cross-backend input copy
    in a split stays live to its last use inside it, n_copies times over under pipeline
    parallelism. Only multi-backend setups reach this. The growable array stays, the cut goes
    back on the constant.

  • FA MMQ overflowed the f16 accumulator with q8_0 K (d83217c). acc is an int32 sum of
    dotPacked4x8EXT results, bounded by d_per_step*4*127*127 with q8_0 on both sides, which
    runs past f16 range at GGML_PREC_DEFAULT and turns the score into +inf. Scales are now
    applied in fp32 before narrowing. This is an upstream bug and it was fixed here once before;
    the rebase onto b10133 dropped it, and the upstream merge then added the kv_view=1 test
    variants that exposed it again. No model path reaches it, both ggml_flash_attn_ext call
    sites force GGML_PREC_F32, but it took the suite from 32695/32733 to clean.

Validation

Hand-cut from an on-box build on gfx1151, the same tree that was tested:

  • test-backend-ops: 32733/32733 on Vulkan0, 0 failures, 2/2 backends. v0.5 reported
    16661/16661; the suite has roughly doubled since, mostly from upstream variants added in the
    merge.
  • gpt-oss-20b MXFP4, pp2048/tg64, r=3, quiet box, same driver on both arms:
    parity with v0.5. The candidate's prefill (1712.3 t/s) sits inside v0.5's own
    run-to-run band (1709.9 to 1723.7), and decode is 79.06 against 79.03 and 79.15.
  • Payload smoke: libraries resolve with no not found, binaries report 10349 (d83217c67),
    generation coherent at 80.8 t/s.

Driver

Unchanged from v0.4.1. Mesa 26.3.0-devel RADV (git-d18d598e), libdrm 2.4.134 built with
--prefix=/usr.

Upgrading

Replace the vulkan/ directory from the tarball, or re-pull
ghcr.io/nathanw1014/strix-halo-llamacpp:vulkan.