Skip to content

v0.6.10 - token-exact MTP rollback returns, plus Ling (bailingmoe3) DSpark support

Choose a tag to compare

@Nathanw1014 Nathanw1014 released this 22 Aug 12:03
· 10 commits to master since this release

v0.6.10 - token-exact MTP rollback returns, plus Ling (bailingmoe3) DSpark support

v0.6.9 stopped the v0.6.8 MTP hang by reverting the full-checkpoint rollback change, which
also brought back the v0.6.7 exactness drift after rejected drafts. v0.6.10 lands the actual
fix and reapplies full checkpoints: the hang was never in the state-save path, it was the
server re-verifying replayed draft tokens after a checkpoint restore, which livelocked the
slot. With that fixed, MTP rollback on hybrid GDN targets (qwen35moe, e.g. Qwen3.6-35B-A3B)
is token-exact again and the long-run stall is gone.

Also in this release:

  • DSpark speculative decoding support for bailingmoe3 (Ling 3.0), cherry-picked from upstream
    llama.cpp PR #27508. The runtime side is complete; it activates once draft GGUFs for
    Ling-3.0-flash are published (inclusionAI has the safetensors up, GGUFs are not out yet).
    The reworked bailingmoe3 forward pass was A/B verified byte-identical to the previous tip
    on Ling-3.0-tiny (greedy, 96 tokens).
  • The RADV coopmat LDS pad of 2 is now gated to RADV >= 25.3. On older drivers it violated
    VUID-08986 (measured and ISA-verified before gating). The bundled driver in this payload is
    newer, so the fast path stays active; builds against distro Mesa older than 25.3 now fall
    back instead of tripping validation.

Smoke results (this tarball, gfx1151)

config status detail
Qwen3.6-35B-A3B MTP Q6_K, 800-token repro works 13.9 s, no stall, coherent
Qwen3.6-35B-A3B MTP Q6_K, 3000-token run works 46.4 s (about 65 tok/s), ran through the former stall horizon to the token limit
Ling-3.0-tiny Q4_K_M, plain decode works greedy output consistent with the pre-release A/B
Qwen3-4B Q6_K dense, 1.5k-token prefill works 1921 tok/s prompt processing, grounded summary

Read before quoting

  • Speed figures are single runs, not the BENCHMARKS.md protocol.
  • The DSpark path for Ling cannot be exercised end to end until draft GGUFs exist; what is
    validated here is the reworked non-speculative forward pass and a clean build.
  • A pre-existing CPU-only divergence in one 800-token exactness matrix cell (at token 776)
    remains open; it predates v0.6.8 and is unrelated to the rollback path.

Changes since v0.6.9

  • server : do not re-verify replayed draft tokens after a checkpoint restore (9c5d899)
  • Reapply "common: use full checkpoints for MTP rollback" (f25eefe)
  • vulkan: gate the RADV coopmat pad 2 on driver >= 25.3 (3ca6f21)
  • model : support DSpark for bailingmoe3 (#27508) (2586f6e)

Payload built from Nathanw1014/llama.cpp@2586f6edd (branch strix-halo-vulkan), bundled Mesa
RADV devenv driver as in v0.6.4 through v0.6.9.