v0.6.10 - token-exact MTP rollback returns, plus Ling (bailingmoe3) DSpark support
v0.6.10 - token-exact MTP rollback returns, plus Ling (bailingmoe3) DSpark support
v0.6.9 stopped the v0.6.8 MTP hang by reverting the full-checkpoint rollback change, which
also brought back the v0.6.7 exactness drift after rejected drafts. v0.6.10 lands the actual
fix and reapplies full checkpoints: the hang was never in the state-save path, it was the
server re-verifying replayed draft tokens after a checkpoint restore, which livelocked the
slot. With that fixed, MTP rollback on hybrid GDN targets (qwen35moe, e.g. Qwen3.6-35B-A3B)
is token-exact again and the long-run stall is gone.
Also in this release:
- DSpark speculative decoding support for bailingmoe3 (Ling 3.0), cherry-picked from upstream
llama.cpp PR #27508. The runtime side is complete; it activates once draft GGUFs for
Ling-3.0-flash are published (inclusionAI has the safetensors up, GGUFs are not out yet).
The reworked bailingmoe3 forward pass was A/B verified byte-identical to the previous tip
on Ling-3.0-tiny (greedy, 96 tokens). - The RADV coopmat LDS pad of 2 is now gated to RADV >= 25.3. On older drivers it violated
VUID-08986 (measured and ISA-verified before gating). The bundled driver in this payload is
newer, so the fast path stays active; builds against distro Mesa older than 25.3 now fall
back instead of tripping validation.
Smoke results (this tarball, gfx1151)
| config | status | detail |
|---|---|---|
| Qwen3.6-35B-A3B MTP Q6_K, 800-token repro | works | 13.9 s, no stall, coherent |
| Qwen3.6-35B-A3B MTP Q6_K, 3000-token run | works | 46.4 s (about 65 tok/s), ran through the former stall horizon to the token limit |
| Ling-3.0-tiny Q4_K_M, plain decode | works | greedy output consistent with the pre-release A/B |
| Qwen3-4B Q6_K dense, 1.5k-token prefill | works | 1921 tok/s prompt processing, grounded summary |
Read before quoting
- Speed figures are single runs, not the BENCHMARKS.md protocol.
- The DSpark path for Ling cannot be exercised end to end until draft GGUFs exist; what is
validated here is the reworked non-speculative forward pass and a clean build. - A pre-existing CPU-only divergence in one 800-token exactness matrix cell (at token 776)
remains open; it predates v0.6.8 and is unrelated to the rollback path.
Changes since v0.6.9
- server : do not re-verify replayed draft tokens after a checkpoint restore (
9c5d899) - Reapply "common: use full checkpoints for MTP rollback" (
f25eefe) - vulkan: gate the RADV coopmat pad 2 on driver >= 25.3 (
3ca6f21) - model : support DSpark for bailingmoe3 (#27508) (
2586f6e)
Payload built from Nathanw1014/llama.cpp@2586f6edd (branch strix-halo-vulkan), bundled Mesa
RADV devenv driver as in v0.6.4 through v0.6.9.