v0.6.6 - quantised KV cache no longer switches off DeepSeek V4 sparsity

v0.6.6 - quantised KV cache no longer switches off DeepSeek V4 sparsity

If you run DeepSeek V4 Flash with a quantised KV cache (-ctk q8_0 -ctv q8_0, or q4_0), every
release through v0.6.5 silently dropped the two DSv4 sparse attention paths back to dense flash
attention: the sparse prefill path and the per-token decode gather both hard-required an f16
cache. Dense attention is O(kv) where the sparse paths are O(active set), so the penalty grows
with context depth. A community report on a 128 GB box measured the prefill half end to end:
5.4 / 22.2 / 39.4 / 52.9 percent slower than f16 KV at source depths 17k / 33k / 67k / 134k.
f16 KV configs were never affected.

Two fixes, both verified on gfx1151:

• Sparse prefill now serves a quantised cache by dequantising it once per op into the f16
  scratch the dense path already uses, with the existing fused dequant+transpose shaders. Every
  type with such a shader rides it: q4_0, q4_1, q5_0, q5_1, q8_0, iq4_nl. Others keep the dense
  fallback, as does GGML_VK_FA_DEQUANT=0.
• Per-token decode (plain autoregressive generation, batch 1): the gather that compacts the
  active set now dequantises rows on the way through instead of leaving flash attention to
  re-decode them once per query block.

Sparse prefill FA op, batch 1024 (test-backend-ops, compressed-KV rows = source depth / 4)

kv rows	f16	q8_0 before	q8_0 now	q4_0 now
5,504	47.7 ms	60.9 ms (1.3x)	46.9 ms	47.1 ms
35,584	51.7 ms	427.6 ms (8.3x)	51.9 ms	52.0 ms

Flat against depth again, within about 1 percent of f16 at every measured point. The dequant
pass itself costs under 1 percent.

Per-token decode FA op, kv=11,008 (us per op)

	f16	q8_0	q4_0
before	114.5	226.9	258.4
now	108.9	109.4	109.1

What this should mean end to end

Applying the community depth curve above, a quantised-KV 128K prefill should land roughly 2x
where v0.6.4/v0.6.5 put it. We cannot re-measure DSv4 end to end on the 64 GB dev box, so
treat that as a prediction: if you have a 128 GB machine, llama-bench or a long-prompt
serving run against v0.6.5 with the same flags would be a welcome confirmation either way.
One public 128K comparison that ran exactly the affected config (q8_0 KV on v0.6.4) is
pepuscz/strix-halo-deepseek-v4-flash;
its input-processing cell should move accordingly.

Read before quoting the numbers

• These are op-level measurements. The end-to-end depth curve quoted is a community
  measurement of the bug on v0.6.3-beta3, not a post-fix re-run.
• Wide verify widths are NOT fixed: at batch 16 (continuous batching, wide speculative
  verify) a quantised cache still declines the compaction gate and pays about 1.7x on the FA
  op against f16. Known, documented, next on the list.
• The fixes change which kernel path runs, not the math: 13,324 FLASH_ATTN_EXT
  test-backend-ops cases pass on the release commit, including the new quantised sparse
  prefill eval cases and the nb=1 quantised perf grid rows.

Changes since v0.6.5

• vulkan: decode quantised K/V inside the DeepSeek V4 per-token gather (7b63cbd6b)
• vulkan: dequantise the cache for the DeepSeek V4 sparse prefill (7b6c61330)

Payload built from Nathanw1014/llama.cpp@7b6c61330 (branch strix-halo-vulkan), bundled Mesa
RADV devenv driver as in v0.6.4/v0.6.5.