D3D12 backend v0.5.5 - fast CPU code on every PC (runtime CPU dispatch)

D3D12 backend v0.5.5 - fast CPU code on every PC (runtime CPU dispatch)

Windows x64, DirectX 12. On llama.cpp master 7592b61 (2026-11-10). Contains everything of v0.5.4.

What is new in v0.5.5

• The CPU code picks the best instruction set when the program starts. Until now every CPU part was plain x86-64 so it would run on any PC. This zip carries 14 variants of the CPU backend in the cpu\ folder (ggml-cpu-x64, sse42, sandybridge, ivybridge, piledriver, haswell, skylakex, cascadelake, cooperlake, cannonlake, icelake, sapphirerapids, zen4, alderlake). At start ggml checks what the CPU supports and loads the best matching one; an old CPU gets the plain x64 one. Unzip the whole zip and keep the cpu folder next to the exes.
• Smaller zip, shared DLLs. The exes are small launchers that use the libllama*.dll and ggml*.dll files of the zip, so the zip is 47.7 MB (v0.5.4: 73.8 MB). vla-cli.exe (robot models) stays the static build of v0.5.4.
• Fix in this layout: llama-server no longer hangs at exit after --help or --version (a static destructor joined a thread while the DLL was unloading; the object is now never destroyed).
• Upstream base and backend are unchanged from v0.5.4.

Speed on a Radeon AI PRO R9700 (DirectX 12, zip smoke)

Model	Tool	Result
Clover-35B-A3B-Preview Q4_K_M, -ngl 99 -lm none -fa 1	llama-bench	pp512 646.82 t/s, tg128 63.69 t/s
Clover-35B-A3B-Preview Q4_K_M	llama-perplexity	PPL 4.2676
diffusiongemma-26B-A4B-it Q4_K_M	llama-diffusion-cli	498.47 ms per step
LLaDA-MoE-7B-A1B-Instruct Q4_K_M	llama-diffusion-cli, block 32, 32 steps	56.45 ms per step

Models that run fully on the GPU do not get faster from the CPU variants (D3D12 A/B, Clover Q4_K_M: pp512 690.09 -> 697.71 t/s, tg128 67.16 -> 67.54 t/s, one run each, inside the run-to-run spread). The smoke numbers are one run each on a warm card.

Speed when the CPU does the work (Ryzen 7 5700G, 8 threads, -ngl 0, DX12 build, exp1088, one run each)

Model	Metric	v0.5.4 (plain x86-64)	v0.5.5 (best CPU variant, haswell here)
qwen2.5-0.5b-instruct Q4_0	pp512	48.21 t/s	747.44 t/s
qwen2.5-0.5b-instruct Q4_0	tg128	35.73 t/s	91.17 t/s
Clover-35B-A3B-Preview Q4_K_M	pp512	10.63 t/s	65.47 t/s
Clover-35B-A3B-Preview Q4_K_M	tg64	5.12 t/s	8.50 t/s

CPU-only numbers move from run to run (an earlier run of the Clover model gave pp512 87.61 t/s with the variants).

Tested

• Radeon AI PRO R9700: every shipped exe of this zip (llama-server, llama-mtmd-cli, llama-cli, llama-completion, llama-perplexity, llama-bench, test-backend-ops, llama-diffusion-cli, llama-clover-diffusion, vla-cli with all 5 robot models): 15/15 steps pass, no NaN/Inf, the CPU backend loaded from cpu\ggml-cpu-<variant>.dll. Zip SHA-256 376bcefe66385cb16261ac4a53cd4b8872c0a31d5e026def1ebb2ba7624a3595.
• llama-server also answered a chat request on the dispatch layout (qwen2.5-0.5b-instruct Q4_0, D3D12 build, "The capital of France is Paris.").
• The .bat files were not run in the test; they call the tested exes. Other GPUs were not tested with this zip.

D3D12 backend v0.5.4 - llama.cpp upstream move to 7592b61d3

D3D12 backend v0.5.4 - llama.cpp upstream move to 7592b61

Windows x64, DirectX 12. On llama.cpp master 7592b61 (2026-11-10). Contains everything of v0.5.3.

What is new in v0.5.4

• Upstream move: llama.cpp 6023692 (2026-11-05) -> 7592b61 (2026-11-10). 116 upstream commits. One merge conflict, in ggml/src/ggml-backend-reg.cpp (the backend load list); the upstream block is kept and our one ggml_backend_load_best line is added after Vulkan. Nothing else of the backend changed, so the shader cache is the one of v0.5.3.
• vla-cli.exe is the v0.5.3 build (not rebuilt; it is a separate static exe). Robot models, DiffusionGemma and Clover work as in v0.5.3.

Speed on a Radeon AI PRO R9700 (DirectX 12)

Model	Tool	Result
Clover-35B-A3B-Preview Q4_K_M, DX12, -ngl 99 -lm none -fa 1	llama-bench (zip smoke)	pp512 635.83 t/s, tg128 66.90 t/s (box hot, hotspot 89 C)
Clover-35B-A3B-Preview Q4_K_M	llama-perplexity	PPL 4.2676 (= v0.5.3)
diffusiongemma-26B-A4B-it Q4_K_M	llama-diffusion-cli	450.53 ms/step
LLaDA-MoE-7B-A1B-Instruct Q4_K_M	llama-diffusion-cli, block 32, 32 steps	52.05 ms/step

Warm A/B on the same box (exp1087, 3 repeats, Clover-35B-A3B Q4_K_M, DX12, -ngl 99 -lm none -fa 1): pp512 693.60 / 692.78 t/s (v0.5.4) against 689.69 / 693.89 t/s (v0.5.3), tg128 67.1 t/s in all four runs: no speed change; the lower smoke pp512 was heat.

Tested

• Radeon AI PRO R9700: every shipped exe of this zip (llama-server, llama-mtmd-cli, llama-cli, llama-completion, llama-perplexity, llama-bench, test-backend-ops, llama-diffusion-cli, llama-clover-diffusion, vla-cli with all 5 robot models): 15/15 steps PASS, no NaN/Inf. Zip SHA-256 d7faca3b87587f9a3f7b6944825b12ffb8a0561ad076d74bbc95a4576415c4a8.
• The .bat files were not run in the test; they call the tested exes. Other GPUs were not tested with this zip.

D3D12 backend v0.5.3 - robot models (VLA), faster DiffusionGemma

D3D12 backend v0.5.3 - robot models (VLA), faster DiffusionGemma

Windows x64, DirectX 12. On llama.cpp master 6023692 (2026-11-05). Contains everything of v0.5.2.

What is new in v0.5.3

• Robot models (vision-language-action) run on DirectX 12. vla-cli.exe is the vla-cli program of vla.cpp (https://github.com/VinRobotics/vla.cpp, commit 1012c05, Apache-2.0) built with this zip's DirectX 12 backend; our changes to vla.cpp (backend selection, an embedded test image, a Windows exit fix) are in vla-cpp-1012c05.patch (attached to this release). It takes a camera image and prints rows of robot actions.
  • robot-download-models.bat downloads 5 small models (ACT, Octo-Small, TurboVLA, SmolVLA, Evo-1; about 4.3 GB) into X:\llamacppdx12\models and checks their SHA-256.
  • robot-test.bat runs all 5 and writes robot-results-<PC name>.zip; see README-robots.md.
• DiffusionGemma another 1.4x faster. The host sampler of llama-diffusion-cli computes the exp of 4 logits at once (SSE2) and sums in double: 0.63 -> 0.45 s per diffusion step on a Radeon AI PRO R9700, same text.

Speed on a Radeon AI PRO R9700 (DX12)

Model	Tool	Result
ACT / Octo-Small / TurboVLA / SmolVLA / Evo-1	vla-cli, median per policy call	10.7 / 13.0 / 23.1 / 146.2 / 557.1 ms
diffusiongemma-26B-A4B-it Q4_K_M	llama-diffusion-cli, 1024 tokens	0.45 s per step, 42.3 tok/s on the test poem (v0.5.2: 0.63 s, 30.0 tok/s)
Clover-35B-A3B-Preview Q4_K_M	llama-bench	pp512 682.24 t/s, tg128 66.05 t/s
Clover-35B-A3B-Preview Q4_K_M	llama-completion	65.16 t/s generation
Clover-35B-A3B-Preview Q4_K_M	llama-clover-diffusion	6.13 t/s
LLaDA-MoE-7B-A1B-Instruct Q4_K_M	llama-diffusion-cli, block 32, 32 steps	50.61 ms per step

Tested

• Radeon AI PRO R9700: every shipped exe of this zip (llama-server, llama-mtmd-cli, llama-cli, llama-completion, llama-perplexity, llama-bench, test-backend-ops ADD 116/116, llama-diffusion-cli, llama-clover-diffusion, vla-cli with all 5 robot models: exit 0, finite actions); Clover perplexity 4.2676 +/- 0.35 (same as v0.5.2).
• The .bat files were not run in the test; they call the tested exes. Other GPUs were not tested with this zip.

3D12 backend v0.4.4 - faster decode, MTP drafting and prompts on the MTT S80, newer llama.cpp

D3D12 backend v0.4.4 - faster decode and MTP drafting on the MTT S80

Windows x64, DirectX 12. On llama.cpp master 785322d (2026-11-01), as v0.4.3.
Changes since v0.4.1 (v0.4.2 and v0.4.3 were never published; their changes are part of this release).

What is new in v0.4.4

• Moore Threads MTT S80: faster generation for Qwen models, much faster MTP / speculative drafting.
  • Qwopus3.5-9B-Coder-MTP Q4_K_M with --spec-type draft-mtp, 3 draft tokens: 18.5 -> 22.3 t/s.
  • Qwen3.5-9B Q4_K_M tg64: 15.8 -> 17.4 t/s; Qwen3.8-27B UD-IQ3_XXS tg128 4.72 -> 4.89 t/s.
  • TOP_K with k <= 32 (the MTP draft sampler, top_k 10 over a 248k vocabulary) runs in two passes instead of a full sort (was ~170 dispatches per draft token).
  • Q6_K matrix-vector product for 1-8 tokens loads every weight byte once.

Also in this release (from v0.4.2 / v0.4.3)

• MTT S80: prompt processing about 40 % faster for IQ models. Qwen3.8-27B UD-IQ3_XXS: pp512 19.3 -> 28.0 t/s, pp4 8.6 -> 10.1 t/s (v0.4.1 -> v0.4.4).
  • The big prompt tile also loads IQ3_XXS, IQ3_S, IQ4_XS, IQ2_S, IQ2_XXS, IQ2_XS and Q2_K weights directly and starts at 16 tokens.
  • Shader buffers are bound through one descriptor table on Moore Threads, as on Intel. Very small models (0.5B) generate about 4 % slower with it; GGML_D3D12_UAV_TABLE=0 switches it off.
• Batches of 5-8 tokens decode each weight once (v0.4.1: 2-4). Radeon AI PRO R9700, glm-4-9b IQ2_M: pp8 +85 %.
• Q6_K keeps the faster matrix-vector path up to 8 tokens on Intel and AMD: Iris Xe Ministral-3-3B Q6_K pp5 5.2 -> 15.1 t/s.
• llama.cpp updated by 124 upstream commits (FP8 E4M3 tensors are reported as unsupported by this backend).

Switches (for A/B tests): GGML_D3D12_TOPK_SMALL=0, GGML_D3D12_Q6K_V2=0, GGML_D3D12_UAV_TABLE=0|1, GGML_D3D12_BT_IQ=0, GGML_D3D12_BT_IQ2=0, GGML_D3D12_BT_MIN=<n>, GGML_D3D12_BT_TN=32|64|128, GGML_D3D12_MV_NC=0.

Tip for MTP models on the MTT S80: --spec-draft-n-max 3 was fastest (5 and 7 draft tokens were slower).

Tested

• MTT S80: every shipped exe of this zip (llama-server, llama-cli, llama-mtmd-cli, llama-completion, llama-perplexity, llama-bench, test-backend-ops), PPL qwen2.5-0.5b 17.29, LFM2-24B-A2B and Qwen3.8-27B benches; test-backend-ops TOP_K 533/533, ARGSORT 98/98, MUL_MAT 2565/2565 of the same source.
• Radeon AI PRO R9700: test-backend-ops MUL_MAT 2565/2565, TOP_K 533/533, ARGSORT 98/98 with the same changes; v0.4.3 source 13 op groups 11,643/11,643.

Not tested: Intel op sweep of this exact build, the other GPUs of the backend (Zhaoxin, Lisuan) - please report problems.

D3D12 backend v0.5.2 - text diffusion models (LLaDA, Dream, DiffusionGemma, Clover), newer llama.cpp

D3D12 backend v0.5.2 - text diffusion models, newer llama.cpp

Windows x64, DirectX 12. On llama.cpp master 6023692 (2026-11-05). Contains everything of v0.4.4 (v0.5.0 and v0.5.1 were never published; their changes are part of this release).

What is new in v0.5.2

• DiffusionGemma about 1.6x faster. The host sampler of llama-diffusion-cli computed the exp of all 262144 vocabulary entries twice per position; it now computes each once (same sampled tokens, same text). DiffusionGemma-26B, 1024 tokens: 0.96 s -> 0.57-0.63 s per diffusion step on a Radeon AI PRO R9700.

Also new since v0.4.4:

• Text diffusion models run on DirectX 12.

  • llama-diffusion-cli.exe: LLaDA / LLaDA-MoE, Dream / Dream-Coder and DiffusionGemma (upstream llama.cpp PR 24423, not merged upstream yet).
  • llama-clover-diffusion.exe: Clover-35B-A3B block diffusion (32-token blocks, denoised in parallel).
  • Ready-made .bat files: diffusiongemma-chat.bat, diffusiongemma-visual.bat, llada-diffusion-visual.bat, dream-diffusion-visual.bat, clover-chat.bat, clover-server.bat, clover-diffusion.bat, clover-diffusion-visual.bat (the -visual ones show the canvas being filled step by step).

• llama.cpp updated by 127 upstream commits (785322d -> 6023692). The backend follows two ggml changes: the RoPE truncate flag, and FLASH_ATTN_EXT with a K/V row list (not supported yet; those nodes run elsewhere).

• Bf16 copies run on the GPU. Models that copy values into bf16 tensors in every layer (Clover-35B-A3B) ran each copy on the CPU (1302 graph splits per token); now 2 splits.

Speed on a Radeon AI PRO R9700 (DX12)

Model	Tool	Result
Clover-35B-A3B-Preview Q4_K_M	llama-bench	pp512 695.89 t/s, tg128 67.29 t/s (before the bf16 fix: 213.75 / 6.29)
Clover-35B-A3B-Preview Q4_K_M	llama-completion	65.3 t/s generation
Clover-35B-A3B-Preview Q4_K_M	llama-clover-diffusion	6.2-16.0 t/s (prompt-dependent)
diffusiongemma-26B-A4B-it Q4_K_M	llama-diffusion-cli, 1024 tokens	0.63 s per step, 30.0 tok/s on the test poem (v0.5.1: 0.96 s per step, 19.7 tok/s)
LLaDA-MoE-7B-A1B-Instruct Q4_K_M	llama-diffusion-cli, block 32, 32 steps	53.5 ms per step

The Vulkan build is still faster for Clover on this card (pp512 3526, tg128 109 t/s).

Tested

• Radeon AI PRO R9700: every shipped exe of this zip (llama-server, llama-mtmd-cli, llama-cli, llama-completion, llama-perplexity, llama-bench, test-backend-ops, llama-diffusion-cli, llama-clover-diffusion); Clover perplexity 4.2676 +/- 0.35 (4 chunks of 512, same as Vulkan within error).
• Same source, R9700: test-backend-ops ROPE 510/510, FLASH_ATTN_EXT 5368/5368, CPY 412/412, SET_ROWS 323/323, ADD 116/116; LLaDA-MoE, Dream-Coder-7B, DiffusionGemma-26B and Clover-35B (AR and diffusion) give correct answers.
• The .bat files were not run in the test; they call the tested exes.

D3D12 backend v0.4.1 - faster generation and speculative decoding on the MTT S80

D3D12 backend v0.4.1 - faster generation and speculative decoding on the MTT S80

Windows x64, DirectX 12. Same base as v0.4.0 (llama.cpp master caa8191, 2026-10-26).

What is new

• Dense IQ models on the Moore Threads MTT S80 decode 2.5x faster. Qwen3.8-27B UD-IQ3_XXS: generation 1.77 -> 4.37 t/s (tg128), prompt 14.7 -> 19.3 t/s (pp512).
  New single-column kernels for IQ3_XXS / IQ3_S / IQ2_*, IQ codebooks in groupshared memory, 8 threads per row on Moore Threads, K-quants in the big prompt tile.
• Batches of 2-4 tokens decode each weight once. Before, a 2-token batch cost as much as 2 single tokens. Qwen3.8-27B on the S80: pp2 4.75 -> 6.40 t/s, pp4 5.03 -> 8.64 t/s.
  This is what makes speculative decoding pay off: with the model's MTP head (mtp-Qwen3.8-27B-Q4_0.gguf, unsloth/Qwen3.8-27B-GGUF folder MTP) and
  --spec-type draft-mtp --spec-draft-n-max 3 --spec-draft-p-min 0.75, generation is about 6.1 t/s instead of 4.4 t/s on the S80 (measured with llama-speculative-simple, 256 tokens, greedy).
  Also faster on other GPUs: Radeon AI PRO R9700 pp4 +44-51 %, Intel Iris Xe pp4 8.2 -> 15.6 t/s (Ministral-3-3B Q6_K).
• AMD discrete GPUs: matvec threads per row chosen by weight type (IQ 8, others 32): glm-4-9b IQ2_M generation 68.0 -> 79.6 t/s on the R9700.

Switches (all default on, for A/B tests): GGML_D3D12_MV_NC=0 (one dispatch per column again), GGML_D3D12_MV_PERCOL=0, GGML_D3D12_MM_TPR=<n>.

Tested

• MTT S80: every shipped exe of this zip (llama-server, llama-cli, llama-mtmd-cli, llama-completion, llama-perplexity, llama-bench, test-backend-ops), PPL qwen2.5-0.5b 17.29, LFM2-24B-A2B and Qwen3.8-27B benches.
• Intel Iris Xe: test-backend-ops of this zip, 13 op groups (MUL_MAT 2520/2520, MUL_MAT_ID 1107/1107, ...).
• Radeon AI PRO R9700: MUL_MAT / MUL_MAT_ID of the same source (test build).

Not tested: the other GPUs of the backend (Zhaoxin, Lisuan) - please report problems.

D3D12 backend v0.4.0 - faster small batches and MoE routing on the MTT S80

ggml-d3d12 : v0.4.0 - faster small batches and MoE routing on the Moore Threads MTT S80
MTT S80: batches of 2-15 tokens (multi-token prediction drafts, speculative decoding, short prompts) of Q4_K / Q6_K and the other types without a multi-column kernel now run the one-column matvec once per column instead of the generic column loop: 2.7-2.9x faster. On by default only on the S80 (vendor 0x1ED5); GGML_D3D12_MV_PERCOL=N sets the column limit on any GPU, 0 turns it off.
All GPUs: the MoE router of models with sigmoid gating, expert bias and weight norm (LFM2 MoE and similar) runs as one fused topk_moe dispatch instead of 7-10 small ones (port of the upstream Vulkan TOPK_MOE_SIGMOID_NORM_BIAS fusion).
Still on upstream llama.cpp master caa8191 (2026-10-26), like v0.3.9.
Measured on the Moore Threads MTT S80 (driver 32.0.2609.1, test build with AVX1 CPU code, llama-bench, -ngl 99):
Qwopus3.5-9B-Coder-MTP Q4_K_M, time per batch: 2 tokens 319 -> 108 ms, 4 tokens 506 -> 191 ms, 8 tokens 986 -> 356 ms; generation (tg128) 15.4 t/s, unchanged.
LFM2-24B-A2B Q4_K_M: GPU dispatches per token 1081 -> 853; generation (tg64) +6-7 % (66.6 vs 71.5 ms per token, 8 alternating runs); 2 tokens 139.7 -> 113.1 ms, 4 tokens 239.8 -> 200.4 ms.
Note for LFM2-24B-A2B Q4_K_M on the 16 GB S80: it needs 13897 MiB of VRAM while Windows grants one process about 12970 MiB; keeping the experts of 4-6 layers on the CPU (llama.cpp --n-cpu-moe 4) costs nothing (tg64 16.05 t/s at 4, 15.6 t/s at 6) and brings it near or under the limit.
Op tests on the S80 (test-backend-ops, D3D120, 4 workers): MUL_MAT 2520/2520, MUL_MAT_ID 1107/1107, TOPK_MOE 576/576, FLASH_ATTN_EXT 5368/5368, ADD, MUL, RMS_NORM, GET_ROWS, SET_ROWS, CPY, SOFT_MAX, ROPE, SSM_CONV, CONCAT, SWIGLU, GEGLU, REGLU, SWIGLU_OAI, GEGLU_ERF, GEGLU_QUICK all pass.
Not run on the Radeon AI PRO R9700 and the Intel Iris Xe for this version (both offline); the small-batch change is off there by default, the router fusion is new for every GPU.

D3D12 backend v0.3.9 - on upstream llama.cpp caa8191df (2026-10-26)

ggml-d3d12 : v0.3.9 - on upstream llama.cpp master caa8191 (2026-10-26)
The backend now sits on upstream ggml-org/llama.cpp master caa8191 (v0.3.8 sat on 88dcc46 of 2026-10-07; 426 upstream commits in between), so the newer model architectures and tools of upstream build with it.
MUL_MAT_ID and ADD_ID treat an expert id of -1 as a skipped slot (upstream #26631): the matching output row is zero for MUL_MAT_ID, nothing is added for ADD_ID. The old kernels read outside the buffers on such ids (device removed on the Radeon AI PRO R9700 in the new upstream tests).
Not implemented on the GPU yet: GGML_OP_TOP_K_MASK (new upstream fused op for the Qwen4exp attention mask), it runs on the CPU.
Measured on the Radeon AI PRO R9700 (test build, AVX1 CPU code; v0.3.8 = the published zip), llama-bench:
gemma-4-E2B Q4_0, generation (tg32): 93.4 / 93.1 t/s vs 93.4 / 92.2 t/s with v0.3.8 (equal).
gemma-4-E2B Q4_0, prompt processing (pp512): 3562 / 3562 t/s vs 3640 / 3734 t/s with v0.3.8 (2-5 % lower).
qwen2.5-0.5b Q4_0, generation (tg64): 219-220 t/s vs 222 t/s with v0.3.8.
Op tests on the R9700 (test-backend-ops, D3D120): ADD, MUL_MAT S0/S5, MUL_MAT_ID S0-S9, ADD_ID, FLASH_ATTN_EXT_S0, SOFT_MAX_S0, ROPE_S0, RMS_NORM, MUL, CPY, SCALE, GET_ROWS, SET_ROWS all pass.

D3D12 backend v0.3.8 - tiled matmul for any K, faster Moore Threads prompt, int8-dot prompt matmul

ggml-d3d12 : v0.3.8 - tiled matmul accepts any K for F32/F16 weights

• Matmuls with F32 or F16 weights whose K (the shared dimension) is not a multiple of 32 and with at least 16 columns used the one-thread-per-output scalar kernel. Small robot-policy models (vla.cpp: ACT, Octo, TurboVLA, Evo-1) have such matmuls everywhere (K = 21, 261, 535, 1025 ...). The tiled prompt kernel now handles them: the last K tile is loaded with guarded scalar loads and zero padding (-DK_TAIL in mul_mat_tiled.hlsl). Quantized weights and K multiples of 32 take the same path as in v0.3.7, so LLM speed is unchanged.
• Default gate: K % 4 != 0. GGML_D3D12_TILED_TAIL=2 uses the tiled kernel for every K that is not a multiple of 32, GGML_D3D12_TILED_TAIL=0 turns it off.
• The zip ships the warmed shader cache of v0.3.1 (d3d12-shader-cache); the new K-tail shader variants compile on first use.
• Also in this pull request since the published v0.3.1 (versions v0.3.2 to v0.3.7): Moore Threads prompt speed-ups, the Q4_0/Q8_0 big tile with int8 dot products, the prompt flash attention opened for every GPU, Intel fixes (see the commit messages of the seven commits).

Measured with vla.cpp robot models (median warm predict, two runs each way, "was" = the v0.3.7 code plus a tiny vectorised ADD/MUL change that gains under 1 %):

GPU / model	was	now
Radeon AI PRO R9700, ACT	98.3 / 98.8 ms	12.6 / 12.6 ms (rows equal)
Radeon AI PRO R9700, Evo-1	840.6 / 846.9 ms	590.7 / 594.6 ms (-30 %)
Radeon AI PRO R9700, TurboVLA	30.5 / 29.6 ms	24.1 / 24.0 ms (rows equal)
Radeon AI PRO R9700, Octo-Small	25.0 / 24.8 ms	22.4 / 21.7 ms
Radeon AI PRO R9700, SmolVLA (K multiple of 4)	161.6 / 161.1 ms	161.4 / 167.9 ms (unchanged)
Intel Iris Xe, Evo-1	34.3 s	15.7 s (-54 %)
Intel Iris Xe, Octo-Small	173.1 / 172.9 ms	82.8 / 83.2 ms
Moore Threads MTT S80, ACT	219.2 ms	65.9 ms
Moore Threads MTT S80, Evo-1	7894 / 7889 ms	5138 / 5136 ms (-35 %)

Correctness: 704 new MUL_MAT test cases (F32 and F16 weights, K = 1..1025 with K % 32 != 0, 16 to 256 rows, 1 to 100 columns, batches, broadcast, permuted) pass 704/704 on the Radeon R9700, Intel Iris Xe and Moore Threads S80 (test-backend-ops, D3D120 against the CPU). Zip smoke: all 7 exes of this exact zip, plus two exit-shape bench runs and a Bonsai-8B-Q1_0 perplexity run, ran on the Radeon with 0 failed runs (qwen2.5-0.5b Q4_0 PPL 17.3638, Bonsai-8B-Q1_0 PPL 12.4121). Not run with this exact zip: Intel and S80 (boxes off after the test run); the code is identical to the tested builds.

D3D12 backend v0.3.1 - faster single-token generation

ggml-d3d12 : v0.3.1 - queue small uploads without a CPU wait, tail readback of the logits

• Faster single-token generation (tg) on small and mid-size models: small uploads (up to 256 KB: token ids, positions, masks) are queued without waiting for the GPU, and the logits are copied behind the graph so reading them back is a memcpy.
• GGML_D3D12_NO_SMALL_UPLOAD=1 and GGML_D3D12_NO_TAIL_READBACK=1 turn the two paths off.
• The zip ships a warmed shader cache (d3d12-shader-cache, 1303 files).

Measured with llama-bench on a Radeon AI PRO R9700 (tg64, mean of 2 runs, v0.3.0 -> v0.3.1; pp128 within +-2 %):

model	v0.3.0	v0.3.1
qwen2.5-0.5b Q4_0	168.2	237.3 (+41 %)
gemma-4-E2B Q4_0	83.5	95.1 (+14 %)
Ministral-3-3B Q6_K	74.3	89.3 (+20 %)

Greedy text (32 tokens) identical to v0.3.0 for qwen2.5-0.5b and Ministral-3-3B.

Tested: all 7 exes of the zip ran once on the R9700 (perplexity qwen2.5-0.5b Q4_0 17.2885, Bonsai-8B Q1_0 12.4120). test-backend-ops, 0 failures: R9700 ADD RMS_NORM SOFT_MAX GET_ROWS UPSCALE CPY ROPE SET_ROWS 1820/1820 and 51 further op groups 4259/4259; Iris Xe MUL_MAT_ID 943/943, MUL_MAT 2519/2519, FLASH_ATTN_EXT 5308/5308. Not run: zip smoke on Intel and on the MTT S80.