Skip to content

D3D12 backend v0.5.2 - text diffusion models (LLaDA, Dream, DiffusionGemma, Clover), newer llama.cpp

Choose a tag to compare

View all tags
@koza-coder koza-coder released this 07 Nov 10:58
v0.5.2
ae60775

D3D12 backend v0.5.2 - text diffusion models, newer llama.cpp

Windows x64, DirectX 12. On llama.cpp master 6023692 (2026-11-05). Contains everything of v0.4.4 (v0.5.0 and v0.5.1 were never published; their changes are part of this release).

What is new in v0.5.2

  • DiffusionGemma about 1.6x faster. The host sampler of llama-diffusion-cli computed the exp of all 262144 vocabulary entries twice per position; it now computes each once (same sampled tokens, same text). DiffusionGemma-26B, 1024 tokens: 0.96 s -> 0.57-0.63 s per diffusion step on a Radeon AI PRO R9700.

Also new since v0.4.4:

  • Text diffusion models run on DirectX 12.

    • llama-diffusion-cli.exe: LLaDA / LLaDA-MoE, Dream / Dream-Coder and DiffusionGemma (upstream llama.cpp PR 24423, not merged upstream yet).
    • llama-clover-diffusion.exe: Clover-35B-A3B block diffusion (32-token blocks, denoised in parallel).
    • Ready-made .bat files: diffusiongemma-chat.bat, diffusiongemma-visual.bat, llada-diffusion-visual.bat, dream-diffusion-visual.bat, clover-chat.bat, clover-server.bat, clover-diffusion.bat, clover-diffusion-visual.bat (the -visual ones show the canvas being filled step by step).

  • llama.cpp updated by 127 upstream commits (785322d -> 6023692). The backend follows two ggml changes: the RoPE truncate flag, and FLASH_ATTN_EXT with a K/V row list (not supported yet; those nodes run elsewhere).

  • Bf16 copies run on the GPU. Models that copy values into bf16 tensors in every layer (Clover-35B-A3B) ran each copy on the CPU (1302 graph splits per token); now 2 splits.

Speed on a Radeon AI PRO R9700 (DX12)

Model Tool Result
Clover-35B-A3B-Preview Q4_K_M llama-bench pp512 695.89 t/s, tg128 67.29 t/s (before the bf16 fix: 213.75 / 6.29)
Clover-35B-A3B-Preview Q4_K_M llama-completion 65.3 t/s generation
Clover-35B-A3B-Preview Q4_K_M llama-clover-diffusion 6.2-16.0 t/s (prompt-dependent)
diffusiongemma-26B-A4B-it Q4_K_M llama-diffusion-cli, 1024 tokens 0.63 s per step, 30.0 tok/s on the test poem (v0.5.1: 0.96 s per step, 19.7 tok/s)
LLaDA-MoE-7B-A1B-Instruct Q4_K_M llama-diffusion-cli, block 32, 32 steps 53.5 ms per step

The Vulkan build is still faster for Clover on this card (pp512 3526, tg128 109 t/s).

Tested

  • Radeon AI PRO R9700: every shipped exe of this zip (llama-server, llama-mtmd-cli, llama-cli, llama-completion, llama-perplexity, llama-bench, test-backend-ops, llama-diffusion-cli, llama-clover-diffusion); Clover perplexity 4.2676 +/- 0.35 (4 chunks of 512, same as Vulkan within error).
  • Same source, R9700: test-backend-ops ROPE 510/510, FLASH_ATTN_EXT 5368/5368, CPY 412/412, SET_ROWS 323/323, ADD 116/116; LLaDA-MoE, Dream-Coder-7B, DiffusionGemma-26B and Clover-35B (AR and diffusion) give correct answers.
  • The .bat files were not run in the test; they call the tested exes.
Assets 5
Loading