Skip to content

D3D12 backend v0.5.3 - robot models (VLA), faster DiffusionGemma

Choose a tag to compare

View all tags
@koza-coder koza-coder released this 07 Nov 12:11
v0.5.3
062a5b8

D3D12 backend v0.5.3 - robot models (VLA), faster DiffusionGemma

Windows x64, DirectX 12. On llama.cpp master 6023692 (2026-11-05). Contains everything of v0.5.2.

What is new in v0.5.3

  • Robot models (vision-language-action) run on DirectX 12. vla-cli.exe is the vla-cli program of vla.cpp (https://github.com/VinRobotics/vla.cpp, commit 1012c05, Apache-2.0) built with this zip's DirectX 12 backend; our changes to vla.cpp (backend selection, an embedded test image, a Windows exit fix) are in vla-cpp-1012c05.patch (attached to this release). It takes a camera image and prints rows of robot actions.
    • robot-download-models.bat downloads 5 small models (ACT, Octo-Small, TurboVLA, SmolVLA, Evo-1; about 4.3 GB) into X:\llamacppdx12\models and checks their SHA-256.
    • robot-test.bat runs all 5 and writes robot-results-<PC name>.zip; see README-robots.md.
  • DiffusionGemma another 1.4x faster. The host sampler of llama-diffusion-cli computes the exp of 4 logits at once (SSE2) and sums in double: 0.63 -> 0.45 s per diffusion step on a Radeon AI PRO R9700, same text.

Speed on a Radeon AI PRO R9700 (DX12)

Model Tool Result
ACT / Octo-Small / TurboVLA / SmolVLA / Evo-1 vla-cli, median per policy call 10.7 / 13.0 / 23.1 / 146.2 / 557.1 ms
diffusiongemma-26B-A4B-it Q4_K_M llama-diffusion-cli, 1024 tokens 0.45 s per step, 42.3 tok/s on the test poem (v0.5.2: 0.63 s, 30.0 tok/s)
Clover-35B-A3B-Preview Q4_K_M llama-bench pp512 682.24 t/s, tg128 66.05 t/s
Clover-35B-A3B-Preview Q4_K_M llama-completion 65.16 t/s generation
Clover-35B-A3B-Preview Q4_K_M llama-clover-diffusion 6.13 t/s
LLaDA-MoE-7B-A1B-Instruct Q4_K_M llama-diffusion-cli, block 32, 32 steps 50.61 ms per step

Tested

  • Radeon AI PRO R9700: every shipped exe of this zip (llama-server, llama-mtmd-cli, llama-cli, llama-completion, llama-perplexity, llama-bench, test-backend-ops ADD 116/116, llama-diffusion-cli, llama-clover-diffusion, vla-cli with all 5 robot models: exit 0, finite actions); Clover perplexity 4.2676 +/- 0.35 (same as v0.5.2).
  • The .bat files were not run in the test; they call the tested exes. Other GPUs were not tested with this zip.
Assets 6
Loading