Skip to content

Conversation

giladgd
Copy link
Contributor

@giladgd giladgd commented Oct 20, 2025

A Microsoft Direct3D12 device appears on some Windows machines, which comes from the Microsoft Dozen (Dzn) Vulkan driver that is built on top of Direct3D 12.

This PR fixes an issue where sometimes both the native device entry and the Direct3D12 device entry are picked up by llama.cpp, and since both point to the same underlying device, after using all the memory of the first device, buffer allocations on the second will always fail.

When a native Vulkan driver is already available for a given device, we now prioritize it over the Dozen driver for reduced overhead.
I found a note on VulkanHub about the Direct3D12 device that was helpful in figuring this out.


Before

ggml_vulkan: Found 4 Vulkan devices:
ggml_vulkan: 0 = Microsoft Direct3D12 (AMD Radeon(TM) Graphics) (Dozen) | uma: 1 | fp16: 1 | bf16: 0 | warp size: 32 | shared memory: 32768 | int dot: 1 | matrix cores: none
ggml_vulkan: 1 = AMD Radeon(TM) Graphics (AMD proprietary driver) | uma: 1 | fp16: 1 | bf16: 0 | warp size: 32 | shared memory: 65536 | int dot: 1 | matrix cores: none
ggml_vulkan: 2 = NVIDIA GeForce RTX 3070 Ti Laptop GPU (NVIDIA) | uma: 0 | fp16: 1 | bf16: 1 | warp size: 32 | shared memory: 49152 | int dot: 1 | matrix cores: NV_coopmat2
ggml_vulkan: 3 = Microsoft Direct3D12 (NVIDIA GeForce RTX 3070 Ti Laptop GPU) (Dozen) | uma: 0 | fp16: 1 | bf16: 0 | warp size: 32 | shared memory: 32768 | int dot: 1 | matrix cores: none

After

ggml_vulkan: Found 2 Vulkan devices:
ggml_vulkan: 0 = AMD Radeon(TM) Graphics (AMD proprietary driver) | uma: 1 | fp16: 1 | bf16: 0 | warp size: 32 | shared memory: 65536 | int dot: 1 | matrix cores: none
ggml_vulkan: 1 = NVIDIA GeForce RTX 3070 Ti Laptop GPU (NVIDIA) | uma: 0 | fp16: 1 | bf16: 1 | warp size: 32 | shared memory: 49152 | int dot: 1 | matrix cores: NV_coopmat2

@giladgd giladgd requested a review from 0cc4m as a code owner October 20, 2025 17:40
@github-actions github-actions bot added Vulkan Issues specific to the Vulkan backend ggml changes relating to the ggml tensor library for machine learning labels Oct 20, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ggml changes relating to the ggml tensor library for machine learning Vulkan Issues specific to the Vulkan backend

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant