feat(windows): add Vulkan GPU detection for Intel Arc and other non-CUDA GPUs

[darthcav]

Summary

Fixes #925 — Docker Model Runner does not use Intel Arc GPU via Vulkan on Windows.

Root cause

CanUseGPU in gpuinfo_windows.go only detects NVIDIA CUDA (amd64) and Qualcomm Adreno OpenCL (arm64). There is no Vulkan path, so Intel Arc and other Vulkan-capable GPUs (AMD without CUDA, etc.) are silently ignored and fall back to the cpu llama.cpp variant.

Changes

gpuinfo_windows.go

• hasVulkanCapableGPU() — uses PCI enumeration (via ghw) to find GPUs that are neither NVIDIA nor Adreno; Intel Arc, AMD GPUs, and similar fall into this category.
• hasVulkan() — probes vulkan-1.dll via syscall.LoadLibrary, mirroring the existing hasOpenCL / OpenCL.dll pattern. Returns true only when a Vulkan-capable GPU is present and the Vulkan runtime is loadable.
• CanUseGPU() — on amd64, after the CUDA check, now also calls hasVulkan().
• hasNVIDIAGPU(), hasSupportedAdrenoGPU(), hasVulkanCapableGPU() — added nil guards for DeviceInfo, Vendor, and Product fields before dereferencing .Name, preventing panics when ghw fails to retrieve full PCI details for a card.

download_windows.go

• Adds canUseVulkan detection in ensureLatestLlamaCpp.
• Priority order: CUDA > Vulkan > CPU (OpenCL stays arm64-only).
• Selects desiredVariant = "vulkan" when Vulkan is detected.

Known limitation / follow-up required

No vulkan image variant of docker/docker-model-backend-llamacpp currently exists on Docker Hub (tags available: cpu, cuda, opencl, rocm, metal, generic). When the vulkan tag is not found, downloadLatestLlamaCpp logs a warning and falls back to the vendored Docker Desktop binary — which already ships ggml-vulkan.dll. A TODO comment in download_windows.go marks this gap.

A follow-up task to publish docker/docker-model-backend-llamacpp:latest-vulkan (a Windows build with the Vulkan backend compiled in) is needed to complete the end-to-end fix.

Test plan

• Build compiles cleanly for GOOS=windows GOARCH=amd64 ✓
• Existing unit tests pass (go test ./pkg/inference/backends/llamacpp/...) ✓
• Manual verification on a Windows machine with Intel Arc GPU: docker model run ai/smollm2 "Test" should show Vulkan backend in logs once the vulkan Docker Hub image variant is published.

🤖 Generated with Claude Code

[sourcery-ai]

Hey - I've found 1 issue, and left some high level feedback:

• In hasVulkanCapableGPU, accessing gpu.DeviceInfo.Vendor.Name and gpu.DeviceInfo.Product.Name assumes these nested fields are always non-nil; consider adding nil checks to avoid panics on unexpected ghw data.
• The Adreno/Qualcomm detection in hasVulkanCapableGPU is case-sensitive while the vendor comparison is lowercased; normalizing product (and possibly using vendor IDs) would make the heuristic more robust and avoid misclassifying GPUs as Vulkan-capable.

Prompt for AI Agents

Please address the comments from this code review:

## Overall Comments
- In `hasVulkanCapableGPU`, accessing `gpu.DeviceInfo.Vendor.Name` and `gpu.DeviceInfo.Product.Name` assumes these nested fields are always non-nil; consider adding nil checks to avoid panics on unexpected `ghw` data.
- The Adreno/Qualcomm detection in `hasVulkanCapableGPU` is case-sensitive while the vendor comparison is lowercased; normalizing `product` (and possibly using vendor IDs) would make the heuristic more robust and avoid misclassifying GPUs as Vulkan-capable.

## Individual Comments

### Comment 1
<location path="pkg/inference/backends/llamacpp/gpuinfo_windows.go" line_range="111-113" />
<code_context>
+	if err != nil {
+		return false, err
+	}
+	for _, gpu := range gpus.GraphicsCards {
+		vendor := strings.ToLower(gpu.DeviceInfo.Vendor.Name)
+		product := gpu.DeviceInfo.Product.Name
+		isNVIDIA := vendor == "nvidia"
+		isAdreno := strings.Contains(product, "Adreno") || strings.Contains(product, "Qualcomm")
</code_context>
<issue_to_address>
**issue (bug_risk):** Guard against nil DeviceInfo/Product to avoid potential panics from ghw.GPU()

ghw may return GraphicsCards where `DeviceInfo`, `Vendor`, or `Product` is nil. Directly accessing `gpu.DeviceInfo.Vendor.Name` and `gpu.DeviceInfo.Product.Name` can therefore panic. Please add nil checks (e.g., skip entries with missing DeviceInfo/Product) before dereferencing these fields.
</issue_to_address>

---

Sourcery is free for open source - if you like our reviews please consider sharing them ✨

• X
• Mastodon
• LinkedIn
• Facebook

_{Help me be more useful! Please click 👍 or 👎 on each comment and I'll use the feedback to improve your reviews.}

[gemini-code-assist]

Code Review

This pull request introduces Vulkan GPU detection for Windows amd64 systems as a fallback when CUDA is not available. It includes logic to identify Vulkan-capable hardware and verify the presence of the Vulkan runtime library. A critical review comment identifies potential nil pointer dereferences in the hardware discovery logic, recommending defensive checks to ensure stability.

[ericcurtin]

@darthcav go may not be your coding language but this PR is perfectly clean :) The important thing is that you tested and confirmed this worked. Did you test it?

[darthcav]

@ericcurtin The problem is that I cannot do a full compilation in my laptop and check it against the local GPU. I would need someone to compile that...

[ericcurtin]

@ericcurtin The problem is that I cannot do a full compilation in my laptop and check it against the local GPU. I would need someone to compile that...

Well you could also use the coding agent you used for this to figure out how to build it.

I don't have an Intel Arc machine, so even if I wanted to, I can't test this.