Misc. bug: Can you disable a backend in llama-server?

[sergeysi779]

Name and Version

llama-cli --version
ggml_cuda_init: failed to initialize CUDA: no CUDA-capable device is detected
ggml_vulkan: Found 2 Vulkan devices:
ggml_vulkan: 0 = AMD Radeon Vega 3 Graphics (RADV RAVEN2) (radv) | uma: 1 | fp16: 1 | bf16: 0 | warp size: 64 | shared memory: 65536 | int dot: 0 | matrix cores: none
ggml_vulkan: 1 = AMD Radeon 540X Series (RADV POLARIS12) (radv) | uma: 0 | fp16: 0 | bf16: 0 | warp size: 64 | shared memory: 65536 | int dot: 0 | matrix cores: none
version: 6970 (7f09a68)
built with cc (Ubuntu 13.3.0-6ubuntu2~24.04) 13.3.0 for x86_64-linux-gnu

Operating systems

No response

Which llama.cpp modules do you know to be affected?

No response

Command line

Problem description & steps to reproduce

I have a laptop with AMD dGPU and nVidia eGPU.
llama.cpp is built with both CUDA and Vulkan backends.
When I use llama.cpp with explicitly set -dev Vulkan1 it tries to initialize CUDA backend first:

llama-server --port 10021 --host 0.0.0.0 -ngl 999 --jinja -fa on -dev Vulkan1 --model qwen3/Qwen3-VL-2B-Instruct-UD-Q4_K_XL.gguf -c 8000 -nr
ggml_cuda_init: failed to initialize CUDA: no CUDA-capable device is detected
ggml_vulkan: Found 2 Vulkan devices:

Is there a way to disable CUDA backend in such cases?
I tried to set CUDA_VISIBLE_DEVICES="", it didn't help.

It's not critical in any way, it's just annoying 10 seconds when loading a model.

First Bad Commit

No response

Relevant log output

[tbocek]

With Vulkan, try GGML_VK_VISIBLE_DEVICES=xy, not sure if it will change the priority though... but maybe worth a test

[sergeysi779]

No, it still tries to initialize CUDA backend with GGML_VK_VISIBLE_DEVICES set:

$ GGML_VK_VISIBLE_DEVICES=1 llama-cli --version
ggml_cuda_init: failed to initialize CUDA: no CUDA-capable device is detected
ggml_vulkan: Found 1 Vulkan devices:
ggml_vulkan: 0 = AMD Radeon 540X Series (RADV POLARIS12) (radv) | uma: 0 | fp16: 0 | bf16: 0 | warp size: 64 | shared memory: 65536 | int dot: 0 | matrix cores: none
version: 6970 (7f09a680a)
built with cc (Ubuntu 13.3.0-6ubuntu2~24.04) 13.3.0 for x86_64-linux-gnu

[taronaeo]

llama.cpp is built with both CUDA and Vulkan backends.

Is this a hard requirement that you must have both CUDA and Vulkan backend support compiled? Do you have a requirement to fulfil here? IMO it's better to compile support for only 1 backend.

[0cc4m]

I don't think it should take that long for the CUDA backend to figure out that it has no devices. Any idea which state it's hanging on? Maybe someone needs to look into it with a profiler.

[sergeysi779]

Is this a hard requirement that you must have both CUDA and Vulkan backend support compiled? Do you have a requirement to fulfil here?

I use a laptop with integrated AMD GPU and nVidia as eGPU, so I compiled it for both Vulkan and CUDA backends to be able to use smaller models when without eGPU.
It's not a hard requirement. I just thought maybe there is a way to disable or select a backend at runtime.