How can I know and change the GPU backend llama-cpp is using?

[mildred]

I'm using llama-server to run the models, and I believe I have llama-cpp built with both Vulkan and ROCm. I have a hardware that now supports both on the latest Fedora.

My question is simple : how can I know which backend llama-cpp is using for inference?

How can I change the GPU backend (choose Vulkan over RONm for instance as the former seems best)?

Is there a way I can see which backend are compiled in?

Thank you

[hoangperry]

All three questions have direct answers:

1. Which backend/device is in use: the startup log says it. Look at the first lines from llama-server — each backend announces its devices as it initializes (e.g. ggml_vulkan: Found 1 Vulkan devices: ... / ROCm lines from the HIP backend), and the model-load output shows which device the layers were offloaded to. If a GPU backend is present, it's preferred over CPU automatically.

2. Listing what's available: llama-server --list-devices prints every device the build can see, tagged by backend (Vulkan0, ROCm0, ...). If both ROCm and Vulkan are compiled in, your 7900-class GPU will typically show up twice — once per backend.

3. Choosing one: pass --device with the device name from that list, e.g. --device Vulkan0 to force Vulkan over ROCm (combine with -ngl 99 as usual). Setting --device none gives you a CPU run for comparison.

4. Which backends were compiled in: the version banner plus the startup init lines effectively tell you (each compiled backend logs its probe). Note that distro/Fedora-style builds increasingly ship backends as runtime-loadable ggml-* modules rather than baked in — in that case what's "available" depends on which backend packages are installed, and --list-devices remains the ground truth.

Practical note for your comparison: benchmark both with llama-bench on your actual model before settling — on RDNA3, Vulkan vs ROCm ranking varies by quant and batch size, and the winner isn't consistent across model sizes.