ODS on Proxmox VM with AMD IGPU RADV RENOIR passthrough

[tigermick]

Update: full diagnosis after further testing

I dug deeper into this and wanted to share what I found, since the crash is reproducible and I think I've isolated the actual failure point.

System

Proxmox VM, AMD Renoir/Lucienne iGPU (gfx90c) passed through via full VFIO PCI passthrough (04:00.0 VGA controller)
RADV driver confirmed working: vulkaninfo correctly detects AMD QEMU Virtual CPU version 2.5+ (RADV RENOIR), Mesa 25.2.8
/dev/dri accessible in container with correct permissions (render/video group)
ODS llama-server backend: Vulkan (not ROCm — ROCm doesn't support gfx90c)

What I ruled out

I tested extensively with llama-cli directly inside the ods-llama-server container to isolate the issue from ODS's own wrapper:

Not model-specific: crashes identically on Qwen3.5-2B (hybrid SSM/attention) and Qwen2.5-1.5B-Instruct (plain attention, no SSM layers) — same failure signature both times.
Not flash attention: -fa off doesn't help.
Not batch size: reducing -ub to 256 doesn't help.
Not mmap: --no-mmap changes the crash from an assert to a silent segfault, but doesn't fix it.
Not GGML_VK_PREFER_HOST_MEMORY=1: no effect, same assert.

The actual failure

With the default -fit (auto memory fitting), both Vulkan0 and Vulkan_Host buffers report 0.00 MiB at load time, and the process segfaults right after llama_context: constructing llama_context.

Adding -fit off (as suggested by llama.cpp's own log message) makes the buffer sizes report correctly (Vulkan0 model buffer size = 934.69 MiB), but then a different, more informative failure appears:

done_getting_tensors: tensor 'token_embd.weight' (q4_K) (and 0 others) cannot be used with preferred buffer type Vulkan_Host, using CPU instead
/ggml/src/ggml-vulkan/ggml-vulkan.cpp:6804: GGML_ASSERT(dst->memory_property_flags & vk::MemoryPropertyFlagBits::eHostCoherent) failed

This points to ggml-vulkan assuming the "host visible" memory heap exposed by the driver always carries the HOST_COHERENT flag when placing tensors in a Vulkan_Host buffer. On this passthrough RADV Renoir setup, that assumption seems to break, and instead of falling back gracefully it hits a hard assert/abort.

Reproduction command (run inside the ods-llama-server container):

llama-cli
-m /models/qwen2.5-1.5b-instruct-q4_k_m.gguf
-p "Hello" -n 10 --device Vulkan0 --ctx-size 2048
-ngl 99 -fa off -fit off --verbose

Questions for maintainers / other users

Is this a known issue with the llama.cpp/Vulkan binary bundled via lemonade for AMD Renoir (gfx90c) / RADV under virtualization?
Is there a way to pin ODS to an earlier llama.cpp build (before the -fit auto-memory-fitting and Vulkan_Host preferred-buffer logic were introduced), as a workaround?
Has anyone gotten Vulkan GPU inference working on this specific chip (Renoir/Lucienne, gfx90c) through ODS, passthrough or bare metal?

Happy to provide any additional logs, vulkaninfo --summary output, or test further combinations if it helps track this down.