This is all just ai-slop testing but the it seems valid on my setup. didnt see anything about this in the docs and you did mention the two pools
Setting GGML_VK_PREFER_HOST_MEMORY=1 gives +5–12% on text-generation throughput with no measurable impact on prompt processing.
The BC-250 exposes two memory heaps — a 5.5 GiB BAR-visible heap (heap 0) and an 11 GiB DEVICE_LOCAL-only heap (heap 1).
From vulkaninfo output:
memoryHeaps: count = 2
memoryHeaps[0]:
size = 5905580032 (0x160000000) (5.50 GiB)
budget = 1318912000 (0x4e9d0000) (1.23 GiB)
usage = 0 (0x00000000) (0.00 B)
flags:
None
memoryHeaps[1]:
size = 11811160064 (0x2c0000000) (11.00 GiB)
budget = 2637819904 (0x9d39f000) (2.46 GiB)
usage = 0 (0x00000000) (0.00 B)
flags: count = 1
MEMORY_HEAP_DEVICE_LOCAL_BIT
So:
- Heap 0: 5.50 GiB, no flags (i.e. not MEMORY_HEAP_DEVICE_LOCAL_BIT) — this is the BAR-visible window.
- Heap 1: 11.00 GiB, MEMORY_HEAP_DEVICE_LOCAL_BIT only — invisible-to-CPU device memory.
And the 11 memory types map onto these heaps with different property combinations. The relevant ones for buffer placement:
memoryTypes[0]: heapIndex=1, DEVICE_LOCAL only ← weights land here by default
memoryTypes[2]: heapIndex=0, HOST_VISIBLE | HOST_COHERENT ← weights land here with PREFER_HOST_MEMORY
memoryTypes[3]: heapIndex=1, DEVICE_LOCAL | HOST_VISIBLE | HOST_COHERENT (rebar)
memoryTypes[5]: heapIndex=0, HOST_VISIBLE | HOST_COHERENT | HOST_CACHED
llama.cpp's default UMA path requests eDeviceLocal first → matches type 0 → lands on heap 1 (11 GiB). With GGML_VK_PREFER_HOST_MEMORY=1 it requests eHostVisible | eHostCoherent first → matches type 2 → lands on heap 0 (5.5 GiB). The +5–12% tg gain is between those two placements on otherwise identical configs.
The 16 GiB total (5.5 + 11) matches mem_info_gtt_total = 17179869184 from /sys/class/drm/card0/device/, confirming both heaps back the same unified GDDR6 pool — they're different views of the same physical memory, not distinct silicon. That's why the gain has to be a cache-attribute or MMU-path effect rather than physical bandwidth.
With GGML_VK_PREFER_HOST_MEMORY=1, weights go to heap 0 (BAR-mapped) and inference is consistently faster — same physical GDDR6 backing both, so it's likely a cache-attribute or MMU-path difference. Models that exceed 5.5 G/btwiB still work — Vulkan falls back to heap 1 for the overflow, and even then there's a small +2% gain.
This is all just ai-slop testing but the it seems valid on my setup. didnt see anything about this in the docs and you did mention the two pools
Setting GGML_VK_PREFER_HOST_MEMORY=1 gives +5–12% on text-generation throughput with no measurable impact on prompt processing.
The BC-250 exposes two memory heaps — a 5.5 GiB BAR-visible heap (heap 0) and an 11 GiB DEVICE_LOCAL-only heap (heap 1).
From vulkaninfo output:
So:
- Heap 0: 5.50 GiB, no flags (i.e. not MEMORY_HEAP_DEVICE_LOCAL_BIT) — this is the BAR-visible window.
- Heap 1: 11.00 GiB, MEMORY_HEAP_DEVICE_LOCAL_BIT only — invisible-to-CPU device memory.
And the 11 memory types map onto these heaps with different property combinations. The relevant ones for buffer placement:
llama.cpp's default UMA path requests eDeviceLocal first → matches type 0 → lands on heap 1 (11 GiB). With GGML_VK_PREFER_HOST_MEMORY=1 it requests eHostVisible | eHostCoherent first → matches type 2 → lands on heap 0 (5.5 GiB). The +5–12% tg gain is between those two placements on otherwise identical configs.
The 16 GiB total (5.5 + 11) matches mem_info_gtt_total = 17179869184 from /sys/class/drm/card0/device/, confirming both heaps back the same unified GDDR6 pool — they're different views of the same physical memory, not distinct silicon. That's why the gain has to be a cache-attribute or MMU-path effect rather than physical bandwidth.
With GGML_VK_PREFER_HOST_MEMORY=1, weights go to heap 0 (BAR-mapped) and inference is consistently faster — same physical GDDR6 backing both, so it's likely a cache-attribute or MMU-path difference. Models that exceed 5.5 G/btwiB still work — Vulkan falls back to heap 1 for the overflow, and even then there's a small +2% gain.