Misc. bug: Running Vulkan backend on Intel DG1 outputs gibberish due to commit 38eaf32

[h9j6k]

Name and Version

$ ./llama-server --version
ggml_vulkan: Found 1 Vulkan devices:
ggml_vulkan: 0 = Intel(R) Iris(R) Xe Graphics (DG1) (Intel open-source Mesa driver) | uma: 0 | fp16: 1 | bf16: 0 | warp size: 32 | shared memory: 65536 | int dot: 1 | matrix cores: none
version: 7079 (416e7c7)
built with cc (Debian 14.3.0-8) 14.3.0 for x86_64-linux-gnu

Operating systems

Linux

Which llama.cpp modules do you know to be affected?

llama-server

Command line

./llama-server -m /path/to/models/ibm-granite_granite-4.0-h-micro-Q4_0.gguf --host HOSTNAME --port PORT -c 16384 -b 8192 -e -ngl 99 -t 8 -n -1 --no-mmap --jinja --kv-unified -fa off -np 2

Problem description & steps to reproduce

I git pull'ed this morning then it went crazy on 416e7c7 So I rolled back all those vulkan related commits one by one until the one right before 38eaf32, it became ok. Then I fast forward'ed to 416e7c7 again and simply git revert 38eaf32, it was still ok. So I think 38eaf32 is the issue?

First Bad Commit

38eaf32

Relevant log output

(test-backend-ops -o MUL_MAT,MUL_MAT_ID)

  4/4 tests passed
  Backend BLAS: OK
Backend 3/3: CPU
  Skipping CPU backend
3/3 backends passed
OK

(model loading and running)

Nothing unusual except output is gibberish

[jeffbolznv]

I don't have any ideas for what could be causing this. Are you on the latest driver?

Please try a couple experiments: (1) Add a call to ggml_vk_synchronize(ctx); at the end of ggml_backend_vk_graph_compute, (2) Change ggml_backend_vk_get_tensor_async not to call ggml_vk_buffer_read_async (just set ret = false;).

[h9j6k]

Thank you. I tried several things and the issue is still there.

Are you on the latest driver?

I am on debian 13 trixie. I tried two drivers, mesa 25.2.6 backport from debian and 25.3 compiled from mesa3d.org release. Same issue as before.

vulkaninfo vulkan sdk 328.1 from lunarG, mesa 25.2.6 from debian

$ vulkaninfo |grep -i version
Vulkan Instance Version: 1.4.328
VK_LAYER_INTEL_nullhw (INTEL NULL HW) Vulkan version 1.1.73, layer version 1:
VK_LAYER_KHRONOS_profiles (Khronos Profiles layer) Vulkan version 1.4.328, layer version 1:
VK_LAYER_KHRONOS_shader_object (Khronos Shader object layer) Vulkan version 1.4.328, layer version 1:
VK_LAYER_KHRONOS_synchronization2 (Khronos Synchronization2 layer) Vulkan version 1.4.328, layer version 1:
VK_LAYER_KHRONOS_validation (Khronos Validation Layer) Vulkan version 1.4.328, layer version 1:
VK_LAYER_KHRONOS_validation (Khronos Validation Layer) Vulkan version 1.4.321, layer version 1:
VK_LAYER_LUNARG_api_dump (LunarG API dump layer) Vulkan version 1.4.328, layer version 2:
VK_LAYER_LUNARG_crash_diagnostic (Crash Diagnostic Layer is a crash/hang debugging tool that helps determines GPU progress in a Vulkan application.) Vulkan version 1.4.328, layer version 1:
VK_LAYER_LUNARG_gfxreconstruct (GFXReconstruct Capture Layer Version 1.0.5) Vulkan version 1.4.328, layer version 4194309:
VK_LAYER_LUNARG_monitor (Execution Monitoring Layer) Vulkan version 1.4.328, layer version 1:
VK_LAYER_LUNARG_screenshot (LunarG image capture layer) Vulkan version 1.4.328, layer version 1:
VK_LAYER_MESA_device_select (Linux device selection layer) Vulkan version 1.4.303, layer version 1:
VK_LAYER_MESA_overlay (Mesa Overlay layer) Vulkan version 1.4.303, layer version 1:
        apiVersion        = 1.4.318 (4211006)
        driverVersion     = 25.2.6 (104865798)
        conformanceVersion:
        VK_KHR_sampler_ycbcr_conversion               : extension revision 14
        samplerYcbcrConversion             = true

Please try a couple experiments

Regarding (1) by your suggestion, what I did was, on l.12902, adding ggml_vk_synchronize(ctx);

static ggml_status ggml_backend_vk_graph_compute(ggml_backend_t backend, ggml_cgraph * cgraph) {                                     
...  
  return GGML_STATUS_SUCCESS;                     
                                                     
  UNUSED(backend);                                
  ggml_vk_synchronize(ctx);  // line 12902, ggml-vulkan.cpp                   
}                      

Regarding (2), what I did was, commenting out bool ret on l.12226 then set bool ret=false;

static void ggml_backend_vk_get_tensor_async(ggml_backend_t backend, const ggml_tensor * tensor, void * data, size_t offset, size_t size) {
 ...
//    bool ret = ggml_vk_buffer_read_async(transfer_ctx, buf, src_offset, data, size);
    bool ret = false; // line 12227, ggml-vulkan.cpp
    // If that failed, copy synchronously through a staging buffer
...
}

[jeffbolznv]

ggml_vk_synchronize(ctx); // line 12902, ggml-vulkan.cpp

This should be before the return.

[h9j6k]

This should be before the return.

My bad!

Tried this alone, it did not make a difference.

l.12899    ggml_vk_synchronize(ctx);
l.12900    return GGML_STATUS_SUCCESS;

Tried both the above and bool ret = false; on l.12227, still the same.

[0cc4m]

Does running with the environment variable GGML_VK_DISABLE_INTEGER_DOT=1 make a difference? Or with GGML_VK_DISABLE_F16=1?

[h9j6k]

Does running with the environment variable GGML_VK_DISABLE_INTEGER_DOT=1 make a difference? Or with GGML_VK_DISABLE_F16=1?

Tried each of them separately and both with Jeff's suggestions enabled, no difference. Also tried them each separately and both on master, no difference :<

[0cc4m]

Thank you, then it's a different issue. That's helpful to rule out.