DeviceLostError on Intel Xe TGL with Vulkan

[dinccey]

Hi, I tried compiling whisper.cpp with Vulkan with these flags

-DGGML_VULKAN=1 -DGGML_BLAS=ON -DGGML_BLAS_VENDOR=OpenBLAS

and everything starts out fine, inference is correct and much faster than CPU, and then this crash occurs:
/home/user/projects/whisper/whisper.cpp/build/ggml/src/libggml-base.so.0(+0x2f65) [0x7efde403cf65] /home/user/projects/whisper/whisper.cpp/build/ggml/src/libggml-base.so.0(ggml_print_backtrace+0x1eb) [0x7efde403d32b] /home/user/projects/whisper/whisper.cpp/build/ggml/src/libggml-base.so.0(+0x14e79) [0x7efde404ee79] /lib64/libstdc++.so.6(+0x1eb9c) [0x7efde0da7b9c] /lib64/libstdc++.so.6(_ZSt10unexpectedv+0x0) [0x7efde0d91d3a] /lib64/libstdc++.so.6(+0x1ee48) [0x7efde0da7e48] /home/user/projects/whisper/whisper.cpp/build/ggml/src/ggml-vulkan/libggml-vulkan.so.0(+0x6c46) [0x7efde1006c46] /home/user/projects/whisper/whisper.cpp/build/ggml/src/ggml-vulkan/libggml-vulkan.so.0(+0xde505) [0x7efde10de505] /home/user/projects/whisper/whisper.cpp/build/ggml/src/ggml-vulkan/libggml-vulkan.so.0(+0xde6f0) [0x7efde10de6f0] /home/user/projects/whisper/whisper.cpp/build/ggml/src/libggml-base.so.0(ggml_backend_sched_synchronize+0x2e) [0x7efde40542ce] /home/user/projects/whisper/whisper.cpp/build/ggml/src/libggml-base.so.0(ggml_backend_sched_graph_compute+0x1c) [0x7efde4057c3c] /home/user/projects/whisper/whisper.cpp/build/src/libwhisper.so.1(+0xb078) [0x7efde4256078] /home/user/projects/whisper/whisper.cpp/build/src/libwhisper.so.1(+0x20b3b) [0x7efde426bb3b] /home/user/projects/whisper/whisper.cpp/build/src/libwhisper.so.1(whisper_full_with_state+0x13c0) [0x7efde4282160] /home/user/projects/whisper/whisper.cpp/build/src/libwhisper.so.1(whisper_full+0xec) [0x7efde4286f7c] ./whisper/whisper.cpp/build/bin/whisper-cli() [0x40490d] /lib64/libc.so.6(+0x35b5) [0x7efde0b985b5] /lib64/libc.so.6(__libc_start_main+0x88) [0x7efde0b98668] ./whisper/whisper.cpp/build/bin/whisper-cli() [0x4079a5] terminate called after throwing an instance of 'vk::DeviceLostError' what(): vk::Device::waitForFences: ErrorDeviceLost Processing failed: Command '['./whisper/whisper.cpp/build/bin/whisper-cli', '-m', './whisper/whisper.cpp/models/ggml-medium.bin', '--language', 'en', '--vad', '--vad-model', './whisper/whisper.cpp/models/ggml-silero-v5.1.2.bin', '-bs', '5', '--entropy-thold', '2.8', '--max-context', '64', '-f', '2d1d99a369771750bb1c62c68a6471cd.wav', '-ovtt']' died with <Signals.SIGABRT: 6>.

Running on Fedora 43 (in toolbox container) with these dependencies:
dnf install vulkan-loader vulkan-loader-devel vulkan-headers vulkan-tools \ mesa-vulkan-drivers glslc vulkan-validation-layers vulkan-validation-layers-devel \ libshaderc-devel

Vulkaninfo summary (in toolbox ct)
`Devices:

GPU0:
apiVersion = 1.4.318
driverVersion = 25.2.6
vendorID = 0x8086
deviceID = 0x9a49
deviceType = PHYSICAL_DEVICE_TYPE_INTEGRATED_GPU
deviceName = Intel(R) Iris(R) Xe Graphics (TGL GT2)
driverID = DRIVER_ID_INTEL_OPEN_SOURCE_MESA
driverName = Intel open-source Mesa driver
driverInfo = Mesa 25.2.6
conformanceVersion = 1.4.0.0
`

Any idea why it happens?

[dinccey]

I think I managed to make it stable by disabling Flash attention. After around 17 hours of transcribing long audio files on 4 different Intel machines (2x UHD 630, 1x Xe TGL, 1x UHD 770) no crashes due to Vulkan happened. With FA enabled (default) I had frequent crashes on Xe and every few hours on the other ones so I'd call Flash attention the culprit that crashes the Vulkan driver for whatever reason. I didn't find another way to disable flash attention, so this is how to do it for whisper.cpp:

1. edit whisper.cpp/examples/cli/cli.cpp
   by replacing line
   bool flash_attn = true;
   to
   bool flash_attn = false;

2. build with Vulkan as usual

The performance is still pretty good and definitely much better than crashing randomly 40 minutes into a file. It is also a lot more power efficient than CPU mode so it is worth it.

Is there an easier way to disable flash attention?