Build failure: llama-cpp (with vulkanSupport = true)

[twice5023]

Nixpkgs version

• Unstable (25.05)

Steps to reproduce

Run nix-shell -p "llama-cpp.override { vulkanSupport = true; }"

or

1. Put this into configuration.nix:

  environment.systemPackages = with pkgs; [
    (llama-cpp.override { vulkanSupport = true; })
  ];

2. Run sudo nixos-rebuild build --upgrade

Can Hydra reproduce this build failure?

No, Hydra cannot reproduce this build failure.

Link to Hydra build job

No response

Relevant log output

unpacking 1 channels...
building Nix...
building the system configuration...
these 13 derivations will be built:
  /nix/store/dr00cv420gm1k1f2dxdibqcd1yjvr0x4-llama-cpp-5416.drv
  /nix/store/7mf684bbi329lmvqphsk7m1b252qc23x-system-path.drv
  /nix/store/9pm1b08s3a10fb9shbp8wdxsz5hpz9b0-dbus-1.drv
  /nix/store/w0ixkpjf1604z5cwb0wlk9xg8nz0l3ic-X-Restart-Triggers-dbus.drv
  /nix/store/mw68d0qfk5rspfa1bdw09ay8hzg53mjf-unit-dbus.service.drv
  /nix/store/fbxdr0r3akblrpqgbzkq9723cxq4n5h6-X-Restart-Triggers-polkit.drv
  /nix/store/nspv62684ypnbv2lw86cv3kwgfjmmhyy-unit-polkit.service.drv
  /nix/store/p9mkwm9niw721p68vmvmxi7x7pakv1hw-unit-accounts-daemon.service.drv
  /nix/store/3sa4gb2jkz0dm06fdy0c7hsz8crvwv2f-system-units.drv
  /nix/store/3vdvlhd23bwfrid85asw6xggs4ljq53g-unit-dbus.service.drv
  /nix/store/z0igvr5ykp74xwbvgvn30pbpcakshjg0-user-units.drv
  /nix/store/wy7lzlvxa5q386hk23n0c2g6ahrffp54-etc.drv
  /nix/store/8pml312zkyq965k3q370g1m9z5alqqjn-nixos-system-a-25.11pre803506.2795c506fe8f.drv
building '/nix/store/dr00cv420gm1k1f2dxdibqcd1yjvr0x4-llama-cpp-5416.drv'...
Running phase: unpackPhase
unpacking source archive /nix/store/ass2w6972dxwhqa2wqqsfjkrizi2siz4-source
source root is source
Running phase: patchPhase
Running phase: updateAutotoolsGnuConfigScriptsPhase
Running phase: configurePhase
fixing cmake files...
cmake flags: -GNinja -DCMAKE_FIND_USE_SYSTEM_PACKAGE_REGISTRY=OFF -DCMAKE_FIND_USE_PACKAGE_REGISTRY=OFF -DCMAKE_EXPORT_NO_PACKAGE_REGISTRY=ON -DCMAKE_BUILD_TYPE=Release -DBUILD_TESTING=OFF -DCMAKE_INSTALL_LOCALEDIR=/nix/store/vvbn4wfcq59f0j3dw2rxi0cz20iz29k3-llama-cpp-5416/share/locale -DCMAKE_INSTALL_LIBEXECDIR=/nix/store/vvbn4wfcq59f0j3dw2rxi0cz20iz29k3-llama-cpp-5416/libexec -DCMAKE_INSTALL_LIBDIR=/nix/store/vvbn4wfcq59f0j3dw2rxi0cz20iz29k3-llama-cpp-5416/lib -DCMAKE_INSTALL_DOCDIR=/nix/store/vvbn4wfcq59f0j3dw2rxi0cz20iz29k3-llama-cpp-5416/share/doc/llama-cpp -DCMAKE_INSTALL_INFODIR=/nix/store/vvbn4wfcq59f0j3dw2rxi0cz20iz29k3-llama-cpp-5416/share/info -DCMAKE_INSTALL_MANDIR=/nix/store/vvbn4wfcq59f0j3dw2rxi0cz20iz29k3-llama-cpp-5416/share/man -DCMAKE_INSTALL_INCLUDEDIR=/nix/store/vvbn4wfcq59f0j3dw2rxi0cz20iz29k3-llama-cpp-5416/include -DCMAKE_INSTALL_SBINDIR=/nix/store/vvbn4wfcq59f0j3dw2rxi0cz20iz29k3-llama-cpp-5416/sbin -DCMAKE_INSTALL_BINDIR=/nix/store/vvbn4wfcq59f0j3dw2rxi0cz20iz29k3-llama-cpp-5416/bin -DCMAKE_INSTALL_NAME_DIR=/nix/store/vvbn4wfcq59f0j3dw2rxi0cz20iz29k3-llama-cpp-5416/lib -DCMAKE_POLICY_DEFAULT_CMP0025=NEW -DCMAKE_FIND_FRAMEWORK=LAST -DCMAKE_STRIP=/nix/store/0fsnicvfpf55nkza12cjnad0w84d6ba7-gcc-wrapper-14.2.1.20250322/bin/strip -DCMAKE_RANLIB=/nix/store/0fsnicvfpf55nkza12cjnad0w84d6ba7-gcc-wrapper-14.2.1.20250322/bin/ranlib -DCMAKE_AR=/nix/store/0fsnicvfpf55nkza12cjnad0w84d6ba7-gcc-wrapper-14.2.1.20250322/bin/ar -DCMAKE_C_COMPILER=gcc -DCMAKE_CXX_COMPILER=g++ -DCMAKE_INSTALL_PREFIX=/nix/store/vvbn4wfcq59f0j3dw2rxi0cz20iz29k3-llama-cpp-5416 -DGGML_NATIVE:BOOL=FALSE -DLLAMA_BUILD_SERVER:BOOL=TRUE -DLLAMA_CURL:BOOL=TRUE -DBUILD_SHARED_LIBS:BOOL=TRUE -DGGML_BLAS:BOOL=FALSE -DGGML_CLBLAST:BOOL=FALSE -DGGML_CUDA:BOOL=FALSE -DGGML_HIP:BOOL=FALSE -DGGML_METAL:BOOL=FALSE -DGGML_RPC:BOOL=FALSE -DGGML_VULKAN:BOOL=TRUE
-- The C compiler identification is GNU 14.2.1
-- The CXX compiler identification is GNU 14.2.1
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: /nix/store/0fsnicvfpf55nkza12cjnad0w84d6ba7-gcc-wrapper-14.2.1.20250322/bin/gcc - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: /nix/store/0fsnicvfpf55nkza12cjnad0w84d6ba7-gcc-wrapper-14.2.1.20250322/bin/g++ - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Found Git: /nix/store/805a5wv1cyah5awij184yfad1ksmbh9f-git-2.49.0/bin/git (found version "2.49.0")
fatal: not a git repository (or any of the parent directories): .git
fatal: not a git repository (or any of the parent directories): .git
-- Setting GGML_NATIVE_DEFAULT to OFF
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD - Success
-- Found Threads: TRUE
-- Warning: ccache not found - consider installing it for faster compilation or disable this warning with GGML_CCACHE=OFF
-- CMAKE_SYSTEM_PROCESSOR: x86_64
-- Including CPU backend
-- Found OpenMP_C: -fopenmp (found version "4.5")
-- Found OpenMP_CXX: -fopenmp (found version "4.5")
-- Found OpenMP: TRUE (found version "4.5")
-- x86 detected
-- Adding CPU backend variant ggml-cpu:  
-- Found Vulkan: /nix/store/nszik1q8ffmvsqk54kbc75dwyxwvi2nm-vulkan-loader-1.4.313.0/lib/libvulkan.so (found version "1.4.313") found components: glslc missing components: glslangValidator
-- Vulkan found
-- GL_KHR_cooperative_matrix supported by glslc
-- GL_NV_cooperative_matrix2 not supported by glslc
-- GL_EXT_integer_dot_product not supported by glslc
-- GL_EXT_bfloat16 not supported by glslc
-- Including Vulkan backend
fatal: not a git repository (or any of the parent directories): .git
fatal: not a git repository (or any of the parent directories): .git
CMake Warning at common/CMakeLists.txt:32 (message):
  Git repository not found; to enable automatic generation of build info,
  make sure Git is installed and the project is a Git repository.


-- Found CURL: /nix/store/v578vkzh0qhzczjvrzf64lqb2c74d5pk-curl-8.13.0/lib/libcurl.so (found version "8.13.0")
-- Configuring done (1.4s)
-- Generating done (0.1s)
CMake Warning:
  Manually-specified variables were not used by the project:

    CMAKE_EXPORT_NO_PACKAGE_REGISTRY
    CMAKE_POLICY_DEFAULT_CMP0025


-- Build files have been written to: /build/source/build
cmake: enabled parallel building
cmake: enabled parallel installing
Running phase: buildPhase
build flags: -j32
[1/237] Creating directories for 'vulkan-shaders-gen'
[2/237] Building CXX object ggml/src/CMakeFiles/ggml-cpu.dir/ggml-cpu/ggml-cpu-hbm.cpp.o
[3/237] No download step for 'vulkan-shaders-gen'
[4/237] Generating build details from Git
-- Found Git: /nix/store/805a5wv1cyah5awij184yfad1ksmbh9f-git-2.49.0/bin/git (found version "2.49.0")
fatal: not a git repository (or any of the parent directories): .git
fatal: not a git repository (or any of the parent directories): .git
[5/237] No update step for 'vulkan-shaders-gen'
[6/237] Building CXX object common/CMakeFiles/build_info.dir/build-info.cpp.o
[7/237] No patch step for 'vulkan-shaders-gen'
[8/237] Building C object examples/gguf-hash/CMakeFiles/sha1.dir/deps/sha1/sha1.c.o
[9/237] Building CXX object ggml/src/CMakeFiles/ggml-base.dir/ggml-threading.cpp.o
[10/237] Building CXX object tools/mtmd/CMakeFiles/llama-minicpmv-cli.dir/deprecation-warning.cpp.o
[11/237] Building CXX object tools/mtmd/CMakeFiles/llama-gemma3-cli.dir/deprecation-warning.cpp.o
[12/237] Building CXX object tools/mtmd/CMakeFiles/llama-llava-cli.dir/deprecation-warning.cpp.o
[13/237] Building C object examples/gguf-hash/CMakeFiles/sha256.dir/deps/sha256/sha256.c.o
[14/237] Building CXX object tools/mtmd/CMakeFiles/llama-qwen2vl-cli.dir/deprecation-warning.cpp.o
[15/237] Linking CXX executable bin/llama-llava-cli
[16/237] Linking CXX executable bin/llama-gemma3-cli
[17/237] Linking CXX executable bin/llama-minicpmv-cli
[18/237] Linking CXX executable bin/llama-qwen2vl-cli
[19/237] Building C object ggml/src/CMakeFiles/ggml-base.dir/ggml-alloc.c.o
[20/237] Building C object examples/gguf-hash/CMakeFiles/xxhash.dir/deps/xxhash/xxhash.c.o
[21/237] Performing configure step for 'vulkan-shaders-gen'
-- The C compiler identification is GNU 14.2.1
-- The CXX compiler identification is GNU 14.2.1
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: /nix/store/0fsnicvfpf55nkza12cjnad0w84d6ba7-gcc-wrapper-14.2.1.20250322/bin/gcc - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: /nix/store/0fsnicvfpf55nkza12cjnad0w84d6ba7-gcc-wrapper-14.2.1.20250322/bin/g++ - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD - Success
-- Found Threads: TRUE
-- Enabling coopmat glslc support
-- Configuring done (0.9s)
-- Generating done (0.0s)
-- Build files have been written to: /build/source/build/ggml/src/ggml-vulkan/vulkan-shaders-gen-prefix/src/vulkan-shaders-gen-build
[22/237] Building CXX object ggml/src/CMakeFiles/ggml-cpu.dir/ggml-cpu/vec.cpp.o
[23/237] Building CXX object ggml/src/CMakeFiles/ggml-cpu.dir/ggml-cpu/llamafile/sgemm.cpp.o
[24/237] Building C object ggml/src/CMakeFiles/ggml-cpu.dir/ggml-cpu/ggml-cpu.c.o
[25/237] Building CXX object ggml/src/CMakeFiles/ggml-base.dir/ggml-opt.cpp.o
[26/237] Building CXX object ggml/src/CMakeFiles/ggml-cpu.dir/ggml-cpu/ggml-cpu-traits.cpp.o
[27/237] Building CXX object ggml/src/CMakeFiles/ggml-base.dir/ggml-backend.cpp.o
[28/237] Building CXX object ggml/src/CMakeFiles/ggml-cpu.dir/ggml-cpu/ggml-cpu.cpp.o
[29/237] Building CXX object ggml/src/CMakeFiles/ggml-cpu.dir/ggml-cpu/amx/amx.cpp.o
[30/237] Building CXX object ggml/src/CMakeFiles/ggml-cpu.dir/ggml-cpu/amx/mmq.cpp.o
[31/237] Building C object ggml/src/CMakeFiles/ggml-base.dir/ggml.c.o
[32/237] Building C object ggml/src/CMakeFiles/ggml-cpu.dir/ggml-cpu/ggml-cpu-quants.c.o
[33/237] Building CXX object ggml/src/CMakeFiles/ggml-cpu.dir/ggml-cpu/binary-ops.cpp.o
[34/237] Building CXX object ggml/src/CMakeFiles/ggml-cpu.dir/ggml-cpu/ggml-cpu-aarch64.cpp.o
[35/237] Building CXX object ggml/src/CMakeFiles/ggml-cpu.dir/ggml-cpu/unary-ops.cpp.o
[36/237] Building CXX object ggml/src/CMakeFiles/ggml-base.dir/gguf.cpp.o
[37/237] Building CXX object ggml/src/CMakeFiles/ggml-cpu.dir/ggml-cpu/ops.cpp.o
[38/237] Performing build step for 'vulkan-shaders-gen'
[1/2] Building CXX object CMakeFiles/vulkan-shaders-gen.dir/vulkan-shaders-gen.cpp.o
[2/2] Linking CXX executable /build/source/build/bin/vulkan-shaders-gen
[39/237] Performing install step for 'vulkan-shaders-gen'
FAILED: ggml/src/ggml-vulkan/vulkan-shaders-gen-prefix/src/vulkan-shaders-gen-stamp/vulkan-shaders-gen-install /build/source/build/ggml/src/ggml-vulkan/vulkan-shaders-gen-prefix/src/vulkan-shaders-gen-stamp/vulkan-shaders-gen-install 
cd /build/source/build/ggml/src/ggml-vulkan/vulkan-shaders-gen-prefix/src/vulkan-shaders-gen-build && /nix/store/wphngc22a7aphbp5pi5jqmqlsqmisgn5-cmake-3.31.6/bin/cmake --install . && /nix/store/wphngc22a7aphbp5pi5jqmqlsqmisgn5-cmake-3.31.6/bin/cmake -E touch /build/source/build/ggml/src/ggml-vulkan/vulkan-shaders-gen-prefix/src/vulkan-shaders-gen-stamp/vulkan-shaders-gen-install
-- Install configuration: ""
CMake Error at cmake_install.cmake:52 (file):
  file INSTALL cannot find "/build/source/build/bin/vulkan-shaders-gen": No
  such file or directory.


[40/237] Building C object ggml/src/CMakeFiles/ggml-base.dir/ggml-quants.c.o
ninja: build stopped: subcommand failed.
error: builder for '/nix/store/dr00cv420gm1k1f2dxdibqcd1yjvr0x4-llama-cpp-5416.drv' failed with exit code 1
error: 1 dependencies of derivation '/nix/store/7mf684bbi329lmvqphsk7m1b252qc23x-system-path.drv' failed to build
error: 1 dependencies of derivation '/nix/store/fbxdr0r3akblrpqgbzkq9723cxq4n5h6-X-Restart-Triggers-polkit.drv' failed to build
error: 1 dependencies of derivation '/nix/store/9pm1b08s3a10fb9shbp8wdxsz5hpz9b0-dbus-1.drv' failed to build
error: 1 dependencies of derivation '/nix/store/p9mkwm9niw721p68vmvmxi7x7pakv1hw-unit-accounts-daemon.service.drv' failed to build
error: 1 dependencies of derivation '/nix/store/w0ixkpjf1604z5cwb0wlk9xg8nz0l3ic-X-Restart-Triggers-dbus.drv' failed to build
error: 1 dependencies of derivation '/nix/store/nspv62684ypnbv2lw86cv3kwgfjmmhyy-unit-polkit.service.drv' failed to build
error: 1 dependencies of derivation '/nix/store/3vdvlhd23bwfrid85asw6xggs4ljq53g-unit-dbus.service.drv' failed to build
error: 1 dependencies of derivation '/nix/store/mw68d0qfk5rspfa1bdw09ay8hzg53mjf-unit-dbus.service.drv' failed to build
error: 3 dependencies of derivation '/nix/store/3sa4gb2jkz0dm06fdy0c7hsz8crvwv2f-system-units.drv' failed to build
error: 1 dependencies of derivation '/nix/store/z0igvr5ykp74xwbvgvn30pbpcakshjg0-user-units.drv' failed to build
error: 4 dependencies of derivation '/nix/store/wy7lzlvxa5q386hk23n0c2g6ahrffp54-etc.drv' failed to build
error: 2 dependencies of derivation '/nix/store/8pml312zkyq965k3q370g1m9z5alqqjn-nixos-system-a-25.11pre803506.2795c506fe8f.drv' failed to build

Additional context

This worked up until at least April 29.

Without vulkanSupport = true it succeeds.

System metadata

• system: "x86_64-linux"
• host os: Linux 6.14.3, NixOS, 25.05 (Warbler), 25.05pre789333.f771eb401a46
• multi-user?: yes
• sandbox: yes
• version: nix-env (Nix) 2.28.2
• channels(root): "nixos"
• nixpkgs: /nix/store/nvw9cf1nffldxzahsbiv7jgfp832i26b-nixos/nixos

Notify maintainers

@dit7ya @elohmeier @philiptaron @xddxdd

---

Note for maintainers: Please tag this issue in your pull request description. (i.e. Resolves #ISSUE.)

I assert that this issue is relevant for Nixpkgs

• I assert that this is a bug and not a support request.
• I assert that this is not a duplicate of an existing issue.
• I assert that I have read the NixOS Code of Conduct and agree to abide by it.

Is this issue important to you?

Add a 👍 reaction to issues you find important.

[cmm]

if it helps anyone, the ustream commit causing this is ggml-org/llama.cpp@09d13d9 (ggml-org/llama.cpp#13263)

[philiptaron]

I won't be able to work on this, but I will gladly test and merge PRs that do.

[Nafanya]

There is a PR in upstream that somewhat fixes the issue

[Nafanya]

With this overlay I'm able to build and run llama.cpp

        (final: prev: {
          llama-cpp = prev.llama-cpp.overrideAttrs (oldAttrs: {
            postPatch =
              (oldAttrs.postPatch or "")
              + ''
                echo "Applying patch to ggml/src/ggml-vulkan/CMakeLists.txt"
                sed -i '/DCMAKE_RUNTIME_OUTPUT_DIRECTORY/d' ggml/src/ggml-vulkan/CMakeLists.txt
              '';
          });
        })

[7c6f434c]

Is anyone here using both Nixpkgs llama.cpp and Windows with a development toolchain set up? I think the upstream PR needs fixing the Windows side…

[theoparis]

I tried updating nixpkgs to see if the latest commit with a merged PR solved it, but I ran into this:

error: Cannot build '/nix/store/l7q2dhq28qxdj5ccn3qa0rmvcd64gcym-llama-cpp-5702.drv'.
       Reason: builder failed with exit code 1.
       Output paths:
         /nix/store/cwhl7ha22pi4cj6bskj9dlgagamq6gai-llama-cpp-5702
       Last 25 log lines:
       >       |                                                                                                                         ^~~~~~
       > /build/source/ggml/src/ggml-vulkan/ggml-vulkan.cpp:2110:48: error: 'matmul_id_bf16_aligned_cm2_len' was not declared in this scope; did you mean 'matmul_id_f16_aligned_cm2_len'?
       >  2110 |             CREATE_MM(pipeline_matmul_id_bf16, matmul_id_bf16, , wg_denoms, warptile, vk_mat_mat_id_push_constants, 4)
       >       |                                                ^~~~~~~~~~~~~~
       > /build/source/ggml/src/ggml-vulkan/ggml-vulkan.cpp:2072:101: note: in definition of macro 'CREATE_MM'
       >  2072 |         ggml_vk_create_pipeline(device, device-> PIPELINE_NAME ->a_l, #NAMELC #F16ACC "_aligned_l", NAMELC ## _aligned ## F16ACC ## _cm2_len, NAMELC ## _aligned ## F16ACC ## _cm2_data, "main", PARAMCOUNT, sizeof(PUSHCONST), l_ ## WG_DENOMS, l_ ## WARPTILE, l_align);   \
       >       |                                                                                                     ^~~~~~
       > /build/source/ggml/src/ggml-vulkan/ggml-vulkan.cpp:2110:48: error: 'matmul_id_bf16_aligned_cm2_data' was not declared in this scope; did you mean 'matmul_id_f16_aligned_cm2_data'?
       >  2110 |             CREATE_MM(pipeline_matmul_id_bf16, matmul_id_bf16, , wg_denoms, warptile, vk_mat_mat_id_push_constants, 4)
       >       |                                                ^~~~~~~~~~~~~~
       > /build/source/ggml/src/ggml-vulkan/ggml-vulkan.cpp:2072:143: note: in definition of macro 'CREATE_MM'
       >  2072 |         ggml_vk_create_pipeline(device, device-> PIPELINE_NAME ->a_l, #NAMELC #F16ACC "_aligned_l", NAMELC ## _aligned ## F16ACC ## _cm2_len, NAMELC ## _aligned ## F16ACC ## _cm2_data, "main", PARAMCOUNT, sizeof(PUSHCONST), l_ ## WG_DENOMS, l_ ## WARPTILE, l_align);   \
       >       |                                                                                                     

[quag]

Adding this patch gets llama-cpp building again for me with Vulkan support.

From 654575ed77996923556940b8d6a7027c4f890436 Mon Sep 17 00:00:00 2001
From: Quag <quaggy@gmail.com>
Date: Sat, 28 Jun 2025 13:40:24 -0700
Subject: [PATCH] Remove GGML_VULKAN_BFLOAT16_GLSLC_SUPPORT

---
 ggml/src/ggml-vulkan/CMakeLists.txt           |  6 ---
 ggml/src/ggml-vulkan/ggml-vulkan.cpp          | 44 -------------------
 .../ggml-vulkan/vulkan-shaders/CMakeLists.txt |  4 --
 .../vulkan-shaders/vulkan-shaders-gen.cpp     |  2 -
 4 files changed, 56 deletions(-)

diff --git a/ggml/src/ggml-vulkan/CMakeLists.txt b/ggml/src/ggml-vulkan/CMakeLists.txt
index b97e7bf9..9cdc62a4 100644
--- a/ggml/src/ggml-vulkan/CMakeLists.txt
+++ b/ggml/src/ggml-vulkan/CMakeLists.txt
@@ -70,12 +70,6 @@ if (Vulkan_FOUND)
         "GGML_VULKAN_INTEGER_DOT_GLSLC_SUPPORT"
     )
 
-    test_shader_extension_support(
-        "GL_EXT_bfloat16"
-        "${CMAKE_CURRENT_SOURCE_DIR}/vulkan-shaders/test_bfloat16_support.comp"
-        "GGML_VULKAN_BFLOAT16_GLSLC_SUPPORT"
-    )
-
     target_link_libraries(ggml-vulkan PRIVATE Vulkan::Vulkan)
     target_include_directories(ggml-vulkan PRIVATE ${CMAKE_CURRENT_BINARY_DIR})
 
diff --git a/ggml/src/ggml-vulkan/ggml-vulkan.cpp b/ggml/src/ggml-vulkan/ggml-vulkan.cpp
index 996ccbf6..93167688 100644
--- a/ggml/src/ggml-vulkan/ggml-vulkan.cpp
+++ b/ggml/src/ggml-vulkan/ggml-vulkan.cpp
@@ -2095,11 +2095,6 @@ static void ggml_vk_load_shaders(vk_device& device) {
         CREATE_MM(PIPELINE_NAME . f32acc, NAMELC, , WG_DENOMS, WARPTILE, PUSHCONST, PARAMCOUNT)   \
 
         CREATE_MM2(pipeline_matmul_f16, matmul_f16, wg_denoms, warptile, vk_mat_mat_push_constants, 3)
-#if defined(GGML_VULKAN_BFLOAT16_GLSLC_SUPPORT)
-        if (device->coopmat_bf16_support) {
-            CREATE_MM(pipeline_matmul_bf16, matmul_bf16, , wg_denoms, warptile, vk_mat_mat_push_constants, 3)
-        }
-#endif
         CREATE_MM2(pipeline_dequant_mul_mat_mat_f16[GGML_TYPE_Q4_0], matmul_q4_0_f16, mmq_wg_denoms, warptile_mmq, vk_mat_mat_push_constants, 3)
         CREATE_MM2(pipeline_dequant_mul_mat_mat_f16[GGML_TYPE_Q4_1], matmul_q4_1_f16, mmq_wg_denoms, warptile_mmq, vk_mat_mat_push_constants, 3)
         CREATE_MM2(pipeline_dequant_mul_mat_mat_f16[GGML_TYPE_Q5_0], matmul_q5_0_f16, mmq_wg_denoms, warptile_mmq, vk_mat_mat_push_constants, 3)
@@ -2121,11 +2116,6 @@ static void ggml_vk_load_shaders(vk_device& device) {
         CREATE_MM2(pipeline_dequant_mul_mat_mat_f16[GGML_TYPE_IQ4_NL],  matmul_iq4_nl_f16,  mmq_wg_denoms, warptile_mmq, vk_mat_mat_push_constants, 3)
 
         CREATE_MM2(pipeline_matmul_id_f16, matmul_id_f16, wg_denoms, warptile, vk_mat_mat_id_push_constants, 4)
-#if defined(GGML_VULKAN_BFLOAT16_GLSLC_SUPPORT)
-        if (device->coopmat_bf16_support) {
-            CREATE_MM(pipeline_matmul_id_bf16, matmul_id_bf16, , wg_denoms, warptile, vk_mat_mat_id_push_constants, 4)
-        }
-#endif
         CREATE_MM(pipeline_dequant_mul_mat_mat_id[GGML_TYPE_Q4_0].f16acc, matmul_id_q4_0_f16, , mmqid_wg_denoms, warptile_mmqid, vk_mat_mat_id_push_constants, 4)
         CREATE_MM(pipeline_dequant_mul_mat_mat_id[GGML_TYPE_Q4_1].f16acc, matmul_id_q4_1_f16, , mmqid_wg_denoms, warptile_mmqid, vk_mat_mat_id_push_constants, 4)
         CREATE_MM(pipeline_dequant_mul_mat_mat_id[GGML_TYPE_Q5_0].f16acc, matmul_id_q5_0_f16, , mmqid_wg_denoms, warptile_mmqid, vk_mat_mat_id_push_constants, 4)
@@ -2179,11 +2169,6 @@ static void ggml_vk_load_shaders(vk_device& device) {
         CREATE_MM(GGML_TYPE_F32, pipeline_matmul_f32_f16, matmul_f32_f16, , wg_denoms, warptile, vk_mat_mat_push_constants, 3, );
         CREATE_MM2(GGML_TYPE_F16, pipeline_matmul_f16, matmul_f16, wg_denoms, warptile, vk_mat_mat_push_constants, 3, );
         CREATE_MM2(GGML_TYPE_F16, pipeline_matmul_f16_f32, matmul_f16_f32, wg_denoms, warptile, vk_mat_mat_push_constants, 3, );
-#if defined(GGML_VULKAN_BFLOAT16_GLSLC_SUPPORT)
-        if (device->coopmat_bf16_support) {
-            CREATE_MM(GGML_TYPE_BF16, pipeline_matmul_bf16, matmul_bf16, , wg_denoms, warptile, vk_mat_mat_push_constants, 3, )
-        }
-#endif
 
         if (device->coopmat_acc_f16_support) {
             CREATE_MM2(GGML_TYPE_Q4_0, pipeline_dequant_mul_mat_mat[GGML_TYPE_Q4_0], matmul_q4_0_f32, mmq_wg_denoms, warptile_mmq, vk_mat_mat_push_constants, 3, );
@@ -2232,11 +2217,6 @@ static void ggml_vk_load_shaders(vk_device& device) {
         CREATE_MM(GGML_TYPE_F32, pipeline_matmul_id_f32, matmul_id_f32_f32, , wg_denoms, warptile, vk_mat_mat_push_constants, 4, _id);
         CREATE_MM2(GGML_TYPE_F16, pipeline_matmul_id_f16, matmul_id_f16, wg_denoms, warptile, vk_mat_mat_push_constants, 4, _id);
         CREATE_MM2(GGML_TYPE_F16, pipeline_matmul_id_f16_f32, matmul_id_f16_f32, wg_denoms, warptile, vk_mat_mat_push_constants, 4, _id);
-#if defined(GGML_VULKAN_BFLOAT16_GLSLC_SUPPORT)
-        if (device->coopmat_bf16_support) {
-            CREATE_MM(GGML_TYPE_BF16, pipeline_matmul_id_bf16, matmul_id_bf16, , wg_denoms, warptile, vk_mat_mat_push_constants, 4, _id);
-        }
-#endif
 
         if (device->coopmat_acc_f16_support) {
             CREATE_MM(GGML_TYPE_Q4_0, pipeline_dequant_mul_mat_mat_id[GGML_TYPE_Q4_0].f16acc, matmul_id_q4_0_f32, _f16acc, mmq_wg_denoms, warptile_mmq, vk_mat_mat_id_push_constants, 4, _id);
@@ -2892,11 +2872,6 @@ static vk_device ggml_vk_get_device(size_t idx) {
             } else if (strcmp("VK_KHR_shader_integer_dot_product", properties.extensionName) == 0 &&
                        !getenv("GGML_VK_DISABLE_INTEGER_DOT_PRODUCT")) {
                 device->integer_dot_product = true;
-#endif
-#if defined(GGML_VULKAN_BFLOAT16_GLSLC_SUPPORT)
-            } else if (strcmp("VK_KHR_shader_bfloat16", properties.extensionName) == 0 &&
-                       !getenv("GGML_VK_DISABLE_BFLOAT16")) {
-                bfloat16_support = true;
 #endif
             }
         }
@@ -3306,25 +3281,6 @@ static vk_device ggml_vk_get_device(size_t idx) {
                     device->coopmat_int_n = prop.NSize;
                     device->coopmat_int_k = prop.KSize;
                 }
-#if defined(VK_KHR_shader_bfloat16) && defined(GGML_VULKAN_BFLOAT16_GLSLC_SUPPORT)
-                if (prop.AType == VK_COMPONENT_TYPE_BFLOAT16_KHR &&
-                    prop.BType == VK_COMPONENT_TYPE_BFLOAT16_KHR &&
-                    prop.CType == VK_COMPONENT_TYPE_FLOAT32_KHR &&
-                    prop.ResultType == VK_COMPONENT_TYPE_FLOAT32_KHR &&
-                    (vk::ScopeKHR)prop.scope == vk::ScopeKHR::eSubgroup
-                ) {
-                    // coopmat sizes not set yet
-                    if (device->coopmat_m == 0) {
-                        device->coopmat_bf16_support = true;
-                        device->coopmat_m = prop.MSize;
-                        device->coopmat_n = prop.NSize;
-                        device->coopmat_k = prop.KSize;
-                    } else if (device->coopmat_m == prop.MSize && device->coopmat_n == prop.NSize && device->coopmat_k == prop.KSize) {
-                        // Only enable if shape is identical
-                        device->coopmat_bf16_support = true;
-                    }
-                }
-#endif
             }
 
             if (device->coopmat_m == 0 || !device->coopmat_acc_f32_support) {
diff --git a/ggml/src/ggml-vulkan/vulkan-shaders/CMakeLists.txt b/ggml/src/ggml-vulkan/vulkan-shaders/CMakeLists.txt
index e1f613fb..3c61a19b 100644
--- a/ggml/src/ggml-vulkan/vulkan-shaders/CMakeLists.txt
+++ b/ggml/src/ggml-vulkan/vulkan-shaders/CMakeLists.txt
@@ -15,10 +15,6 @@ if (GGML_VULKAN_INTEGER_DOT_GLSLC_SUPPORT)
     add_compile_definitions(GGML_VULKAN_INTEGER_DOT_GLSLC_SUPPORT)
     message(STATUS "Enabling dot glslc support")
 endif()
-if (GGML_VULKAN_BFLOAT16_GLSLC_SUPPORT)
-    add_compile_definitions(GGML_VULKAN_BFLOAT16_GLSLC_SUPPORT)
-    message(STATUS "Enabling bfloat16 glslc support")
-endif()
 if (GGML_VULKAN_SHADER_DEBUG_INFO)
     add_compile_definitions(GGML_VULKAN_SHADER_DEBUG_INFO)
     message(STATUS "Enabling shader debug info")
diff --git a/ggml/src/ggml-vulkan/vulkan-shaders/vulkan-shaders-gen.cpp b/ggml/src/ggml-vulkan/vulkan-shaders/vulkan-shaders-gen.cpp
index c63345ec..c9252203 100644
--- a/ggml/src/ggml-vulkan/vulkan-shaders/vulkan-shaders-gen.cpp
+++ b/ggml/src/ggml-vulkan/vulkan-shaders/vulkan-shaders-gen.cpp
@@ -349,9 +349,7 @@ void matmul_shaders(bool fp16, bool matmul_id, bool coopmat, bool coopmat2, bool
         std::string to_float_type = (coopmat || coopmat2) ? "uintBitsToBFloat16EXT" : "bf16_to_fp32";
 
         // If bfloat16 is not supported, then only compile the scalar (promote to fp32) shader
-#if !defined(GGML_VULKAN_BFLOAT16_GLSLC_SUPPORT)
         if (!(coopmat || coopmat2))
-#endif
         {
             string_to_spv(shader_name + "_bf16_aligned", source_name, merge_maps(base_dict, {{"FLOAT_TYPE", FLOAT_TYPE("bf16")}, {"TO_FLOAT_TYPE", to_float_type}, {"DATA_A_BF16", "1"}, {"LOAD_VEC_A", load_vec_a},           {"LOAD_VEC_B", "4"}, {"B_TYPE", coopmat2 ? "bfloat16_t" : "u16vec4"},   {"D_TYPE", "float"}, {"B_IS_FLOAT", "1"}, {"ALIGNED", "1"}}), fp16, coopmat, coopmat2, f16acc);
             string_to_spv(shader_name + "_bf16",         source_name, merge_maps(base_dict, {{"FLOAT_TYPE", FLOAT_TYPE("bf16")}, {"TO_FLOAT_TYPE", to_float_type}, {"DATA_A_BF16", "1"}, {"LOAD_VEC_A", load_vec_a_unaligned},                      {"B_TYPE", coopmat2 ? "bfloat16_t" : "uint16_t"},                          {"D_TYPE", "float"}, {"B_IS_FLOAT", "1"}}),                   fp16, coopmat, coopmat2, f16acc);
-- 
2.49.0

[kalvinarts]

As an alternative the official repo has a flake.nix. Just set it in your inputs like this:

llama-cpp = {
  url = "github:ggml-org/llama.cpp/b5896";
};

Then in your environment.systemPakages add:

inputs.llama-cpp.packages.x86_64-linux.vulkan

EDIT: Wasn't able to build the latest version but this one: b5896 worked for me.

[arbv]

That is still an issue.

Also, I think it would have been great to have a separate llama-cpp-vulkan package to avoid building it on lower-end devices (like SBCs), but that is a completely different discussion.

[7c6f434c]

Right now, I suppose we need #422031 to get to unstable? What shaderc version do you get on the checkout you use?

[arbv]

@7c6f434c I cannot test it right now, but your suggestion should indeed resolve the issue. I am using an older unstable branch right now (from weeks ago).

[7c6f434c]

Today's might still not be fresh enough, but as the staging cycle converges, this change will eventually propagate