Skip to content

Only request VK_KHR_shader_bfloat16 extension if supported - #28155

Merged
ggerganov merged 1 commit into
ggml-org:masterfrom
nobodywho-ooo:fix-bfloat16-extension
Sep 2, 2026
Merged

Only request VK_KHR_shader_bfloat16 extension if supported#28155
ggerganov merged 1 commit into
ggml-org:masterfrom
nobodywho-ooo:fix-bfloat16-extension

Conversation

@madsmtm

@madsmtm madsmtm commented Sep 1, 2026

Copy link
Copy Markdown
Contributor

Overview

My friend's Windows machine has an AMD Radeon(TM) 890M Graphics iGPU which doesn't support bf16.

In some cases though the code will currently still set device->coopmat_bf16_support = true, which later on causes it to request the VK_KHR_shader_bfloat16 extension (even though it isn't reported as supported), and the following error to be reported:

llama_model_load: error loading model: vk::PhysicalDevice::createDevice: ErrorExtensionNotPresent

This PR adds a bfloat16_support check to ensure that we don't try to do this if VK_KHR_shader_bfloat16 isn't supported. The fix has been tested by building with this diff applied, and successfully loading the model in this case.

Fixes #28161.

Requirements

  • I have read and agree with the contributing guidelines
  • AI usage disclosure: YES, used for debugging the issue, I've written the code and PR myself.

@madsmtm
madsmtm requested a review from a team as a code owner September 1, 2026 10:03
@github-actions github-actions Bot added Vulkan Issues specific to the Vulkan backend ggml changes relating to the ggml tensor library for machine learning labels Sep 1, 2026
@madsmtm

madsmtm commented Sep 1, 2026

Copy link
Copy Markdown
Contributor Author

CC @0cc4m @jeffbolznv

@ggml-gh-bot

ggml-gh-bot Bot commented Sep 1, 2026

Copy link
Copy Markdown

Hi @madsmtm, thanks for your contribution!

Per our contribution guidelines, the automated PR checker found the following issue(s) that need your attention:

  • Maintainers cannot push to this PR: Please enable Allow edits by maintainers. If this PR comes from an organization-owned fork, that option is not available on GitHub; please re-open the PR from a fork owned by your personal account.

  • Multiple open PRs from a new contributor: We limit new contributors (those without a previously merged PR) to 1 open PR at a time. You currently have 2 open PRs.


Please note that maintainers reserve the right to make final decisions on PRs. If you believe there is a mistake, please comment below.

@ggml-gh-bot ggml-gh-bot Bot added the draft PR will be changed to draft by github-actions bot label Sep 1, 2026
@0cc4m

0cc4m commented Sep 1, 2026

Copy link
Copy Markdown
Contributor

Open an issue with logs and description of what goes wrong instead.

@madsmtm

madsmtm commented Sep 1, 2026

Copy link
Copy Markdown
Contributor Author

Sure, I've opened #28161.

@0cc4m

0cc4m commented Sep 1, 2026

Copy link
Copy Markdown
Contributor

Thank you, now I understood the problem. Your fix is basically correct, the one thing missing is to remove the duplicated device_extensions.push_back("VK_KHR_shader_bfloat16"); block that is added if coopmat_bfloat16_support is set.

@0cc4m 0cc4m reopened this Sep 1, 2026
@0cc4m 0cc4m linked an issue Sep 1, 2026 that may be closed by this pull request
@madsmtm
madsmtm force-pushed the fix-bfloat16-extension branch from e5b507c to 29fed0e Compare September 1, 2026 13:50
@madsmtm

madsmtm commented Sep 1, 2026

Copy link
Copy Markdown
Contributor Author

Ofc, I missed that the extension is already added earlier

@0cc4m 0cc4m added merge ready A maintainer can use this label to indicate that they consider the changes final and ready to merge. and removed draft PR will be changed to draft by github-actions bot labels Sep 2, 2026
@ggerganov
ggerganov merged commit 56dd815 into ggml-org:master Sep 2, 2026
26 of 30 checks passed
@madsmtm
madsmtm deleted the fix-bfloat16-extension branch September 2, 2026 09:40
fewtarius pushed a commit to fewtarius/CachyLLama that referenced this pull request Sep 5, 2026
thecodacus pushed a commit to thecodacus/llama.cpp that referenced this pull request Sep 7, 2026
SteelPh0enix pushed a commit to SteelPh0enix/llama.cpp-qwen4exp that referenced this pull request Sep 8, 2026
SteelPh0enix pushed a commit to SteelPh0enix/llama.cpp-qwen4exp that referenced this pull request Sep 8, 2026
x1250 pushed a commit to x1250/llama.cpp that referenced this pull request Sep 9, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ggml changes relating to the ggml tensor library for machine learning merge ready A maintainer can use this label to indicate that they consider the changes final and ready to merge. Vulkan Issues specific to the Vulkan backend

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Eval bug: VK_KHR_shader_bfloat16 is requested even when unsupported

3 participants