Skip to content

[SPIRV] Add a means of turning off/on tests for specific spirv drivers#678

Merged
farzonl merged 6 commits intollvm:mainfrom
farzonl:update_api-query
Jan 31, 2026
Merged

[SPIRV] Add a means of turning off/on tests for specific spirv drivers#678
farzonl merged 6 commits intollvm:mainfrom
farzonl:update_api-query

Conversation

@farzonl
Copy link
Copy Markdown
Member

@farzonl farzonl commented Jan 28, 2026

In the testing of KosmicKrisp Many tests for MoltenVK were XFAIL resulting in unexpected passes. This change makes those tests XFAIL for just MoltenVK.

Results

OFFLOADTEST_GPU_NAME=KosmicKrisp ninja -C ../builds/llvm_relwithdebinfo_offload check-hlsl-clang-vk

Total Discovered Tests: 329
  Unsupported      :  72 (21.88%)
  Passed           : 186 (56.54%)
  Expectedly Failed:  71 (21.58%)

OFFLOADTEST_GPU_NAME=KosmicKrisp ninja -C ../builds/llvm_relwithdebinfo_offload check-hlsl-vk 

Total Discovered Tests: 329
  Unsupported      :  59 (17.93%)
  Passed           : 251 (76.29%)
  Expectedly Failed:  19 (5.78%)

OFFLOADTEST_GPU_NAME=MoltenVK ninja -C ../builds/llvm_relwithdebinfo_offload check-hlsl-vk

Total Discovered Tests: 329
  Unsupported      : 117 (35.56%)
  Passed           : 189 (57.45%)
  Expectedly Failed:  23 (6.99%)

OFFLOADTEST_GPU_NAME=MoltenVK ninja -C ../builds/llvm_relwithdebinfo_offload check-hlsl-clang-vk

Total Discovered Tests: 329
  Unsupported      : 128 (38.91%)
  Passed           : 134 (40.73%)
  Expectedly Failed:  67 (20.36%)

Comment thread test/lit.cfg.py Outdated
Comment thread test/Feature/Textures/Texture2D.SRVToUAV.array.test.yaml Outdated
Comment thread test/Feature/CBuffer/array-dynamic-index.test Outdated
Copy link
Copy Markdown
Contributor

@tex3d tex3d left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Comment thread lib/API/VK/Device.cpp
" (" + std::string(DriverProps.driverName, DriverNameSz) + ")";
#endif
DriverName = std::string(DriverProps.driverName, DriverNameSz);
Description += " (" + DriverName + ")";
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would it be better to leave description alone and just print the driver name as well in api-query?

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So we might not need to do this string append on linux vulkan but we do need to do it on apple vulkan right now the device description is the same across vulkan instances

vulkaninfo --summary | grep deviceName
        deviceName         = Apple M3 Max
        deviceName         = Apple M3 Max
vulkaninfo --summary | grep driverName 
        driverName         = MoltenVK
        driverName         = KosmicKrisp

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

alternatively we could have adapter-regex do matching on description and device name? I dunno if that is better though?

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh, I see. You want a combined string for the filtering mechanism. Okay, that's probably fine.


# Unimplemented: https://github.com/llvm/offload-test-suite/issues/664
# XFAIL: DirectX || Metal
# XFAIL: DirectX || Metal || Vulkan && Darwin
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this the right issue? It's specifically about DirectX and Metal support for samplers being unimplemented.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

From what I can tell this is a Metal issue and so MoltenVK runs into the same problem. I didn't look to deeply into why KosmicKrisp is also failing.

# GetDimensions on Vulkan (DXC) returns [ 16, 16 ] instead of [ 8, 5 ]
# except on AMD https://github.com/llvm/offload-test-suite/issues/523
# XFAIL: DXC && Vulkan && !AMD
# XFAIL: DXC && Vulkan && !Darwin && !AMD
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does the comment need to be updated here?

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah I'm thinking it would be easier to update this to

XFAIL: DXC && Vulkan && (Nvidia || Intel) 

Because its passing on Mac across both KosmicKrisp and MoltenVK

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I didn't because I actually don't have an integrated intel gpu to test on and the intel CI is down right now so I don't know how to confirm this.

Comment thread lib/API/VK/Device.cpp
" (" + std::string(DriverProps.driverName, DriverNameSz) + ")";
#endif
DriverName = std::string(DriverProps.driverName, DriverNameSz);
Description += " (" + DriverName + ")";
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh, I see. You want a combined string for the filtering mechanism. Okay, that's probably fine.

Comment thread tools/api-query/api-query.cpp
Comment thread test/lit.cfg.py Outdated
config.available_features.add(API)
config.available_features.add(compiler)
config.available_features.add(config.offloadtest_os)
if device["Driver"]:
Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This seems to only make sense for mesa drivers. On linux we get names

driverName         = NVK
driverName         = llvmpipe
driverName         = venus

But on windows it is:

driverName         = Intel Corporation
driverName         = NVIDIA
driverName         = AMD proprietary driver

I think for now we should limit this to Mac and maybe expand it to Linux later. It seems mostly useless on Windows.

@farzonl farzonl merged commit 2fcc0c9 into llvm:main Jan 31, 2026
5 of 12 checks passed
@farzonl farzonl deleted the update_api-query branch January 31, 2026 23:50
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants