Usage of atomicAdd() causes shader compilation error

[JonathanWoollett-Light]

Using .spv file of a compiled shader which includes the function atomicAdd(), in this case specifically:

atomicAdd(outputs[gl_GlobalInvocationID.y],sum);

When run (with cargo run --release) leads to the error:

thread 'main' panicked at 'failed to create shader module: Shader error: Compilation failed: ', C:\Users\jonat\.rustup\toolchains\stable-x86_64-pc-windows-msvc\lib/rustlib/src/rust\src\libstd\macros.rs:16:9
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
error: process didn't exit successfully: `target\release\test-project.exe` (exit code: 101)

It does not cause an error when the GLSL code is compiled to SPIR-V.

Compiled with the command:

C:/VulkanSDK/1.2.148.1/Bin/glslc C:/Users/jonat/Projects/test-project/glsl/sgemv2.comp -o C:/Users/jonat/Projects/test-project/spir-v/sgemv2.spv --target-env=vulkan1.1`

Notably enabling the extenson does not cause an error (using #extension GL_EXT_shader_atomic_float: enable is fine, it is only when adding the usage of atomicAdd() does the error occur).

Test project: test-project.zip

[kvark]

Trying to reproduce this on http://shader-playground.timjones.io/, I'm getting:

WARNING: D:\local\Temp\7f4faf02-ac48-4682-a9f3-cb5d0f40b045.tmp:3: '#extension' : extension not supported: GL_EXT_shader_atomic_float

[JonathanWoollett-Light]

So having recreated this I get the same result. I'm not sure how this relates to what I could/should change in my usage of the shader in my project.

[kvark]

Could you run the application via Visual Studio? To do that, you can just "Open Project" and point it to already compiled executable. What this gives you is D3D12 validation output, which will say what exactly is the problem with the generated shader.

[JonathanWoollett-Light]

I have not done this before so I may have done this wrong, but as you said I opened a new project by selecting the release executable then pressing start in visual studio to run it.

The resulting debug terminal output:

'test-project.exe' (Win32): Loaded 'C:\Users\jonat\Projects\test-project\target\release\test-project.exe'. Symbols loaded.
'test-project.exe' (Win32): Loaded 'C:\Windows\System32\ntdll.dll'. 
'test-project.exe' (Win32): Loaded 'C:\Windows\System32\kernel32.dll'. 
'test-project.exe' (Win32): Loaded 'C:\Windows\System32\KernelBase.dll'. 
'test-project.exe' (Win32): Loaded 'C:\Windows\System32\advapi32.dll'. 
'test-project.exe' (Win32): Loaded 'C:\Windows\System32\msvcrt.dll'. 
'test-project.exe' (Win32): Loaded 'C:\Windows\System32\sechost.dll'. 
'test-project.exe' (Win32): Loaded 'C:\Windows\System32\rpcrt4.dll'. 
'test-project.exe' (Win32): Loaded 'C:\Windows\System32\ucrtbase.dll'. 
'test-project.exe' (Win32): Loaded 'C:\Windows\System32\D3DCompiler_47.dll'. 
'test-project.exe' (Win32): Loaded 'C:\Windows\System32\msvcp140.dll'. 
'test-project.exe' (Win32): Loaded 'C:\Windows\System32\vcruntime140_1.dll'. 
'test-project.exe' (Win32): Loaded 'C:\Windows\System32\vcruntime140.dll'. 
'test-project.exe' (Win32): Loaded 'C:\Windows\System32\cryptsp.dll'. 
'test-project.exe' (Win32): Loaded 'C:\Windows\System32\cryptbase.dll'. 
'test-project.exe' (Win32): Loaded 'C:\Windows\System32\bcryptprimitives.dll'. 
'test-project.exe' (Win32): Loaded 'C:\Windows\System32\vulkan-1.dll'. 
'test-project.exe' (Win32): Loaded 'C:\Windows\System32\cfgmgr32.dll'. 
'test-project.exe' (Win32): Loaded 'C:\Windows\System32\gdi32.dll'. 
'test-project.exe' (Win32): Loaded 'C:\Windows\System32\win32u.dll'. 
'test-project.exe' (Win32): Loaded 'C:\Windows\System32\gdi32full.dll'. 
'test-project.exe' (Win32): Loaded 'C:\Windows\System32\msvcp_win.dll'. 
'test-project.exe' (Win32): Loaded 'C:\Windows\System32\user32.dll'. 
'test-project.exe' (Win32): Loaded 'C:\Windows\System32\imm32.dll'. 
'test-project.exe' (Win32): Loaded 'C:\Windows\System32\dxgi.dll'. 
'test-project.exe' (Win32): Loaded 'C:\Windows\System32\DXCore.dll'. 
'test-project.exe' (Win32): Loaded 'C:\Windows\System32\combase.dll'. 
'test-project.exe' (Win32): Loaded 'C:\Windows\System32\ResourcePolicyClient.dll'. 
'test-project.exe' (Win32): Unloaded 'C:\Windows\System32\ResourcePolicyClient.dll'
'test-project.exe' (Win32): Loaded 'C:\Windows\System32\kernel.appcore.dll'. 
'test-project.exe' (Win32): Loaded 'C:\Windows\System32\DriverStore\FileRepository\u0355311.inf_amd64_183b8d63847c90cf\B355199\amdvlk64.dll'. 
'test-project.exe' (Win32): Loaded 'C:\Windows\System32\setupapi.dll'. 
'test-project.exe' (Win32): Loaded 'C:\Windows\System32\winmm.dll'. 
'test-project.exe' (Win32): Loaded 'C:\Windows\System32\bcrypt.dll'. 
'test-project.exe' (Win32): Loaded 'C:\Windows\System32\winmmbase.dll'. 
'test-project.exe' (Win32): Loaded 'C:\Windows\System32\winmmbase.dll'. 
'test-project.exe' (Win32): Loaded 'C:\Windows\System32\winmmbase.dll'. 
'test-project.exe' (Win32): Unloaded 'C:\Windows\System32\winmmbase.dll'
'test-project.exe' (Win32): Unloaded 'C:\Windows\System32\winmmbase.dll'
'test-project.exe' (Win32): Loaded 'C:\Windows\System32\amdihk64.dll'. Module was built without symbols.
'test-project.exe' (Win32): Loaded 'C:\Windows\System32\shlwapi.dll'. 
'test-project.exe' (Win32): Loaded 'C:\ProgramData\obs-studio-hook\graphics-hook64.dll'. 
'test-project.exe' (Win32): Loaded 'C:\Windows\System32\dwmapi.dll'. 
'test-project.exe' (Win32): Loaded 'C:\Windows\System32\devobj.dll'. 
'test-project.exe' (Win32): Loaded 'C:\Windows\System32\wintrust.dll'. 
'test-project.exe' (Win32): Loaded 'C:\Windows\System32\msasn1.dll'. 
'test-project.exe' (Win32): Loaded 'C:\Windows\System32\crypt32.dll'. 
'test-project.exe' (Win32): Loaded 'C:\Windows\System32\D3D12.dll'. 
'test-project.exe' (Win32): Loaded 'C:\Windows\System32\DXGIDebug.dll'. 
'test-project.exe' (Win32): Loaded 'C:\Windows\System32\DriverStore\FileRepository\u0355311.inf_amd64_183b8d63847c90cf\B355199\amdxc64.dll'. 
'test-project.exe' (Win32): Loaded 'C:\Windows\System32\version.dll'. 
Failed to open pipe
Hooked to process: test-project.exe
(half life scientist) everything..  seems to be in order
'test-project.exe' (Win32): Loaded 'C:\Windows\System32\uxtheme.dll'. 
'test-project.exe' (Win32): Loaded 'C:\Windows\System32\msctf.dll'. 
'test-project.exe' (Win32): Loaded 'C:\Windows\System32\oleaut32.dll'. 
'test-project.exe' (Win32): Loaded 'C:\Windows\System32\D3DSCache.dll'. 
'test-project.exe' (Win32): Loaded 'C:\Windows\System32\twinapi.appcore.dll'. 
'test-project.exe' (Win32): Loaded 'C:\Windows\System32\userenv.dll'. 
'test-project.exe' (Win32): Loaded 'C:\Windows\System32\profapi.dll'. 
'test-project.exe' (Win32): Loaded 'C:\Windows\System32\SHCore.dll'. 
'test-project.exe' (Win32): Loaded 'C:\Windows\System32\rmclient.dll'. 
'test-project.exe' (Win32): Loaded 'C:\Windows\System32\windows.storage.dll'. 
'test-project.exe' (Win32): Loaded 'C:\Windows\System32\powrprof.dll'. 
'test-project.exe' (Win32): Loaded 'C:\Windows\System32\umpdc.dll'. 
'test-project.exe' (Win32): Loaded 'C:\Windows\System32\dxilconv.dll'. 
'test-project.exe' (Win32): Unloaded 'C:\Windows\System32\dxilconv.dll'
'test-project.exe' (Win32): Loaded 'C:\Windows\System32\d3d10warp.dll'. 
'test-project.exe' (Win32): Loaded 'C:\Windows\System32\dxilconv.dll'. 
'test-project.exe' (Win32): Unloaded 'C:\Windows\System32\dxilconv.dll'
'test-project.exe' (Win32): Unloaded 'C:\Windows\System32\version.dll'
'test-project.exe' (Win32): Unloaded 'C:\Windows\System32\DriverStore\FileRepository\u0355311.inf_amd64_183b8d63847c90cf\B355199\amdxc64.dll'
'test-project.exe' (Win32): Unloaded 'C:\Windows\System32\d3d10warp.dll'
Exception thrown at 0x00007FFDC555A719 in test-project.exe: Microsoft C++ exception:  ?? ::st_panic at memory location 0x000000D776FFE9C8.
'test-project.exe' (Win32): Unloaded 'C:\Windows\System32\userenv.dll'
'test-project.exe' (Win32): Unloaded 'C:\Windows\System32\rmclient.dll'
'test-project.exe' (Win32): Unloaded 'C:\Windows\System32\twinapi.appcore.dll'
'test-project.exe' (Win32): Unloaded 'C:\Windows\System32\SHCore.dll'
'test-project.exe' (Win32): Unloaded 'C:\Windows\System32\profapi.dll'
'test-project.exe' (Win32): Unloaded 'C:\Windows\System32\umpdc.dll'
'test-project.exe' (Win32): Unloaded 'C:\Windows\System32\powrprof.dll'
'test-project.exe' (Win32): Unloaded 'C:\Windows\System32\windows.storage.dll'
'test-project.exe' (Win32): Unloaded 'C:\Windows\System32\D3DSCache.dll'
'test-project.exe' (Win32): Unloaded 'C:\Windows\System32\D3D12.dll'
'test-project.exe' (Win32): Unloaded 'C:\Windows\System32\dwmapi.dll'
'test-project.exe' (Win32): Unloaded 'C:\Windows\System32\shlwapi.dll'
'test-project.exe' (Win32): Unloaded 'C:\Windows\System32\amdihk64.dll'
'test-project.exe' (Win32): Unloaded 'C:\Windows\System32\winmmbase.dll'
'test-project.exe' (Win32): Unloaded 'C:\Windows\System32\winmm.dll'
'test-project.exe' (Win32): Unloaded 'C:\Windows\System32\devobj.dll'
'test-project.exe' (Win32): Unloaded 'C:\Windows\System32\msasn1.dll'
'test-project.exe' (Win32): Unloaded 'C:\Windows\System32\crypt32.dll'
'test-project.exe' (Win32): Unloaded 'C:\Windows\System32\wintrust.dll'
'test-project.exe' (Win32): Unloaded 'C:\Windows\System32\setupapi.dll'
'test-project.exe' (Win32): Unloaded 'C:\Windows\System32\DriverStore\FileRepository\u0355311.inf_amd64_183b8d63847c90cf\B355199\amdvlk64.dll'
The thread 0x3a24 has exited with code 101 (0x65).
The thread 0x14dc has exited with code 101 (0x65).
The thread 0x358 has exited with code 101 (0x65).
The thread 0x1950 has exited with code 101 (0x65).
The thread 0x1b94 has exited with code 101 (0x65).
DXGI WARNING: Process is terminating. Using simple reporting. Please call ReportLiveObjects() at runtime for standard reporting. [ STATE_CREATION WARNING #0: ]
DXGI WARNING: Live Producer at 0x00000255BA0B9248, Refcount: 5. [ STATE_CREATION WARNING #0: ]
DXGI WARNING: 	Live Object at 0x00000255BA17EF70, Refcount: 1. [ STATE_CREATION WARNING #0: ]
DXGI WARNING: 	Live Object at 0x00000255C47663D0, Refcount: 1. [ STATE_CREATION WARNING #0: ]
DXGI WARNING: 	Live Object at 0x00000255C4772710, Refcount: 1. [ STATE_CREATION WARNING #0: ]
DXGI WARNING: 	Live Object at 0x00000255C47BCF30, Refcount: 1. [ STATE_CREATION WARNING #0: ]
DXGI WARNING: Live                         Object :      4 [ STATE_CREATION WARNING #0: ]
The program '[12236] test-project.exe' has exited with code 101 (0x65).

[kvark]

Looks like it's failing before even reaching D3D12. There is only SPIRV-Cross in there.
Too bad we can't reproduce it on the shader playground because of that warning :/

[JonathanWoollett-Light]

Are you able to recreate the errors by running the test project/executable?

[kvark]

Did this now. I incorrectly assumed you were on D3D12, but you really are running on Vulkan backend.
What you are missing is the log output (and Vulkan validation layers, if you don't have them installed). Just hooking up env_logger would get you the following output:

[2020-09-16T13:57:19Z ERROR gfx_backend_vulkan]
VALIDATION [UNASSIGNED-CoreValidation-Shader-InconsistentSpirv (0)] : SPIR-V module not valid: Invalid SPIR-V binary version 1.3 for target environment SPIR-V 1.0 (under Vulkan 1.0 semantics).
object info: (type: UNKNOWN, hndl: 0)

[2020-09-16T13:57:19Z ERROR gfx_backend_vulkan]
VALIDATION [UNASSIGNED-CoreValidation-Shader-ExceedDeviceLimit (0)] : Shader requires flag VK_SHADER_STAGE_COMPUTE_BIT set in VkPhysicalDeviceSubgroupProperties::supportedStages but it is not set on the device
object info: (type: UNKNOWN, hndl: 0)

[2020-09-16T13:57:19Z ERROR gfx_backend_vulkan]
VALIDATION [UNASSIGNED-CoreValidation-Shader-ExceedDeviceLimit (0)] : Shader requires flag VK_SUBGROUP_FEATURE_BASIC_BIT set in VkPhysicalDeviceSubgroupProperties::supportedOperations but it is not set on the device
object info: (type: UNKNOWN, hndl: 0)

[2020-09-16T13:57:19Z ERROR gfx_backend_vulkan]
VALIDATION [UNASSIGNED-CoreValidation-Shader-ExceedDeviceLimit (0)] : Shader requires flag VK_SHADER_STAGE_COMPUTE_BIT set in VkPhysicalDeviceSubgroupProperties::supportedStages but it is not set on the device
object info: (type: UNKNOWN, hndl: 0)

[2020-09-16T13:57:19Z ERROR gfx_backend_vulkan]
VALIDATION [UNASSIGNED-CoreValidation-Shader-ExceedDeviceLimit (0)] : Shader requires flag VK_SUBGROUP_FEATURE_ARITHMETIC_BIT set in VkPhysicalDeviceSubgroupProperties::supportedOperations but it is not set on the device
object info: (type: UNKNOWN, hndl: 0)

TL;DR: you are trying to use the subgroup operations. WebGPU doesn't support them yet, and we don't have a native-only extension either. See gpuweb/gpuweb#954
I think we can roll something out in the restricted version, as suggested in this PR, for native-only.
As for this issue, I don't think there is more to do. We surely need to handle that case more gracefully, checking what shader needs, and erroring out appropriately. This is a more general task of validation and shader introspection, it's not specific to atomics.

[JonathanWoollett-Light]

you are trying to use the subgroup operations. WebGPU doesn't support them yet

Im not sure if this is accurate, maybe you meant to say it doesn't support atomic operations?

The shader in the project I shared uses the subgroup operation subgroupInclusiveAdd() and when not implementing atomicAdd() it works fine (atomicAdd is not required in all cases, and thus it can be tested without it).

I also have a variety of other shaders which implement subgroupAdd() and work.

[kvark]

Perhaps, it's my machine only that doesn't support the subgroups? Could you hook up env_logger and check the vulkan validation messages on your machine?

[JonathanWoollett-Light]

I'm not very unfamiliar with using Vulkan validaiton layers, having never really used them.

Using LunarG VulkanSDK and selecting a validation layer through it is how I've previously done it, although even this I have not really used.

So I'm not sure how to do what you ask.

Just in case this is along the lines of what you where looking for, this is the output when enabling the standard validation layer in VulkanSDK:

Started
thread 'main' panicked at 'failed to create shader module: Shader error: Compilation failed: ', C:\Users\jonat\.rustup\toolchains\stable-x86_64-pc-windows-msvc\lib/rustlib/src/rust\src\libstd\macros.rs:16:9
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
VUID-vkDestroyDevice-device-00378(ERROR / SPEC): msgNum: 1901072314 - Validation Error: [ VUID-vkDestroyDevice-device-00378 ] Object 0: handle = 0x1f692cedfa0, type = VK_OBJECT_TYPE_DEVICE; Object 1: handle = 0x1f69d50a520, type = VK_OBJECT_TYPE_COMMAND_BUFFER; | MessageID = 0x71500fba | OBJ ERROR : For VkDevice 0x1f692cedfa0[], VkCommandBuffer 0x1f69d50a520[] has not been destroyed. The Vulkan spec states: All child objects created on device must have been destroyed prior to destroying device (https://vulkan.lunarg.com/doc/view/1.2.148.0/windows/1.2-extensions/vkspec.html#VUID-vkDestroyDevice-device-00378)
    Objects: 2
        [0] 0x1f692cedfa0, type: 3, name: NULL
        [1] 0x1f69d50a520, type: 6, name: NULL
VUID-vkDestroyDevice-device-00378(ERROR / SPEC): msgNum: 1901072314 - Validation Error: [ VUID-vkDestroyDevice-device-00378 ] Object 0: handle = 0x1f692cedfa0, type = VK_OBJECT_TYPE_DEVICE; Object 1: handle = 0x1f69d51e900, type = VK_OBJECT_TYPE_COMMAND_BUFFER; | MessageID = 0x71500fba | OBJ ERROR : For VkDevice 0x1f692cedfa0[], VkCommandBuffer 0x1f69d51e900[] has not been destroyed. The Vulkan spec states: All child objects created on device must have been destroyed prior to destroying device (https://vulkan.lunarg.com/doc/view/1.2.148.0/windows/1.2-extensions/vkspec.html#VUID-vkDestroyDevice-device-00378)
    Objects: 2
        [0] 0x1f692cedfa0, type: 3, name: NULL
        [1] 0x1f69d51e900, type: 6, name: NULL
VUID-vkDestroyDevice-device-00378(ERROR / SPEC): msgNum: 1901072314 - Validation Error: [ VUID-vkDestroyDevice-device-00378 ] Object 0: handle = 0x1f692cedfa0, type = VK_OBJECT_TYPE_DEVICE; Object 1: handle = 0x1f69d6cae00, type = VK_OBJECT_TYPE_COMMAND_BUFFER; | MessageID = 0x71500fba | OBJ ERROR : For VkDevice 0x1f692cedfa0[], VkCommandBuffer 0x1f69d6cae00[] has not been destroyed. The Vulkan spec states: All child objects created on device must have been destroyed prior to destroying device (https://vulkan.lunarg.com/doc/view/1.2.148.0/windows/1.2-extensions/vkspec.html#VUID-vkDestroyDevice-device-00378)
    Objects: 2
        [0] 0x1f692cedfa0, type: 3, name: NULL
        [1] 0x1f69d6cae00, type: 6, name: NULL
VUID-vkDestroyDevice-device-00378(ERROR / SPEC): msgNum: 1901072314 - Validation Error: [ VUID-vkDestroyDevice-device-00378 ] Object 0: handle = 0x1f692cedfa0, type = VK_OBJECT_TYPE_DEVICE; Object 1: handle = 0x1f69d698450, type = VK_OBJECT_TYPE_COMMAND_BUFFER; | MessageID = 0x71500fba | OBJ ERROR : For VkDevice 0x1f692cedfa0[], VkCommandBuffer 0x1f69d698450[] has not been destroyed. The Vulkan spec states: All child objects created on device must have been destroyed prior to destroying device (https://vulkan.lunarg.com/doc/view/1.2.148.0/windows/1.2-extensions/vkspec.html#VUID-vkDestroyDevice-device-00378)
    Objects: 2
        [0] 0x1f692cedfa0, type: 3, name: NULL
        [1] 0x1f69d698450, type: 6, name: NULL
VUID-vkDestroyDevice-device-00378(ERROR / SPEC): msgNum: 1901072314 - Validation Error: [ VUID-vkDestroyDevice-device-00378 ] Object 0: handle = 0x1f692cedfa0, type = VK_OBJECT_TYPE_DEVICE; Object 1: handle = 0x1f69d68e260, type = VK_OBJECT_TYPE_COMMAND_BUFFER; | MessageID = 0x71500fba | OBJ ERROR : For VkDevice 0x1f692cedfa0[], VkCommandBuffer 0x1f69d68e260[] has not been destroyed. The Vulkan spec states: All child objects created on device must have been destroyed prior to destroying device (https://vulkan.lunarg.com/doc/view/1.2.148.0/windows/1.2-extensions/vkspec.html#VUID-vkDestroyDevice-device-00378)
    Objects: 2
        [0] 0x1f692cedfa0, type: 3, name: NULL
        [1] 0x1f69d68e260, type: 6, name: NULL
VUID-vkDestroyDevice-device-00378(ERROR / SPEC): msgNum: 1901072314 - Validation Error: [ VUID-vkDestroyDevice-device-00378 ] Object 0: handle = 0x1f692cedfa0, type = VK_OBJECT_TYPE_DEVICE; Object 1: handle = 0x1f69d4e1d60, type = VK_OBJECT_TYPE_COMMAND_BUFFER; | MessageID = 0x71500fba | OBJ ERROR : For VkDevice 0x1f692cedfa0[], VkCommandBuffer 0x1f69d4e1d60[] has not been destroyed. The Vulkan spec states: All child objects created on device must have been destroyed prior to destroying device (https://vulkan.lunarg.com/doc/view/1.2.148.0/windows/1.2-extensions/vkspec.html#VUID-vkDestroyDevice-device-00378)
    Objects: 2
        [0] 0x1f692cedfa0, type: 3, name: NULL
        [1] 0x1f69d4e1d60, type: 6, name: NULL
VUID-vkDestroyDevice-device-00378(ERROR / SPEC): msgNum: 1901072314 - Validation Error: [ VUID-vkDestroyDevice-device-00378 ] Object 0: handle = 0x1f692cedfa0, type = VK_OBJECT_TYPE_DEVICE; Object 1: handle = 0x1f692d23070, type = VK_OBJECT_TYPE_COMMAND_BUFFER; | MessageID = 0x71500fba | OBJ ERROR : For VkDevice 0x1f692cedfa0[], VkCommandBuffer 0x1f692d23070[] has not been destroyed. The Vulkan spec states: All child objects created on device must have been destroyed prior to destroying device (https://vulkan.lunarg.com/doc/view/1.2.148.0/windows/1.2-extensions/vkspec.html#VUID-vkDestroyDevice-device-00378)
    Objects: 2
        [0] 0x1f692cedfa0, type: 3, name: NULL
        [1] 0x1f692d23070, type: 6, name: NULL
VUID-vkDestroyDevice-device-00378(ERROR / SPEC): msgNum: 1901072314 - Validation Error: [ VUID-vkDestroyDevice-device-00378 ] Object 0: handle = 0x1f692cedfa0, type = VK_OBJECT_TYPE_DEVICE; Object 1: handle = 0x1f69d4f6140, type = VK_OBJECT_TYPE_COMMAND_BUFFER; | MessageID = 0x71500fba | OBJ ERROR : For VkDevice 0x1f692cedfa0[], VkCommandBuffer 0x1f69d4f6140[] has not been destroyed. The Vulkan spec states: All child objects created on device must have been destroyed prior to destroying device (https://vulkan.lunarg.com/doc/view/1.2.148.0/windows/1.2-extensions/vkspec.html#VUID-vkDestroyDevice-device-00378)
    Objects: 2
        [0] 0x1f692cedfa0, type: 3, name: NULL
        [1] 0x1f69d4f6140, type: 6, name: NULL
VUID-vkDestroyDevice-device-00378(ERROR / SPEC): msgNum: 1901072314 - Validation Error: [ VUID-vkDestroyDevice-device-00378 ] Object 0: handle = 0x1f692cedfa0, type = VK_OBJECT_TYPE_DEVICE; Object 1: handle = 0x1f693082cc0, type = VK_OBJECT_TYPE_COMMAND_BUFFER; | MessageID = 0x71500fba | OBJ ERROR : For VkDevice 0x1f692cedfa0[], VkCommandBuffer 0x1f693082cc0[] has not been destroyed. The Vulkan spec states: All child objects created on device must have been destroyed prior to destroying device (https://vulkan.lunarg.com/doc/view/1.2.148.0/windows/1.2-extensions/vkspec.html#VUID-vkDestroyDevice-device-00378)
    Objects: 2
        [0] 0x1f692cedfa0, type: 3, name: NULL
        [1] 0x1f693082cc0, type: 6, name: NULL
VUID-vkDestroyDevice-device-00378(ERROR / SPEC): msgNum: 1901072314 - Validation Error: [ VUID-vkDestroyDevice-device-00378 ] Object 0: handle = 0x1f692cedfa0, type = VK_OBJECT_TYPE_DEVICE; Object 1: handle = 0x1f69d6b6a20, type = VK_OBJECT_TYPE_COMMAND_BUFFER; | MessageID = 0x71500fba | OBJ ERROR : For VkDevice 0x1f692cedfa0[], VkCommandBuffer 0x1f69d6b6a20[] has not been destroyed. The Vulkan spec states: All child objects created on device must have been destroyed prior to destroying device (https://vulkan.lunarg.com/doc/view/1.2.148.0/windows/1.2-extensions/vkspec.html#VUID-vkDestroyDevice-device-00378)
    Objects: 2
        [0] 0x1f692cedfa0, type: 3, name: NULL
        [1] 0x1f69d6b6a20, type: 6, name: NULL
VUID-vkDestroyDevice-device-00378(ERROR / SPEC): msgNum: 1901072314 - Validation Error: [ VUID-vkDestroyDevice-device-00378 ] Object 0: handle = 0x1f692cedfa0, type = VK_OBJECT_TYPE_DEVICE; Object 1: handle = 0x1f69d6ac830, type = VK_OBJECT_TYPE_COMMAND_BUFFER; | MessageID = 0x71500fba | OBJ ERROR : For VkDevice 0x1f692cedfa0[], VkCommandBuffer 0x1f69d6ac830[] has not been destroyed. The Vulkan spec states: All child objects created on device must have been destroyed prior to destroying device (https://vulkan.lunarg.com/doc/view/1.2.148.0/windows/1.2-extensions/vkspec.html#VUID-vkDestroyDevice-device-00378)
    Objects: 2
        [0] 0x1f692cedfa0, type: 3, name: NULL
        [1] 0x1f69d6ac830, type: 6, name: NULL
VUID-vkDestroyDevice-device-00378(ERROR / SPEC): msgNum: 1901072314 - Validation Error: [ VUID-vkDestroyDevice-device-00378 ] Object 0: handle = 0x1f692cedfa0, type = VK_OBJECT_TYPE_DEVICE; Object 1: handle = 0x1f69d4dbb90, type = VK_OBJECT_TYPE_COMMAND_BUFFER; | MessageID = 0x71500fba | OBJ ERROR : For VkDevice 0x1f692cedfa0[], VkCommandBuffer 0x1f69d4dbb90[] has not been destroyed. The Vulkan spec states: All child objects created on device must have been destroyed prior to destroying device (https://vulkan.lunarg.com/doc/view/1.2.148.0/windows/1.2-extensions/vkspec.html#VUID-vkDestroyDevice-device-00378)
    Objects: 2
        [0] 0x1f692cedfa0, type: 3, name: NULL
        [1] 0x1f69d4dbb90, type: 6, name: NULL
VUID-vkDestroyDevice-device-00378(ERROR / SPEC): msgNum: 1901072314 - Validation Error: [ VUID-vkDestroyDevice-device-00378 ] Object 0: handle = 0x1f692cedfa0, type = VK_OBJECT_TYPE_DEVICE; Object 1: handle = 0x1f69d684070, type = VK_OBJECT_TYPE_COMMAND_BUFFER; | MessageID = 0x71500fba | OBJ ERROR : For VkDevice 0x1f692cedfa0[], VkCommandBuffer 0x1f69d684070[] has not been destroyed. The Vulkan spec states: All child objects created on device must have been destroyed prior to destroying device (https://vulkan.lunarg.com/doc/view/1.2.148.0/windows/1.2-extensions/vkspec.html#VUID-vkDestroyDevice-device-00378)
    Objects: 2
        [0] 0x1f692cedfa0, type: 3, name: NULL
        [1] 0x1f69d684070, type: 6, name: NULL
VUID-vkDestroyDevice-device-00378(ERROR / SPEC): msgNum: 1901072314 - Validation Error: [ VUID-vkDestroyDevice-device-00378 ] Object 0: handle = 0x1f692cedfa0, type = VK_OBJECT_TYPE_DEVICE; Object 1: handle = 0x1f69d4ebf50, type = VK_OBJECT_TYPE_COMMAND_BUFFER; | MessageID = 0x71500fba | OBJ ERROR : For VkDevice 0x1f692cedfa0[], VkCommandBuffer 0x1f69d4ebf50[] has not been destroyed. The Vulkan spec states: All child objects created on device must have been destroyed prior to destroying device (https://vulkan.lunarg.com/doc/view/1.2.148.0/windows/1.2-extensions/vkspec.html#VUID-vkDestroyDevice-device-00378)
    Objects: 2
        [0] 0x1f692cedfa0, type: 3, name: NULL
        [1] 0x1f69d4ebf50, type: 6, name: NULL
VUID-vkDestroyDevice-device-00378(ERROR / SPEC): msgNum: 1901072314 - Validation Error: [ VUID-vkDestroyDevice-device-00378 ] Object 0: handle = 0x1f692cedfa0, type = VK_OBJECT_TYPE_DEVICE; Object 1: handle = 0x1f69d500330, type = VK_OBJECT_TYPE_COMMAND_BUFFER; | MessageID = 0x71500fba | OBJ ERROR : For VkDevice 0x1f692cedfa0[], VkCommandBuffer 0x1f69d500330[] has not been destroyed. The Vulkan spec states: All child objects created on device must have been destroyed prior to destroying device (https://vulkan.lunarg.com/doc/view/1.2.148.0/windows/1.2-extensions/vkspec.html#VUID-vkDestroyDevice-device-00378)
    Objects: 2
        [0] 0x1f692cedfa0, type: 3, name: NULL
        [1] 0x1f69d500330, type: 6, name: NULL
VUID-vkDestroyDevice-device-00378(ERROR / SPEC): msgNum: 1901072314 - Validation Error: [ VUID-vkDestroyDevice-device-00378 ] Object 0: handle = 0x1f692cedfa0, type = VK_OBJECT_TYPE_DEVICE; Object 1: handle = 0x1f69d6a2640, type = VK_OBJECT_TYPE_COMMAND_BUFFER; | MessageID = 0x71500fba | OBJ ERROR : For VkDevice 0x1f692cedfa0[], VkCommandBuffer 0x1f69d6a2640[] has not been destroyed. The Vulkan spec states: All child objects created on device must have been destroyed prior to destroying device (https://vulkan.lunarg.com/doc/view/1.2.148.0/windows/1.2-extensions/vkspec.html#VUID-vkDestroyDevice-device-00378)
    Objects: 2
        [0] 0x1f692cedfa0, type: 3, name: NULL
        [1] 0x1f69d6a2640, type: 6, name: NULL
VUID-vkDestroyDevice-device-00378(ERROR / SPEC): msgNum: 1901072314 - Validation Error: [ VUID-vkDestroyDevice-device-00378 ] Object 0: handle = 0x1f692cedfa0, type = VK_OBJECT_TYPE_DEVICE; Object 1: handle = 0x1f69d514710, type = VK_OBJECT_TYPE_COMMAND_BUFFER; | MessageID = 0x71500fba | OBJ ERROR : For VkDevice 0x1f692cedfa0[], VkCommandBuffer 0x1f69d514710[] has not been destroyed. The Vulkan spec states: All child objects created on device must have been destroyed prior to destroying device (https://vulkan.lunarg.com/doc/view/1.2.148.0/windows/1.2-extensions/vkspec.html#VUID-vkDestroyDevice-device-00378)
    Objects: 2
        [0] 0x1f692cedfa0, type: 3, name: NULL
        [1] 0x1f69d514710, type: 6, name: NULL
VUID-vkDestroyDevice-device-00378(ERROR / SPEC): msgNum: 1901072314 - Validation Error: [ VUID-vkDestroyDevice-device-00378 ] Object 0: handle = 0x1f692cedfa0, type = VK_OBJECT_TYPE_DEVICE; Object 1: handle = 0x1f69d3e6510, type = VK_OBJECT_TYPE_COMMAND_BUFFER; | MessageID = 0x71500fba | OBJ ERROR : For VkDevice 0x1f692cedfa0[], VkCommandBuffer 0x1f69d3e6510[] has not been destroyed. The Vulkan spec states: All child objects created on device must have been destroyed prior to destroying device (https://vulkan.lunarg.com/doc/view/1.2.148.0/windows/1.2-extensions/vkspec.html#VUID-vkDestroyDevice-device-00378)
    Objects: 2
        [0] 0x1f692cedfa0, type: 3, name: NULL
        [1] 0x1f69d3e6510, type: 6, name: NULL
VUID-vkDestroyDevice-device-00378(ERROR / SPEC): msgNum: 1901072314 - Validation Error: [ VUID-vkDestroyDevice-device-00378 ] Object 0: handle = 0x1f692cedfa0, type = VK_OBJECT_TYPE_DEVICE; Object 1: handle = 0x1f69d528af0, type = VK_OBJECT_TYPE_COMMAND_BUFFER; | MessageID = 0x71500fba | OBJ ERROR : For VkDevice 0x1f692cedfa0[], VkCommandBuffer 0x1f69d528af0[] has not been destroyed. The Vulkan spec states: All child objects created on device must have been destroyed prior to destroying device (https://vulkan.lunarg.com/doc/view/1.2.148.0/windows/1.2-extensions/vkspec.html#VUID-vkDestroyDevice-device-00378)
    Objects: 2
        [0] 0x1f692cedfa0, type: 3, name: NULL
        [1] 0x1f69d528af0, type: 6, name: NULL
VUID-vkDestroyDevice-device-00378(ERROR / SPEC): msgNum: 1901072314 - Validation Error: [ VUID-vkDestroyDevice-device-00378 ] Object 0: handle = 0x1f692cedfa0, type = VK_OBJECT_TYPE_DEVICE; Object 1: handle = 0x1f69d6c0c10, type = VK_OBJECT_TYPE_COMMAND_BUFFER; | MessageID = 0x71500fba | OBJ ERROR : For VkDevice 0x1f692cedfa0[], VkCommandBuffer 0x1f69d6c0c10[] has not been destroyed. The Vulkan spec states: All child objects created on device must have been destroyed prior to destroying device (https://vulkan.lunarg.com/doc/view/1.2.148.0/windows/1.2-extensions/vkspec.html#VUID-vkDestroyDevice-device-00378)
    Objects: 2
        [0] 0x1f692cedfa0, type: 3, name: NULL
        [1] 0x1f69d6c0c10, type: 6, name: NULL
VUID-vkDestroyDevice-device-00378(ERROR / SPEC): msgNum: 1901072314 - Validation Error: [ VUID-vkDestroyDevice-device-00378 ] Object 0: handle = 0x1f692cedfa0, type = VK_OBJECT_TYPE_DEVICE; Object 1: handle = 0x5d11410000000004, type = VK_OBJECT_TYPE_BUFFER; | MessageID = 0x71500fba | OBJ ERROR : For VkDevice 0x1f692cedfa0[], VkBuffer 0x5d11410000000004[] has not been destroyed. The Vulkan spec states: All child objects created on device must have been destroyed prior to destroying device (https://vulkan.lunarg.com/doc/view/1.2.148.0/windows/1.2-extensions/vkspec.html#VUID-vkDestroyDevice-device-00378)
    Objects: 2
        [0] 0x1f692cedfa0, type: 3, name: NULL
        [1] 0x5d11410000000004, type: 9, name: NULL
VUID-vkDestroyDevice-device-00378(ERROR / SPEC): msgNum: 1901072314 - Validation Error: [ VUID-vkDestroyDevice-device-00378 ] Object 0: handle = 0x1f692cedfa0, type = VK_OBJECT_TYPE_DEVICE; Object 1: handle = 0xa540ac0000000009, type = VK_OBJECT_TYPE_BUFFER; | MessageID = 0x71500fba | OBJ ERROR : For VkDevice 0x1f692cedfa0[], VkBuffer 0xa540ac0000000009[] has not been destroyed. The Vulkan spec states: All child objects created on device must have been destroyed prior to destroying device (https://vulkan.lunarg.com/doc/view/1.2.148.0/windows/1.2-extensions/vkspec.html#VUID-vkDestroyDevice-device-00378)
    Objects: 2
        [0] 0x1f692cedfa0, type: 3, name: NULL
        [1] 0xa540ac0000000009, type: 9, name: NULL
VUID-vkDestroyDevice-device-00378(ERROR / SPEC): msgNum: 1901072314 - Validation Error: [ VUID-vkDestroyDevice-device-00378 ] Object 0: handle = 0x1f692cedfa0, type = VK_OBJECT_TYPE_DEVICE; Object 1: handle = 0x4c5b620000000007, type = VK_OBJECT_TYPE_BUFFER; | MessageID = 0x71500fba | OBJ ERROR : For VkDevice 0x1f692cedfa0[], VkBuffer 0x4c5b620000000007[] has not been destroyed. The Vulkan spec states: All child objects created on device must have been destroyed prior to destroying device (https://vulkan.lunarg.com/doc/view/1.2.148.0/windows/1.2-extensions/vkspec.html#VUID-vkDestroyDevice-device-00378)
    Objects: 2
        [0] 0x1f692cedfa0, type: 3, name: NULL
        [1] 0x4c5b620000000007, type: 9, name: NULL
VUID-vkDestroyDevice-device-00378(ERROR / SPEC): msgNum: 1901072314 - Validation Error: [ VUID-vkDestroyDevice-device-00378 ] Object 0: handle = 0x1f692cedfa0, type = VK_OBJECT_TYPE_DEVICE; Object 1: handle = 0xf8ce070000000002, type = VK_OBJECT_TYPE_BUFFER; | MessageID = 0x71500fba | OBJ ERROR : For VkDevice 0x1f692cedfa0[], VkBuffer 0xf8ce070000000002[] has not been destroyed. The Vulkan spec states: All child objects created on device must have been destroyed prior to destroying device (https://vulkan.lunarg.com/doc/view/1.2.148.0/windows/1.2-extensions/vkspec.html#VUID-vkDestroyDevice-device-00378)
    Objects: 2
        [0] 0x1f692cedfa0, type: 3, name: NULL
        [1] 0xf8ce070000000002, type: 9, name: NULL
VUID-vkDestroyDevice-device-00378(ERROR / SPEC): msgNum: 1901072314 - Validation Error: [ VUID-vkDestroyDevice-device-00378 ] Object 0: handle = 0x1f692cedfa0, type = VK_OBJECT_TYPE_DEVICE; Object 1: handle = 0x6dc7200000000005, type = VK_OBJECT_TYPE_IMAGE; | MessageID = 0x71500fba | OBJ ERROR : For VkDevice 0x1f692cedfa0[], VkImage 0x6dc7200000000005[] has not been destroyed. The Vulkan spec states: All child objects created on device must have been destroyed prior to destroying device (https://vulkan.lunarg.com/doc/view/1.2.148.0/windows/1.2-extensions/vkspec.html#VUID-vkDestroyDevice-device-00378)
    Objects: 2
        [0] 0x1f692cedfa0, type: 3, name: NULL
        [1] 0x6dc7200000000005, type: 10, name: NULL
VUID-vkDestroyDevice-device-00378(ERROR / SPEC): msgNum: 1901072314 - Validation Error: [ VUID-vkDestroyDevice-device-00378 ] Object 0: handle = 0x1f692cedfa0, type = VK_OBJECT_TYPE_DEVICE; Object 1: handle = 0x948acd0000000008, type = VK_OBJECT_TYPE_DEVICE_MEMORY; | MessageID = 0x71500fba | OBJ ERROR : For VkDevice 0x1f692cedfa0[], VkDeviceMemory 0x948acd0000000008[] has not been destroyed. The Vulkan spec states: All child objects created on device must have been destroyed prior to destroying device (https://vulkan.lunarg.com/doc/view/1.2.148.0/windows/1.2-extensions/vkspec.html#VUID-vkDestroyDevice-device-00378)
    Objects: 2
        [0] 0x1f692cedfa0, type: 3, name: NULL
        [1] 0x948acd0000000008, type: 8, name: NULL
VUID-vkDestroyDevice-device-00378(ERROR / SPEC): msgNum: 1901072314 - Validation Error: [ VUID-vkDestroyDevice-device-00378 ] Object 0: handle = 0x1f692cedfa0, type = VK_OBJECT_TYPE_DEVICE; Object 1: handle = 0x3ba5830000000006, type = VK_OBJECT_TYPE_DEVICE_MEMORY; | MessageID = 0x71500fba | OBJ ERROR : For VkDevice 0x1f692cedfa0[], VkDeviceMemory 0x3ba5830000000006[] has not been destroyed. The Vulkan spec states: All child objects created on device must have been destroyed prior to destroying device (https://vulkan.lunarg.com/doc/view/1.2.148.0/windows/1.2-extensions/vkspec.html#VUID-vkDestroyDevice-device-00378)
    Objects: 2
        [0] 0x1f692cedfa0, type: 3, name: NULL
        [1] 0x3ba5830000000006, type: 8, name: NULL
VUID-vkDestroyDevice-device-00378(ERROR / SPEC): msgNum: 1901072314 - Validation Error: [ VUID-vkDestroyDevice-device-00378 ] Object 0: handle = 0x1f692cedfa0, type = VK_OBJECT_TYPE_DEVICE; Object 1: handle = 0x983e60000000003, type = VK_OBJECT_TYPE_DEVICE_MEMORY; | MessageID = 0x71500fba | OBJ ERROR : For VkDevice 0x1f692cedfa0[], VkDeviceMemory 0x983e60000000003[] has not been destroyed. The Vulkan spec states: All child objects created on device must have been destroyed prior to destroying device (https://vulkan.lunarg.com/doc/view/1.2.148.0/windows/1.2-extensions/vkspec.html#VUID-vkDestroyDevice-device-00378)
    Objects: 2
        [0] 0x1f692cedfa0, type: 3, name: NULL
        [1] 0x983e60000000003, type: 8, name: NULL
VUID-vkDestroyDevice-device-00378(ERROR / SPEC): msgNum: 1901072314 - Validation Error: [ VUID-vkDestroyDevice-device-00378 ] Object 0: handle = 0x1f692cedfa0, type = VK_OBJECT_TYPE_DEVICE; Object 1: handle = 0x731f0f000000000a, type = VK_OBJECT_TYPE_IMAGE_VIEW; | MessageID = 0x71500fba | OBJ ERROR : For VkDevice 0x1f692cedfa0[], VkImageView 0x731f0f000000000a[] has not been destroyed. The Vulkan spec states: All child objects created on device must 
have been destroyed prior to destroying device (https://vulkan.lunarg.com/doc/view/1.2.148.0/windows/1.2-extensions/vkspec.html#VUID-vkDestroyDevice-device-00378)
    Objects: 2
        [0] 0x1f692cedfa0, type: 3, name: NULL
        [1] 0x731f0f000000000a, type: 14, name: NULL
VUID-vkDestroyDevice-device-00378(ERROR / SPEC): msgNum: 1901072314 - Validation Error: [ VUID-vkDestroyDevice-device-00378 ] Object 0: handle = 0x1f692cedfa0, type = VK_OBJECT_TYPE_DEVICE; Object 1: handle = 0x83d4ee000000000b, type = VK_OBJECT_TYPE_DESCRIPTOR_SET_LAYOUT; | MessageID = 0x71500fba | OBJ ERROR : For VkDevice 0x1f692cedfa0[], VkDescriptorSetLayout 0x83d4ee000000000b[] has not been destroyed. The Vulkan spec states: All child objects created on device must have been destroyed prior to destroying device (https://vulkan.lunarg.com/doc/view/1.2.148.0/windows/1.2-extensions/vkspec.html#VUID-vkDestroyDevice-device-00378)
    Objects: 2
        [0] 0x1f692cedfa0, type: 3, name: NULL
        [1] 0x83d4ee000000000b, type: 20, name: NULL
VUID-vkDestroyDevice-device-00378(ERROR / SPEC): msgNum: 1901072314 - Validation Error: [ VUID-vkDestroyDevice-device-00378 ] Object 0: handle = 0x1f692cedfa0, type = VK_OBJECT_TYPE_DEVICE; Object 1: handle = 0xe81828000000000d, type = VK_OBJECT_TYPE_DESCRIPTOR_SET; | MessageID = 0x71500fba | OBJ ERROR : For VkDevice 0x1f692cedfa0[], VkDescriptorSet 0xe81828000000000d[] has not been destroyed. The Vulkan spec states: All child objects created on device must have been destroyed prior to destroying device (https://vulkan.lunarg.com/doc/view/1.2.148.0/windows/1.2-extensions/vkspec.html#VUID-vkDestroyDevice-device-00378)
    Objects: 2
        [0] 0x1f692cedfa0, type: 3, name: NULL
        [1] 0xe81828000000000d, type: 23, name: NULL
VUID-vkDestroyDevice-device-00378(ERROR / SPEC): msgNum: 1901072314 - Validation Error: [ VUID-vkDestroyDevice-device-00378 ] Object 0: handle = 0x1f692cedfa0, type = VK_OBJECT_TYPE_DEVICE; Object 1: handle = 0xd76249000000000c, type = VK_OBJECT_TYPE_DESCRIPTOR_POOL; | MessageID = 0x71500fba | OBJ ERROR : For VkDevice 0x1f692cedfa0[], VkDescriptorPool 0xd76249000000000c[] has not been destroyed. The Vulkan spec states: All child objects created on device must have been destroyed prior to destroying device (https://vulkan.lunarg.com/doc/view/1.2.148.0/windows/1.2-extensions/vkspec.html#VUID-vkDestroyDevice-device-00378)
    Objects: 2
        [0] 0x1f692cedfa0, type: 3, name: NULL
        [1] 0xd76249000000000c, type: 22, name: NULL
VUID-vkDestroyDevice-device-00378(ERROR / SPEC): msgNum: 1901072314 - Validation Error: [ VUID-vkDestroyDevice-device-00378 ] Object 0: handle = 0x1f692cedfa0, type = VK_OBJECT_TYPE_DEVICE; Object 1: handle = 0x2aefa40000000001, type = VK_OBJECT_TYPE_COMMAND_POOL; | MessageID = 0x71500fba | OBJ ERROR : For VkDevice 0x1f692cedfa0[], VkCommandPool 0x2aefa40000000001[] has not been destroyed. The Vulkan spec states: All child objects created on device must have been destroyed prior to destroying device (https://vulkan.lunarg.com/doc/view/1.2.148.0/windows/1.2-extensions/vkspec.html#VUID-vkDestroyDevice-device-00378)
    Objects: 2
        [0] 0x1f692cedfa0, type: 3, name: NULL
        [1] 0x2aefa40000000001, type: 25, name: NULL
error: process didn't exit successfully: `target\release\test-project.exe` (exit code: 101)
PS C:\Users\jonat\Projects\test-project>

[kvark]

Interesting. So on your machine there appear to be no validation errors reported by VVL, and it just fails to create the pipeline.

[JonathanWoollett-Light]

I'm not quite sure of the state of this issue what should be done to move forward here, is this simply a bug you are looking into? Is there anything else I can do here?

[kvark]

First step - figuring out what exactly happens there, and why we end up with "Compilation failed".
If it's SPIRV-Cross, second step would be filing an issue to them and providing your shader.

[JonathanWoollett-Light]

First step - figuring out what exactly happens there, and why we end up with "Compilation failed".

Apologies for my inexperience here, but I have no idea how I would do this (or even where I should look to learn how to do this, even being able to see the result of validation layers I don't know how to interpret the output).

[kvark]

Thank you for not hesitating to reach out. We'll be happy to guide you through.
In this case, "Compilation failed" is a message gfx code prints. So by "figuring out what exactly happens here" I meant just doing step-by-step debugging to see what caused the condition. Are you able to break the execution at the relevant create_xxx_pipeline and step through it?

[JonathanWoollett-Light]

Can do (stepping through it all rn).

[JonathanWoollett-Light]

So stepping through and into everything, it seems to crash when stepping into mod::device_create_shader_module.

Just to make it clear as possible, this function:

pub fn device_create_shader_module<B: GfxBackend>(
        &self,
        device_id: id::DeviceId,
        source: pipeline::ShaderModuleSource,
        id_in: Input<G, id::ShaderModuleId>,
    ) -> Result<id::ShaderModuleId, pipeline::CreateShaderModuleError> {

The debugger highlights the line of the function name, then when stepped into, crashes.

Just to clarify are you unable to replicate the error on your system? (I just updated the test project to make sure it's right: test-project.zip)

[kvark]

Something in compile_shader_module returned an error. If you could step through and see, what exactly, that would be helpful.

[JonathanWoollett-Light]

So after hitting this breakpoint:

And stepping into, it results in:

Stepping into again results in the notification just reopening and no progress. Stepping over has the same result.

Pressing continue however results in:

Pressing continue again at this point results in:

Here the program has excited.

Playing around with breakpoints a bit more I am able to step through the function a bit more, ultimately it steps through until hitting the same breakpoint (at the function name) and exciting by opening this file this time:

[kvark]

Hmm, I don't understand. You aren't able to step by step within this function? Perhaps, you are working with a release build?
I don't know how VSCode debugging experience is, I've had some success with MSVS, XCode, and gdb/lldb.

[JonathanWoollett-Light]

What happens when you run the project on your machine with cargo run (if you maybe have slightly older version please download it again from here)?

[kvark]

As I mentioned in #564 (comment), running your test_project (after hooking up env_logger) gives me validation errors. Now tested on 2 totally different machines. Perhaps, your validation layers are just old, and they don't catch that? Or you are using a modifier shader or something, not what you shared.

[JonathanWoollett-Light]

Perhaps, your validation layers are just old, and they don't catch that?

In #564 (comment) the outputs here are not the result of using env_logger.

(after hooking up env_logger)

Are you reffering to this https://github.com/gfx-rs/wgpu-rs#simple-setup? So just setting this up (Without setting RUST_LOG, I have no idea how to set it or what to set it to) results in (when run with cargo run):

[0.483317 ERROR]()(no module):
VALIDATION [VUID-vkDestroyDevice-device-00378 (1901072314)] : Validation Error: [ VUID-vkDestroyDevice-device-00378 ] Object 0: handle = 0x28a25eec220, type = VK_OBJECT_TYPE_DEVICE; Object 1: handle = 0xe81828000000000d, type = VK_OBJECT_TYPE_DESCRIPTOR_POOL; | MessageID = 0x71500fba | OBJ ERROR : For VkDevice 0x28a25eec220[], VkDescriptorPool 0xe81828000000000d[] has not been destroyed. The Vulkan spec states: All child objects created on device must have been destroyed prior to destroying device (https://vulkan.lunarg.com/doc/view/1.2.148.0/windows/1.2-extensions/vkspec.html#VUID-vkDestroyDevice-device-00378)
object info: (type: DEVICE, hndl: 2792365146656), (type: DESCRIPTOR_POOL, hndl: 16724161196705447949)


[0.484899 ERROR]()(no module):
VALIDATION [VUID-vkDestroyDevice-device-00378 (1901072314)] : Validation Error: [ VUID-vkDestroyDevice-device-00378 ] Object 0: handle = 0x28a25eec220, type = VK_OBJECT_TYPE_DEVICE; Object 1: handle = 0xf8ce070000000002, type = VK_OBJECT_TYPE_COMMAND_POOL; | MessageID = 0x71500fba | OBJ ERROR : For VkDevice 0x28a25eec220[], VkCommandPool 0xf8ce070000000002[] has not been destroyed. The Vulkan spec states: All child objects created on device must have been destroyed prior to destroying device (https://vulkan.lunarg.com/doc/view/1.2.148.0/windows/1.2-extensions/vkspec.html#VUID-vkDestroyDevice-device-00378)
object info: (type: DEVICE, hndl: 2792365146656), (type: COMMAND_POOL, hndl: 17928274863189917698)

error: process didn't exit successfully: `target\debug\test-project.exe` (exit code: 101)
PS C:\Users\jonat\Projects\test-project>     

Is this familiar?

[kvark]

Yep, it looks like you aren't getting the errors I'm getting. What version of the Vulkan SDK do you have installed? I have "vulkan-validation-layers, version: 1.2.141.0, release: 9".

[JonathanWoollett-Light]

I beleive I have the latest version (from https://vulkan.lunarg.com/sdk/home).

Vulkan configurator->Tools->Vulkan Info:

Vulkan Instance Version: 1.2.148

But in #564 (comment) it is not running (or atleast I have not manually done anything involving it, I haven't opened the Vulkan configurator), I'm unsure why it would matter to the result of env_logger?

[kvark]

I don't know how your system is configured. Generally, we intercept the validation messages and put them into log. You can only see the log via something that re-routes it to the console, like env_logger. The fact that you are seeing the messages without env_logger means that you configured the validation layers to always print out errors to the console, regardless of what the app thinks.

Anyhow, I can't reproduce the situation you have without triggering validation errors. Perhaps, the test case you shared is not the one you are running?

[JonathanWoollett-Light]

To give an overview:

This is the test project (this should be same as other links, but this is just to be safe): test-project.zip

I am ofc excluding warnings in outputs here.

This is the simple output (without Vulkan configurator running in background and with wgpu_subscriber::initialize_default_subscriber(None); commented out):

Started
thread 'main' panicked at 'failed to create shader module: Shader error: Compilation failed: ', C:\Users\jonat\.rustup\toolchains\stable-x86_64-pc-windows-msvc\lib/rustlib/src/rust\src\libstd\macros.rs:16:9
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
error: process didn't exit successfully: `target\debug\test-project.exe` (exit code: 101)
PS C:\Users\jonat\Projects\test-project> 

This is output with env_logger with cargo run: https://hastebin.com/gomopesege.md (it is a little long to put as code here)
(I am aware this has changed since my last comment, I'm not sure why)

This is output with env_logger with cargo run --release: https://hastebin.com/alopajepax.md

This is output with Vulkan configurator in background using the standard validation layer: https://hastebin.com/unomasojud.rb (requires running using cargo run --release):

[JonathanWoollett-Light]

I hate to be annoying (this issue has already reached quite the length), but I have no idea how to move forward with this bug.
I greatly appreciate your attention here (I know it's perhaps a more frustrating issue than most).

Could you cexplain how you got the D3DCompile errors in #564 (comment)?

[kvark]

@JonathanWoollett-Light I did mention that it's useful to hook up env_logger (or any other way to see the log) multiple times, but your latest update on the test case still doesn't have this. Once you hook it up (i.e just call env_logger::init at start of main()), and set the wgpu::BackendBit::DX12, you'll see the error in the console output.

[JonathanWoollett-Light]

The difference leading to the errors you mention in #564 (comment) is setting wgpu::BackendBit::DX12. Doing this outputs the error error X4010: Unsigned integer divide by zero.

Perhaps you assumed by mentioning D3DCompile I would know to set wgpu::BackendBit::DX12? (instead of wgpu::BackendBit::PRIMARY)

The outputs of 2 logging approaches when wgpu::BackendBit::DX12 is set (specifically error X4010: Unsigned integer divide by zero is on the line thread 'main' panicked at...):

wgpu_subscriber::initialize_default_subscriber(None); (this approach comes from the repo page here)(as was setup in latest test project):

PS C:\Users\jonat\Projects\test-project> cargo run --release
   Compiling test-project v0.1.0 (C:\Users\jonat\Projects\test-project)
    Finished release [optimized] target(s) in 11.33s
     Running `target\release\test-project.exe`
Started
x.len(): 16
internal buffer byte size: 32
[0.166222 ERROR]()(no module): D3DCompile error 80004005
thread 'main' panicked at 'Compute pipeline creation error: Err(Shader(CompilationFailed("C:\\Users\\jonat\\Projects\\test-project\\Shader@0x000002368D9A95E0(23,17-25): warning X3556: integer divides may be much slower, try using uints if possible.\nC:\\Users\\jonat\\Projects\\test-project\\Shader@0x000002368D9A95E0(32,17-31): error X4010: Unsigned integer divide by zero\nC:\\Users\\jonat\\Projects\\test-project\\Shader@0x000002368D9A95E0(32,17-31): warning X3556: integer divides may be much slower, try using uints if possible.\n\u{0}")))', C:\Users\jonat\.rustup\toolchains\stable-x86_64-pc-windows-msvc\lib/rustlib/src/rust\src\libstd\macros.rs:16:9
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
[0.181608 ERROR]()(no module): Heaps not completely freed before drop. Utilization: MemoryHeapUtilization { utilization: MemoryUtilization { used: 4194304, effective: 196608 }, size: 7701849907 }
[0.193378 ERROR]()(no module): Memory leak: SizeEntry(65536) is still used
[0.194108 ERROR]()(no module): Memory leak: SizeEntry(524288) is still used
[0.194750 ERROR]()(no module): Memory leak: SizeEntry(131072) is still used
[0.208460 ERROR]()(no module): Memory leak: SizeEntry(1048576) is still used
[0.209856 ERROR]()(no module): Memory leak: SizeEntry(262144) is still used
[0.210580 ERROR]()(no module): Memory leak: SizeEntry(2097152) is still used
[0.211240 ERROR]()(no module): DescriptorAllocator is dropped
error: process didn't exit successfully: `target\release\test-project.exe` (exit code: 101)
PS C:\Users\jonat\Projects\test-project> 

env_logger::init();:

PS C:\Users\jonat\Projects\test-project> cargo run
   Compiling test-project v0.1.0 (C:\Users\jonat\Projects\test-project)
    Finished dev [unoptimized + debuginfo] target(s) in 3.85s
     Running `target\debug\test-project.exe`
Started
x.len(): 16
internal buffer byte size: 32
[2020-10-14T20:28:41Z ERROR gfx_backend_dx12::device] D3DCompile error 80004005
thread 'main' panicked at 'Compute pipeline creation error: Err(Shader(CompilationFailed("C:\\Users\\jonat\\Projects\\test-project\\Shader@0x0000024DFD815FF0(23,17-25): warning X3556: integer divides may be much slower, try using uints if possible.\nC:\\Users\\jonat\\Projects\\test-project\\Shader@0x0000024DFD815FF0(32,17-31): error X4010: Unsigned integer divide by zero\nC:\\Users\\jonat\\Projects\\test-project\\Shader@0x0000024DFD815FF0(32,17-31): warning X3556: integer divides may be much slower, try using uints if possible.\n\u{0}")))', C:\Users\jonat\.rustup\toolchains\stable-x86_64-pc-windows-msvc\lib/rustlib/src/rust\src\libstd\macros.rs:16:9
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
[2020-10-14T20:28:41Z ERROR gfx_memory::heaps] Heaps not completely freed before drop. Utilization: MemoryHeapUtilization { utilization: MemoryUtilization { used: 4194304, effective: 196608 }, size: 7701849907 }
[2020-10-14T20:28:41Z ERROR gfx_memory::allocator::general] Memory leak: SizeEntry(65536) is still used
[2020-10-14T20:28:41Z ERROR gfx_memory::allocator::general] Memory leak: SizeEntry(524288) is still used
[2020-10-14T20:28:41Z ERROR gfx_memory::allocator::general] Memory leak: SizeEntry(131072) is still used
[2020-10-14T20:28:41Z ERROR gfx_memory::allocator::general] Memory leak: SizeEntry(1048576) is still used
[2020-10-14T20:28:41Z ERROR gfx_memory::allocator::general] Memory leak: SizeEntry(262144) is still used
[2020-10-14T20:28:41Z ERROR gfx_memory::allocator::general] Memory leak: SizeEntry(2097152) is still used
[2020-10-14T20:28:41Z ERROR gfx_descriptor::allocator] DescriptorAllocator is dropped
error: process didn't exit successfully: `target\debug\test-project.exe` (exit code: 101)
PS C:\Users\jonat\Projects\test-project> 

[kvark]

Bummer, it's my bad. I got confused about the backends again. I keep thinking that it's the D3D12 you are having issues with... Sorry! And I appreciate you bearing with me here 👍

So, testing your latest code on Vulkan, I get the following validation error:

VALIDATION [UNASSIGNED-CoreValidation-Shader-InconsistentSpirv (7060244)] : Validation Error: [ UNASSIGNED-CoreValidation-Shader-InconsistentSpirv ] Object 0: handle = 0x558a2059b260, type = VK_OBJECT_TYPE_DEVICE; | MessageID = 0x6bbb14 | SPIR-V module not valid: Invalid capability operand: 6033

object info: (type: DEVICE, hndl: 94051736597088)

This would perfectly explain the "failed to create shader module" error you are having. I wonder why you are not seeing this error? Maybe your validation layers are older?

[JonathanWoollett-Light]

Maybe your validation layers are older?

How would I check this?

Just to be as clear as possible, when running the project (cargo run) right now using env_logger::init(); and wgpu::BackendBit::VULKAN I get (worth being clear for this output I haven't done anything else other than simply add the line env_logger::init();):

PS C:\Users\jonat\Projects\test-project> cargo run
   Compiling test-project v0.1.0 (C:\Users\jonat\Projects\test-project)
    Finished dev [unoptimized + debuginfo] target(s) in 3.90s
     Running `target\debug\test-project.exe`
Started
x.len(): 16
internal buffer byte size: 32
thread 'main' panicked at 'failed to create shader module: Shader error: Compilation failed: ', C:\Users\jonat\.rustup\toolchains\stable-x86_64-pc-windows-msvc\lib/rustlib/src/rust\src\libstd\macros.rs:16:9
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
[2020-10-14T20:53:50Z ERROR gfx_memory::heaps] Heaps not completely freed before drop. Utilization: MemoryHeapUtilization { utilization: MemoryUtilization { used: 4194304, effective: 768 }, size: 268435456 }
[2020-10-14T20:53:50Z ERROR gfx_memory::allocator::general] Memory leak: SizeEntry(256) is still used
[2020-10-14T20:53:50Z ERROR gfx_memory::allocator::general] Memory leak: SizeEntry(2048) is still used
[2020-10-14T20:53:50Z ERROR gfx_memory::allocator::general] Memory leak: SizeEntry(16384) is still used
[2020-10-14T20:53:50Z ERROR gfx_memory::allocator::general] Memory leak: SizeEntry(131072) is still used
[2020-10-14T20:53:50Z ERROR gfx_memory::allocator::general] Memory leak: SizeEntry(32768) is still used
[2020-10-14T20:53:50Z ERROR gfx_memory::allocator::general] Memory leak: SizeEntry(262144) is still used
[2020-10-14T20:53:50Z ERROR gfx_memory::allocator::general] Memory leak: SizeEntry(65536) is still used
[2020-10-14T20:53:50Z ERROR gfx_memory::allocator::general] Memory leak: SizeEntry(524288) is still used
[2020-10-14T20:53:50Z ERROR gfx_memory::allocator::general] Memory leak: SizeEntry(1048576) is still used
[2020-10-14T20:53:50Z ERROR gfx_memory::allocator::general] Memory leak: SizeEntry(2097152) is still used
[2020-10-14T20:53:50Z ERROR gfx_descriptor::allocator] DescriptorAllocator is dropped
[2020-10-14T20:53:50Z ERROR gfx_backend_vulkan]
    VALIDATION [VUID-vkDestroyDevice-device-00378 (1901072314)] : Validation Error: [ VUID-vkDestroyDevice-device-00378 ] Object 0: handle = 0x1707cd81570, type = VK_OBJECT_TYPE_DEVICE; Object 1: handle = 0x3ba5830000000006, name = Storage Buffer Outputs, type = VK_OBJECT_TYPE_BUFFER; | MessageID = 0x71500fba | OBJ ERROR : For VkDevice 0x1707cd81570[], VkBuffer 0x3ba5830000000006[Storage Buffer Outputs] has not been destroyed. The Vulkan spec states: All child objects created on device must have been destroyed prior to destroying device (https://vulkan.lunarg.com/doc/view/1.2.148.0/windows/1.2-extensions/vkspec.html#VUID-vkDestroyDevice-device-00378)
    object info: (type: DEVICE, hndl: 1582642500976), (type: BUFFER, hndl: 4297985455418245126, name: Storage Buffer Outputs)

[2020-10-14T20:53:50Z ERROR gfx_backend_vulkan]
    VALIDATION [VUID-vkDestroyDevice-device-00378 (1901072314)] : Validation Error: [ VUID-vkDestroyDevice-device-00378 ] Object 0: handle = 0x1707cd81570, type = VK_OBJECT_TYPE_DEVICE; Object 1: handle = 0x983e60000000003, name = Storage Buffer X, type = VK_OBJECT_TYPE_BUFFER; | MessageID = 0x71500fba | OBJ ERROR : For VkDevice 0x1707cd81570[], VkBuffer 0x983e60000000003[Storage Buffer X] has not been destroyed. The Vulkan spec states: All child objects created on device must have been destroyed prior to destroying device (https://vulkan.lunarg.com/doc/view/1.2.148.0/windows/1.2-extensions/vkspec.html#VUID-vkDestroyDevice-device-00378)
    object info: (type: DEVICE, hndl: 1582642500976), (type: BUFFER, hndl: 685644455964835843, name: Storage Buffer X)

[2020-10-14T20:53:50Z ERROR gfx_backend_vulkan]
    VALIDATION [VUID-vkDestroyDevice-device-00378 (1901072314)] : Validation Error: [ VUID-vkDestroyDevice-device-00378 ] Object 0: handle = 0x1707cd81570, type = VK_OBJECT_TYPE_DEVICE; Object 1: handle = 0x6dc7200000000005, name = Storage Buffer Internal, type = VK_OBJECT_TYPE_BUFFER; | MessageID = 0x71500fba | OBJ ERROR : For VkDevice 0x1707cd81570[], VkBuffer 0x6dc7200000000005[Storage Buffer Internal] has not been destroyed. The Vulkan spec states: All child objects created on device must have been destroyed prior to destroying device (https://vulkan.lunarg.com/doc/view/1.2.148.0/windows/1.2-extensions/vkspec.html#VUID-vkDestroyDevice-device-00378)
    object info: (type: DEVICE, hndl: 1582642500976), (type: BUFFER, hndl: 7910326454871654405, name: Storage Buffer Internal)

[2020-10-14T20:53:50Z ERROR gfx_backend_vulkan]
    VALIDATION [VUID-vkDestroyDevice-device-00378 (1901072314)] : Validation Error: [ VUID-vkDestroyDevice-device-00378 ] Object 0: handle = 0x1707cd81570, type = VK_OBJECT_TYPE_DEVICE; Object 1: handle = 0x5d11410000000004, type = VK_OBJECT_TYPE_DEVICE_MEMORY; | MessageID = 0x71500fba | OBJ ERROR : For VkDevice 0x1707cd81570[], VkDeviceMemory 0x5d11410000000004[] has not been destroyed. The Vulkan spec states: All child objects created on device must have been destroyed prior to destroying device (https://vulkan.lunarg.com/doc/view/1.2.148.0/windows/1.2-extensions/vkspec.html#VUID-vkDestroyDevice-device-00378)
    object info: (type: DEVICE, hndl: 1582642500976), (type: DEVICE_MEMORY, hndl: 6706212788387184644)

[2020-10-14T20:53:50Z ERROR gfx_backend_vulkan] 
    VALIDATION [VUID-vkDestroyDevice-device-00378 (1901072314)] : Validation Error: [ VUID-vkDestroyDevice-device-00378 ] Object 0: handle = 0x1707cd81570, type = VK_OBJECT_TYPE_DEVICE; Object 1: handle = 0x4c5b620000000007, type = VK_OBJECT_TYPE_DESCRIPTOR_SET_LAYOUT; | MessageID = 0x71500fba | OBJ ERROR : For VkDevice 0x1707cd81570[], VkDescriptorSetLayout 0x4c5b620000000007[] has not been destroyed. The Vulkan spec states: All child objects created on device must have been destroyed prior to destroying device (https://vulkan.lunarg.com/doc/view/1.2.148.0/windows/1.2-extensions/vkspec.html#VUID-vkDestroyDevice-device-00378)
    object info: (type: DEVICE, hndl: 1582642500976), (type: DESCRIPTOR_SET_LAYOUT, hndl: 5502099121902714887)

[2020-10-14T20:53:50Z ERROR gfx_backend_vulkan]
    VALIDATION [VUID-vkDestroyDevice-device-00378 (1901072314)] : Validation Error: [ VUID-vkDestroyDevice-device-00378 ] Object 0: handle = 0x1707cd81570, type = VK_OBJECT_TYPE_DEVICE; Object 1: handle = 0xa540ac0000000009, type = VK_OBJECT_TYPE_DESCRIPTOR_SET; | MessageID = 0x71500fba | OBJ ERROR : For VkDevice 0x1707cd81570[], VkDescriptorSet 0xa540ac0000000009[] has not been destroyed. The Vulkan spec states: All child objects created on device must have been destroyed prior to destroying device (https://vulkan.lunarg.com/doc/view/1.2.148.0/windows/1.2-extensions/vkspec.html#VUID-vkDestroyDevice-device-00378)
    object info: (type: DEVICE, hndl: 1582642500976), (type: DESCRIPTOR_SET, hndl: 11907706530767568905)

[2020-10-14T20:53:50Z ERROR gfx_backend_vulkan]
    VALIDATION [VUID-vkDestroyDevice-device-00378 (1901072314)] : Validation Error: [ VUID-vkDestroyDevice-device-00378 ] Object 0: handle = 0x1707cd81570, type = VK_OBJECT_TYPE_DEVICE; Object 1: handle = 0x948acd0000000008, type = VK_OBJECT_TYPE_DESCRIPTOR_POOL; | MessageID = 0x71500fba | OBJ ERROR : For VkDevice 0x1707cd81570[], VkDescriptorPool 0x948acd0000000008[] has not been destroyed. The Vulkan spec states: All child objects created on device must have been destroyed prior to destroying device (https://vulkan.lunarg.com/doc/view/1.2.148.0/windows/1.2-extensions/vkspec.html#VUID-vkDestroyDevice-device-00378)
    object info: (type: DEVICE, hndl: 1582642500976), (type: DESCRIPTOR_POOL, hndl: 10703592864283099144)

[2020-10-14T20:53:50Z ERROR gfx_backend_vulkan]
    VALIDATION [VUID-vkDestroyDevice-device-00378 (1901072314)] : Validation Error: [ VUID-vkDestroyDevice-device-00378 ] Object 0: handle = 0x1707cd81570, type = VK_OBJECT_TYPE_DEVICE; Object 1: handle = 0xf8ce070000000002, type = VK_OBJECT_TYPE_COMMAND_POOL; | MessageID = 0x71500fba | OBJ ERROR : For VkDevice 0x1707cd81570[], VkCommandPool 0xf8ce070000000002[] has not been destroyed. The Vulkan spec states: All child objects created on device must have been destroyed prior to destroying device (https://vulkan.lunarg.com/doc/view/1.2.148.0/windows/1.2-extensions/vkspec.html#VUID-vkDestroyDevice-device-00378)
    object info: (type: DEVICE, hndl: 1582642500976), (type: COMMAND_POOL, hndl: 17928274863189917698)

error: process didn't exit successfully: `target\debug\test-project.exe` (exit code: 101)
PS C:\Users\jonat\Projects\test-project> 

[kvark]

Here are possible things for you to do:

1. install the latest Vulkan LunarG SDK, it has the validation layers
2. build/install https://github.com/KhronosGroup/SPIRV-Tools and just plain try to validate the SPIR-V you are feeding

[JonathanWoollett-Light]

1. install the latest Vulkan LunarG SDK, it has the validation layers

So I 've had this installed for quite a while, there are a bunch of configs for validation layers and I'm not sure what is relevant. Is there a particular validaiton layer config to go for?

The configs:

2. build/install https://github.com/KhronosGroup/SPIRV-Tools and just plain try to validate the SPIR-V you are feeding

So downloading the windows release from here and running the validator spriv-val.exe leads to:

C:\Users\jonat>C:\Users\jonat\Desktop\SPIRV-Tools-master-windows-x64-Release\bin\spirv-val.exe C:\Users\jonat\Projects\test-project\spir-v\atomicArraySum.spv
error: line 8: Invalid capability operand: 6033
error: line 8: Invalid capability operand: 6033

C:\Users\jonat>

I have no idea what these errors mean or what to do with them (and I can't find any reference or explanation).

[kvark]

Is there a particular validaiton layer config to go for?

This all is relatively new. The "Standard" should work, at least.

I have no idea what these errors mean or what to do with them (and I can't find any reference or explanation).

Ok, so I think we cleared up the unknowns by now. If you are wondering why wgpu fails to create a shader module, the answer is that the SPIR-V you feed is invalid. wgpu doesn't intend to validate SPIR-V, but it will validate WGSL when it's ready. You can't expect wgpu to work correctly if the shaders on the input are invalid. wgpu doesn't know or care about the way you got the SPIR-V (i.e. about your GLSL and such).

The tool you used to produce SPIR-V (glslc) has a bug, since it generated invalid SPIR-V. This module uses the capability CapabilityAtomicFloat32AddEXT that requires an extension, and when you request it to generate a 1.0 module, the logic gets confused. You can file it on their repository, wherever it is.

From your side, stripping the code of atomic adds should let the tool to produce the module without that capability required.

[JonathanWoollett-Light]

The tool you used to produce SPIR-V (glslc) has a bug, since it generated invalid SPIR-V. This module uses the capability CapabilityAtomicFloat32AddEXT that requires an extension, and when you request it to generate a 1.0 module, the logic gets confused. You can file it on their repository, wherever it is.

So glslc.exe is a part of VulkanSDK (C:/VulkanSDK/1.2.148.1/Bin/glslc).

I can't find where to report bugs for VulkanSDK, any idea?

[kvark]

According to https://github.com/google/shaderc:

glslc wraps around core functionality in glslang and SPIRV-Tools.

So it's either shaderc repo or https://github.com/KhronosGroup/glslang

[dneto0]

Issue google/shaderc#1140 was filed against Shaderc. The root cause is that the copy of spirv-val that you have is out of date, and did not have information about the SPV_EXT_shader_atomic_float_add extension.

Verison 1.2.154 of the Vulkan SDK was released in early October, and should have an update spirv-val.

[JonathanWoollett-Light]

@kvark Does #564 (comment) seem like something that makes sense that could be fixed?

[grovesNL]

@JonathanWoollett-Light wgpu doesn't include spirv-val. This comment means you would need to update spirv-val locally (i.e. probably updating Vulkan SDK, or however spirv-val is installed on your computer)

[JonathanWoollett-Light]

@grovesNL So I updated Vulkand SDK from 1.2.148.1 to 1.2.154, and used:

C:/VulkanSDK/1.2.154.1/Bin/glslc C:/Users/jonat/Projects/test-project/glsl/atomicArraySum.comp -o C:/Users/jonat/Projects/test-project/spir-v/atomicArraySum.spv --target-env=vulkan1.1

to recompile the shader in the test project.

The error given by attempting to run the program (cargo run) remains unchanged:

thread 'main' panicked at 'failed to create shader module: Shader error: Compilation failed: ', C:\Users\jonat\.rustup\toolchains\stable-x86_64-pc-windows-msvc\lib/rustlib/src/rust\src\libstd\macros.rs:16:9

The errors given by the validator also remain unchanged:

C:\Users\jonat>C:\Users\jonat\Desktop\SPIRV-Tools-master-windows-x64-Release\bin\spirv-val.exe C:\Users\jonat\Projects\test-project\spir-v\atomicArraySum.spv
error: line 8: Invalid capability operand: 6033
error: line 8: Invalid capability operand: 6033

This doesn't seem to have had any affect, am I missing something here?

[kvark]

@dneto0 is there an issue to track with regards to SPV_EXT_shader_atomic_float_add support in spirv-val?

[JonathanWoollett-Light]

@kvark I have no idea where this issue stands right now. Where does the issue lie? Is it that webgpu is using an outdated version of something? (@dneto0 takes quite a while to reply from past experience)

[kvark]

@JonathanWoollett-Light wgpu doesn't use spirv-val for the Vulkan backend. We pretty much pass-through the SPIR-V shader you give us to the driver, plus doing reflection to see what it does. Then the validation errors catch the problem with SPIR-V shader, so it's about how you are generating it. @dneto0 noted that your way of producing SPIR-V indirectly relies on spirv-val with a bug that was recently fixed. I asked for an upstream issue to get more information...

Interestingly, google/shaderc#1140 (comment) reports no evidence of the issue on latest toolchain. So maybe your experiment didn't do the same thing?

If I were you, I'd probably start cutting out the stuff from the shader (especially use of new extensions) to see where it starts to break, just to get a better understanding of the problem. In any way, this will only become wgpu issue once SPIR-V validation succeeds on those shader modules.

[JonathanWoollett-Light]

@kvark So in updating to use the latest https://github.com/KhronosGroup/SPIRV-Tools download version using spir_val and glslc gives the outputs:

C:\Users\jonat>C:/SPIRV-Tools/bin/spirv-val --version
SPIRV-Tools v2020.6 v2020.5-89-gf7da5277
Targets:
  SPIR-V 1.0
  SPIR-V 1.1
  SPIR-V 1.2
  SPIR-V 1.3
  SPIR-V 1.4
  SPIR-V 1.5
  SPIR-V 1.2 (under OpenCL 2.2 Full Profile semantics)
  SPIR-V 1.0 (under Vulkan 1.0 semantics)
  SPIR-V 1.3 (under Vulkan 1.1 semantics)
  SPIR-V 1.4 (under Vulkan 1.1 semantics)
  SPIR-V 1.3 (under WIP WebGPU semantics)

C:\Users\jonat>C:/SPIRV-Tools/bin/glslc --version
shaderc v2020.4-dev v2020.3-5-gfcafd5b
spirv-tools v2020.6 v2020.5-89-gf7da5277
glslang 10-11.0.0-6-g142cb87f

Target: SPIR-V 1.0

C:\Users\jonat>C:/SPIRV-Tools/bin/glslc C:/Users/jonat/Projects/test-project/glsl/atomicArraySum.comp -o C:/Users/jonat/Projects/test-project/spir-v/atomicArraySum.spv --target-env=vulkan1.1

C:\Users\jonat>C:/SPIRV-Tools/bin/spirv-val C:/Users/jonat/Projects/test-project/spir-v/atomicArraySum.spv

C:\Users\jonat>C:/SPIRV-Tools/bin/glslc C:/Users/jonat/Projects/test-project/glsl/atomicArraySum.comp -o C:/Users/jonat/Projects/test-project/spir-v/atomicArraySum.spv --target-env=vulkan1.0

C:\Users\jonat>C:/SPIRV-Tools/bin/spirv-val C:/Users/jonat/Projects/test-project/spir-v/atomicArraySum.spv

C:\Users\jonat>C:/SPIRV-Tools/bin/glslc C:/Users/jonat/Projects/test-project/glsl/atomicArraySum.comp -o C:/Users/jonat/Projects/test-project/spir-v/atomicArraySum.spv

C:\Users\jonat>C:/SPIRV-Tools/bin/spirv-val C:/Users/jonat/Projects/test-project/spir-v/atomicArraySum.spv

Assuming I have done this correctly, the shader has been correctly compiled, does this not then imply an error in wgpu?

If I were you, I'd probably start cutting out the stuff from the shader (especially use of new extensions) to see where it starts to break

It is any usage of atomicAdd which causes the errors.

[kvark]

Whan you are passing this shader to wgpu, do you get any validation errors (like the one in #564 (comment))?

[JonathanWoollett-Light]

@kvark Yes, I believe the exact same validation errors as in #564 (comment).

PS C:\Users\jonat\Projects\test-project> cargo run
   Compiling test-project v0.1.0 (C:\Users\jonat\Projects\test-project)
    Finished dev [unoptimized + debuginfo] target(s) in 3.67s
     Running `target\debug\test-project.exe`
Started
x.len(): 16
internal buffer byte size: 32
thread 'main' panicked at 'failed to create shader module: Shader error: Compilation failed: ', C:\Users\jonat\.rustup\toolchains\stable-x86_64-pc-windows-msvc\lib/rustlib/src/rust\src\libstd\macros.rs:16:9
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
[2020-11-12T00:01:54Z ERROR gfx_memory::heaps] Heaps not completely freed before drop. Utilization: MemoryHeapUtilization { utilization: MemoryUtilization { used: 4194304, effective: 768 }, size: 268435456 }
[2020-11-12T00:01:54Z ERROR gfx_memory::allocator::general] Memory leak: SizeEntry(256) is still used
[2020-11-12T00:01:54Z ERROR gfx_memory::allocator::general] Memory leak: SizeEntry(2048) is still used
[2020-11-12T00:01:54Z ERROR gfx_memory::allocator::general] Memory leak: SizeEntry(16384) is still used
[2020-11-12T00:01:54Z ERROR gfx_memory::allocator::general] Memory leak: SizeEntry(131072) is still used
[2020-11-12T00:01:54Z ERROR gfx_memory::allocator::general] Memory leak: SizeEntry(32768) is still used
[2020-11-12T00:01:54Z ERROR gfx_memory::allocator::general] Memory leak: SizeEntry(262144) is still used
[2020-11-12T00:01:54Z ERROR gfx_memory::allocator::general] Memory leak: SizeEntry(65536) is still used
[2020-11-12T00:01:54Z ERROR gfx_memory::allocator::general] Memory leak: SizeEntry(524288) is still used
[2020-11-12T00:01:54Z ERROR gfx_memory::allocator::general] Memory leak: SizeEntry(1048576) is still used
[2020-11-12T00:01:54Z ERROR gfx_memory::allocator::general] Memory leak: SizeEntry(2097152) is still used
[2020-11-12T00:01:54Z ERROR gfx_descriptor::allocator] DescriptorAllocator is dropped
[2020-11-12T00:01:54Z ERROR gfx_backend_vulkan]
    VALIDATION [VUID-vkDestroyDevice-device-00378 (1901072314)] : Validation Error: [ VUID-vkDestroyDevice-device-00378 ] Object 0: handle = 0x1b41edac230, type = VK_OBJECT_TYPE_DEVICE; Object 1: handle = 0x3ba5830000000006, name = Storage Buffer Outputs, type = VK_OBJECT_TYPE_BUFFER; | MessageID = 0x71500fba | OBJ ERROR : For VkDevice 0x1b41edac230[], VkBuffer 0x3ba5830000000006[Storage Buffer Outputs] has not been destroyed. The Vulkan spec states: All child objects created on device must have been destroyed prior to destroying device (https://vulkan.lunarg.com/doc/view/1.2.154.1/windows/1.2-extensions/vkspec.html#VUID-vkDestroyDevice-device-00378)
    object info: (type: DEVICE, hndl: 1873123394096), (type: BUFFER, hndl: 4297985455418245126, name: Storage Buffer Outputs)

[2020-11-12T00:01:54Z ERROR gfx_backend_vulkan]
    VALIDATION [VUID-vkDestroyDevice-device-00378 (1901072314)] : Validation Error: [ VUID-vkDestroyDevice-device-00378 ] Object 0: handle = 0x1b41edac230, type = VK_OBJECT_TYPE_DEVICE; Object 1: handle = 0x983e60000000003, name = Storage Buffer X, 
type = VK_OBJECT_TYPE_BUFFER; | MessageID = 0x71500fba | OBJ ERROR : For VkDevice 0x1b41edac230[], VkBuffer 0x983e60000000003[Storage Buffer X] has not been destroyed. The Vulkan spec states: All child objects created on device must have been destroyed prior to destroying device (https://vulkan.lunarg.com/doc/view/1.2.154.1/windows/1.2-extensions/vkspec.html#VUID-vkDestroyDevice-device-00378)
    object info: (type: DEVICE, hndl: 1873123394096), (type: BUFFER, hndl: 685644455964835843, name: Storage Buffer X)

[2020-11-12T00:01:54Z ERROR gfx_backend_vulkan]
    VALIDATION [VUID-vkDestroyDevice-device-00378 (1901072314)] : Validation Error: [ VUID-vkDestroyDevice-device-00378 ] Object 0: handle = 0x1b41edac230, type = VK_OBJECT_TYPE_DEVICE; Object 1: handle = 0x6dc7200000000005, name = Storage Buffer Internal, type = VK_OBJECT_TYPE_BUFFER; | MessageID = 0x71500fba | OBJ ERROR : For VkDevice 0x1b41edac230[], VkBuffer 0x6dc7200000000005[Storage Buffer Internal] has not been destroyed. The Vulkan spec states: All child objects created on device must 
have been destroyed prior to destroying device (https://vulkan.lunarg.com/doc/view/1.2.154.1/windows/1.2-extensions/vkspec.html#VUID-vkDestroyDevice-device-00378)
    object info: (type: DEVICE, hndl: 1873123394096), (type: BUFFER, hndl: 7910326454871654405, name: Storage Buffer Internal)

[2020-11-12T00:01:54Z ERROR gfx_backend_vulkan] 
    VALIDATION [VUID-vkDestroyDevice-device-00378 (1901072314)] : Validation Error: [ VUID-vkDestroyDevice-device-00378 ] Object 0: handle = 0x1b41edac230, type = VK_OBJECT_TYPE_DEVICE; Object 1: handle = 0x5d11410000000004, type = VK_OBJECT_TYPE_DEVICE_MEMORY; | MessageID = 0x71500fba | OBJ ERROR : For VkDevice 0x1b41edac230[], VkDeviceMemory 0x5d11410000000004[] has not been destroyed. The Vulkan spec states: All child objects created on device must have been destroyed prior to destroying device (https://vulkan.lunarg.com/doc/view/1.2.154.1/windows/1.2-extensions/vkspec.html#VUID-vkDestroyDevice-device-00378)
    object info: (type: DEVICE, hndl: 1873123394096), (type: DEVICE_MEMORY, hndl: 6706212788387184644)

[2020-11-12T00:01:54Z ERROR gfx_backend_vulkan]
    VALIDATION [VUID-vkDestroyDevice-device-00378 (1901072314)] : Validation Error: [ VUID-vkDestroyDevice-device-00378 ] Object 0: handle = 0x1b41edac230, type = VK_OBJECT_TYPE_DEVICE; Object 1: handle = 0x4c5b620000000007, type = VK_OBJECT_TYPE_DESCRIPTOR_SET_LAYOUT; | MessageID = 0x71500fba | OBJ ERROR : For VkDevice 0x1b41edac230[], VkDescriptorSetLayout 0x4c5b620000000007[] has not been destroyed. The Vulkan spec states: All child objects created on device must have been destroyed prior to destroying device (https://vulkan.lunarg.com/doc/view/1.2.154.1/windows/1.2-extensions/vkspec.html#VUID-vkDestroyDevice-device-00378)
    object info: (type: DEVICE, hndl: 1873123394096), (type: DESCRIPTOR_SET_LAYOUT, hndl: 5502099121902714887)

[2020-11-12T00:01:54Z ERROR gfx_backend_vulkan]
    VALIDATION [VUID-vkDestroyDevice-device-00378 (1901072314)] : Validation Error: [ VUID-vkDestroyDevice-device-00378 ] Object 0: handle = 0x1b41edac230, type = VK_OBJECT_TYPE_DEVICE; Object 1: handle = 0xa540ac0000000009, type = VK_OBJECT_TYPE_DESCRIPTOR_SET; | MessageID = 0x71500fba | OBJ ERROR : For VkDevice 0x1b41edac230[], VkDescriptorSet 0xa540ac0000000009[] has not been destroyed. The Vulkan spec states: All child objects created on device must have been destroyed prior to destroying 
device (https://vulkan.lunarg.com/doc/view/1.2.154.1/windows/1.2-extensions/vkspec.html#VUID-vkDestroyDevice-device-00378)
    object info: (type: DEVICE, hndl: 1873123394096), (type: DESCRIPTOR_SET, hndl: 11907706530767568905)

[2020-11-12T00:01:54Z ERROR gfx_backend_vulkan]
    VALIDATION [VUID-vkDestroyDevice-device-00378 (1901072314)] : Validation Error: [ VUID-vkDestroyDevice-device-00378 ] Object 0: handle = 0x1b41edac230, type = VK_OBJECT_TYPE_DEVICE; Object 1: handle = 0x948acd0000000008, type = VK_OBJECT_TYPE_DESCRIPTOR_POOL; | MessageID = 0x71500fba | OBJ ERROR : For VkDevice 0x1b41edac230[], VkDescriptorPool 0x948acd0000000008[] has not been destroyed. The Vulkan spec states: All child objects created on device must have been destroyed prior to destroying device (https://vulkan.lunarg.com/doc/view/1.2.154.1/windows/1.2-extensions/vkspec.html#VUID-vkDestroyDevice-device-00378)
    object info: (type: DEVICE, hndl: 1873123394096), (type: DESCRIPTOR_POOL, hndl: 10703592864283099144)

[2020-11-12T00:01:54Z ERROR gfx_backend_vulkan]
    VALIDATION [VUID-vkDestroyDevice-device-00378 (1901072314)] : Validation Error: [ VUID-vkDestroyDevice-device-00378 ] Object 0: handle = 0x1b41edac230, type = VK_OBJECT_TYPE_DEVICE; Object 1: handle = 0xf8ce070000000002, type = VK_OBJECT_TYPE_COMMAND_POOL; | MessageID = 0x71500fba | OBJ ERROR : For VkDevice 0x1b41edac230[], VkCommandPool 0xf8ce070000000002[] has not been destroyed. The Vulkan spec states: All child objects created on device must have been destroyed prior to destroying device (https://vulkan.lunarg.com/doc/view/1.2.154.1/windows/1.2-extensions/vkspec.html#VUID-vkDestroyDevice-device-00378)
    object info: (type: DEVICE, hndl: 1873123394096), (type: COMMAND_POOL, hndl: 17928274863189917698)

error: process didn't exit successfully: `target\debug\test-project.exe` (exit code: 101)
PS C:\Users\jonat\Projects\test-project>

[kvark]

No, these are unrelated. I was asking about "CoreValidation-Shader-InconsistentSpirv", and it looks like you are no longer getting it, yet the shader still fails to compile. Can you share the updated test case/app?

[JonathanWoollett-Light]

Updated project: test-project.zip

[kvark]

Thank you! I had another look at this. The problem is that the SPIRV shader you are feeding to us requires AtomicFloat32AddEXT capability. This can only be enabled by VK_EXT_shader_atomic_float extension that we are neither requesting or enabling today.

The Vulkan validation layers are not comprehensive, and they miss this problem. But it's still invalid usage of Vulkan.

Notably, the WebGPU group hasn't discussed anything with regards to floating point atomics yet. We could expose a native-only feature for this capability.

[JonathanWoollett-Light]

Notably, the WebGPU group hasn't discussed anything with regards to floating point atomics yet. We could expose a native-only feature for this capability.

I believe it is quite a fundemental component required for many compute operations. It would be great if it could be used.

[kvark]

Filed gfx-rs/wgpu#1020 and added an entry to https://github.com/gfx-rs/wgpu/wiki/Extensions
Thank you for your patience and all the help investigating this!

[JonathanWoollett-Light]

@kvark Could you guess at any eta for gfx-rs/wgpu#1020? (Apologies if this comes off as rude it would just be useful to know)

[kvark]

@JonathanWoollett-Light there is no ETA. It's a feature that hasn't been requested before, and not in the spec, so it's a classy thing for "scratch the itch"-driven development, where somebody who needs it contributes the feature and maintains it (there isn't much to maintain here, fortunately).
Since the change involves gfx-rs changes, I suggest to wait for gfx-rs/wgpu#1027 first, which should happen next week.