Validation error: "Semaphore must not have any pending operations"

[TP-David]

Launching vsgdraw in vsgExamples with debugLayer=true prints this validation error multiple times, then it goes away:

Validation Error: [ VUID-vkAcquireNextImageKHR-semaphore-01779 ] | MessageID = 0x5717e75b
vkAcquireNextImageKHR(): Semaphore must not have any pending operations.
The Vulkan spec states: If semaphore is not VK_NULL_HANDLE, it must not have any uncompleted signal or wait operations pending (https://vulkan.lunarg.com/doc/view/1.4.309.0/windows/antora/spec/latest/chapters/VK_KHR_surface/wsi.html#VUID-vkAcquireNextImageKHR-semaphore-01779)
Objects: 1
    [0] VkSemaphore 0xb12fb2000000002c

Rolling back to v1.1.2 in March 2024 still gives the error, showing that this has been around for a long time.
This is not solved by #1559.

GPU is NVIDIA RTX 2000 Ada Generation Laptop GPU, on other hardware the error doesn't appear.

Is this a known issue @robertosfield?

[robertosfield]

I have just tried:

vsgdraw -d

And don't see any Vulkan Validation Layer errors. If I run with -a then I see the API Layer enabled so it looks like the Validation Layer should be active as it's enabled by the same WindowTraits(CommandLine&) constructor.

This is with VSG & vsgExamples master. Which version are you using?

[TP-David]

vsgExamples, VulkanSceneGraph and vsgXChange are all on master.

Machine is using Windows 11, our team doesn't see this on certain machines so it may be GPU specific.

[robertosfield]

What hardware and drivers are being used on the different machines that do/don't show this validation error?

[AnyOldName3]

This looks very similar to #1558, but with a slightly different message ID and VUID, so while it might have a related root cause, I wouldn't have expected #1559 to fix it.

Like with #1558, I'm pretty sure the reason Robert's not hitting it is that he's running Kubuntu 24.04 (unless that's changed recently), and that provides version 1.3.275.0 of the Vulkan SDK, whereas, looking at your logs, it seems you're running 1.4.309.0, which has more checks. Hardware and drivers shouldn't make a difference to which validation errors are emitted (unless the application changes behaviour based on the hardware or tries to do things that the hardware doesn't support like using absent extensions), but the version of the Vulkan SDK absolutely can. I guess maybe timings, which could be device-specific, might make a difference to validation errors, but usually in my experience, the validation layers are pretty good at consistently emitting errors that are theoretically dependent on timing.

The version you're using doesn't yet have the check that #1559 fixes, so I wouldn't expect it to make any difference to validation errors on your machine, just potentially eliminate synchronisation bugs (that don't even seem to happen with desktop Vulkan implementations).

On my machine, where I'm using the 1.4.321.1 SDK, I don't see the validation error you're getting, so there's a possibility that there was a check added after 1.3.275.0 but before 1.4.309.0 that had a false positive that was fixed before 1.4.321.1. I'd suggest maybe trying the latest version (1.4.328.1 at time of writing) and seeing what it detects. I'd also suggest checking which versions of the SDK the machines that you've got available are using in case you find that there's an obvious version range where the error is emitted.

[TP-David]

I do get varying results with different Vulkan SDK versions using vsgdraw -d, I'm changing the path to VkLayer_khronos_validation.json using regedit in location Computer\HKEY_LOCAL_MACHINE\SOFTWARE\Khronos\Vulkan\ExplicitLayers:

1.3.268.0: No validation errors
1.3.283.0: Constantly prints VUID-vkAcquireNextImageKHR-semaphore-01779 without stopping
1.4.309.0: See image above, errors stop after a second or so
1.4.328.1: Along with VUID-vkAcquireNextImageKHR-semaphore-01779 for a few seconds like 1.4.309.0, VUID-vkQueueSubmit-pSignalSemaphores-00067 is also printed a couple of times. If I switch to the AnyOldName3-fix-semaphore-reuse branch, the latter is fixed.

[AnyOldName3]

On my machine, the validation layers default to printing an indentical error only ten times, so that might be part of why the errors stop after a while.

By the way, if you find editing the registry key a pain, if you run the Vulkan Configurator that comes with the SDK, it's got a dropdown to pick from the versions of the validation layers that you've got installed.

[TP-David]

@AnyOldName3 I have a proposed fix for this error (01779), by creating a semaphore for each image, do you have any input about potential drawbacks?

See commit: b4d04d5

[AnyOldName3]

As I don't remember noticing anything odd with the image acquisition semaphores when I was investigating semaphore issues a few months ago, and can't reproduce the validation errors with a newer Vulkan SDK, I'm still not entirely sure what the failure case is, and therefore what needs doing to prevent it. All the calls I logged showed that the image acquisition semaphores were always waited on before another call was made that would signal them, and that a fence was waited for that couldn't be triggered until after the semaphore's wait had finished before the semaphore was reused. If I had to guess what the cause is, I'd assume that it's something flaky in the mechanism that ensures the right fence is blocking the right semaphore, and if that's the problem, then the fix is going to involve connecting those together more effectively rather than just adding more semaphores, otherwise all it can do is make the symptoms less likely rather than address the root cause.

I had some logging that would tell me the details when calls that interacted with Vulkan synchronisation happened without burying them among other calls like happens with API dumping, which would have been helpful for investigating this, but unfortunately, I don't seem to have saved the patch, so I can't send it to you.

[robertosfield]

@AnyOldName3 I have a proposed fix for this error (01779), by creating a semaphore for each image, do you have any input about potential drawbacks?

See commit: b4d04d5

@TP-David could you generate a PR with this commit and I'll do a review of it tomorrow morning. Thanks.

[robertosfield]

I have implemented a combination of the approach @AnyOldName3 took in your PR #1559 and @TP-David took in his [commit](commit: b4d04d5)commit: b4d04d5. Could you test the FrameSemaphores branch. Thanks.

[robertosfield]

I have gone ahead and merged with master: #1639

Merry Christmas!