Garbled framebuffer output on Nvidia

[fintelia]

Description
Running Terra on Nvidia GPUs produces garbled framebuffer output and some Vulkan validation errors.

This was first reported as fintelia/terra#13.

Repro steps
Either clone and run Terra or just use the trace file attached below.

Extra materials

trace.zip

Validation errors:

[UNASSIGNED-CoreValidation-DrawState-InvalidImageLayout] Validation Error: [ UNASSIGNED-CoreValidation-DrawState-InvalidImageLayout ] Object 0: handle = 0x7fb69a0af848, type = VK_OBJECT_TYPE_COMMAND_BUFFER; | MessageID = 0x4dae5635 | Submitted command buffer expects VkImage 0x22d000000022d[] (subresource: aspectMask 0x1 array layer 0, mip level 0) to be in layout VK_IMAGE_LAYOUT_GENERAL--instead, current layout is VK_IMAGE_LAYOUT_TRANSFER_DST_OPTIMAL.
[VUID-VkRenderPassBeginInfo-framebuffer-03210] Validation Error: [ VUID-VkRenderPassBeginInfo-framebuffer-03210 ] Object 0: handle = 0x8970000000897, type = VK_OBJECT_TYPE_RENDER_PASS; | MessageID = 0xcc018073 | VkRenderPassBeginInfo: Image view #0 created from an image with usage set as 0x11, but image info #0 used to create the framebuffer had usage set as 0x10 The Vulkan spec states: If framebuffer was created with a VkFramebufferCreateInfo::flags value that included VK_FRAMEBUFFER_CREATE_IMAGELESS_BIT, each element of the pAttachments member of a VkRenderPassAttachmentBeginInfo structure included in the pNext chain must be a VkImageView of an image created with a value of VkImageCreateInfo::usage equal to the usage member of the corresponding element of VkFramebufferAttachmentsCreateInfo::pAttachments used to create framebuffer (https://www.khronos.org/registry/vulkan/specs/1.2-extensions/html/vkspec.html#VUID-VkRenderPassBeginInfo-framebuffer-03210)
[VUID-VkRenderPassBeginInfo-framebuffer-03210] Validation Error: [ VUID-VkRenderPassBeginInfo-framebuffer-03210 ] Object 0: handle = 0x8970000000897, type = VK_OBJECT_TYPE_RENDER_PASS; | MessageID = 0xcc018073 | VkRenderPassBeginInfo: Image view #1 created from an image with usage set as 0x27, but image info #1 used to create the framebuffer had usage set as 0x20 The Vulkan spec states: If framebuffer was created with a VkFramebufferCreateInfo::flags value that included VK_FRAMEBUFFER_CREATE_IMAGELESS_BIT, each element of the pAttachments member of a VkRenderPassAttachmentBeginInfo structure included in the pNext chain must be a VkImageView of an image created with a value of VkImageCreateInfo::usage equal to the usage member of the corresponding element of VkFramebufferAttachmentsCreateInfo::pAttachments used to create framebuffer (https://www.khronos.org/registry/vulkan/specs/1.2-extensions/html/vkspec.html#VUID-VkRenderPassBeginInfo-framebuffer-03210)

Platform
Ubuntu 20.10
nvidia-driver-460
latest wgpu

[kvark]

Looks very similar to #1172

[kvark]

Running terra on Linux/NV with VVL 1.2.162.0, I'm seeing no validation errors.
I hooked up env_logger to it, and still the only thing I'm seeing is regular output, no validation issues.

Given the story of #1172, I wonder if the validation layers are just buggy? Alternatively, we are doing something unsafe and broken, but it somehow works.

[kvark]

As for your trace, it tries to create a swapchain of size 800x600, and then resizes to 1920x1080. This is impossible to replay on my machine atm. Perhaps, a trace without resizing could help, in case there are issues with debug output on terra for some reason.

[fintelia]

Here is a trace without the resize: trace.zip

On my machine, this results in the screenshot below. Notice that most 8x8 blocks have seemingly random noise, but the bottom row and a handful scattered throughout the rest of the image seem normal. The other image is what the output should look like (I use lower res textures so the trace file isn't 400MB...)

Image from Nvidia GPU:

Correct image (via swiftshader):

[fintelia]

Also to clarify, are you using the proprietary Nvidia drivers?

[kvark]

I'm using the proprietary driver, yes, and I can see the garbled output. I just don't see the validatation errors.
It could be that the validation checks are broken in latest VVL, but the garbled output itself is concerning, obviously. It's just not clear if we are doing anything wrong.

[kvark]

@fintelia could you run the wgpu-rs examples on that machine and see if any validation errors show up?

[fintelia]

I get these validations errors from the hello-triangle example, although the rendered output seems fine:

RDOC 036810: [11:17:45]          vk_core.cpp(3563) - Warning - [VUID-VkRenderPassBeginInfo-framebuffer-03210] Validation Error: [ VUID-VkRenderPassBeginInfo-framebuffer-03210 ] Object 0: handle = 0x1e300000001e3, type = VK_OBJECT_TYPE_RENDER_PASS; | MessageID = 0xcc018073 | VkRenderPassBeginInfo: Image view #0 created from an image with usage set as 0x17, but image info #0 used to create the framebuffer had usage set as 0x10 The Vulkan spec states: If framebuffer was created with a VkFramebufferCreateInfo::flags value that included VK_FRAMEBUFFER_CREATE_IMAGELESS_BIT, each element of the pAttachments member of a VkRenderPassAttachmentBeginInfo structure included in the pNext chain must be a VkImageView of an image created with a value of VkImageCreateInfo::usage equal to the usage member of the corresponding element of VkFramebufferAttachmentsCreateInfo::pAttachments used to create framebuffer (https://www.khronos.org/registry/vulkan/specs/1.2-extensions/html/vkspec.html#VUID-VkRenderPassBeginInfo-framebuffer-03210)
RDOC 036810: [11:17:45]          vk_core.cpp(3563) - Warning - [UNASSIGNED-CoreValidation-DrawState-InvalidImageLayout] Validation Error: [ UNASSIGNED-CoreValidation-DrawState-InvalidImageLayout ] Object 0: handle = 0x7f01f8a80798, type = VK_OBJECT_TYPE_COMMAND_BUFFER; | MessageID = 0x4dae5635 | Submitted command buffer expects VkImage 0x1d700000001d7[] (subresource: aspectMask 0x1 array layer 0, mip level 0) to be in layout VK_IMAGE_LAYOUT_GENERAL--instead, current layout is VK_IMAGE_LAYOUT_TRANSFER_DST_OPTIMAL.
RDOC 036810: [11:17:45]          vk_core.cpp(3563) - Warning - [VUID-VkRenderPassBeginInfo-framebuffer-03210] Validation Error: [ VUID-VkRenderPassBeginInfo-framebuffer-03210 ] Object 0: handle = 0x1e300000001e3, type = VK_OBJECT_TYPE_RENDER_PASS; | MessageID = 0xcc018073 | VkRenderPassBeginInfo: Image view #0 created from an image with usage set as 0x17, but image info #0 used to create the framebuffer had usage set as 0x10 The Vulkan spec states: If framebuffer was created with a VkFramebufferCreateInfo::flags value that included VK_FRAMEBUFFER_CREATE_IMAGELESS_BIT, each element of the pAttachments member of a VkRenderPassAttachmentBeginInfo structure included in the pNext chain must be a VkImageView of an image created with a value of VkImageCreateInfo::usage equal to the usage member of the corresponding element of VkFramebufferAttachmentsCreateInfo::pAttachments used to create framebuffer (https://www.khronos.org/registry/vulkan/specs/1.2-extensions/html/vkspec.html#VUID-VkRenderPassBeginInfo-framebuffer-03210)
RDOC 036810: [11:17:45]          vk_core.cpp(3563) - Warning - [UNASSIGNED-CoreValidation-DrawState-InvalidImageLayout] Validation Error: [ UNASSIGNED-CoreValidation-DrawState-InvalidImageLayout ] Object 0: handle = 0x7f01f8a9a038, type = VK_OBJECT_TYPE_COMMAND_BUFFER; | MessageID = 0x4dae5635 | Submitted command buffer expects VkImage 0x1d700000001d7[] (subresource: aspectMask 0x1 array layer 0, mip level 0) to be in layout VK_IMAGE_LAYOUT_GENERAL--instead, current layout is VK_IMAGE_LAYOUT_TRANSFER_DST_OPTIMAL.
RDOC 036810: [11:17:45]          vk_core.cpp(3563) - Warning - [VUID-VkRenderPassBeginInfo-framebuffer-03210] Validation Error: [ VUID-VkRenderPassBeginInfo-framebuffer-03210 ] Object 0: handle = 0x1e300000001e3, type = VK_OBJECT_TYPE_RENDER_PASS; | MessageID = 0xcc018073 | VkRenderPassBeginInfo: Image view #0 created from an image with usage set as 0x17, but image info #0 used to create the framebuffer had usage set as 0x10 The Vulkan spec states: If framebuffer was created with a VkFramebufferCreateInfo::flags value that included VK_FRAMEBUFFER_CREATE_IMAGELESS_BIT, each element of the pAttachments member of a VkRenderPassAttachmentBeginInfo structure included in the pNext chain must be a VkImageView of an image created with a value of VkImageCreateInfo::usage equal to the usage member of the corresponding element of VkFramebufferAttachmentsCreateInfo::pAttachments used to create framebuffer (https://www.khronos.org/registry/vulkan/specs/1.2-extensions/html/vkspec.html#VUID-VkRenderPassBeginInfo-framebuffer-03210)
RDOC 036810: [11:17:45]          vk_core.cpp(3563) - Warning - [UNASSIGNED-CoreValidation-DrawState-InvalidImageLayout] Validation Error: [ UNASSIGNED-CoreValidation-DrawState-InvalidImageLayout ] Object 0: handle = 0x7f01f93a9618, type = VK_OBJECT_TYPE_COMMAND_BUFFER; | MessageID = 0x4dae5635 | Submitted command buffer expects VkImage 0x1d700000001d7[] (subresource: aspectMask 0x1 array layer 0, mip level 0) to be in layout VK_IMAGE_LAYOUT_GENERAL--instead, current layout is VK_IMAGE_LAYOUT_TRANSFER_DST_OPTIMAL.

I should add that I'm getting these logs by capturing a frame in RenderDoc and seeing what validation issues it reports.

[kvark]

@fintelia could you please investigate a tiny bit further: check the Vulkan objects in that RenderDoc capture, and see if the descriptor values for image usage are indeed matching what validation reports. I'm not seeing us doing the wrong here in the code, so I'm a bit puzzled.

[kvark]

Perhaps, you could share the capture here (zipped) as well, so that I can look at it and see if anything is different.

[fintelia]

Here's the RenderDoc capture: hello-triangle-capture.zip.

I'm not 100% confident, but as best I can tell the swapchain usages seem to be right.