AMD renoir/Linux/wayland: bgfx vulkan backend does not start and falls back to opengl

[belegdol]

Hello,
I have originally reported this against mesa:
https://gitlab.freedesktop.org/mesa/mesa/-/issues/4727
In brief, starting with mesa-21.0 (Fedora 34), mame will crash if one attempts to use the bgfx vulkan backend (./mame -bgfx_backend vulkan -bgfx_debug -video bgfx). With bgfx synced to upstream (PR #8035), bgfx does not even start the vulkan backend and falls back to opengl.
This can be worked around by using MESA_VK_WSI_PRESENT_MODE=relaxed. I have checked bgfx examples and these appear to run on vulkan without the need for the environmental variable mentioned earlier.

[belegdol]

As of 28bbea0 the crash is gone. As with my PR, without any environmental variables set, vulkan backend does not start and bgfx falls back to opengl.

[belegdol]

For easy reference, the feedback from mesa devs was that the mame usage was invalid.

[belegdol]

#8469 does not improve this. It did not advertise it would, but we could have been lucky.

[belegdol]

Running with debugging enabled shows the following errors at the time of vulkan -> gl fallback:

../../../../../3rdparty/bgfx/src/renderer_vk.cpp (7256): BGFX findSurfaceFormat: Surface format RGBA8 not found! Defaulting to BGRA8.
../../../../../3rdparty/bgfx/src/renderer_vk.cpp (6911): BGFX Create swapchain error: vkGetSwapchainImagesKHR: numSwapchainImages 5 > countof(m_backBufferColorImage) 4.
../../../../../3rdparty/bgfx/src/renderer_vk.cpp (6489): BGFX Create swap chain error: creating swapchain and image views failed -3: VK_ERROR_INITIALIZATION_FAILED
../../../../../3rdparty/bgfx/src/renderer_vk.cpp (6521): BGFX errorState 1
../../../../../3rdparty/bgfx/src/renderer_vk.cpp (1857): BGFX Init error: creating swap chain failed -3: VK_ERROR_INITIALIZATION_FAILED.
../../../../../3rdparty/bgfx/src/renderer_vk.cpp (1971): BGFX errorState 4
../../../../../3rdparty/bgfx/src/renderer_vk.cpp (628): BGFX ---E-            Instance, Validation, 0: Validation Error: [ VUID-vkDestroySurfaceKHR-surface-01266 ] Object 0: handle = 0x7fd85c021be8, type = VK_OBJECT_TYPE_INSTANCE; | MessageID = 0xbed408e4 | vkDestroySurfaceKHR() called before its associated VkSwapchainKHR was destroyed. The Vulkan spec states: All VkSwapchainKHR objects created for surface must have been destroyed prior to destroying surface (https://www.khronos.org/registry/vulkan/specs/1.2-extensions/html/vkspec.html#VUID-vkDestroySurfaceKHR-surface-01266)

Full output is available here:
https://gist.github.com/belegdol/279cb26c38f6175b5aa3f46ab0ab94cf

[belegdol]

It turns it only happens when running under XWayland. When the entire session is switched to X11 the vulkan renderer starts without the need for extra switches.

[belegdol]

This is fixed by bkaradzic/bgfx@be2c110. Should I make a PR with just this patch, or rather resync bgfx with upstream so that we do not diverge too much?