Vulkan broken on Linux by #3994

[zekesonxx]

Title. Vulkan works on 51a2b43, but not on fbceec4 (#3994).

F RSX: Could not find a vulkan compatible GPU driver. Your GPU(s) may not support Vulkan, or you need to install the vulkan runtime and drivers

Tested on Arch Linux, kernel 4.13.12, Nvidia drivers 386.34, Wayland version 1.14, xfce4 (Wayland not in use, obviously).

Issue created with @hcorion's love and support.

[hcorion]

I can also reproduce, Arch Linux, NVIDIA GTX 1070, driver version 387.34, KDE Plasma 5.11.5. Wayland is installed, but not in use.

Tagging @myfreeweb

[hcorion]

Works with the latest AppImage downloaded from https://rpcs3.net/download , presumably because the AppImages don't build with wayland.

[valpackett]

huh. Looks like this error happens when this function fails

rpcs3/rpcs3/Emu/RSX/VK/VKHelpers.h

Line 1204 in 5959411

uint32_t createInstance(const char *app_name, bool fast = false)

not much was changed here, why would it fail on nvidia specifically? Are the VK_KHR_XLIB_SURFACE_EXTENSION_NAME etc. defines somehow wrong?

[kd-11]

Maybe we just need to edit the cmake file to not enable wayland unless explicitly requested via config. When wayland is not available, vulkan runs fine. I'm pretty sure nvidia does not support wayland last I checked.

[kd-11]

Ugh

#else
				VK_KHR_XLIB_SURFACE_EXTENSION_NAME,
#endif
#ifdef VK_USE_PLATFORM_WAYLAND_KHR
				VK_KHR_WAYLAND_SURFACE_EXTENSION_NAME,
#endif

Only one should be requested at a time, not both. Either wayland or X but not wayland and X

[kd-11]

Retest with kd-11@73442a7

[kd-11]

A potentially better way of doing it kd-11@5369d2a
To enable wayland, pass USE_WAYLAND_EGL=ON to cmake, otherwise it should use X
I dont have wayland on my setup to actually test this

[valpackett]

Oh. Yeah, right, that's a list where everything must be present. I'll make it a runtime decision.

(BTW, what nvidia doesn't support is GBM – Wayland compositors can work on their proprietary driver if they implement EGLStreams. They just didn't implement the Wayland surface, it seems.)

[kd-11]

Yea, its best to compile globally with xlib and have a cmake arg for wayland support, at least until support is universal. All vulkan drivers on linux support xlib surface but not all support wayland, and it might be that way for a while.

[valpackett]

No, I mean, it's best to check supported extensions at runtime: #4048

[greentop]

PR #4048 corrects the Vulkan problem also seen in Ubuntu 16.04 using Nvidia 387.34.

[kd-11]

Closing as fixed