Add Wayland support to the Steam overlay

[flibitijibibo]

Your system information

• Steam client version (build number or date): Jul 20 2021, at 22:25:57
• Distribution (e.g. Ubuntu): Fedora 34
• Opted into Steam client beta?: No
• Have you checked for system updates?: Yes

Please describe your issue in as much detail as possible:

The Steam overlay currently depends on GLX and X WSI for OpenGL and Vulkan respectively. As a result, when running a Wayland-native application the overlay will not appear.

For OpenGL, gameoverlayrenderer.so overrides glX* but not egl*. For Vulkan, steamoverlayvulkanlayer.so only overrides the functions for VK_KHR_xcb_surface and VK_KHR_xlib_surface, not VK_KHR_wayland_surface.

This will take time to write (mostly for input handling I imagine) so a workaround would be for Steam to set SDL_VIDEODRIVER to x11 if it's unset, being sure to only do this for native applications.

Steps for reproducing this issue:

1. Take any game with a recent SDL version (my go-to is the public Steam beta for FEZ)
2. Set SDL_VIDEODRIVER=wayland %command% /gldevice:XXX as a launch option, where XXX can be OpenGL or Vulkan
3. Observe no overlay

[smcv]

Related to this, ValveSoftware/steam-runtime#473 (comment) mentions that in at least some configurations, the overlay is loaded into a native Wayland app, but causes the app to crash when it is opened.

[flibitijibibo]

It's starting to look very much like 2.0.22 will be the SDL release that defaults to Wayland... our only blockers are this and NVIDIA finishing their driver support, the latter being actively worked on as we speak.

As a stepping stone to native support, it might be a good idea for the Steam client to check SDL_VIDEODRIVER and set it to x11 if it's unset? That way we can have Wayland be the default in SDL without having to wait for the Steam team to catch up.

[murlakatamenka]

I've been playing Axiom Verge recently (native, FNA game) and Steam overlay shows up but nothing is rendered. For example, achievement popups has nothing on them, just a flat background. Is it due to this issue?
I'll launch the game with SDL_VIDEODRIVER=wayland and see if it works better.

[flibitijibibo]

I've been playing Axiom Verge recently (native, FNA game) and Steam overlay shows up but nothing is rendered. For example, achievement popups has nothing on them, just a flat background. Is it due to this issue? I'll launch the game with SDL_VIDEODRIVER=wayland and see if it works better.

#7889

[murlakatamenka]

@flibitijibibo thanks, Ethan, that's exactly the issue.
You can add the list of games you have there.

[smcv]

I see Fedora has patched their SDL to make SDL default to native Wayland (which I think is premature, but I'm not a Fedora developer, so it's not up to me)...

As a stepping stone to native support, it might be a good idea for the Steam client to check SDL_VIDEODRIVER and set it to x11 if it's unset?

... so I think it would be a good idea for Steam to do this, as a stopgap until the gameoverlayrenderer has Wayland support.

[Conan-Kudo]

I see Fedora has patched their SDL to make SDL default to native Wayland (which I think is premature, but I'm not a Fedora developer, so it's not up to me)...

From my point of view, it's important so that the Wayland backend is actively used and developers wind up encountering it. Otherwise people will continue to develop their applications in a way where the backend won't work.