Game is not visible in SteamOS gamescope session when using Steam Runtime in Flatpak

[R1kaB3rN]

Your system information

• Steam Runtime Version: sniper_platform_0.20240423.85483
• Distribution (e.g. Ubuntu 18.04): SteamOS 3.6.3
• Have you checked for system updates?: [Yes/No] Yes
• What compatibility tool are you using?: [None / Steam Linux Runtime / Proton 5.13+ / older Proton]

Please describe your issue in as much detail as possible:

Game launchers like Heroic and Lutris distribute their applications to the Steam Deck as a Flatpak, and users add those launchers as a non-Steam game to utilize the SteamOS gamecope session features before running their games. However, when running games in Flatpak that uses its own bundle of the Steam Runtime within the SteamOS gamescope session, the game window will not be displayed and the client will just hang with the Steam logo spinner spinning indefinitely. On the other hand, when running the game in Desktop Mode, the game window will be displayed as expected. The same applies when distributing the game launcher as a native application instead as a Flatpak, where the window will be displayed in both sessions. Currently, this is a big issue for users of those launchers, causing them to revert to Wine instead of Proton to run their games.

SLR log when running the application (Flatpak) in the SteamOS gamescope session:
https://gist.github.com/R1kaB3rN/f27ec679c4ed6d1250c979a4ea33e84f

SLR log when running the application (native) in the SteamOS gamescope session:
https://gist.github.com/R1kaB3rN/abe95b52d11eef25269cc21e1175db64

Steps for reproducing this issue:

1. Enter Desktop Mode on SteamOS
2. Download the umu-launcher Flatpak on the Steam Deck
3. tar -xaf umu-flatpak-test.tar.gz
4. flatpak --user remote-add --no-gpg-verify umu-repo umu-repo
5. flatpak --user install umu-repo org.openwinecomponents.umu.umu-launcher
6. Create this script and name it test.sh
7. Add the script as a non-Steam game in the Steam client
8. Enter SteamOS gamescope session
9. Run the test.sh

[smcv]

When you say "Steam Runtime", do you mean the container runtime that involves pressure-vessel/, the same thing that's distributed on Steam as steamapps/common/SteamLinuxRuntime_sniper?

In the logs attached to this report, it seems that you're using winecfg as a simple substitute for an actual game. Is that correct?

I don't see any error messages in the logs you attached that would indicate that winecfg has crashed or failed to start, or that pressure-vessel has failed before it got as far as launching winecfg. Instead, it looks to me as though it is probably starting OK, but gamescope just isn't making the main window visible the way it is meant to.

If you use a ssh session to inspect the process tree while the "game" is running, do you see a winecfg process running?

If you do, please get its process ID (let's say 12345) and look at /proc/12345/maps with less or similar. Do you see that it has mapped gameoverlayrenderer.so into memory? I would expect it to appear at a path that might look something like /tmp/pressure-vessel-libs-VO37M2/lib64/gameoverlayrenderer.so (which will probably be a symbolic link into /home/deck/.local/share/Steam).

I suspect this might be a problem with the logic in Gamescope that aims to only display the main game window (full-screen), and not display anything that is not the main game window. If that logic relies on the gameoverlayrenderer.so getting LD_PRELOADed into the game executable (does it?), then it wouldn't surprise me if either Flatpak, pressure-vessel, or the combination of the two will interfere with that.

Am I correct to think that the truth table goes like this?

• installed unsandboxed at OS level, run in desktop session, uses its own bundled SteamLinuxRuntime_sniper + a third-party Proton: OK
• installed unsandboxed at OS level, run in Gamescope session as a non-Steam game, uses its own bundled SteamLinuxRuntime_sniper + a third-party Proton: OK, https://gist.github.com/R1kaB3rN/abe95b52d11eef25269cc21e1175db64
• installed as a Flatpak app, run in desktop session, uses its own bundled SteamLinuxRuntime_sniper + a third-party Proton: OK
• installed as a Flatpak app, run in Gamescope session as a non-Steam game, uses its own bundled SteamLinuxRuntime_sniper + a third-party Proton: does not appear, https://gist.github.com/R1kaB3rN/f27ec679c4ed6d1250c979a4ea33e84f
• installed by any mechanism, run by any means, uses Steam's official SteamLinuxRuntime_sniper + any Proton: not tested
• installed by any mechanism, run by any means, uses any SteamLinuxRuntime_sniper + Steam's official Proton: not tested
• installed by any mechanism, run by any means, uses any SteamLinuxRuntime_sniper, native Linux executable: not tested
• installed by any mechanism, run by any means, does not use SteamLinuxRuntime_sniper, native Linux executable: not tested

If that's the case, then I think the first obvious things to try would be to eliminate SteamLinuxRuntime_sniper and Proton by using some simple native Linux executable like xterm or zenity as your "game" payload:

• installed unsandboxed at OS level, run in desktop session, no SteamLinuxRuntime_sniper, no Proton, native Linux executable
• installed unsandboxed at OS level, run in Gamescope session as a non-Steam game, no SteamLinuxRuntime_sniper, no Proton, native Linux executable
• installed as a Flatpak app, run in desktop session, no SteamLinuxRuntime_sniper, no Proton, native Linux executable
• installed as a Flatpak app, run in Gamescope session as a non-Steam game, , no SteamLinuxRuntime_sniper, no Proton, native Linux executable

because that will help to narrow down whether this problem is unique to SteamLinuxRuntime_sniper and/or Proton.

Download the umu-launcher Flatpak on the Steam Deck

How is this compiled? What does it contain, and what Flatpak sandboxing parameters does it have?

[R1kaB3rN]

When you say "Steam Runtime", do you mean the container runtime that involves pressure-vessel/, the same thing that's distributed on Steam as steamapps/common/SteamLinuxRuntime_sniper?

Yes, by Steam Runtime, I mean only the contents contained within the steamrt3 image binary release at this endpoint. The application (umu-launcher) does not use the Scout runtime and only uses the runtime contents of SteamLinuxRuntime_sniper.tar.xz, which includes files such as pressure-vessel and sniper_platform_0.20240423.85483 without modifying its functionality in any way.

In the logs attached to this report, it seems that you're using winecfg as a simple substitute for an actual game. Is that correct?

That's correct.

If you use a ssh session to inspect the process tree while the "game" is running, do you see a winecfg process running?

Yes, I see the winecfg process running and here's the contents of the maps file for that process:
https://gist.github.com/R1kaB3rN/82a6ff2f03a11a0120f3bc36beacdf7a

Also, the above log was generated after building the Flatpak application on the Steam Deck via flatpak-builder with this manifest instead of the umu-launcher tarball I had referenced earlier, which has also been built using that manifest. Here were the steps used to build it:

1. git clone https://github.com/Open-Wine-Components/umu-launcher.git
2. cd packaging/flatpak
3. flatpak-builder --user --install builddir org.openwinecomponents.umu.umu-launcher.yml
4. Run flatpak install when prompted for dependencies

Though, for that tarball I had referenced, I do not know the exact command GloriousEggroll had used to build it nor the system that it was built on. However, you should still run into the same issue.

Am I correct to think that the truth table goes like this?

That's correct. For comparison, here's the maps contents of the glxgears executable that has been copied to the home directory using the Flatpak umu-launcher application:
https://gist.github.com/R1kaB3rN/53d44f5b2bfa926e94b195db09d6a447

Of course, the window still does not display.

Now, here's the contents of maps and an SLR log using the native umu-launcher application for glxgears. In this case, for some reason, window displays as expected in the SteamOS gamescope session.

Please let me know if I can assist you with any more information or testing, and if there's a possible workaround or solution for this in the Steam Runtime. However, if a workaround within the Runtime is not possible and it's gamescope like you suspected, then do you have an idea what the appropriate logic would be?

[smcv]

For comparison, here's the maps contents of the glxgears executable that has been copied to the home directory using the Flatpak umu-launcher application: https://gist.github.com/R1kaB3rN/53d44f5b2bfa926e94b195db09d6a447

Of course, the window still does not display.

I'm not seeing why you say "of course" here: it was not obvious to me whether this one would display a window or not. Is there a reason why it was obvious to you that it would not work?

If this one did not work, but should have worked, then that's really useful information, because it probably means we can eliminate Proton from consideration, and maybe SLR too. That considerably reduces the number of moving parts involved (although there are still a lot).

Every time Steam runs a game (whether it's a Steam or non-Steam game), you should see something that looks like this being logged somewhere, except instead of being broken across multiple lines like this, it will be one very long line (this one came from me adding gnome-calculator as a non-Steam game on Ubuntu):

/bin/sh\0-c\0/home/me/.local/share/Steam/ubuntu12_32/reaper SteamLaunch AppID=xxxx -- \
/home/me/.local/share/Steam/ubuntu12_32/steam-launch-wrapper -- \
"gnome-calculator"\0

It would be useful if you could match up each of the interesting test scenarios with the command-line that gets logged, starting with /bin/sh\0-c\0. What I can't tell you, unfortunately, is where you will find that command-line. Depending on which SteamOS and Steam versions you have (and whether they are beta or stable versions), it might be in either $XDG_RUNTIME_DIR/gamescope.log, the systemd Journal, ~/.steam/steam/logs/console-linux.txt, or ~/.steam/steam/logs/console_log.txt.

Also, for any test scenario that involves umu-launcher running anything, if you can make it log the command-line or argv that it launches (including any SteamLinuxRuntime_sniper/run or SteamLinuxRuntime_sniper/_v2-entry-point or similar wrappers that it might have been using), please do that and mention it in your test results. And, if SLR is involved, please get a log from SLR as you did in the initial issue report.

I can't immediately see what this test scenario is, in terms similar to what I said in #1341 (comment). It might be one of:

• installed as a Flatpak app, run in Gamescope, no SteamLinuxRuntime_sniper, no Proton, native Linux executable: program runs but window does not appear (?)

or

• installed as a Flatpak app, run in Gamescope, uses UMU's bundled SteamLinuxRuntime_sniper version 0.20240423.85483, no Proton, native Linux executable: program runs but window does not appear (?)

or it might be something else? Please clarify?

[smcv]

Now, here's the contents of maps and an SLR log using the native umu-launcher application for glxgears. In this case, for some reason, window displays as expected in the SteamOS gamescope session.

I think that this is:

• installed as a Flatpak app, run in Gamescope, uses UMU's bundled SteamLinuxRuntime_sniper version 0.20240423.85483, uses UMU's bundled Proton (?!), native Linux executable: OK

Proton is a mechanism for running Windows executables, so I'm surprised that it succeeds when asked to launch a native Linux executable... but apparently it does, so that's nice? And in this case you say that the glxgears window does appear.

glxgears is a pure OpenGL application with no Vulkan, so this rules out the Gamescope Vulkan WSI being significant. (I think?)

What was the command that was run by Steam (logged starting with /bin/sh\0-c\0, as above) in this scenario?

[smcv]

if a workaround within the Runtime is not possible and it's gamescope like you suspected, then do you have an idea what the appropriate logic would be?

Sorry, no, I am not a Gamescope developer.

[smcv]

To get a comparison vs. the scenario discussed in #1341 (comment), it could be useful to try it with a version of umu-launcher that has been hacked so that it does not add any SteamLinuxRuntime_sniper or Proton components to the command-line, but instead just launches your /home/deck/Games/glxgears executable directly.

Obviously this would break umu-launcher's ability to run Windows games under Proton, so you would want to roll back to a more normal version immediately (unless you added a check to make this code path be guarded by an environment variable or something): but it would be a useful step to either confirm that SLR and Proton are significant, or find that they are irrelevant. Either way, we will have gained important information.

[GloriousEggroll]

I've managed to resolve this. It's not a problem with the runtime so much as it was a problem with the window properties missing the ID that gamescope and steam expect. In combination with the layer order going out of whack when focus is lost.

When a non-steam application is launched in steam, steam assigns it a specific value for its window property. For example we'll say 12345. It also does this same process for steam games, but for steam games it assigns a window property STEAM_GAME=12345. It then informs gamescope of this ID, and gamescope stores it in a few variables.

This means a main application would get assigned window property 12345 -- but anything launched underneath in flatpak for some reason would not, thus whatever was launched from say lutris flatpak would never be brought into focus.

The variable in gamescope that controls the layer ordering is GAMESCOPECTRL_BASELAYER_APPID, and the layer order should typically be "number, 12345, number, 769" with 12345 being our app and 769 being steam. When focus is lost, the order changes to "number, number, 769, 12345"

So the first step is to correct the layer order. The second step is to make sure the game window gets assigned STEAM_GAME=12345 window property. Additionally, there are two displays, :0 is the main steam window and :1 is where our game window is generated).

1: Get the layer order from display :0 :
xprop -d :0 -root | grep GAMESCOPECTRL_BASELAYER_APPID | cut -d "=" -f 2

ex output:

413091, 413090, 769, 12345

This is the incorrect order when a window is out of focus. To bring it in focus we need to change this order to:

413091, 12345, 413090, 769

For reference you can launch a steam game and check again to verify this is the correct order used.

2. Use whatever you like to rearrange the order, be it bash script or python or whatever:

413091, 12345, 413090, 769

3. Assign the new order:

xprop -d :0 -root -f GAMESCOPECTRL_BASELAYER_APPID 32co -set GAMESCOPECTRL_BASELAYER_APPID "413091, 12345, 413090, 769"

4. Get the game window ID from display :1. It's usually the first entry in the list with 'steam_app'. This can be tricky because it loads steamcompmgr first, then later populates the rest. I had to do a while loop for checking this:

$ xwininfo -d :1 -root -tree

xwininfo: Window id: 0x385 (the root window) (has no name)

  Root window id: 0x385 (the root window) (has no name)
  Parent window id: 0x0 (none)
     20 children:
     0x1200001 "Ys Origin": ("steam_app_207350" "steam_app_207350")  800x600+0+0  +0+0
        1 child:
        0xc00038 (has no name): ()  800x600+0+0  +0+0
     0x1200003 "Hidden D3D Window": ("steam_app_207350" "steam_app_207350")  800x600+0+0  +0+0
        1 child:
        0xc00073 (has no name): ()  800x600+0+0  +0+0
     0xc00064 (has no name): ()  1x1+0+0  +0+0
     0x1400002 "Default IME": ("steam_app_207350" "steam_app_207350")  1x1+0+0  +0+0
     0x1400001 "ActiveMovie Window": ("steam_app_207350" "steam_app_207350")  320x240+0+0  +0+0
     0x800008 (has no name): ("steam_app_207350" "steam_app_207350")  1x1+0+0  +0+0
     0xc00023 (has no name): ()  1x1+-1+-1  +-1+-1
     0x1000001 "Input": ("steam_app_207350" "steam_app_207350")  112x27+4+4  +4+4
     0x1000002 "Default IME": ("steam_app_207350" "steam_app_207350")  1x1+0+0  +0+0
     0xe00009 (has no name): ()  1x1+-1+-1  +-1+-1
     0xa00001 (has no name): ()  1x1+0+0  +0+0
     0x800007 (has no name): ("steam_app_207350" "steam_app_207350")  1x1+0+0  +0+0
     0x800006 (has no name): ("steam_app_207350" "steam_app_207350")  1x1+0+0  +0+0
     0x800005 (has no name): ("steam_app_207350" "steam_app_207350")  1x1+0+0  +0+0
     0x800004 (has no name): ("steam_app_207350" "steam_app_207350")  1x1+0+0  +0+0
     0x800002 (has no name): ("steam_app_207350" "steam_app_207350")  160x27+0+0  +0+0
     0x800003 "Default IME": ("steam_app_207350" "steam_app_207350")  1x1+0+0  +0+0
     0x600035 (has no name): ()  1x1+-1+-1  +-1+-1
     0x800001 (has no name): ()  800x600+0+0  +0+0
     0x200001 "steamcompmgr": ()  1x1+0+0  +0+0

In our case the window ID is here:

0x1200001 "Ys Origin": ("steam_app_207350" "steam_app_207350")  800x600+0+0  +0+0

5. Assign the expected ID (12345) to the game window:

xprop -d :1 -id 0x1200001 -f STEAM_GAME 32c -set STEAM_GAME 12345

And this should bring your game in focus with steam overlays working.

IMPORTANT NOTE: The ID HAS to be the original one steam assigned. If you use a random ID then gamescope will focus it fine, but steam will not recognize it since steam expects the one it generated and there's no way to tell steam it has changed, and therefore your steam overlays will not work because they will be underneath the game window.

It should also be noted that if games have their own configuration tool that launches before the game, sometimes this can re-set the invalid layer order. I had to add a monitor thread to umu to check periodically if the order is incorrect and to re-correct it. I also added another check to find if the display :1 window id list has changed and if so to re-apply the STEAM_GAME id. An example of this would be the configuration tool for Akiba's Trip which launches before the game itself.

[sysbite]

This seems to be it indeed #1341 (comment). Looks like a hassle though to do this for each game. This needs automation or a fix imo, especially if that value could change with patches or restarts (not sure how it's generated).

[GloriousEggroll]

This seems to be it indeed #1341 (comment). Looks like a hassle though to do this for each game. This needs automation or a fix imo, especially if that value could change with patches or restarts (not sure how it's generated).

we've automated it in umu, so flatpaks like lutris and heroic can use it without further action needed. right now the main kicker is it needs xwininfo and xprop to do so. I tried using python-xlib but it was problematic when fetching window IDs. (anyway that's off topic for this issue)

-edit-

I managed to get this working using python-xlib alone, which is much faster than relying on subprocess command calls.

[smcv]

It might be clearer what is going on if someone with suitable permissions retitles this bug from "Game will not launch" to "Game window not visible" - as far as I can see, it's launching and executing code fine, but the problem is all about whether Gamescope brings it to the foreground or not.

[WhiteBahamut]

I managed to get this working using python-xlib alone, which is much faster than relying on subprocess command calls.

Anything for people test? If binaries and instuctions on how to get the proper logs are provided I could test with wow (lutris steam library link and started game ui). Right now I am suckt on 8-26 and would happily try out a potential fix