Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Runtime Build ID 9027669 fails to launch games (Borderlands 3, Wonderlands) #516

Closed
apocalyptech opened this issue Jul 20, 2022 · 36 comments
Assignees

Comments

@apocalyptech
Copy link

Your system information

  • Steam Runtime Version: Build ID: 9027669
  • Distribution (e.g. Ubuntu 18.04): Arch Linux
  • Link to your full system information (Help -> System Information) in a Gist: https://gist.github.com/apocalyptech/8744d4206a621a2cac7c14289395fd88
  • Have you checked for system updates?: [Yes/No] Yes
  • What compatibility tool are you using?: [None / Steam Linux Runtime / Proton 5.13+ / older Proton] Steam Linux Runtime - Soldier, a couple different GE Proton builds
  • If you are using Steam Linux Runtime for native Linux games: What versions are listed in SteamLinuxRuntime/VERSIONS.txt? n/a
  • If you are using Steam Linux Runtime, or Proton 5.13 or newer: What versions are listed in SteamLinuxRuntime_soldier/VERSIONS.txt?

#Name Version Runtime Runtime_Version Comment
depot 0.20220629.59 # Overall version number
pressure-vessel 0.20220624.0
scripts v0.20220620.0-0-g22dad8e # Entry point scripts, etc.
soldier 0.20220628.0 soldier 0.20220628.0 # soldier_platform_0.20220628.0/

Please describe your issue in as much detail as possible:

Steam installed a new Steam Linux Runtime for me today (Build ID: 9027669), and I'm unable to launch either Borderlands 3 or Tiny Tina's Wonderlands (the only two games installed on this root). There's a different error with each (and they each use a different GE Proton), but with similar errors: essentially complaining about not finding files which are, nevertheless, on the system. For Wonderlands, I think the game itself launches, but it shows a dialog saying "Unable to initialize SteamAPI - Please make sure Steam is running and you are logged in to an account entitled to the game," and the following shows up on the console:

dlopen failed trying to load:
/home/bl3/.steam/sdk64/steamclient.so
with error:
/home/bl3/.steam/sdk64/steamclient.so: cannot open shared object file: No such file or directory
[S_API] SteamAPI_Init(): Sys_LoadModule failed to load: /home/bl3/.steam/sdk64/steamclient.so
[S_API FAIL] SteamAPI_Init() failed

That file does, indeed, exist, though:

[bl3@arrakis ~]$ ls -l /home/bl3/.steam/sdk64/steamclient.so
-rwxrwxr-x 1 bl3 bl3 33786666 Jul 18 13:03 /home/bl3/.steam/sdk64/steamclient.so*
[bl3@arrakis ~]$ ldd /home/bl3/.steam/sdk64/steamclient.so
		linux-vdso.so.1 (0x00007ffc86933000)
		libdl.so.2 => /usr/lib/libdl.so.2 (0x00007f9e35ae9000)
		libpthread.so.0 => /usr/lib/libpthread.so.0 (0x00007f9e35ae4000)
		librt.so.1 => /usr/lib/librt.so.1 (0x00007f9e35adf000)
		/usr/lib64/ld-linux-x86-64.so.2 (0x00007f9e35b44000)
		libm.so.6 => /usr/lib/libm.so.6 (0x00007f9e33919000)
		libc.so.6 => /usr/lib/libc.so.6 (0x00007f9e33600000)
[bl3@arrakis ~]$ /games/bl3_steam/games/steamapps/common/SteamLinuxRuntime_soldier/pressure-vessel/bin/pressure-vessel-adverb ldd /home/bl3/.steam/sdk64/steamclient.so
		linux-vdso.so.1 (0x00007fff0f1d2000)
		libdl.so.2 => /usr/lib/libdl.so.2 (0x00007fc9117a0000)
		libpthread.so.0 => /usr/lib/libpthread.so.0 (0x00007fc91179b000)
		librt.so.1 => /usr/lib/librt.so.1 (0x00007fc911796000)
		/usr/lib64/ld-linux-x86-64.so.2 (0x00007fc9117fb000)
		libm.so.6 => /usr/lib/libm.so.6 (0x00007fc90f519000)
		libc.so.6 => /usr/lib/libc.so.6 (0x00007fc90f200000)

Borderlands 3, in contrast, doesn't launch at all, with the following showing up on the console:

pressure-vessel-adverb[61952]: E: Failed to execute child process "/games/bl3_steam/root/compatibilitytools.d/Proton-5.21-GE-1/proton" (No such file or directory)

As with steamclient.so above, that file does exist:

[bl3@arrakis ~]$ ls -l /games/bl3_steam/root/compatibilitytools.d/Proton-5.21-GE-1/proton
-rwxr-xr-x 1 bl3 bl3 40954 Nov 19  2020 /games/bl3_steam/root/compatibilitytools.d/Proton-5.21-GE-1/proton*

This behavior happens with the client_beta branch of Steam Linux Runtime as well (Build ID: 9157627), but I can revert to the previous_release branch (Build ID: 8860828) and launch both games just fine without problems. (They were also working fine ~12 hours ago, before that Steam Linux Runtime upgrade happened.)

Debug logs as per reporting-steamlinuxruntime-bugs.md: https://gist.github.com/30a62ace83342519a92aa91da6f20dd5

Steps for reproducing this issue:

  1. Update to Build ID: 9027669
  2. Attempt to run either of the games installed in this root
@TTimo
Copy link
Collaborator

TTimo commented Jul 20, 2022

Hello @apocalyptech - have you tried with Proton Experimental and latest Proton 7? It could be specific to GE.

@apocalyptech
Copy link
Author

apocalyptech commented Jul 20, 2022

I have not, since the ingame videos apparently don't work on non-GE Proton builds. I suppose I could give it a try, though I'm generally loathe to rock the boat with swapping out Proton versions. I don't have any faith that doing so won't wipe out savegames or settings, etc, so I tend to keep the old versions around for quite awhile (hence why my BL3's still using an old 5.21).

@TTimo
Copy link
Collaborator

TTimo commented Jul 20, 2022

I cannot reproduce with Tiny Tina on Manjaro with Proton Experimental (it's a new install, I didn't have the game). I note that videos are playing too. I don't have a Proton GE to check against atm.

@smcv
Copy link
Contributor

smcv commented Jul 21, 2022

Are there any symbolic links in the path to /home/bl3/.steam/sdk64/steamclient.so? Please paste the output of:

ls -ld /home
ls -ld /home/bl3
ls -ld /home/bl3/.steam
ls -ld /home/bl3/.steam/sdk64
ls -ld /home/bl3/.steam/sdk64/steamclient.so

and the equivalents for /games/bl3_steam/root/compatibilitytools.d/GE-Proton7-20/proton [edit: and also /games/bl3_steam/root/compatibilitytools.d/Proton-5.21-GE-1/proton].

@smcv
Copy link
Contributor

smcv commented Jul 21, 2022

This behavior happens with the client_beta branch of Steam Linux Runtime as well (Build ID: 9157627), but I can revert to the previous_release branch (Build ID: 8860828) and launch both games just fine without problems.

The build IDs you quoted here are releases of SteamLinuxRuntime_soldier, listed as "Steam Linux Runtime - soldier" in the Steam UI. There is another component called SteamLinuxRuntime, listed as "Steam Linux Runtime" in the Steam UI, which is not the same thing. Both were updated yesterday, and both are involved in running a native Linux game in the container runtime.

I think you are saying that varying the version of "Steam Linux Runtime - soldier" causes this issue to appear or disappear, and varying the version of "Steam Linux Runtime" doesn't have any effect. Is that correct? Like this:

  • "Steam Linux Runtime - soldier" (SteamLinuxRuntime_soldier directory on disk, app 1391110)
    • bad: build ID 9157627, labelled depot 0.20220720.62 in SteamLinuxRuntime_soldier/VERSIONS.txt, in client_beta since yesterday
    • bad: build ID 9027669, labelled depot 0.20220629.59 in SteamLinuxRuntime_soldier/VERSIONS.txt, default branch since yesterday
    • good: build ID 8860828, labelled depot 0.20220602.55 in SteamLinuxRuntime_soldier/VERSIONS.txt, previous_release branch since yesterday
  • "Steam Linux Runtime" (SteamLinuxRuntime directory on disk, app 1070560)
    • branch/version does not matter for this issue, only "Steam Linux Runtime - soldier" matters

Or if that's not correct, please confirm what you are varying to make this issue appear or disappear.

@smcv
Copy link
Contributor

smcv commented Jul 21, 2022

Oh, are these both Windows games running under Proton? If so, then yes, varying "Steam Linux Runtime - soldier" would have an effect, and varying "Steam Linux Runtime" would not (and you probably don't even have "Steam Linux Runtime" installed).

Another data point that would be useful would be to install a small, known-to-work Windows game for which you don't care about saved games etc. into the same prefix, swap that game between official Proton and unofficial Proton-GE, and report what happens (in particular whether it gets the same failure mode as one or both of these Borderlands-derived games).

The Expendabros is a relatively small free-to-play Windows game which I often use for testing.

@smcv
Copy link
Contributor

smcv commented Jul 21, 2022

A more detailed log for each of the Proton GE versions would also be useful: in addition to what you did to capture https://gist.github.com/apocalyptech/30a62ace83342519a92aa91da6f20dd5, please set STEAM_LINUX_RUNTIME_VERBOSE=1.

If this is a problem with not all symlink targets being imported into the container, then the results I would expect to see with a known-good game like Expendabros would be something like this:

  • old SLR_soldier: OK
  • new SLR_soldier:
    • GE-Proton7-20: same failure mode as Tiny Tina's Wonderlands due to using the same soldier and Proton versions
    • Proton-5.21-GE-1: same failure mode as Borderlands 3 due to using the same soldier and Proton versions
    • official Proton: probably OK?

@TTimo
Copy link
Collaborator

TTimo commented Jul 21, 2022

@smcv yeah they are both Proton titles / soldier SLR. I tested both Tiny Tina and Borderlands 3 on Manjaro and they work fine against official Proton. This may be a regression specific to GE's setup.

@apocalyptech
Copy link
Author

Hello - thanks for the responses. Answers inlined below!

Are there any symbolic links in the path to /home/bl3/.steam/sdk64/steamclient.so? Please paste the output of:

ls -ld /home
ls -ld /home/bl3
ls -ld /home/bl3/.steam
ls -ld /home/bl3/.steam/sdk64
ls -ld /home/bl3/.steam/sdk64/steamclient.so

and the equivalents for /games/bl3_steam/root/compatibilitytools.d/GE-Proton7-20/proton [edit: and also /games/bl3_steam/root/compatibilitytools.d/Proton-5.21-GE-1/proton].

There's a pretty incredible utility called namei which I discovered not too long ago which is perfect for this kind of reporting, btw. Here you go (in summary: /home/bl3/.steam/sdk64 is a symlink over to /games/bl3_steam/root/linux64 but everything else is just a dir):

[bl3@arrakis ~]$ namei -l /home/bl3/.steam/sdk64/steamclient.so
f: /home/bl3/.steam/sdk64/steamclient.so
drwxr-xr-x root root /
drwxr-xr-x root root home
drwxr-xr-x bl3  bl3  bl3
drwxrwxr-x bl3  bl3  .steam
lrwxrwxrwx bl3  bl3  sdk64 -> /games/bl3_steam/root/linux64
drwxr-xr-x root root   /
drwxr-xr-x pez  pez    games
drwxr-xr-x bl3  bl3    bl3_steam
drwxr-xr-x bl3  bl3    root
drwxrwxr-x bl3  bl3    linux64
-rwxrwxr-x bl3  bl3  steamclient.so
[bl3@arrakis ~]$ namei -l /games/bl3_steam/root/compatibilitytools.d/GE-Proton7-20/proton
f: /games/bl3_steam/root/compatibilitytools.d/GE-Proton7-20/proton
drwxr-xr-x root root /
drwxr-xr-x pez  pez  games
drwxr-xr-x bl3  bl3  bl3_steam
drwxr-xr-x bl3  bl3  root
drwxrwxr-x bl3  bl3  compatibilitytools.d
drwxr-xr-x bl3  bl3  GE-Proton7-20
-rwxr-xr-x bl3  bl3  proton
[bl3@arrakis ~]$ namei -l /games/bl3_steam/root/compatibilitytools.d/Proton-5.21-GE-1/proton
f: /games/bl3_steam/root/compatibilitytools.d/Proton-5.21-GE-1/proton
drwxr-xr-x root root /
drwxr-xr-x pez  pez  games
drwxr-xr-x bl3  bl3  bl3_steam
drwxr-xr-x bl3  bl3  root
drwxrwxr-x bl3  bl3  compatibilitytools.d
drwxrwxr-x bl3  bl3  Proton-5.21-GE-1
-rwxr-xr-x bl3  bl3  proton

I think you are saying that varying the version of "Steam Linux Runtime - soldier" causes this issue to appear or disappear, and varying the version of "Steam Linux Runtime" doesn't have any effect. Is that correct? Like this:

"Steam Linux Runtime - soldier" (SteamLinuxRuntime_soldier directory on disk, app 1391110)
    bad: build ID 9157627, labelled depot 0.20220720.62 in SteamLinuxRuntime_soldier/VERSIONS.txt, in client_beta since yesterday
    bad: build ID 9027669, labelled depot 0.20220629.59 in SteamLinuxRuntime_soldier/VERSIONS.txt, default branch since yesterday
    good: build ID 8860828, labelled depot 0.20220602.55 in SteamLinuxRuntime_soldier/VERSIONS.txt, previous_release branch since yesterday

Yep, that is correct - reverting to the previous_release branch of SteamLinuxRuntime_soldier is the only way to play Proton-based games on this install at the moment.

Oh, are these both Windows games running under Proton? If so, then yes, varying "Steam Linux Runtime - soldier" would have an effect, and varying "Steam Linux Runtime" would not (and you probably don't even have "Steam Linux Runtime" installed).

Another data point that would be useful would be to install a small, known-to-work Windows game for which you don't care about saved games etc. into the same prefix, swap that game between official Proton and unofficial Proton-GE, and report what happens (in particular whether it gets the same failure mode as one or both of these Borderlands-derived games).

Ah indeed, right you are -- dunno why I didn't think of that before. I just picked Witcheye since I'd recently played it, and while that game does sort-of launch, it too has seemingly-identical problems. From its console:

dlopen failed trying to load:
/home/bl3/.steam/sdk64/steamclient.so
with error:
/home/bl3/.steam/sdk64/steamclient.so: cannot open shared object file: No such file or directory
[S_API] SteamAPI_Init(): Sys_LoadModule failed to load: /home/bl3/.steam/sdk64/steamclient.so
[S_API FAIL] SteamAPI_Init() failed

... and various game components clearly didn't load, such as some opening text, textures on the "skip cutscene" buttons, and then the main menu never loads; it just gets stuck on a loading screen. All rather indicative of the EXE running but maybe being unable to load files. That's on both Proton 7.0 (from Steam) and Proton Experimental. As with BL3 and TTWL, reverting to the old Soldier does the trick, though.

A more detailed log for each of the Proton GE versions would also be useful: in addition to what you did to capture https://gist.github.com/apocalyptech/30a62ace83342519a92aa91da6f20dd5, please set STEAM_LINUX_RUNTIME_VERBOSE=1.

Here's that log of me running Witcheye with, I believe, Proton Experimental: https://gist.github.com/15ef21b289f63682a56cdd081bdc63c5

@TTimo
Copy link
Collaborator

TTimo commented Jul 22, 2022

default has been rolled back to previous_release while we work on the problem

@smcv
Copy link
Contributor

smcv commented Jul 22, 2022

The situation is now:

As far as I can tell so far, the problem scenario that triggers the regression involves having a path that we want to share with the container environment, for which either the path itself is a symbolic link, or one of the directories "above" it is a symbolic link. The container runtime has code to handle this situation, but it regressed in the last few weeks, most likely as a result of changes made for interoperability with FEX-Emu.

@smcv
Copy link
Contributor

smcv commented Jul 22, 2022

The symptoms you observed with Wonderlands + GE-Proton7-20 are consistent with #518. I've been able to reproduce the same situation as #518 and I'm testing some possible solutions, which I hope will also work here.

The symptoms you observed with Witcheye + Proton Experimental also seem very similar to #518, so, again, I'm hoping the solution will be the same.

The symptoms you observed with Borderlands 3 + Proton-5.21-GE-1 are not the same, which means there might be something different going on. Because you're varying more than one thing at a time (game and also Proton version), and didn't attach a log for this scenario, it's hard to tell which factor is most important to trigger this.

Please could you get a log with STEAM_LINUX_RUNTIME_VERBOSE=1 and the client_beta branch of SteamLinuxRuntime_soldier, together with Proton-5.21-GE-1, and a "known good" game other than Borderlands 3 in the same Steam installation - perhaps Expendabros or Witcheye? I want to narrow down whether this is triggered by something that is special/different about Proton-5.21-GE-1, or by something that is special/different about Borderlands 3.

It would also be useful to see a similar log from Borderlands 3 with the client_beta branch of SteamLinuxRuntime_soldier, still with Proton-5.21-GE-1 as per your original report.

It would also be interesting to see a similar log for Borderlands 3 but with the compat tool switched to Proton Experimental, but since you've said you're concerned about compatibility between versions, only do this if you've backed up the saved games and compatdata first - and this is a lower priority than the other things I asked for in this comment.

@smcv
Copy link
Contributor

smcv commented Jul 22, 2022

I think I have a solution for #517, #518, and the Wonderlands/GE-Proton7-20 and Witcheye/Proton Experimental parts of this issue, which will go through more testing and review next week.

The same change might also solve your Borderlands 3/Proton-5.21-GE-1 scenario, but I don't yet understand what is different in that scenario, so the only way to find out is to try it...

If you are comfortable with testing unreleased software, you can try it out as follows:

  • switch to the client_beta branch
  • test your game and reproduce the bug
  • replace SteamLinuxRuntime_soldier/pressure-vessel/ with the result of unpacking this pressure-vessel build (from !466)
  • test your game again, and hopefully the bug will no longer be present

It would be useful if you could share a debug log recorded with this test version, especially if it doesn't work. The same procedure you used to get your Witcheye log is suitable.

@apocalyptech
Copy link
Author

Please could you get a log with STEAM_LINUX_RUNTIME_VERBOSE=1 and the client_beta branch of SteamLinuxRuntime_soldier, together with Proton-5.21-GE-1, and a "known good" game other than Borderlands 3 in the same Steam installation - perhaps Expendabros or Witcheye? I want to narrow down whether this is triggered by something that is special/different about Proton-5.21-GE-1, or by something that is special/different about Borderlands 3.

Howdy, sure thing, here's Witcheye + client_beta + Proton-5.21-GE-1: https://gist.github.com/3837598a2160589b7229379cf5e9f625

Interestingly, Witcheye fails in the same way that it did on Proton 7 + Proton Experimental: it launches but various assets of the game don't load in, and it gets stuck on a loading screen before the main menu.

It would also be useful to see a similar log from Borderlands 3 with the client_beta branch of SteamLinuxRuntime_soldier, still with Proton-5.21-GE-1 as per your original report.

Here 'tis: BL3 + client_beta + Proton 5.21-GE-1: https://gist.github.com/0e0177635c7446c3826e018a26938cf3

I actually don't see that "Failed to execute child process" on the console when grabbing the logs this way, but that is present down at the bottom of the log.

If you are comfortable with testing unreleased software, you can try it out as follows:

Sure thing! It looks like that build actually probably does the trick, both for the weirder 5.21-GE that I was using, and for Proton Experimental:

Witcheye + client_beta + !466 + Proton-5.21-GE-1: Works great! - https://gist.github.com/30131a26826b85114631328169a0d2a2
Witcheye + client_beta + !466 + Proton Experimental: Works great! - https://gist.github.com/b7ca4e58a6bf37e0e3808bdda8b6b03e
BL3 + client_beta + !466 + Proton-5.21-GE-1: Works great! - https://gist.github.com/4e53bfcc0ebad2eefd06573effdf8c03

I'll stick with that for the time being and let you know if I run into any further issues.

@smcv
Copy link
Contributor

smcv commented Jul 25, 2022

It looks like that build actually probably does the trick, both for the weirder 5.21-GE that I was using, and for Proton Experimental

Great, that's what I was hoping. The fix for this should be in the next beta. If you stay on the client_beta branch, you should get upgraded to it automatically.

Witcheye fails in the same way that it did on Proton 7 + Proton Experimental

Witcheye and Wonderlands seem to be the same root cause as #518: the regression involving symlinks results in the Steamworks API failing to load. Different games have different symptoms when the Steamworks API is missing or broken: some fail completely (e.g. if they do a license check against Steam, like #518), some run with reduced functionality, and some are unaffected because they don't use the Steamworks API for anything important (or at all).

I actually don't see that "Failed to execute child process" on the console when grabbing the logs this way, but that is present down at the bottom of the log.

That's normal: enabling the logging mechanism captures everything that would previously have gone to the terminal where you ran Steam, from the container setup code or any component below it (including Proton and the game), so that we get all the necessary information in one place and in the correct order.

@smcv
Copy link
Contributor

smcv commented Jul 25, 2022

something that is special/different about Borderlands 3

Looking at your BL3 logs, there is indeed something different going on here, which is not the same as the other games (or the other users with different versions of this regression).

Witcheye and Wonderlands are working the way they are meant to. We can see Steam telling the container runtime which compatibility tool you have asked it to use:

15:19:51.018168: pressure-vessel-wrap[425497]: D: 	'STEAM_COMPAT_INSTALL_PATH=/games/bl3_steam/games/steamapps/common/Witcheye'
15:19:51.018180: pressure-vessel-wrap[425497]: D: 	'STEAM_COMPAT_LIBRARY_PATHS=/games/bl3_steam/games/steamapps'
...
15:19:51.018223: pressure-vessel-wrap[425497]: D: 	'STEAM_COMPAT_TOOL_PATHS=/games/bl3_steam/games/steamapps/common/Proton - Experimental:/games/bl3_steam/games/steamapps/common/SteamLinuxRuntime_soldier'

pressure-vessel is meant to make sure these end up visible in the container, even if they are outside your home directory, outside your default Steam game library and so on, so that Proton can work. The regression that you reported was partially breaking this, but in general it works.

But for BL3, some of these are set to empty values:

14:59:48.306202: pressure-vessel-wrap[418948]: D: 	'STEAM_COMPAT_INSTALL_PATH=/games/bl3_steam/games/steamapps/common/Borderlands 3'
14:59:48.306214: pressure-vessel-wrap[418948]: D: 	'STEAM_COMPAT_LIBRARY_PATHS='
...
14:59:48.306250: pressure-vessel-wrap[418948]: D: 	'STEAM_COMPAT_TOOL_PATHS='

I think this is why you saw a different, worse failure mode for this regression with BL3 (Proton not found) than for the other two games you tried (Proton works correctly, but Steamworks API not found). pressure-vessel takes all the compatibility tools that it is told about and makes them available in the container; but that didn't work for BL3, because when running BL3, it was not told that they exist, so it didn't know that it needs to look for them.

When pressure-vessel is working correctly (after fixing the regression), it happens to work in your specific case, because your Proton 5.21 compat tool happens to be installed in /games/bl3_steam/root/compatibilitytools.d/, which gets into the container as part of /games/bl3_steam/root. However, if this was an official Proton build installed in a non-default Steam game library, it might not have appeared in the container.

Is this something you have configured yourself, for example by putting STEAM_COMPAT_LIBRARY_PATHS= and/or STEAM_COMPAT_TOOL_PATHS= in the game's Launch Options?

Or is Borderlands 3 special in some way, like perhaps being a non-Steam game that you have copied into a Steam directory, or some other special property that isn't true for Wonderlands and Witcheye?

@smcv
Copy link
Contributor

smcv commented Jul 27, 2022

The fix for this should be in the next beta. If you stay on the client_beta branch, you should get upgraded to it automatically.

This has gone out in SteamLinuxRuntime_soldier build ID 9200891. Specifically, what you want to see in VERSIONS.txt is pressure-vessel 0.20220726.0 or later. After upgrading, please verify integrity on "Steam Linux Runtime - soldier" to clear out any remnants of an unofficial pressure-vessel build.

This version should be a full solution for the failure to start Wonderlands and Witcheye.

I am still waiting for more information about what is different about Borderlands 3 in your setup: that part is not 100% solved, but the updated runtime will work around it in practice.

@apocalyptech
Copy link
Author

I am still waiting for more information about what is different about Borderlands 3 in your setup: that part is not 100% solved, but the updated runtime will work around it in practice.

Yeah, sorry about that! Haven't had an opportunity to look into this any more -- it might be until the weekend before I'm able to give it another go. I'll let you know what I find, though!

@smcv
Copy link
Contributor

smcv commented Jul 27, 2022

Haven't had an opportunity to look into this any more

No problem. I'm confident that the main practical problem here is solved, but I'd prefer to get a better idea of what's happening with BL3 in your logs before closing the issue, in case it's something important that will break for other people.

@smcv
Copy link
Contributor

smcv commented Aug 4, 2022

This has gone out in SteamLinuxRuntime_soldier build ID 9200891.

This version has now been promoted to the default branch, and no longer requires using a beta.

Let's leave this issue open until we have a better understanding of what's going on with the empty STEAM_COMPAT_LIBRARY_PATHS and STEAM_COMPAT_TOOL_PATHS in BL3 in your installation.

@apocalyptech
Copy link
Author

apocalyptech commented Aug 5, 2022

Hello! Apologies for the radio silence on my end; Real Life's been busy.

Anyway, I thought I'd had a few things on my plate to try, but I see really all you wanted was some info on the game's install, etc. There wouldn't've been anything funky about the game install itself -- that was just done via Steam as usual. There's no launch options set for the game, either:

screenshot-2022-08-05-13_54_54

Now that I've looked a bit more closely at those slr-*.log files myself, I agree that it's pretty mystifying. I cobbled together a quick script to strip out timestamps+pids for easier diffing between 'em, and it doesn't seem to show much significant to me, at least:

Both games are still working fine, on the latest SLR-soldier.

Let me know if there's something else I can dig into here!

Edit: could that GE build be doing game-specific fixes itself? Maybe that's the culprit here... In the end, I was thinking about getting this prefix upgraded to something more current anyway (be it a Steam-provided Proton or a more current GE). I could go ahead and pull the trigger on that, if you'd like that datapoint. I'll leave it for the time being, in case there's something specific you'd like me to poke at with the current config.

@smcv
Copy link
Contributor

smcv commented Aug 8, 2022

There's no launch options set for the game, either

This seems quite mysterious. @TTimo, do you know of anything inside Steam itself that could make it set $STEAM_COMPAT_LIBRARY_PATHS and $STEAM_COMPAT_TOOL_PATHS to empty strings, instead of the correct :-separated lists of paths?

could that GE build be doing game-specific fixes itself?

I don't see how it could have this effect even if it wanted to. The way this works is that Steam runs SLR_soldier, and tells SLR_soldier to run Proton (Proton-GE in your case) inside the container. At the time we run the SLR_soldier entry point, Proton has not yet had an opportunity to mess with the environment.

You can see this at the top of your log: the SLR_soldier entry point was run as

_v2-entry-point \
/games/bl3_steam/root/compatibilitytools.d/Proton-5.21-GE-1/proton \
waitforexitandrun \
/games/bl3_steam/games/steamapps/common/Borderlands\ 3/OakGame/Binaries/Win64/Borderlands3.exe

which means "start the container, and then inside the container, run .../proton waitforexitandrun .../Borderlands3.exe".

(In any case there is no reason why Proton-GE would want to clear out the values of those variables: interfering with the communication channel between Steam and SLR_soldier is only going to break games, and won't fix them.)

@TTimo
Copy link
Collaborator

TTimo commented Aug 8, 2022

Those environment variables will be empty if Proton-GE does not enable session mode in it's toolmanifest.vdf ("use_sessions" "1")

@TTimo
Copy link
Collaborator

TTimo commented Aug 8, 2022

You may be able to workaround the issue in Proton-GE by settings STEAM_COMPAT_FORCE_SESSIONS before running Steam.

@apocalyptech
Copy link
Author

Those environment variables will be empty if Proton-GE does not enable session mode in it's toolmanifest.vdf ("use_sessions" "1")

Hm, looks like that is set in that file, btw. And I was using the same GE install for Witcheye, which does have those vars:

$ cat /games/bl3_steam/root/compatibilitytools.d/Proton-5.21-GE-1/toolmanifest.vdf
"manifest"
{
  "version" "2"
  "commandline" "/proton %verb%"
  "require_tool_appid" "1391110"
  "use_sessions" "1"
}

You may be able to workaround the issue in Proton-GE by settings STEAM_COMPAT_FORCE_SESSIONS before running Steam.

Looks like that didn't change anything, btw, apart from that env var showing up in the log output. Here's a timestamp-and-PID-stripped version of the log (for easier diffing with the previous ones, if you like) - https://gist.github.com/4a253854a27bdd5393f5e4f8b6a1a8fb

(And just to clarify: the game's working just fine for me -- the absence of those vars doesn't seem to be causing any apparent issues ATM.)

@smcv
Copy link
Contributor

smcv commented Aug 9, 2022

Those environment variables will be empty if Proton-GE does not enable session mode in it's toolmanifest.vdf ("use_sessions" "1")

I think we want the Steam client to set $STEAM_COMPAT_LIBRARY_PATHS and $STEAM_COMPAT_TOOL_PATHS whenever we're using compat tools (at least SLR, but ideally any compat tool at all), whether the compat tool enables "session mode" or not. SLR needs those environment variables so that it can tell what it needs to share with the container. @TTimo, would it be possible to make that change in future Steam clients? I've opened steamrt/tasks#150 to track this internally.

@smcv
Copy link
Contributor

smcv commented Aug 9, 2022

You may be able to workaround the issue in Proton-GE by settings STEAM_COMPAT_FORCE_SESSIONS before running Steam.

Looks like that didn't change anything, btw, apart from that env var showing up in the log output.

How exactly did you set this environment variable? I ask because setting it in Launch Options probably will not work. Because it alters the behaviour of the Steam client, it needs to be in the Steam client's execution environment, more like this:

$ export STEAM_COMPAT_FORCE_SESSIONS=1
$ steam

(And just to clarify: the game's working just fine for me -- the absence of those vars doesn't seem to be causing any apparent issues ATM.)

It's working fine for you because you happen to have your Proton-GE build installed inside the Steam installation root (the directory that ~/.steam/root points to), and SLR always makes sure the Steam installation root is available inside its container.

My concern is that if someone was in almost your situation, but using an official build of Proton, and they had configured a custom Steam library like /large-disk-drive/SteamLibrary and installed the official Proton build into that custom library, SLR would be relying on Steam telling it how to find that custom library via STEAM_COMPAT_TOOL_PATHS and STEAM_COMPAT_LIBRARY_PATHS - but if Steam doesn't do that reliably, then SLR will become unreliable. To avoid that happening, I'd like to understand why those two variables are getting cleared, so we can stop doing that.

@smcv
Copy link
Contributor

smcv commented Aug 9, 2022

I was using the same GE install for Witcheye

It's still a mystery to me why BL3 and Witcheye are behaving differently here, with what ought to be the same settings.

@apocalyptech
Copy link
Author

How exactly did you set this environment variable? I ask because setting it in Launch Options probably will not work. Because it alters the behaviour of the Steam client, it needs to be in the Steam client's execution environment, more like this:

Indeed -- I'd launched like so:

[bl3@arrakis ~]$ STEAM_LINUX_RUNTIME_LOG=1 STEAM_LINUX_RUNTIME_VERBOSE=1 STEAM_COMPAT_FORCE_SESSIONS=1 steam
steam.sh[1853916]: Running Steam on arch rolling 64-bit

Not exactly the same as using export beforehand, but I believe those two are functionally equivalent? I could export the var first, instead, if you like.

@TTimo
Copy link
Collaborator

TTimo commented Aug 9, 2022

Hm, looks like that is set in that file, btw. And I was using the same GE install for Witcheye, which does have those vars:

Thanks. It looked like an easy possibility for those to not get set when I had a look in the Steam client code, so I wanted to make sure.

@smcv
Copy link
Contributor

smcv commented Aug 9, 2022

Not exactly the same as using export beforehand, but I believe those two are functionally equivalent?

Yes they are, and the way you tried is fine.

@apocalyptech
Copy link
Author

Hm, complicating matters slightly here is that an update was just pushed to Borderlands 3 today, and now the game appears to not launch for me. Whether that's got something to do with the weirdness we've been looking into or something screwy with the update (or perhaps some new, hitherto unnoticed issue with my setup) I couldn't say. I'd been keeping the game installed with that old GE-5.21 install in case there was anything else you wanted me to check out, but I could always go ahead and move that over to using something more modern, to see if that'd clear this new problem up for me. (Wonderlands and Witcheye continue to launch just fine for me.)

The behavior at launch now, btw, is that I get a little Windows dialog saying "Steam must be running in order to launch this game. Please restart the game after starting the Steam client." Then once I hit "OK" on that, I get a brief splash screen for the game and it exits. So the game can't find Steam, apparently.

Here's a fresh SLR log, once again stripped of timestamps + pids, for easier diffing (I should maybe add in something to normalize the tempdir names so that those don't show up in diffs, too...): https://gist.github.com/b338c6f28ba04ec3ef815805a3410bc2

Not much stands out to me in there -- it looks basically just like my previous BL3 launch logs. The one difference which I did see was this, near the very end:

--- slr-bl3-app397540-t20220805T140010.log      2022-08-18 12:02:23.530545540 -0500
+++ slr-bl3-post_patch_no_launch-app397540-t20220818T120025.log 2022-08-18 12:02:23.600544859 -0500
-Loading Library: Z:/games/bl3_steam/games/steamapps/common/Borderlands 3/Engine/Binaries/ThirdParty/Steamworks/Steamv147/Win64/steam_api64.dll
+Loading Library: Z:/games/bl3_steam/games/steamapps/common/Borderlands 3/Engine/Binaries/ThirdParty/Steamworks/Steamv154/Win64/steam_api64.dll
+Loading Library: C:\windows\system32\winex11.drv

... so they've bumped up their steam_api64.dll version inbetween patches. So I suspect that this new problem has nothing to do with what we've been talking about here, and may just be a problem with the game itself.

Anyway, I bring it up in case it turns out to be important anyway. Cheers!

@apocalyptech
Copy link
Author

Okay, well, I ended up biting the bullet and upgrading BL3 to a newer Proton (just used the same one I'd been using for Wonderlands, for simplicity's sake), since I maintain various modding things for BL3 and wanted to be able to launch it. Bringing that up to GE Proton-7.20 did let the game launch properly again, but interestingly, it still does not have those two environment variables specified -- STEAM_COMPAT_LIBRARY_PATHS and STEAM_COMPAT_TOOL_PATHS appear to still be blank. I updated my slr-log-stripping script to turn tmp-<randomchars> into a literal tmp-RANDOM instead, for even easier diffing:

Here's my last failed launch from 5.21, stripped out for diffs: https://gist.github.com/19152a0fc9766b6fcf9205ed115c14d4
And the working launch with 7.20, also stripped for diffs: https://gist.github.com/531be1a44051b4b1b3b06bc2b1020b33
Witcheye comparison (from 5.21, where it does have those vars): https://gist.github.com/d10be5cff2fd96cc5a06d783377bd2ff

As before, nothing really pops out at me, apart from those vars being mysteriously blank for BL3. I'll give it a go on an actually official Proton within the next day or two here.

@apocalyptech
Copy link
Author

Okay, and odder still, I did try this on Proton Experimental tonight as well, and even in that environment, it seems BL3's missing those env vars: https://gist.github.com/0b7e36fd7873bf86d348e77cf5b81f2e

@TTimo
Copy link
Collaborator

TTimo commented May 25, 2023

In case you are still running into this, the next steam beta update has a change that may fix the problem. Would be curious to know if that helped.

@TTimo TTimo assigned TTimo and unassigned smcv May 25, 2023
@kisak-valve
Copy link
Member

Closing pending feedback.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

4 participants