Windows Games Won't Run, Seem to not detect the presence of Steam (though it launched them)

[BlackJar72]

I'm running Kubuntu 22.04, and currently Steam Client built July 19, 2022 at 20:51:26 (API v20). I have opted in for beta, but changing the setting to opt out did not fix the problem. Checking for updates currently reveals that Steam is up to date.

All games ran correctly in the recent past; while I didn't actively test them on July 19 as there seemed to be no problem, as of then I can confirm that Dark Souls remastered was working. No changes have been made to my system since things were working.

The Problem

As of this morning (July 20th) most Windows games do not work:

• Dark Souls: Remastered -- error message "Failed to initialize Steam"
• Bioshock -- error message "Game cannot launch unless Steam is running"
• The Sims 4 -- error Message "Try relaunching game to continue linking accounts"
• Skyrim and Thief: Deadly Shadow -- no error message, quietly crash or exit within about 1 second.

These are launched with differnt proton versions, and I didn't not change any of the compatibility setting before the problem, so that is not it.

All these were launched from Steam, so Steam is clearly running and they should have access to it.

Games I've tried with native Linux builds (Ziggurat 2, Cities: Skylines, and Caverns of Evil) run without a problem, as does The Sims 3 interestingly. I have no way of knowing if these games do or do not use Steam DRM other than that Caverns of Evil does not since that is my own feeble attempt at making a game. I suspect that The Sims 3 may not and that that is why it works, but just a guess since I have no way of knowing about its coding. Since I don't know this I cannot guarantee it is only Windows games or if these coincidentally are not requiring Steam, but the pattern is most Windows games don't work while natively Linux games do.

I expect that games to run, they do not and give some interesting and telling error message.

My best guess from the nature of the error messages is that Windows games are no longer detecting Steam and are being shut down as a result.

Steps for reproducing this issue:

1. Start the Steam client
2. Try to launch any of the previously listed game
3. Game does not start and may give one of the error message above (depending on the game)

[kisak-valve]

Hello @BlackJar72, in general, issues Windows games run with Proton should be reported to the Proton issue tracker, however, this sounds similar to #516, so let's ponder this issue as a Steam Linux Runtime - Soldier container issue until there's a stronger hint that the issue is elsewhere. Proton 5.13 and newer use this container runtime.

Please give https://github.com/ValveSoftware/steam-runtime/blob/master/doc/reporting-steamlinuxruntime-bugs.md#essential-information a read and share the requested information.

[BlackJar72]

Dark Souls was run with Proton 7.0-3, the others with Proton 6.21-GE-1; changing Dark Souls to Proton Experiment does not change anything, though the version was not changed between working and not.

Anyway, more info:

From VERSIONS.txt

#Name	Version		Runtime	Runtime_Version	Comment
SteamLinuxRuntime	v0.20201124.0-2-g0f9c98f			# Entry point scripts, etc.
pressure-vessel	0.20201203.0+srt1	scout	0.20201203.1	# pressure-vessel-bin.tar.gz
soldier	0.20201203.0	soldier	0.20201203.0	# com.valvesoftware.SteamRuntime.Platform-amd64,i386-soldier-runtime.tar.gz

For a log:

https://www.mediafire.com/file/ele6odjdac3rct9/pressure-vessel.log/file

System Info:

https://www.mediafire.com/file/508mcmcj659pf3v/SystemInfoSteam.txt/file

[kisak-valve]

Thanks, one more quick question before we'll need to hear from a runtime maintainer, if you go to Steam Linux Runtime - Soldier in Steam's library view -> Properties -> Betas and select the previous_release beta, do the games run as expected?

[BlackJar72]

Thanks.
Yes, at least for Dark Souls, it runs fine.
I haven't test all those games (I suspect its the same problem, but I can test-run them if you like).

[smcv]

soldier 0.20201203.0

Wait, what? That's very old. Are you sure you're looking in the right place?

[smcv]

According to the system info, you're actually using soldier 0.20220628.0, which seems more correct. I think perhaps you're finding an outdated VERSIONS.txt somewhere else?

[smcv]

I think this is the root cause:

"soldier runtime container" information:
{
  "can-write-uinput" : true,
  "steam-installation" : {
    "path" : null,
    "data_path" : null,
    "bin32_path" : null,
    "steamscript_path" : "/usr/games/steam",
    "steamscript_version" : "1.0.0.74-1ubuntu2/Ubuntu",
    "issues" : [
      "cannot-find",
      "dot-steam-steam-not-symlink",
      "cannot-find-data",
      "dot-steam-steam-not-directory",
      "dot-steam-root-not-symlink",
      "dot-steam-root-not-directory",
      "missing-steam-uri-handler",
      "unexpected-steam-uri-handler"
    ]
  },

specifically the dot-steam- bits. Outside the container, ~/.steam/steam and ~/.steam/root are correctly symlinks to /opt/home/jared/Steam/debian-installation, but inside the container, for some reason they're missing.

There is code in pressure-vessel that is meant to make all these work, and it's working here in my testing, but it seems like it isn't working correctly for you. I'll look into this in more detail tomorrow and try to work out what could be going wrong.

It would be very useful to know whether there are any symbolic links involved in paths like /opt/home/jared/Steam/debian-installation, the name of your home directory, ~/.steam and so on. Seeing /opt/home makes me suspect that /home might be a symbolic link to /opt/home?

[BlackJar72]

/home/jared/0pt/Steam
...is the real directory where everything is but the steam uses installation the default...
/home/jared/.steam
...and is a symlink to...
/home/jared/0pt/Steam

I set it up this way to have games on a large HDD instead of a smaller SDD used for OS and where /home is also located and I didn't know of a way to change where Steam actually put its files (so I used a symlink to make the default point to where I wanted them to be).

/home is real, /opt/home is also real (/opt mounts my HDD), /opt/home/jared and /opt/home/jared/Steam each have a symlink pointing to them from inside /home/jared, though only the one from /home/jared/.steam to /opt/home/jared/Steam directly should actually be used by Steam.

TLDR: /home/jared is real, but /home/jared/.steam is a symlike to /opt/home/jared/Steam

EDIT: It more convoluted than I realize. /home/jared/0pt points to /opt/home/jared while /home/jared/.steam points to /home/jared/0pt/Steam -- so a symlink to a directory through a symlink (I think this evolved over several moves of files).

[smcv]

Thanks, I'm trying to replicate your setup on a test machine so I can track down where the regression is.

For information, Steam has a slightly unusual setup where it "owns" two conceptually separate directories. The packages shipped by Debian/Ubuntu, which are not official from Valve's point of view, usually combine those directories (decreasing perceived clutter but increasing confusion), and the use of the path ~/.steam/debian-installation is the compromise I was able to reach with the co-maintainer of those packages to avoid other things getting broken.

The Steam installation directory is typically ~/.local/share/Steam, or in your case ~/.steam/debian-installation. It's the direct Linux equivalent of C:\Steam on Windows. By default all your games and other large content go here (although like on Windows, you can also add secondary Steam libraries, possibly on other disks, and move games into those). This directory can be relocated to another drive or partition, and Steam is designed to cope more-or-less gracefully with that (which is why Debian/Ubuntu are able to redirect it into ~/.steam/debian-installation).

The ~/.steam directory is used by the Steamworks API as an indirection point through which to find various Steam components: if the Steam installation directory is like C:\Steam, then ~/.steam is more like C:\Users\me\Local Application Data\Steam or the Windows registry. It is normally meant to be a "real" directory, not a symlink. It contains some symlinks, notably ~/.steam/root and ~/.steam/steam which both point to the Steam installation directory, and some small configuration/state files. It doesn't contain anything large, except for ~/.steam/debian-installation on systems where the Debian/Ubuntu packages were used.

I'll look into whether we can document a more official way to relocate the Steam installation (which is the only one that really needs moving, since it's the only one that contains anything large), without relocating all of ~/.steam (which can cause problems).

I set it up this way to have games on a large HDD instead of a smaller SDD used for OS and where /home is also located

Please don't change your setup right now, because I'm looking into solutions for this regression, and I want to be able to confirm that they've worked correctly for your situation.

However, advice for anyone discovering this comment later:

The better-supported way to relocate to a location with more space is to leave the actual Steam installation where it is, add a secondary game library with more space (those instructions describe Windows but it's the same on Linux), use Steam's UI to move existing games to the new game library, and set the new game library as the default for newly-downloaded games.

If you are rearranging the filesystem hierarchy more generally, having symlinks where applications don't expect to see a symlink can cause a lot of weird problems, especially when containers are involved. It's usually more reliable to use bind mounts to rearrange the filesystem layout in a way that is more transparent to applications, even applications that are doing complicated things (like Steam/Proton).

[smcv]

The update that triggered this regression has been rolled back by copying the previous_release branch to the default branch. The situation is now:

• client_beta branch: soldier depot 0.20220720.62, has this regression (but should still work for people whose installation paths don't involve symlinks)
• default branch: soldier depot 0.20220602.55, believed to be unaffected
• previous_release: soldier depot 0.20220602.55, believed to be unaffected

[smcv]

I was able to reproduce this situation, and I think I have a solution, which will go through more testing and review next week.

If you are comfortable with testing unreleased software, you can try it out as follows:

• switch to the client_beta branch
• test an affected game and reproduce the bug
• replace SteamLinuxRuntime_soldier/pressure-vessel/ with the result of unpacking this pressure-vessel build (from !466)
• test your game again, and hopefully the bug will no longer be present

It would be useful if you could share a debug log recorded with this test version, especially if it doesn't work. The same procedure you used to get https://www.mediafire.com/file/ele6odjdac3rct9/pressure-vessel.log/file is suitable (I think you ran PRESSURE_VESSEL_VERBOSE=1 steam and recorded the output).

[BlackJar72]

client_beta branch? Am I supposed to compile my own test client? If so I'll have to do it latter this afternoon, but would be happy to do so.

[kisak-valve]

No, you do not need to compile anything to test. In Steam's library view, search for Steam Linux Runtime - Soldier, then right click on it -> Properties -> Betas -> select client_beta from the dropdown list. After that @smcv is asking you to confirm the issue is present and then grab the linked pre-release variant and try that as well.

[BlackJar72]

OK, sorry -- I was in a hurry and didn't read it carefully. Will do, I'll be back as soon as I have something to report.

[BlackJar72]

Nope, sorry, I got "Failed to initialize Steam" again.

[smcv]

Please collect a new log using the prerelease? (As well as the attempt at fixing the issue, it also has more logging in the affected code so I can see what else is going wrong.)

[BlackJar72]

OK
https://www.mediafire.com/file/rntropm3wqeytjm/pressure-vessel-2.log/file

[smcv]

That log looks like it came from the ordinary version from the client_beta branch, not the prerelease build linked above. Are you sure you were using the prerelease? I would have expected to see the extra debug messages in the log.

To try the prerelease:

• download https://gitlab.steamos.cloud/steamrt/steam-runtime-tools/-/jobs/95803/artifacts/raw/_build/production/pressure-vessel-bin+src.tar.gz
• unpack it (you should get a pressure-vessel directory containing bin, lib, etc. subdirectories)
• rename or delete the SteamLinuxRuntime_soldier/pressure-vessel/ directory
• replace it with the pressure-vessel directory you unpacked from pressure-vessel-bin+src.tar.gz

To stop using the prerelease, use the "Verify integrity" button in Steam on "Steam Linux Runtime - soldier".

[BlackJar72]

OK, that worked, my game ran fine. Thanks!

In case you need it:
https://www.mediafire.com/file/ka6o6homs0xvfrq/pressure-vessel-4.log/file

BTW, I figured out where the wrong VERSIONS.txt came from -- it seems I have two steamapps directories, and old one under .steam and new one under .steam/debian_installation -- at first I was going to the wrong one, I think from an older Steam Client version before an update shuffled things around.

[smcv]

The fixed version has gone out in the client_beta branch as SteamLinuxRuntime_soldier build ID 9200891. Specifically, what you want to see in VERSIONS.txt is pressure-vessel 0.20220726.0 or later. After upgrading, please verify integrity on "Steam Linux Runtime - soldier" to clear out any remnants of an unofficial pressure-vessel build.

I believe the situation as of today is:

• client_beta branch: soldier depot 0.20220727.64, the issue is fixed
• default branch: soldier depot 0.20220602.55, too old for this issue to exist
• previous_release: soldier depot 0.20220602.55, too old for this issue to exist

so @kisak-valve can close this issue as soon as the beta has been confirmed to fix it (unusually, we do not have to wait for it to be promoted to the default branch).

[BlackJar72]

What I get for VERSIONS.txt (under /opt/home/jared/Steam/debian_instalation, symlinked at my .steam/debain_instalation) is:

#Name	Version		Runtime	Runtime_Version	Comment
depot	0.20220727.64			# Overall version number
pressure-vessel	0.20220726.0			
scripts	v0.20220726.0-0-ga110829			# Entry point scripts, etc.
soldier	0.20220726.0	soldier	0.20220726.0	# soldier_platform_0.20220726.0/

Everything seems to work fine.

I didn't see any problems, but just in case you need it here is the output:
https://www.mediafire.com/file/t9r74t4u0wi6u5v/pressure-vessel-5.log/file

Let me know if there is anything else I can test or do.

I'll hold off on changing anything (such as using the bind mounting technique for linking directories) in case you need me to do any more tests.

[smcv]

Thanks, this all looks like it's working well. @kisak-valve, I think we can close this issue now.

[smcv]

You can clean up the filesystem layout by using bind-mounts instead of symlinks if you want, I don't need any more test results.

For your information, the intended layout is:

• You have a Steam installation in some location of your choice (referred to as STEAMROOT in the scripts, typically ~/.local/share/Steam or ~/.steam/debian-installation, yours is currently /opt/home/jared/Steam/debian-installation) and it's a real directory (not a symlink, but might be a bind-mount if you want it to be)
• ~/.steam is a real directory (not a symlink, ideally not a bind-mount either)
• ~/.steam contains symlinks bin, bin32, bin64, root, sdk32, sdk64, steam which are normally created automatically by Steam itself:
  • root and steam both point to $STEAMROOT
  • bin points to $HOME/.steam/bin32
  • bin32 points to $STEAMROOT/ubuntu12_32
  • bin64 points to $STEAMROOT/ubuntu12_64
  • sdk32 points to $STEAMROOT/linux32
  • sdk64 points to $STEAMROOT/linux64

If you exit from Steam completely, move the Steam installation to wherever you want it to be, and then run steam.sh by its absolute path (for example /opt/home/jared/Steam/debian-installation/steam.sh), it should sort out the symlinks in ~/.steam automatically.

[kisak-valve]

Thanks for retesting @BlackJar72, closing.

[smcv]

The fixed version has gone out in the client_beta branch as SteamLinuxRuntime_soldier build ID 9200891.

This is now in the default branch.