[Feature Request] Look for compatibility tools in system-wide location

[dreamer]

Your system information

• Steam client version (build number or date): Jun 6 2019, at 00:22:35
• Distribution: Fedora
• Opted into Steam client beta?: Yes
• Have you checked for system updates?: Yes

Feature request

At the moment, Steam looks for compatibility tools in the following locations:

<any game library>  # for stable Proton releases
~/.local/share/Steam/compatibilitytools.d  # or wherever XDG_DATA_DIR points to
~/.steam/root/compatibilitytools.d  # usually root dir links to real location of Steam dir
~/.steam/compatibilitytools.d  # as last resort, I guess

My request is to add at least one system-wide path to this list, e.g.:

/usr/share/steam/compatibilitytools.d

and, optionally:

/usr/local/share/steam/compatibilitytools.d

Why?

If Steam respected these directories when scanning *.vdf files, then community-created compatibility tools could be distributed through system packages - simplifying the distribution of such tools.

More specifically, I want to be able to package my tool (steam-dos) this way, so users could install a single package instead of all dependencies by hand and then unpacking a tarball.

Other tools, that could use that directory: community-maintained Proton forks (e.g. proton-tkg), or other (not existing yet) tools, e.g. for launching games through ScummVM or GZDoom.

Details

To avoid any possibility of conflict with Proton, Steam should filter-out any tool with the same internal name or user-visible name as compatibility tools already installed in user library.

Alternative

There's a potential solution for allowing distribution of such tools through Steam store (as Proton itself is distributed), BUT… after reading Steamworks rules [1], I must conclude it is not allowed. Compatibility tools do not fall into any of the categories listed on Support page [2]. There's also no way for me to recuperate cost of sign-up fee when providing compatibility tool at no cost.

[1] https://partner.steamgames.com/newpartner/?signup_type=1 paragraph "Accepted types of content"
[2] https://help.steampowered.com/en/wizard/HelpWithPublishing?issueid=925

[sizeofbool]

My request is to add at least one system-wide path to this list, e.g.:

/usr/share/steam/compatibilitytools.d

and, optionally:

/usr/local/share/steam/compatibilitytools.d

This is a bad idea, because these folders inaccessible without root priveleges and nobody wants to run Steam Client as root.
"compatibilitytools.d" folder is perfectly capable to hold as many different custom Proton buiilds as you want without need for them to actually be inside this folder.
To achieve this, you should use "compatibilitytool.vdf" config file to describe and redirect Steam Client to any valid custom Proton build location using "install_path" directive.

[dreamer]

@sizeofbool Read my feature request (with understanding) before commenting. This is about packaging compatibility tools (different than official Proton) through distribution packages, not about "running Steam as root".

[Faalagorn]

@sizeofbool From what I know, it's more similar akin to having wine binaries installed system-wide (to be read-only for user to copy the files for each prefix) and only making prefixes writable.

Either way, this request sounds like a good idea for me, I'd love to have unofficial proton version managed by my package manager instead of having to manually update it (which boasts both the compatibility and security since expoits would be fixed along the system update).

EDIT: Alternatively, another possible way to possibly distribute the items would be as steam workshop item for Proton, but that would require Steam to add workshop support for Proton.

[sizeofbool]

@sizeofbool From what I know, it's more similar akin to having wine binaries installed system-wide (to be read-only for user to copy the files for each prefix) and only making prefixes writable.

This is what I've tried to say. I have a custom Proton builds located on different disk and not in my user's home directory.
By simply creating a new "vdf" file, or by editing old one, you can point Steam Client to that location and it doesn't matter where actual custom Proton build files are stored.
This functionality is already implemented in Steam Client.

Additionally, I doubt official package distribution staff of your distro will be interested in packaging and maintaining custom Proton builds for Steam Client. Which leads to requirement to use unofficial repositories, adding external untrusted signing keys, which could be potentially harmful and insecure.
You will be installing arbitrary packages from untrusted sources with root privileges and this is why I thought it will be a bad idea.
It is so easy to sneak in some rogue package into your system from unofficial repos to replace trusted and official one with the same name.

[dreamer]

Which leads to requirement to use unofficial repositories (blah, blah)

This is up to users to decide, and many Linux distributions won't have a problem packaging such tools as long as they are free and open source.

But again, @sizeofbool READ BEFORE LEAVING COMMENTS. In my feature request I wrote:

I want to be able to package my tool (steam-dos) this way, so users could install a single package instead of all dependencies by hand and then unpacking a tarball. Other tools, that could use that directory: community-maintained Proton forks (e.g. proton-tkg), or other (not existing yet) tools, e.g. for launching games through ScummVM or GZDoom.

[gasinvein]

This could be very useful for Steam flatpak. We want the ability to add custom Proton builds as app extensions for the Steam app. Currently our only option is to copy the extensions files to the default, per-user location, which is ugly and unreliable. Support for system-wide (or app-wide in flatpak case) compability tools location would solve this.
See flathub/com.valvesoftware.Steam#349 for details.
If someone consider implementing this, please note that flatpak uses the /app prefix.

[dreamer]

Ok, there was no real response from Valve for almost a month now… maybe I'll start poking a bit.

@Plagman what do you think about this feature request? In one form it could be simply a new evironment variable (STEAM_COMPATIBILITY_TOOLS_D?) pointing to the directory where Steam would additionally look for compatibiliy tools' *.vdf files; distribution packages would be in charge of providing value for this variable (just like they do for STEAM_FRAME_FORCE_CLOSE right now).

[Plagman]

Thanks for the suggestion; the next Beta cycle will include support for this. The paths are searched in that order:

/usr/share/steam/compatibilitytools.d
/usr/local/share/steam/compatibilitytools.d
Colon-separated global paths in $STEAM_EXTRA_COMPAT_TOOLS_PATHS
~/.steam/root/compatibilitytools.d (steam install folder symlink)

Any tool with the same name as another will take precedence over the one registered prior to it, so the ones in the Steam install folder will always win.

[ghost]

That's why we all love valve ! by the way, the steam-dos application is really cool !

[dreamer]

Are there any updates on this feature? I am monitoring Steam Beta releases and keep test compat. tools installed just in case it was released without official announcement, but it does not work still.

[Freso]

@dreamer It seems to be working for me with the most recent Steam client beta. Is it not working for you still? (The new/current beta cycle has a noticeably different UI compared to previous Steam client versions, so it’s pretty easy to tell if you’re on it or not.)

[dreamer]

Yes, it seems to be working (it was released with new Steam Library Beta). I haven't tested all edge cases yet, though - so didn't want to declare it fixed prematurely.

Few days ago I implemented new make install target for Boxtron (dreamer/boxtron@8b7b2af) - please note, it is on "next" branch - not released yet - I need to write guide for packagers where to place the files, to avoid some headaches for end-users (and myself) in the future.

I'll update this issue when my testing will be concluded, but it looks good :)

[dreamer]

So finally, I had some time to finish testing. I went over the following scenarios:

• installation in /usr/local tree
• installation in /usr tree
• installation in directory included in STEAM_EXTRA_COMPAT_TOOLS_PATHS
• installation in a directory included in STEAM_EXTRA_COMPAT_TOOLS_PATHS where paths include whitespace characters

Scenarios were repeated for *.vdf file inside tool dir (as compatibilitytool.vdf with relative installation dir '.') and as a separate file (boxtron.vdf with an absolute path to installation directory). Also, I verified that tool installed in user library overrides system-wide installation.

It all worked perfectly :) (testing was performed on Fedora 30 with the package from RPM Fusion repo).

The only caveat is: this feature is available only on Steam Beta for now - but it's good enough for community to start work on packaging compatibility tools.

If anyone wants to verify it as well, then I just released Boxtron 0.5.3, which can be installed system-wide with sudo make install (remember to install all dependencies before using it).

@Plagman thanks!
@kisak-valve should I close this feature request now or do you prefer to keep it open until feature is officially released?

[kisak-valve]

Hello @dreamer, this shipped in the 2019-10-30 Steam client update.

Closing as implemented.