gconv modules fail to load from /usr/lib64/gconv on Void Linux + SLR #680

oreo639 · 2024-07-14T20:07:32Z

Your system information

Steam Runtime Version:
Distribution (e.g. Ubuntu 18.04): Void Linux
Link to your full system information (Help -> Steam Runtime Diagnostics) in a Gist: https://gist.github.com/oreo639/ffa8929b50054dbdc7f134f6f20b614e
Have you checked for system updates?: Yes
What compatibility tool are you using?: Steam Linux Runtime and Proton 5.13+
What versions are listed in steamapps/common/SteamLinuxRuntime/VERSIONS.txt?
N/A
What versions are listed in steamapps/common/SteamLinuxRuntime_soldier/VERSIONS.txt?

#Name	Version		Runtime	Runtime_Version	Comment
depot	0.20240618.92327			# Overall version number
pressure-vessel	0.20240610.0	scout		# pressure-vessel-bin.tar.gz
scripts	0.20240610.0			# from steam-runtime-tools
soldier	0.20240618.92327	soldier	0.20240618.92327	# soldier_platform_0.20240618.92327/

What versions are listed in steamapps/common/SteamLinuxRuntime_sniper/VERSIONS.txt?

#Name	Version		Runtime	Runtime_Version	Comment
depot	0.20240618.92328			# Overall version number
pressure-vessel	0.20240610.0	scout		# pressure-vessel-bin.tar.gz
scripts	0.20240610.0			# from steam-runtime-tools
sniper	0.20240618.92328	sniper	0.20240618.92328	# sniper_platform_0.20240618.92328/

Please describe your issue in as much detail as possible:

Attempting to run 64-bit software that relies on iconv(), including Easy Anticheat, causes iconv() to fail.

On Void linux the $(libdir) used when compiling glibc is /usr/lib{bits} which results in glibc attempting to search for modules in /usr/lib64/gconv, however, pressure-vessel only mounts the gconv modules to /usr/lib/gconv and /usr/lib32/gconv leaving /usr/lib64/gconv as an empty directory causing iconv() to fail to find any gconv modules.

Steps for reproducing this issue:

Compile glibc with --libdir=/usr/lib{bits}: https://github.com/void-linux/void-packages/blob/master/srcpkgs/glibc/template#L67
Install steam and the steam linux runtime
Download the following example and compile with gcc iconv.c -o iconv: https://gist.github.com/lesstif/626451b603e7e8a8d6b65df8eced29f9
Run "$HOME/.local/share/Steam/steamapps/common/SteamLinuxRuntime_soldier"/_v2-entry-point --verb=waitforexitandrun -- ./iconv and note that it outputs "not supported code"
Note that the above command works if you add GCONV_PATH="/usr/lib/gconv" to the beginning.

See also: void-linux/void-packages#41388 (comment)

The text was updated successfully, but these errors were encountered:

smcv · 2024-07-15T11:12:36Z

To confirm some facts about how your system is working, please capture a verbose log from a Void Linux system. The usual way to do this is to set the launch options of a game that uses SLR or Proton to STEAM_LINUX_RUNTIME_VERBOSE=1 STEAM_LINUX_RUNTIME_LOG=1 %command%, then launch that game, then look for .../SteamLinuxRuntime_soldier/var/slr-latest.log or .../SteamLinuxRuntime_sniper/var/slr-latest.log depending on which SLR/Proton version the game uses.

Or if you're more comfortable with invoking SLR more directly, you can use a command like:

STEAM_LINUX_RUNTIME_VERBOSE=1 \
$HOME/.local/share/Steam/steamapps/common/SteamLinuxRuntime_soldier'/_v2-entry-point \
--verb=waitforexitandrun -- ./iconv

and redirect its output to a log file.

smcv · 2024-07-15T11:15:06Z

The Steam Linux Runtime framework currently supports several models for how the multiarch/multilib ${libdir} could be arranged (in each case below I've mentioned 32- and 64-bit in that order):

Multiarch (Debian family): /usr/lib/i386-linux-gnu and /usr/lib/x86_64-linux-gnu
FHS multilib (Red Hat family, etc.): /usr/lib and /usr/lib64
Arch-style multilib: /usr/lib32 and /usr/lib
ClearLinux-style multilib: /usr/lib32 and /usr/lib64
Cross-compiler-style multilib (Exherbo): /usr/i686-linux-gnu/lib and /usr/x86_64-linux-gnu/lib

... and then assumes that the directory for gconv modules is ${libdir}/gconv in all cases.

It looks as though Void Linux is using a mixture of the Arch and ClearLinux layouts: the actual shared libraries are like Arch (32-bit ${libdir} is /usr/lib32, 64-bit ${libdir} is /usr/lib), but gconv modules are like ClearLinux (they're loaded from /usr/lib32 or /usr/lib64, and never from /usr/lib). Is that accurate?

okawo80085 · 2024-07-15T11:37:00Z

It looks as though Void Linux is using a mixture of the Arch and ClearLinux layouts: the actual shared libraries are like Arch (32-bit ${libdir} is /usr/lib32, 64-bit ${libdir} is /usr/lib), but gconv modules are like ClearLinux (they're loaded from /usr/lib32 or /usr/lib64, and never from /usr/lib). Is that accurate?

Not exactly, on Void Linux all of the above are present

$ ls -l /usr
...
drwxr-xr-x 168 root root 176128 Jul 12 21:00 lib
drwxr-xr-x  15 root root  24576 Jul  6 23:58 lib32
lrwxrwxrwx   1 root root      3 Feb 12 19:13 lib64 -> lib
...

However /usr/lib64 is a symlink to /usr/lib, /usr/lib32/gconv is present as well

If its recognizing the system as ClearLinux, is it failing to follow a symlink? If its recognizing the system as Arch what is it missing?

okawo80085 · 2024-07-15T11:41:52Z

To confirm some facts about how your system is working, please capture a verbose log from a Void Linux system. The usual way to do this is to set the launch options of a game that uses SLR or Proton to STEAM_LINUX_RUNTIME_VERBOSE=1 STEAM_LINUX_RUNTIME_LOG=1 %command%, then launch that game, then look for .../SteamLinuxRuntime_soldier/var/slr-latest.log or .../SteamLinuxRuntime_sniper/var/slr-latest.log depending on which SLR/Proton version the game uses.

Or if you're more comfortable with invoking SLR more directly, you can use a command like:
STEAM_LINUX_RUNTIME_VERBOSE=1 \
$HOME/.local/share/Steam/steamapps/common/SteamLinuxRuntime_soldier'/_v2-entry-point \
--verb=waitforexitandrun -- ./iconv
and redirect its output to a log file.

Will do later

smcv · 2024-07-15T11:45:07Z

pressure-vessel doesn't really specifically recognize the system as anything (that would scale really poorly - we'd have to implement specific code for everyone's favourite distro, and there are hundreds of major and minor variants). Instead, it assumes that the realpath() of a library reflects the canonical path that the distro prefers to use.

This is usually true on most distros - for instance on Arch Linux (which has the same on-disk physical layout, with a symlink lib64 -> lib), the canonical path that is hard-coded into libraries that want to load plugins is something like /usr/lib/gconv.

The difference here is that on Void Linux it seems like the realpath() of glibc will be /usr/lib/libc.so.6 or similar (just like Arch), but unlike Arch, glibc wants to load its plugins from the hard-coded path /usr/lib64/gconv which doesn't match that.

I think the shortest path to fixing this is likely to be teaching pressure-vessel to set GCONV_PATH itself, internally; so that wherever your gconv modules turn up inside the container, glibc will be forced to load them.

if you add GCONV_PATH="/usr/lib/gconv" to the beginning

I think a better workaround would be GCONV_PATH=/usr/lib/gconv:/usr/lib32/gconv, which would also work for 32-bit games.

If you set the Launch Options of a game that uses EAC to

GCONV_PATH=/usr/lib/gconv:/usr/lib32/gconv %command%

does that work around the EAC issue? If yes, then that would be the short term workaround that I would suggest.

smcv · 2024-07-15T11:49:46Z

@kisak-valve or @oreo639, please could you retitle this to "gconv modules fail to load from /usr/lib64/gconv on Void Linux + SLR" to set its scope?

This looks like a distro-specific issue affecting Void Linux only (and maybe a few other distros that are unusual in the same way), and the suggested workaround is hopefully correct for Void Linux but would not work for other major distro families. I don't want to get non-Void-Linux users replying to this issue with unrelated EAC problems, that would just cause confusion and make it take longer to fix anything!

smcv · 2024-07-15T13:26:50Z

If Void Linux developers want to make it not be an outlier in this way, the way to achieve that would be to ensure that the path you hard-code into your glibc as the place it wants to look for its gconv modules is the one that doesn't involve traversing a symlink: /usr/lib32/gconv for 32-bit (same as now), but /usr/lib/gconv for 64-bit (replacing the current /usr/lib64/gconv). That would result in this particular part of Void Linux being "the same shape" as e.g. Arch, which is something that already works.

I don't know whether that's an option or whether it would break some other aspect of the OS design (this is the first time I've really looked at Void Linux!) but it's a simplification that might be helpful in general.

As a side benefit, that would also make loading the modules very very slightly more efficient (although likely not enough to show up in anything except a highly artificial benchmark).

oreo639 · 2024-07-16T11:05:53Z

I don't know whether that's an option or whether it would break some other aspect of the OS design (this is the first time I've really looked at Void Linux!) but it's a simplification that might be helpful in general.

Not really feasible afaict. Arch handles it by just having lib32-* packages specifically for 64-bit multiarch.
Our -32bit packages are generated while building packages for the i686 target so if we made /usr/lib for glibc i686, then that would break glibc-32bit (since /usr/lib is 64-bit on multiarch), currently we specify /usr/lib32 but on i686, that is a symlink, for example.

The other option to fix it on our end would be to make /usr/lib32 and /usr/lib64 not symlinks and make /usr/lib a symlink, although that would require rebuilding everything.

smcv · 2024-07-16T12:13:03Z

Our -32bit packages are generated while building packages for the i686 target so if we made /usr/lib for glibc i686, then that would break glibc-32bit (since /usr/lib is 64-bit on multiarch), currently we specify /usr/lib32 but on i686, that is a symlink, for example.

I see the problem. You want the architectures to be "symmetrical", so that you can use literally the same binaries for a purely i686 system that are also used as the 32-bit compatibility layer on an x86_64 system (like e.g. Debian does); but you also want the directory that physically contains the libraries for the system's primary architecture to be /usr/lib (like e.g. Arch does). I don't see a way to reconcile those two demands without sometimes having to load modules from a path that is a symlink, but that means SLR can't automatically understand which of the paths you consider to be the canonical one.

The other option to fix it on our end would be to make /usr/lib32 and /usr/lib64 not symlinks and make /usr/lib a symlink, although that would require rebuilding everything.

In our taxonomy of "which OS has which quirks?", that would result in the OS being "the same shape" as ClearLinux and Solus, rather than Arch; but I realise the need for a mass rebuild probably makes it unachievable.

smcv · 2024-07-17T16:59:40Z

On an affected system, with any GCONV_PATH workarounds removed, please try replacing SteamLinuxRuntime_sniper/pressure-vessel/ and/or SteamLinuxRuntime_soldier/pressure-vessel/ (whichever runtime is in use) with the result of unpacking this: https://gitlab.steamos.cloud/steamrt/steam-runtime-tools/-/jobs/618685/artifacts/raw/_build/pressure-vessel-bin.tar.gz

For Proton/EAC users, it's likely to be SteamLinuxRuntime_sniper/pressure-vessel/ that is the important one (used by Proton 8.0 and up). For manual testing, hopefully you know which one you're using. If in doubt, replace both.

To return to an official build of pressure-vessel, use Verify Integrity on the "Steam Linux Runtime 3.0 (sniper)" and "Steam Linux Runtime 2.0 (soldier)" tools in your Steam library.

(This ends up setting GCONV_PATH=/usr/lib/gconv:/usr/lib32/gconv inside the container, but does it automatically. Test-build taken from !726 v2.)

oreo639 · 2024-07-17T18:49:25Z

Unfortunately, it seems that when passing GCONV_PATH=/usr/lib/gconv:/usr/lib32/gconv to a 32-bit application, it tries to open /usr/lib/gconv modules, fails, and then gives up. The same happens when passing /usr/lib32/gconv to a 64-bit application.

glibc reads the configuration files for all the specified gconv paths (with the default ones last), however it seems to always try to open the module from the first path possible and simply fail if that module doesn't load.

smcv · 2024-07-18T11:09:49Z

Ugh, that's really annoying, and seems like an oversight in glibc.

As a plan B, it might be possible to make /usr/lib64/gconv a symlink to /usr/lib/gconv, which I think would make Void Linux's glibc happy?

okawo80085 · 2024-07-18T12:19:31Z

/usr/lib64/gconv is already a symlink to /usr/lib/gconv on void, doesn't seem to make it more happy :/

smcv · 2024-07-18T12:38:31Z

That's the situation on the host system, but I'm talking about the situation that the container runtime framework sets up inside the container (which is done programmatically). I'm looking into whether it's feasible to achieve that.

smcv · 2024-07-18T12:39:36Z

While I'm looking into that, the information I asked for in #680 (comment) would still be a useful thing for someone to capture, so we have a better understanding of Void Linux and how it interacts with the container runtime framework.

caszuu · 2024-07-18T13:26:48Z

Here's the gist of invoking iconv binary on my machine

okawo80085 · 2024-07-18T13:41:28Z

While I'm looking into that, the information I asked for in #680 (comment) would still be a useful thing for someone to capture, so we have a better understanding of Void Linux and how it interacts with the container runtime framework.

I'll do that as soon as i can, but no eta cuz we are getting unstable blackouts here

smcv · 2024-07-18T14:12:33Z

No need now, @caszuu provided equivalent information (I'm assuming their system is the same as yours in all the ways that matter here).

smcv · 2024-07-18T16:13:55Z

Here's a new build to try: https://gitlab.steamos.cloud/steamrt/steam-runtime-tools/-/jobs/620187/artifacts/raw/_build/pressure-vessel-bin.tar.gz (from !726 v3). Same test procedure as described in #680 (comment).

Whether this works or not, a new log as per #680 (comment) with the version under test, similar to the one @caszuu provided in #680 (comment), would be useful information.

If all goes well, on Void Linux this should end up putting symlinks to the host system's /usr/lib/gconv at both /usr/lib/gconv and /usr/lib64/gconv inside the container, ensuring that whichever of those paths Void Linux's glibc wants to use, it works. The log should say something like:

pressure-vessel-wrap[...]: D: Checking for gconv in /usr/lib64/gconv
...
pressure-vessel-wrap[...]: D: Checking for gconv in /usr/lib/gconv
...
pressure-vessel-wrap[...]: D: Checking for gconv in /usr/lib32/gconv
...
pressure-vessel-wrap[...]: D: Making provider gconv modules visible in container
pressure-vessel-wrap[...]: D: Removing "${container}//usr/lib/gconv"
pressure-vessel-wrap[...]: D: Creating symlink "${container}//usr/lib/gconv" -> "/run/host/usr/lib/gconv"
pressure-vessel-wrap[...]: D: Removing "${container}//usr/lib32/gconv"
pressure-vessel-wrap[...]: D: Creating symlink "${container}//usr/lib32/gconv" -> "/run/host/usr/lib32/gconv"
pressure-vessel-wrap[...]: D: Removing "${container}//usr/lib64/gconv"
pressure-vessel-wrap[...]: D: Creating symlink "${container}//usr/lib64/gconv" -> "/run/host/usr/lib64/gconv"

caszuu · 2024-07-18T22:46:53Z

Can confirm that the iconv binary (both 64 and 32 bit) and EAC games work as expected when using the patch!

@oreo639 Also unrelated to Steam, the glibc-32bit package seems be missing a gconv-modules-extra.conf without which the iconv binary fails to run. Should i make an issue about this?

oreo639 · 2024-07-19T00:11:07Z

Whether this works or not, a new log as per #680 (comment) with the version under test, similar to the one @caszuu provided in #680 (comment), would be useful information.

Seems to work: https://gist.github.com/oreo639/26fd5da965b64c4d01c6a40d8b4011c7

Also unrelated to Steam, the glibc-32bit package seems be missing a gconv-modules-extra.conf without which the iconv binary fails to run. Should i make an issue about this?

Thanks for bringing that up. I was confused how I had it but I realize what happened.
Should be resolved: void-linux/void-packages@4c973e6

kisak-valve added the Pressure Vessel label Jul 14, 2024

kisak-valve changed the title ~~gconv modules fail to load from /usr/lib64/gconv~~ gconv modules fail to load from /usr/lib64/gconv on Void Linux + SLR Jul 15, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

gconv modules fail to load from /usr/lib64/gconv on Void Linux + SLR #680

gconv modules fail to load from /usr/lib64/gconv on Void Linux + SLR #680

oreo639 commented Jul 14, 2024 •

edited

Loading

smcv commented Jul 15, 2024

smcv commented Jul 15, 2024

okawo80085 commented Jul 15, 2024

okawo80085 commented Jul 15, 2024

smcv commented Jul 15, 2024

smcv commented Jul 15, 2024

smcv commented Jul 15, 2024

oreo639 commented Jul 16, 2024 •

edited

Loading

smcv commented Jul 16, 2024

smcv commented Jul 17, 2024 •

edited

Loading

oreo639 commented Jul 17, 2024 •

edited

Loading

smcv commented Jul 18, 2024

okawo80085 commented Jul 18, 2024 •

edited

Loading

smcv commented Jul 18, 2024

smcv commented Jul 18, 2024

caszuu commented Jul 18, 2024 •

edited

Loading

okawo80085 commented Jul 18, 2024

smcv commented Jul 18, 2024

smcv commented Jul 18, 2024 •

edited

Loading

caszuu commented Jul 18, 2024

oreo639 commented Jul 19, 2024 •

edited

Loading

gconv modules fail to load from /usr/lib64/gconv on Void Linux + SLR #680

gconv modules fail to load from /usr/lib64/gconv on Void Linux + SLR #680

Comments

oreo639 commented Jul 14, 2024 • edited Loading

Your system information

Please describe your issue in as much detail as possible:

Steps for reproducing this issue:

smcv commented Jul 15, 2024

smcv commented Jul 15, 2024

okawo80085 commented Jul 15, 2024

okawo80085 commented Jul 15, 2024

smcv commented Jul 15, 2024

smcv commented Jul 15, 2024

smcv commented Jul 15, 2024

oreo639 commented Jul 16, 2024 • edited Loading

smcv commented Jul 16, 2024

smcv commented Jul 17, 2024 • edited Loading

oreo639 commented Jul 17, 2024 • edited Loading

smcv commented Jul 18, 2024

okawo80085 commented Jul 18, 2024 • edited Loading

smcv commented Jul 18, 2024

smcv commented Jul 18, 2024

caszuu commented Jul 18, 2024 • edited Loading

okawo80085 commented Jul 18, 2024

smcv commented Jul 18, 2024

smcv commented Jul 18, 2024 • edited Loading

caszuu commented Jul 18, 2024

oreo639 commented Jul 19, 2024 • edited Loading

oreo639 commented Jul 14, 2024 •

edited

Loading

oreo639 commented Jul 16, 2024 •

edited

Loading

smcv commented Jul 17, 2024 •

edited

Loading

oreo639 commented Jul 17, 2024 •

edited

Loading

okawo80085 commented Jul 18, 2024 •

edited

Loading

caszuu commented Jul 18, 2024 •

edited

Loading

smcv commented Jul 18, 2024 •

edited

Loading

oreo639 commented Jul 19, 2024 •

edited

Loading