Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Linux] Only setup Mojang JVM on glibc standard C library #110

Closed
JamiKettunen opened this issue Oct 22, 2022 · 13 comments · Fixed by #112
Closed

[Linux] Only setup Mojang JVM on glibc standard C library #110

JamiKettunen opened this issue Oct 22, 2022 · 13 comments · Fixed by #112
Assignees
Labels
bug Something isn't working
Milestone

Comments

@JamiKettunen
Copy link
Contributor

JamiKettunen commented Oct 22, 2022

Describe the bug
Mojang JVM binaries/libraries even on non-glibc systems where they're useless and don't work.

To Reproduce
PortableMC version: 3.1.1
Python version: 3.10.8
Add-ons: none
Steps to reproduce the behavior: portablemc start on Void Linux (musl) or Alpine Linux; the same steps to run work on Void Linux (glibc)

Expected behavior
PortableMC should not try to download (and use) glibc-linked Mojang JVM on e.g. musl libc Linux systems; it should've tried to use the java executable from $PATH perhaps or make you provide it via portablemc start --jvm ....

One possible solution to avoid downloading and using Mojang's JVM is to check platform.libc_ver(), as it outputs the following with the different standard C libraries:

glibc

>>> import platform
>>> platform.libc_ver()
('glibc', '2.36')

musl

>>> import platform
>>> platform.libc_ver()
('libc', '')

Additional context

$ portablemc start
[  OK  ] Resolved version 1.19.2.    
[  OK  ] Loaded version JAR.    
[  OK  ] Checked 3390 assets.
[  OK  ] Loaded pretty logger.
[  OK  ] Loaded 54 libraries. 
[  OK  ] Loaded Mojang Java 17.0.3.
[  OK  ] Username: XXXXXXXX (XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX)
Traceback (most recent call last):
  File "/bin/portablemc", line 8, in <module>
    sys.exit(main())
  File "/usr/lib/python3.10/site-packages/portablemc/cli.py", line 145, in main
    cmd(handler, ns)
  File "/usr/lib/python3.10/site-packages/portablemc/cli.py", line 314, in cmd
    handler(ns, new_context(ns))
  File "/usr/lib/python3.10/site-packages/portablemc/cli.py", line 484, in cmd_start
    start.start()
  File "/usr/lib/python3.10/site-packages/portablemc/__init__.py", line 780, in start
    self.runner([
  File "/usr/lib/python3.10/site-packages/portablemc/__init__.py", line 795, in default_runner
    subprocess.run(args, cwd=cwd)
  File "/usr/lib/python3.10/subprocess.py", line 503, in run
    with Popen(*popenargs, **kwargs) as process:
  File "/usr/lib/python3.10/subprocess.py", line 971, in __init__
    self._execute_child(args, executable, preexec_fn, close_fds,
  File "/usr/lib/python3.10/subprocess.py", line 1847, in _execute_child
    raise child_exception_type(errno_num, err_msg, err_filename)
FileNotFoundError: [Errno 2] No such file or directory: '/home/void/.minecraft/jvm/java-runtime-gamma/bin/java'
$ file /home/void/.minecraft/jvm/java-runtime-gamma/bin/java
/home/void/.minecraft/jvm/java-runtime-gamma/bin/java: ELF 64-bit LSB pie executable, x86-64, version 1 (SYSV), dynamically linked, interpr
eter /lib64/ld-linux-x86-64.so.2, for GNU/Linux 2.6.18, not stripped
$ ldd /home/void/.minecraft/jvm/java-runtime-gamma/bin/java
        /lib64/ld-linux-x86-64.so.2 (0x7f00ac125000)
        libz.so.1 => /lib/libz.so.1 (0x7f00ac107000)
        libjli.so => /home/void/.minecraft/jvm/java-runtime-gamma/bin/../lib/libjli.so (0x7f00ac0f6000)
        libpthread.so.0 => /lib64/ld-linux-x86-64.so.2 (0x7f00ac125000)
        libdl.so.2 => /lib64/ld-linux-x86-64.so.2 (0x7f00ac125000)
        libc.so.6 => /lib64/ld-linux-x86-64.so.2 (0x7f00ac125000)
Error relocating /home/void/.minecraft/jvm/java-runtime-gamma/bin/../lib/libjli.so: __strdup: symbol not found
Error relocating /home/void/.minecraft/jvm/java-runtime-gamma/bin/../lib/libjli.so: __rawmemchr: symbol not found
$ file /lib64/ld-linux-x86-64.so.2
/lib64/ld-linux-x86-64.so.2: cannot open `/lib64/ld-linux-x86-64.so.2' (No such file or directory)
$ file /lib64/ld-musl-x86_64.so.1 
/lib64/ld-musl-x86_64.so.1: symbolic link to /usr/lib64/libc.so 
@JamiKettunen
Copy link
Contributor Author

JamiKettunen commented Oct 22, 2022

On musl you also have to launch versions >=1.19 of the game with --lwjgl 3.2.3 or else it'll just crash with LWJGL 3.3.1 (or 3.3.0) at least as also documented by Alpine Linux; perhaps that also could be somehow integrated when we settle on the way to detect glibc.

@mindstorm38 mindstorm38 added this to the Version 3.1.2 milestone Oct 22, 2022
@mindstorm38 mindstorm38 added the bug Something isn't working label Oct 22, 2022
@mindstorm38 mindstorm38 self-assigned this Oct 22, 2022
@mindstorm38
Copy link
Owner

Thank you for this report! The auto JVM install definitely needs a rework for all these edge cases (non-glibc, non-supported os, ...).

@JamiKettunen
Copy link
Contributor Author

JamiKettunen commented Oct 22, 2022

One more thing I noticed, it seems the downloaded /tmp/lwjglvoid/3.2.3-build-13/libglfw.so on Void Linux (musl) as tested on my aarch64 device is also linked against glibc; do we have an option to use similar hacks to MultiMC and use system provided GLFW? Perhaps an --glfw system option would make sense to do that.
crash-2022-10-22_18.42.35-client.txt
(fwiw the aarch64 musl dynamic program loader is ld-musl-aarch64.so.1)

Results from working x86_64-musl environment:

$ file /tmp/lwjglvoid/3.2.3-build-13/libglfw.so 
/tmp/lwjglvoid/3.2.3-build-13/libglfw.so: ELF 64-bit LSB shared object, x86-64,
 version 1 (SYSV), dynamically linked, BuildID[sha1]=90d12f7a666a599abebd2913973dfec9ef6ca71c, stripped
$ ldd /tmp/lwjglvoid/3.2.3-build-13/libglfw.so 
        ldd (0x7f0f5afd0000)
        libm.so.6 => ldd (0x7f0f5afd0000)
        libdl.so.2 => ldd (0x7f0f5afd0000)
        libX11.so.6 => /lib/libX11.so.6 (0x7f0f5ac27000)
        libpthread.so.0 => ldd (0x7f0f5afd0000)
        libc.so.6 => ldd (0x7f0f5afd0000)
        libxcb.so.1 => /lib/libxcb.so.1 (0x7f0f5abfd000)
        libXau.so.6 => /lib/libXau.so.6 (0x7f0f5abf8000)
        libXdmcp.so.6 => /lib/libXdmcp.so.6 (0x7f0f5abf0000)

Results from broken aarch64-musl environment:

$ file /tmp/lwjglvoid/3.2.3-build-13/libglfw.so
/tmp/lwjglvoid/3.2.3-build-13/libglfw.so: ELF 64-bit LSB shared object, ARM aarch64,
 version 1 (SYSV), dynamically linked, BuildID[sha1]=b7de3b39ffec9b2985f2257e4bbc1c69ec77fa55, stripped
$ ldd /tmp/lwjglvoid/3.2.3-build-13/libglfw.so
	ldd (0x7f9b196000)
	libdl.so.2 => ldd (0x7f9b196000)
	libX11.so.6 => /lib/libX11.so.6 (0x7f9afde000)
	libpthread.so.0 => ldd (0x7f9b196000)
	libc.so.6 => ldd (0x7f9b196000)
Error loading shared library ld-linux-aarch64.so.1: No such file or directory (needed by /tmp/lwjglvoid/3.2.3-build-13/libglfw.so)
	libxcb.so.1 => /lib/libxcb.so.1 (0x7f9afaa000)
	libXau.so.6 => /lib/libXau.so.6 (0x7f9af96000)
	libXdmcp.so.6 => /lib/libXdmcp.so.6 (0x7f9af80000)

@mindstorm38
Copy link
Owner

Is this only the case with GLFW or with the entire LWJGL suite? Also, I wonder if modifying -Djava.library.path would work or if this can collide with native-oriented JAR library files. Could you tell me if these native JAR file are installed (.minecraft\libraries\org\lwjgl and jar files ending containing natives)?

@JamiKettunen
Copy link
Contributor Author

JamiKettunen commented Oct 22, 2022

Could you tell me if these native JAR file are installed (.minecraft\libraries\org\lwjgl and jar files ending containing natives)?

Looks like that is the case, the /tmp/lwjglvoid/3.2.3-build-13/libglfw.so sha256sum matches the one contained in lwjgl-glfw-3.2.3-natives-linux-arm64.jar (/linux/arm64/org/lwjgl/glfw/libglfw.so):

$ find ~/.minecraft/libraries/org/lwjgl -name '*.jar'
.../lwjgl-stb/3.2.3/lwjgl-stb-3.2.3-natives-linux-arm32.jar
.../lwjgl-stb/3.2.3/lwjgl-stb-3.2.3-natives-linux-arm64.jar
.../lwjgl-stb/3.2.3/lwjgl-stb-3.2.3.jar
.../lwjgl-stb/3.2.3/lwjgl-stb-3.2.3-natives-linux.jar
.../lwjgl-openal/3.2.3/lwjgl-openal-3.2.3-natives-linux-arm64.jar
.../lwjgl-openal/3.2.3/lwjgl-openal-3.2.3-natives-linux.jar
.../lwjgl-openal/3.2.3/lwjgl-openal-3.2.3-natives-linux-arm32.jar
.../lwjgl-openal/3.2.3/lwjgl-openal-3.2.3.jar
.../lwjgl-glfw/3.2.3/lwjgl-glfw-3.2.3.jar
.../lwjgl-glfw/3.2.3/lwjgl-glfw-3.2.3-natives-linux.jar
.../lwjgl-glfw/3.2.3/lwjgl-glfw-3.2.3-natives-linux-arm64.jar
.../lwjgl-glfw/3.2.3/lwjgl-glfw-3.2.3-natives-linux-arm32.jar
.../lwjgl-jemalloc/3.2.3/lwjgl-jemalloc-3.2.3-natives-linux-arm32.jar
.../lwjgl-jemalloc/3.2.3/lwjgl-jemalloc-3.2.3-natives-linux.jar
.../lwjgl-jemalloc/3.2.3/lwjgl-jemalloc-3.2.3.jar
.../lwjgl-jemalloc/3.2.3/lwjgl-jemalloc-3.2.3-natives-linux-arm64.jar
.../lwjgl-opengl/3.2.3/lwjgl-opengl-3.2.3-natives-linux-arm32.jar
.../lwjgl-opengl/3.2.3/lwjgl-opengl-3.2.3.jar
.../lwjgl-opengl/3.2.3/lwjgl-opengl-3.2.3-natives-linux.jar
.../lwjgl-opengl/3.2.3/lwjgl-opengl-3.2.3-natives-linux-arm64.jar
.../lwjgl/3.2.3/lwjgl-3.2.3.jar
.../lwjgl/3.2.3/lwjgl-3.2.3-natives-linux-arm64.jar
.../lwjgl/3.2.3/lwjgl-3.2.3-natives-linux.jar
.../lwjgl/3.2.3/lwjgl-3.2.3-natives-linux-arm32.jar
.../lwjgl-tinyfd/3.2.3/lwjgl-tinyfd-3.2.3-natives-linux.jar
.../lwjgl-tinyfd/3.2.3/lwjgl-tinyfd-3.2.3-natives-linux-arm64.jar
.../lwjgl-tinyfd/3.2.3/lwjgl-tinyfd-3.2.3-natives-linux-arm32.jar
.../lwjgl-tinyfd/3.2.3/lwjgl-tinyfd-3.2.3.jar

Is this only the case with GLFW or with the entire LWJGL suite?

Extracting them with find -name '*-natives-linux-arm64.jar' -exec jar xf {} \;, then running ldd on all the files returned by find -name '*.so' the situation is this:

 OK  ./linux/arm64/org/lwjgl/opengl/liblwjgl_opengl.so
 OK  ./linux/arm64/org/lwjgl/liblwjgl.so
 OK  ./linux/arm64/org/lwjgl/tinyfd/liblwjgl_tinyfd.so
 OK  ./linux/arm64/org/lwjgl/stb/liblwjgl_stb.so
FAIL ./linux/arm64/org/lwjgl/jemalloc/libjemalloc.so
FAIL ./linux/arm64/org/lwjgl/glfw/libglfw.so
FAIL ./linux/arm64/org/lwjgl/glfw/libglfw_wayland.so
FAIL ./linux/arm64/org/lwjgl/openal/libopenal.so

The FAILed ones all dynamically link to ld-linux-aarch64.so.1 (glibc).

Also, I wonder if modifying -Djava.library.path would work or if this can collide with native-oriented JAR library files.

I'll get back to you if I figure out how this works, installing jemalloc-devel glfw-devel libopenal-devel in my case on Void Linux provides the files (e.g. libjemalloc.so) under /usr/lib for all the libraries that should be replaced by system versions.

If you have tips please do tell, I'm not exactly sure what I'm doing right now (and I couldn't figure out how the file(s) end up in /tmp/lwjgl$USER/3.2.3-build-13/ for example) ^^

@mindstorm38
Copy link
Owner

This temporary directory is created by lwjgl itself, it's where all shared objects are extracted from *natives* JAR files before being dynamically linked by LWJGL.

@JamiKettunen
Copy link
Contributor Author

Well I think I got the game running but believe I've sadly hit issues with the drivers for the hardware as the mainline Linux kernel port for the device isn't quite yet ready for prime time :p

Here's the hack of a diff that did the trick (java process still running without quitting after 15 minutes) for now:

--- a/src/core/portablemc/__init__.py
+++ b/src/core/portablemc/__init__.py
@@ -775,7 +775,7 @@ class Start:
                             with open(path.join(bin_dir, native_name), "wb") as native_file:
                                 shutil.copyfileobj(native_zip_file, native_file)
 
-        self.args_replacements["natives_directory"] = bin_dir
+        self.args_replacements["natives_directory"] = "/usr/lib"
 
         self.runner([
             *replace_list_vars(self.jvm_args, self.args_replacements),
--- a/src/core/portablemc/cli.py
+++ b/src/core/portablemc/cli.py
@@ -697,6 +697,9 @@ def fix_lwjgl_version(version: Version, lwjgl_version: str):
         for lwjgl_os, lwjgl_classifiers in lwjgl_natives.items():
             for lwjgl_classifier in lwjgl_classifiers:
                 classifier_path = f"org/lwjgl/{lwjgl_lib}/{lwjgl_version}/{lwjgl_lib}-{lwjgl_version}-{lwjgl_classifier}.jar"
+                if lwjgl_classifier == "natives-linux-arm64" and (lwjgl_lib == "lwjgl-jemalloc" or lwjgl_lib == "lwjgl-openal" or lwjgl_lib == "lwjgl-glfw"):
+                    print_task("WARN", "start.version.system_lwjgl_native", {"lwjgl_lib": f"{lwjgl_lib}-{lwjgl_version}-{lwjgl_classifier}.jar"}, done=True)
+                    continue
                 classifier_url = f"{maven_repo_url}/{classifier_path}"
                 meta_libraries.append({
                     "downloads": {
@@ -1172,6 +1175,7 @@ messages = {
     "start.version.resolving": "Resolving version {version}... ",
     "start.version.resolved": "Resolved version {version}.",
     "start.version.fixed.lwjgl": "Fixed LWJGL version to {version}",
+    "start.version.system_lwjgl_native": "Using system provided libraries instead of {lwjgl_lib}.",
     "start.version.jar.loading": "Loading version JAR... ",
     "start.version.jar.loaded": "Loaded version JAR.",
     f"start.version.error.{VersionError.NOT_FOUND}": "Version {version} not found.",

Especially that natives_directory hack is ugly as -Djava.library.path=/dir1:/dir2 should be supported to include multiple ones (and is used by libfliteWrapper.so which missing doesn't appear to be fatal) and f"/usr/lib:{bin_dir}" didn't seem to help, needs investigation later.

I had to run with portablemc start --jvm java --lwjgl 3.2.3 1.16.5 as it seems the mesa freedreno Adreno 540 GPU drivers only support up to OGL 3.1 / GLSL 1.40. For versions >=1.17 forcegl20 from modrinth apparently should work but I didn't try yet to avoid another possible point of failure.

In case of GLXBadFBConfig error from LWJGL export MESA_GL_VERSION_OVERRIDE=4.5 (from #62 (comment)) could also help, but in my case as seen above exposed the outdated OpenGL version my GPU driver is capable of.

Here's an image of it "running" with the display not refreshing altogether for currently unknown reason with no obvious logs anywhere:
image
The game isn't the issue as even trying to log into GNOME X11 instead of Wayland session gives similar symptoms.

@mindstorm38
Copy link
Owner

Insane! I'll try to implement all of this in the API, for the CLI I think of a --bin /usr/lib/ or something similar, with options like --no-natives lwjgl-glfw to remove specific natives JARs from the class path. For the JVM, the CLI will just ask the user to manually enter --jvm on weird systems.

@mindstorm38
Copy link
Owner

f"/usr/lib:{bin_dir}" didn't seem to help

Anoying...

@JamiKettunen
Copy link
Contributor Author

It ended up actually being a problem with the custom patched mesa packages I ran; restoring everything mesa related to Void-provided versions from repository fixed the display suddenly stopping refreshing (with no logs!) on seemingly anything X11/XWayland related stuff, so it indeed launches! ^^

Now time for me to start investigating if the issue also happens on unpatched mesa 22.2.2 (currently installed is 22.1.7) or possibly latest git version as well.

Let me know of any PRs etc. to test that include things I noticed here on both x86_64-musl as well as aarch64-musl!

@JamiKettunen
Copy link
Contributor Author

JamiKettunen commented Oct 23, 2022

Another thing about using the system provided native libs: could we somehow make PortableMC pick the correct versioned one (or make user specify it) so we don't rely on unversioned .so symlinks existing under /usr/lib or whatever?

In my case it looks for libjemalloc.so, libglfw.so & libopenal.so and these symlinks on many distributions are provided by development subpackages like jemalloc-devel on Void, libjemalloc-dev on Debian-based and jemalloc-dev on Alpine-based systems.

It would be great if portablemc can just add unversined .so symlinks for e.g. libjemalloc.so.2, libglfw.so.3.3 (or libglfw.so.3) and libopenal.so.1.22.2 (or libopenal.so.1) to the args_replacements["natives_directory"] when specifying those should use the system libraries.

I just tested this theory with the following diff for __init__.py (instead of what I posted previously) and it worked too!

--- a/src/core/portablemc/__init__.py
+++ b/src/core/portablemc/__init__.py
@@ -776,6 +776,8 @@ class Start:
                                 shutil.copyfileobj(native_zip_file, native_file)
 
         self.args_replacements["natives_directory"] = bin_dir
+        print(bin_dir)
+        input()
 
         self.runner([
             *replace_list_vars(self.jvm_args, self.args_replacements),

When I now run portablemc it prints e.g. /home/void/.minecraft/bin/<whatever> and after the following:

bindir=/home/void/.minecraft/bin/<whatever>
ln -s /usr/lib/libjemalloc.so.2    $bindir/libjemalloc.so
ln -s /usr/lib/libglfw.so.3.3      $bindir/libglfw.so
ln -s /usr/lib/libopenal.so.1.22.2 $bindir/libopenal.so

And pressing enter the game happily launched as well, so we could definitely implement something like this to avoid having to mess with self.args_replacements["natives_directory"] at all ^^

@Ristovski
Copy link
Contributor

Ristovski commented Oct 23, 2022

Speaking of glfw, having a way to override the used lwjgl libraries with the system ones would also theoretically allow running Minecraft completely natively on Wayland "for free" (as opposed to it using the "emulated" X server on Wayland), given the system glfw is built with native Wayland support.

Example: https://github.com/Admicos/minecraft-wayland#step-1-setting-up-multimc-to-use-the-system-glfw

@mindstorm38 mindstorm38 linked a pull request Oct 25, 2022 that will close this issue
mindstorm38 added a commit that referenced this issue Nov 2, 2022
* Added many arguments both in API and CLI to support fine-grained library exclusion and binaries inclusion.
* Added code coverage for test pipeline
mindstorm38 added a commit that referenced this issue Nov 9, 2022
* Added many arguments both in API and CLI to support fine-grained library exclusion and binaries inclusion.
* Added code coverage for test pipeline
@mindstorm38
Copy link
Owner

@JamiKettunen Released!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants