Version 0.12.0 compiled with GraalVM fails to call libsignal-client

[selfbc]

I typically use GraalVM EE, but I loaded the most recent free version (Java 17 x64 Linux) from https://www.graalvm.org/downloads/ to verify that something funky wasn't going on with the EE version.

 Java version: 17.0.8+9-LTS, vendor version: Oracle GraalVM 17.0.8+9.1
 Graal compiler: optimization level: 2, target machine: compatibility, PGO: ML-inferred

After compiling, when attempting to use signal-cli I get the following error:

WARN  Manager - Failed to call libsignal-client: Can't load library: /tmp/15313113408514434891libsignal_jni.so
Missing required native library dependency: libsignal-client

Please note that compiling signal-cli with GraalVM has worked for me on all previous versions to include v0.11.11.

I did see the following warnings during compilation:

Warning: Method org.whispersystems.signalservice.internal.push.RegistrationSessionMetadataJson.<init>(String, Integer, Integer, Integer, boolean, List, boolean, int, DefaultConstructorMarker) not found.
Warning: Field org.whispersystems.signalservice.internal.push.SignalServiceProtos$SyncMessage$Blocked.uuids_ not found.
Warning: Field org.whispersystems.signalservice.internal.storage.protos.AccountRecord$PinnedConversation$Contact.uuid_ not found.

Please advise.

[m-ueberall]

Though it doesn't explicitly say so in the warning above, I'd check if the version of libsignal_jni.so you're using is compatible with your environment. Since the above file should still exist in /tmp, try to compare the following outputs:

# objdump -T /tmp/15313113408514434891libsignal_jni.so|grep GLIBC|sed -e 's|.*GLIBC_||g' -e 's| .*||g'|sort -n|uniq
# ldd --version|head -1

The largest version number of the first output specifies the minimal version of glibc required to successfully load the library; the second one displays the version you have. This has been discussed in issue #643; see the wiki for a list of pre-compiled libraries that work with older versions of glibc (starting from 2.28 at the time of writing).

Regarding the warnings: These can be safely ignored (see discussion #1254).

[selfbc]

I doubled checked and objdump on libsignal_jni.so shows that glibc matches the environment. The precompiled versions from AsamK are not, and give an error indicating as such. This is one of the reasons that I've always built from source. I'm not sure what all changed with this version, but I can still compile v0.11.11 and it works just fine with both GraalVM EE and the latest free GraalVM. This issue has started with v0.12.0.

[m-ueberall]

If you're willing/able to execute "downstream" binaries in your environment (after backing up your data), you could give signal-cli_ubuntu2004_amd64.gz or signal-cli_ubuntu2004_arm64.gz a try depending on your host architecture (see https://media.projektzentrisch.de/temp/signal-cli/).
These work without problems here and have been compiled using the following toolchain:

% cat libsignal_jni_so0300_ubuntu2004_amd64.meta
LIBSIGNAL=v0.30.0
LIBSIGNAL_GITREV=3b7f3173cc4431bc4c6e55f6182a37229b2db6fd
LLVM=v1:16.0.6-4~ubuntu20.04+projektzentrisch.de+1
CARGO="v1.73.0-nightly (d78bbf4bd 2023-08-03)"
% diff libsignal_jni_so0300_ubuntu2004_amd64.meta libsignal_jni_so0300_ubuntu2004_arm64.meta                   
% cat signal-cli0120_ubuntu2004_amd64.meta
SIGNALCLI=v0.12.0
SIGNALCLI_GITREV=29923cf930dda65e32375c0bc1b195a3ae24f748
LIBSIGNAL=v0.30.0
BINUTILS=v2.41-3ubuntu1~ubuntu20.04+projektzentrisch.de+1
GCC=v13.2.0-1ubuntu1~ubuntu20.04+projektzentrisch.de+1
TOOLCHAIN="GraalVM 22.3.3 Java 17 CE (17.0.8+7-jvmci-22.3-b22); gcc (linux, x86_64, 13.2.0)"
% diff signal-cli0120_ubuntu2004_amd64.meta signal-cli0120_ubuntu2004_arm64.meta
6c6
< TOOLCHAIN="GraalVM 22.3.3 Java 17 CE (17.0.8+7-jvmci-22.3-b22); gcc (linux, x86_64, 13.2.0)"
---
> TOOLCHAIN="GraalVM 22.3.3 Java 17 CE (17.0.8+7-jvmci-22.3-b22); gcc (linux, aarch64, 13.2.0)"

[m-ueberall]

@selfbc: I'm able to reproduce your warning if I try to build a (completely) static binary using GraalVM (by passing the options "--static --libc=musl" to native-image).
As long as you don't need to use the functionality of the embedded jni library, everything works and you can, e.g., display the help/version information or pass messages to a running/working instance using the dbus interface. Once you try to directly send a message using the static binary, however, you'll see

% echo "Hello, world" | signal-cli0120_ubuntu2004_amd64_static -a +49nnnnnnnnnnnn send --group 9xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx= --message-from-stdin 
WARN  Manager - Failed to call libsignal-client: Can't load library: /tmp/9840499431333150785libsignal_jni.so
Missing required native library dependency: libsignal-client

(That's because you'd need to use a statically linked libsignal_jni.a version of the library here; IMHO, GraalVM/native-image should complain and refuse to work instead of presenting you with a non-working binary in the first place.)

IIRC, you might also see the above error message if you "simply" forget to correctly reference the native library during the build (again, native-image would produce a binary that didn't work)--I saw this a couple of months ago while commenting on issue #1222/discussion #1230.

[selfbc]

Does it work for you using the new GraalVM release (see https://medium.com/graalvm/a-new-graalvm-release-and-new-free-license-4aab483692f5 and https://www.graalvm.org/downloads/)? I have traditionally used GraalVM EE but snagged the latest free version (referenced above) when I started encountering this issue. I could attempt to find and download the older version of GraalVM that you're referencing, but I'd hope that @AsamK would want it to work with the new GraalVM.

[m-ueberall]

@selfbc: I triggered a test build using the same pre-built libsignal_jni.so, signal-cli codebase along with the current Oracle GraalVM release for JDK17 …

 Java version: 17.0.8+9-LTS, vendor version: Oracle GraalVM 17.0.8+9.1
 Graal compiler: optimization level: 2, target machine: x86-64-v3, PGO: ML-inferred
 C compiler: gcc (linux, x86_64, 13.2.0)

… but it looks like it'll take considerably longer than the previous build runs due to the default optimization level (and I unfortunately chose a rather smallish amd64 build instance). I'll compare the results later tonight and keep you posted.

[m-ueberall]

The results are in: Oracle GraalVM 17.0.8+9.1 works as well; with the above optimization level, it's slower (but not massively–the first run which took ages failed because it eventually "ran out of memory", which is really strange given 24GB of RAM and the stated requirements of consecutive runs which were all successful), uses less memory during the build ("Peak RSS" showed 4.09GB vs. 6.61GB), and produces a larger executable while emitting the same/slightly reworded warnings as GraalVM 22.3.3 Java 17 CE.

# ll -rt /opt/signal-cli/bin/*amd64*
-rwxr-xr-x 1 signal-cli signal-cli 78977160 2023-08-12 08:11:56 /opt/signal-cli/bin/signal-cli0120_ubuntu2004_amd64*
-rwxr-xr-x 1 signal-cli signal-cli 92919992 2023-08-17 14:11:13 /opt/signal-cli/bin/signal-cli0120_ubuntu2004_amd64__oracle*

Comparing the output of /usr/bin/time --verbose when sending a single message–either directly or by contacting another instance via the dbus interface–is more or less meaningless if the machine is not totally idle (and which host ever is?);
it looks like optimization shaves off a couple of milliseconds in the case of signal-cli, but the resulting binary is even larger (the libsignal_jni.so library had been stripped beforehand; the resulting binaries above are unstripped, but stripping them saves less than 200KB each!) ...

@selfbc: I did not try to select the "target machine: compatibility" build mode (wouldn't know how to do that without thoroughly reading the documentation–OTOH, given the above comparison, I really can't think of a reason for not using the CE version for signal-cli). However, I suspect that you already tried different settings on your end, so the underlying cause of your problem must lie elsewhere.

EDIT: The option I was looking for would have been -march=compatibility.

[selfbc]

Rebuilding everything (to include libsignal) got it working. I'm still not completely sure what was causing the issue, but I am running a fairly hardened system to include kernel hardening with Grsecurity.

After getting it mostly working, I ran into this read receipt store error that I'm hoping is the query that AsamK just fixed. He hasn't responded to me in some time though, so my only hope is trial and error. Thanks for all of your help on this, it is definitely appreciated.

It's great to have folks like you when the authors of software aren't so responsive and said named software is often riddled with bugs.