[question] Difference with Graal for native-image builds (babashka)

[icp1994]

Hi, I hope it's okay to ask free-form question here although the issue template mentions Slack.

I built https://github.com/babashka/babashka in Linux following their build info. When I try to run ./bb -e '(babashka.process/exec "ls")' with the built binary, I get the error message Message: exec is not supported in non-GraalVM environments. After looking into it, I found it generated from here after this call. Since this error is not present in the upstream provided release binaries, I was wondering if this is due do differences for Graal & Mandrel.

[Karm]

Hmm, ProcessProperties.java#L171, it's not Truffle, it's sdk. Can you confirm that it is not a casspath problem with the sdk? Does it really fail with Mandrel? I can give it a try later...

[icp1994]

I just built twice on amd64 Linux with

• export GRAALVM_HOME=~/graalvm-ce-java17-22.3.1
• export GRAALVM_HOME=~/mandrel-java17-22.3.1.0-Final

The process/exec command works with the generated binary from the first build but not the second.

[github-actions]

This issue appears to be stale because it has been open 30 days with no activity. This issue will be closed in 7 days unless Stale label is removed, a new comment is made, or not-Stale label is added.

[leahneukirchen]

I'd also like to hear an answer on that.

[jerboaa]

I don't think this has anything to do with GraalVM vs. Mandrel (unless I'm missing something obvious). Please report this to the babashka maintainers. Also keep in mind that Mandrel exists as a native-image engine for Quarkus primarily. So I'm not sure the babashka (whatever it is) workflow has any chance of working as I know nothing about it.

Simple non-babashka reproducer:

$ cat ProcessPropertiesTest.java 
import java.nio.file.Path;

import org.graalvm.nativeimage.ProcessProperties;

public class ProcessPropertiesTest {

	public static void main(String[] args) {
		ProcessProperties.exec(Path.of("/usr/bin/ls"));
	}

}
$ $GRAALVM_HOME/bin/javac -cp $GRAALVM_HOME/lib/jvmci/graal-sdk.jar:. ProcessPropertiesTest.java
$ $GRAALVM_HOME/bin/native-image ProcessPropertiesTest
[...]
$ ./processpropertiestest 
Exec.class  exec-example  Exec.java  processpropertiestest  processpropertiestest.build_artifacts.txt  ProcessPropertiesTest.class  ProcessPropertiesTest.java
$ ls
Exec.class  exec-example  Exec.java  processpropertiestest  processpropertiestest.build_artifacts.txt  ProcessPropertiesTest.class  ProcessPropertiesTest.java

We get the same output (compare to plain ls). Works fine with mandrel-java17-22.3.1.0-Final. So I'm not sure what is needed to reproduce this issue. So far it looks like a babashka issue or something else entirely. If you can reproduce it outside babashka, please let us know and we can have a look.

[jerboaa]

Looking at the linked issue void-linux/void-packages#42812 please check this thing is actually using native-image. For example ProcessProperties doesn't work in JVM mode:

$ $GRAALVM_HOME/bin/java -cp $GRAALVM_HOME/lib/jvmci/graal-sdk.jar:. ProcessPropertiesTest
Exception in thread "main" java.lang.Error: The class ImageSingletons can only be used when building native images, i.e., when using the native-image command.
	at org.graalvm.nativeimage.impl.ImageSingletonsSupport.checkInstalled(ImageSingletonsSupport.java:69)
	at org.graalvm.nativeimage.impl.ImageSingletonsSupport.get(ImageSingletonsSupport.java:63)
	at org.graalvm.nativeimage.ImageSingletons.lookup(ImageSingletons.java:86)
	at org.graalvm.nativeimage.ProcessProperties.exec(ProcessProperties.java:160)
	at ProcessPropertiesTest.main(ProcessPropertiesTest.java:8)

[leahneukirchen]

Ok, thanks. I use Mandrel instead of GraalVM to build Babashka because I didn't get GraalVM to build on native Linux/musl.

The build log says:

GraalVM Native Image: Generating 'bb' (executable)...
...
[1/7] Initializing...                                                                                    (5.2s @ 0.14GB)
 Version info: 'GraalVM 22.3.1.0-Final Java 17 Mandrel Distribution'
 Java version info: '17.0.5+7-void-r2'
 C compiler: gcc (unknown, x86_64, 12.2.0)
 Garbage collector: Serial GC
 1 user-specific feature(s)
 - InitAtBuildTimeFeature
...
[2/7] Performing analysis...  [*********]                                       
                        (27.3s @ 1.28GB)
  16,601 (93.47%) of 17,761 classes reachable
  25,798 (54.55%) of 47,291 fields reachable
  72,405 (65.64%) of 110,313 methods reachable
     787 classes,   648 fields, and 7,639 methods registered for reflection
      66 classes,    76 fields, and    58 methods registered for JNI access
       6 native libraries: dl, m, pthread, rt, stdc++, z
[3/7] Building universe...                                                      
                         (4.3s @ 1.93GB)
[4/7] Parsing methods...      [**]                                              
                         (2.6s @ 2.23GB)
[5/7] Inlining methods...     [***]                                             
                         (1.2s @ 1.53GB)
[6/7] Compiling methods...    [****]                                            
                        (16.3s @ 2.74GB)
[7/7] Creating image...                                                         
                         (5.8s @ 0.86GB)
  34.55MB (40.54%) for code area:    52,449 compilation units
  37.83MB (44.38%) for image heap:  394,003 objects and 157 resources
  12.86MB (15.08%) for other data
  85.24MB in total
--------------------------------------------------------------------------------
----------------------------------------
Top 10 packages in code area:                               Top 10 object types 
in image heap:
   2.89MB sci.impl                                             7.38MB java.lang.
Class
   1.64MB sun.security.ssl                                     6.96MB byte[] for
 code metadata
   1.33MB java.util                                            2.94MB java.lang.
String
   1.19MB clojure                                              2.77MB byte[] for
 general heap data
   1.03MB clojure.lang                                         2.57MB byte[] for
 java.lang.String
 890.44KB jdk.internal.net.http                                1.62MB byte[] for embedded resources
 755.45KB java.lang.invoke                                     1.39MB com.oracle.svm.core.hub.DynamicHubCompanion
 740.93KB com.sun.crypto.provider                              1.04MB byte[] for reflection metadata
 739.82KB clojure.core                                         1.04MB java.lang.Object[]
 661.65KB java.util.concurrent                               924.16KB clojure.lang.Symbol
  22.39MB for 436 more packages                                8.54MB for 5478 more object types
------------------------------------------------------------------------------------------------------------------------
                        4.5s (6.6% of total time) in 106 GCs | Peak RSS: 4.24GB | CPU load: 9.88
------------------------------------------------------------------------------------------------------------------------
Produced artifacts:
 /builddir/babashka-1.2.174/bb (executable)
 /builddir/babashka-1.2.174/bb.build_artifacts.txt (txt)
========================================================================================================================
Finished generating 'bb' in 1m 7s.

I think this uses native-image.

[leahneukirchen]

I can verify the pure Java example works as well. But native-image is called with --module-path /usr/lib/jvm/mandrel17/lib/jvmci/graal-sdk.jar:... so it should pickup the right class too I think. (There's only one graal-sdk.jar that could be used.)

[leahneukirchen]

Found the culprit now: Babashka is compiled into an uberjar first, which doesn't have ProcessProperties in the classpath. Thus a compile-time check disables it, and then native-image never picks it up.

However, GraalVM seems to have it in the default classpath, so there's a difference still maybe.

Anyway, forcing graal-sdk.jar into the classpath fixes it and then it works fine.

[jerboaa]

However, GraalVM seems to have it in the default classpath, so there's a difference still maybe.

Yes, that's the difference. GraalVM is a jlinked JDK with graalvm modules part of it. That includes the sdk[1]. Mandrel treats OpenJDK as immutable and only adds needed jars to the classpath/modulepath as needed. So you've found the correct fix by adding it to the classpath.

[1] GraalVM CE

$ java --list-modules | grep sdk
org.graalvm.sdk

[jerboaa]

I'd suggest to talk to the babashka maintainers whether or not they want to support the Mandrel config.