native-image shell script requires a modded JDK to work.

[zakkak]

Describe the issue
When building Graal bits for use with openJDK, native-image needs to pass the following arguments to the jvm --upgrade-module-path ${location}/../../jvmci/graal.jar --add-modules "org.graalvm.truffle,org.graalvm.sdk" --module-path ${location}/../../truffle/truffle-api.jar:${location}/../../jvmci/graal-sdk.jar and -J--add-exports=jdk.internal.vm.ci/jdk.vm.ci.code=jdk.internal.vm.compiler to native-image.

Steps to reproduce the issue
Please include both build steps as well as run steps:

curl -sL https://github.com/AdoptOpenJDK/openjdk11-upstream-binaries/releases/download/jdk-11.0.9%2B2/OpenJDK11U-jdk_x64_linux_11.0.9_2_ea.tar.gz -o jdk.tar.gz
curl -sL https://github.com/AdoptOpenJDK/openjdk11-upstream-binaries/releases/download/jdk-11.0.9%2B2/OpenJDK11U-static-libs_x64_linux_11.0.9_2_ea.tar.gz -o jdk-static-libs.tar.gz
mkdir -p openjdk
tar xf jdk.tar.gz -C openjdk --strip-components=1
tar xf jdk-static-libs.tar.gz -C openjdk --strip-components=1
export JAVA_HOME=$(pwd)/openjdk
git clone --depth 1 https://github.com/oracle/graal
pushd graal/substratevm
mx --components="Native Image" build
popd

export GRAAL_REPO=./graal
export MANDREL_JDK=./mandrel

cp -r ${JAVA_HOME} ${MANDREL_JDK}

mkdir ${MANDREL_JDK}/lib/svm
cp ${GRAAL_REPO}/substratevm/mxbuild/dists/jdk1.8/library-support.jar ${MANDREL_JDK}/lib/svm

mkdir ${MANDREL_JDK}/lib/svm/builder
cp ${GRAAL_REPO}/substratevm/mxbuild/dists/jdk11/{svm,pointsto}.jar ${MANDREL_JDK}/lib/svm/builder
cp ${GRAAL_REPO}/substratevm/mxbuild/dists/jdk1.8/objectfile.jar ${MANDREL_JDK}/lib/svm/builder

mkdir ${MANDREL_JDK}/languages
cp ${GRAAL_REPO}/truffle/mxbuild/dists/jdk11/truffle-nfi.jar ${MANDREL_JDK}/languages

mkdir ${MANDREL_JDK}/lib/graalvm
cp ${GRAAL_REPO}/substratevm/mxbuild/dists/jdk1.8/svm-driver.jar ${MANDREL_JDK}/lib/graalvm

mkdir ${MANDREL_JDK}/lib/jvmci
cp ${GRAAL_REPO}/sdk/mxbuild/dists/jdk11/graal-sdk.jar ${MANDREL_JDK}/lib/jvmci
cp ${GRAAL_REPO}/compiler/mxbuild/dists/jdk11/graal.jar ${MANDREL_JDK}/lib/jvmci

mkdir ${MANDREL_JDK}/lib/truffle
cp ${GRAAL_REPO}/truffle/mxbuild/dists/jdk11/truffle-api.jar ${MANDREL_JDK}/lib/truffle

mkdir -p ${MANDREL_JDK}/lib/svm/clibraries/linux-amd64/include
cp ${GRAAL_REPO}/substratevm/src/com.oracle.svm.native.libchelper/include/amd64cpufeatures.h ${MANDREL_JDK}/lib/svm/clibraries/linux-amd64/incl
ude
cp ${GRAAL_REPO}/substratevm/src/com.oracle.svm.native.libchelper/include/aarch64cpufeatures.h ${MANDREL_JDK}/lib/svm/clibraries/linux-amd64/in
clude
cp ${GRAAL_REPO}/substratevm/src/com.oracle.svm.libffi/include/svm_libffi.h ${MANDREL_JDK}/lib/svm/clibraries/linux-amd64/include
cp ${GRAAL_REPO}/truffle/src/com.oracle.truffle.nfi.native/include/trufflenfi.h ${MANDREL_JDK}/lib/svm/clibraries/linux-amd64/include
cp ${GRAAL_REPO}/substratevm/mxbuild/linux-amd64/src/com.oracle.svm.native.libchelper/amd64/liblibchelper.a ${MANDREL_JDK}/lib/svm/clibraries/l
inux-amd64
cp ${GRAAL_REPO}/substratevm/mxbuild/linux-amd64/src/com.oracle.svm.native.jvm.posix/amd64/libjvm.a ${MANDREL_JDK}/lib/svm/clibraries/linux-amd
64

mkdir ${MANDREL_JDK}/lib/svm/bin
cp ${GRAAL_REPO}/sdk/mxbuild/linux-amd64/native-image.image-bash/native-image ${MANDREL_JDK}/lib/svm/bin/native-image

ln -s ../lib/svm/bin/native-image ${MANDREL_JDK}/bin/native-image

./mandrel/bin/native-image -version

Note that the cp commands just copy graalvm bits in the openjdk filesystem adding them where native-image expects to find them.

Describe GraalVM and your environment:

• GraalVM version: 16a2a12
• JDK major version: 11
• OS: Linux
• Architecture: AMD64

More details

./mandrelJDK/bin/native-image --native-image-info --verbose HelloWorld
Exception in thread "main" java.lang.NoClassDefFoundError: org/graalvm/word/UnsignedWord
        at com.oracle.svm.driver.NativeImage.getPlatform(NativeImage.java:104)
        at com.oracle.svm.driver.NativeImage.<clinit>(NativeImage.java:101)
        at com.oracle.svm.driver.NativeImage$JDK9Plus.main(NativeImage.java:1753)
Caused by: java.lang.ClassNotFoundException: org.graalvm.word.UnsignedWord
        at java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(BuiltinClassLoader.java:581)
        at java.base/jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(ClassLoaders.java:178)
        at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:522)
        ... 3 more

Applying the following patch to ./mandrelJDK/lib/svm/bin/native-image makes it work.

diff --git a/graal/sdk/mxbuild/linux-amd64/native-image.image-bash/native-image b/mandrel/lib/svm/bin/native-image
index 992d85b..11d365a 100755
--- a/graal/sdk/mxbuild/linux-amd64/native-image.image-bash/native-image
+++ b/mandrel/lib/svm/bin/native-image
@@ -108,4 +108,4 @@ if [[ "${VERBOSE_GRAALVM_LAUNCHERS}" == "true" ]]; then
     set -x
 fi
 
-exec "${location}/../../../bin/java" --add-exports=java.base/jdk.internal.module=ALL-UNNAMED -XX:+UnlockExperimentalVMOptions -XX:+EnableJVMCI "${jvm_args[@]}" -cp "${cp}" 'com.oracle.svm.driver.NativeImage$JDK9Plus' "${launcher_args[@]}"
+exec "${location}/../../../bin/java" --upgrade-module-path ${location}/../../jvmci/graal.jar --add-modules "org.graalvm.truffle,org.graalvm.sdk" --module-path ${location}/../../truffle/truffle-api.jar:${location}/../../jvmci/graal-sdk.jar --add-exports=java.base/jdk.internal.module=ALL-UNNAMED -XX:+UnlockExperimentalVMOptions -XX:+EnableJVMCI "${jvm_args[@]}" -cp "${cp}" 'com.oracle.svm.driver.NativeImage$JDK9Plus' -J--add-exports=jdk.internal.vm.ci/jdk.vm.ci.code=jdk.internal.vm.compiler "${launcher_args[@]}"

Results after patch:

$ ./mandrel/bin/native-image HelloWorld                                
[helloworld:238051]    classlist:   1,176.86 ms,  1.00 GB
[helloworld:238051]        (cap):     627.54 ms,  1.00 GB
[helloworld:238051]        setup:   1,760.69 ms,  1.58 GB
[helloworld:238051]     (clinit):     222.38 ms,  1.92 GB
[helloworld:238051]   (typeflow):   4,243.92 ms,  1.92 GB
[helloworld:238051]    (objects):   3,886.14 ms,  1.92 GB
[helloworld:238051]   (features):     219.58 ms,  1.92 GB
[helloworld:238051]     analysis:   8,828.33 ms,  1.92 GB
[helloworld:238051]     universe:     319.31 ms,  1.92 GB
[helloworld:238051]      (parse):     922.83 ms,  1.92 GB
[helloworld:238051]     (inline):   1,381.27 ms,  2.42 GB
[helloworld:238051]    (compile):   7,252.56 ms,  2.42 GB
[helloworld:238051]      compile:   9,986.78 ms,  2.42 GB
[helloworld:238051]        image:   1,291.74 ms,  2.42 GB
[helloworld:238051]        write:     156.78 ms,  2.42 GB
[helloworld:238051]      [total]:  23,738.09 ms,  2.42 GB

$ ./helloworld 
Hello world!!!

[zakkak]

What would be the best way to get the flags there without disrupting other use cases?

[olpaw]

@zakkak is this still relevant? Can you still reproduce this problem?

[olpaw]

/../graal/sdk/mxbuild/linux-amd64/native-image.image-bash/native-image is a bash-launcher for the native-image program (in production native-image is itself compiled to a native image but as part of bootstrapping we also run it as bash-launcher. The generation of such bash launchers is performed by mx. See https://github.com/oracle/graal/blob/master/sdk/mx.sdk/vm/launcher_template.sh

[zakkak]

Hi @olpaw,

This is still relevant, but the instructions were outdated. I have now updated the steps to reproduce the issue.

In Mandrel we rely on the bash-launcher for the production native-image as well. Till now the only major difference we have noticed is the lack of the image build server when using the launcher. Are there any other differences you can think of that we should be aware of?

Would patching https://github.com/oracle/graal/blob/master/sdk/mx.sdk/vm/launcher_template.sh to include the additional parameters that we need be OK with graal? Do you foresee any implications?

Thanks

[olpaw]

Would patching https://github.com/oracle/graal/blob/master/sdk/mx.sdk/vm/launcher_template.sh to include the additional
parameters that we need be OK with graal? Do you foresee any implications?

Hmm, I guess so. Do you agree @dougxc ?

[dougxc]

Sorry, I'm rusty on launcher_template.sh so I leave that call up to you Paul.

[olpaw]

@zakkak I see no problems with making the launcher template mechanism flexible enough to suite your needs.
Keep in mind that we also need to support Windows. So whatever you add should also work with https://github.com/oracle/graal/blob/master/sdk/mx.sdk/vm/launcher_template.cmd (or at least not break it). The instantiations of those templates happens via GraalVmBashLauncherBuildTask. See https://github.com/oracle/graal/blob/master/sdk/mx.sdk/mx_sdk_vm_impl.py#L1719
cc @ansalond

[ansalond]

If I understand correctly, the underlying issue is that you are packaging things differently. For example, in Mandrel org.graalvm.word.UnsignedWord is part of lib/jvmci/graal-sdk.jar while in GraalVM it is part of jmods/org.graalvm.sdk.jmod.

Your proposed command line is correct only on Mandrel, not on GraalVM. Therefore, at run time, our launchers should adapt the command line to the current JDK layout. Alternatively, would it be acceptable to keep patching the native-image launcher as part of your build pipeline?

Minor -- I think the truffle-nfi.jar is misplaced:

- mkdir ${MANDREL_JDK}/languages
- cp ${GRAAL_REPO}/truffle/mxbuild/dists/jdk11/truffle-nfi.jar ${MANDREL_JDK}/languages
+ mkdir -p ${MANDREL_JDK}/languages/nfi
+ cp ${GRAAL_REPO}/truffle/mxbuild/dists/jdk11/truffle-nfi.jar ${MANDREL_JDK}/languages/nfi

[zakkak]

If I understand correctly, the underlying issue is that you are packaging things differently. For example, in Mandrel org.graalvm.word.UnsignedWord is part of lib/jvmci/graal-sdk.jar while in GraalVM it is part of jmods/org.graalvm.sdk.jmod.

Correct.

I have tested packaging mandrel using makegraaljdk as part of the packaging process to perform the jlinking for me and it went well, however it looks like makegraaljdk does more than what we need/want, so I will try to extract the parts that we need.

Thanks for the pointer @ansalond
Since I can fix this issue in Mandrel's packaging I will close this issue.

Out of curiosity, if I was to make mandrel packaging more graal-like would you be able to share the scripts you use for packaging GraalVM CE?

[ansalond]

Out of curiosity, if I was to make mandrel packaging more graal-like would you be able to share the scripts you use for packaging GraalVM CE?

Sure! Our CI scripts are not easy to read, but all the relevant code is open source. For example, the builder that builds and uploads a JDK8 GraalVM CE image and the CE installables for Linux is defined here:
https://github.com/oracle/graal/blob/master/vm/ci_common/common.hocon#L304-L317

Simplifying a bit and assuming that you want to build only the base image (which does not include native-image):

$ cd graal/vm
$ mx --env ce graalvm-show  # check what you are about to build
$ mx --env ce build
$ mx --env ce graalvm-home  # path to the new GraalVM image

For a small GraalVM image that includes Native Image and its dependencies:

$ mx --dy /substratevm --components="Native Image" --native-images=false graalvm-show
GraalVM distribution: GRAALVM_83A3F67E14_JAVA8
Version: 20.2.0-dev
Config name: None
Components:
 - Native Image ('ni', /svm)
 - SubstrateVM ('svm', /svm)
 - Native Image licence files ('nil', /svm)
 - GraalVM compiler ('cmp', /graal)
 - Truffle Macro ('tflm', /truffle)
 - Truffle NFI ('nfi', /nfi)
 - Truffle ('tfl', /truffle)
 - Graal SDK ('sdk', /graalvm)
Launchers:
 - native-image (bash)
Libraries:
 - libnative-image-agent.so (skipped)
Installables:
 - NATIVE_IMAGE_INSTALLABLE_SLIBNATIVE-IMAGE-AGENT.SO_JAVA8
No standalone
$ mx --dy /substratevm --components="Native Image" --native-images=false build
$ mx --dy /substratevm --components="Native Image" --native-images=false graalvm-home

This last GraalVM image does not match any of our registered configurations, therefore it has a generated name (GRAALVM_83A3F67E14_JAVA8 rather than, e.g., GRAALVM_CE_JAVA8).
To register a new GraalVM config, add a line here:
https://github.com/oracle/graal/blob/master/vm/mx.vm/mx_vm.py#L82-L95

These steps are documented here.

Feel free to ping me if you have questions or suggestions.