Enable cross-compilation of native libraries

[loicottet]

I'm working on making the LLVM backend of Graal output ARM programs and a big blocker is that the native C libraries (libjvm, liblibchelper, libstrictmath notably) are compiled by default for the host platform only.

It would be nice to be able to select the target architecture when running mx build and have the binaries output to the <os>-aarch64 folder instead of the current <os>-amd64.

[vjovanov]

Once cross-compilation is fully supported we will also have to ship those with the distribution.

[johanvos]

+1 on this.
The current property in suite.py for doing this is called os_arch but it only checks the (build) os. It would be helpful to have both a targetOs and targetArch entity and have the output indeed in <targetOs>-<targetArch>.
If nothing is specified, the native libs should be built for targetOs = currentOs and targetArch = currentArch.

[vjovanov]

@gilles-duboscq this is a proposal:
As the first step, we need:

1. To introduce --target-arch in mx build. For example, we would have mx build --target-arch=aarch64. Here it is important to do this right: the tripled used by ninja is complicated so we need to design this in a way that it is extensible in the future (e.g., what if ARM needs a different ABI or a different floating point strategy than the host).
2. This flag would be used in the mx.get_arch instead of the host (when set). Here, we need to find all usages of mx.get_arch and
3. The ninja build would use a specific GCC to build for the target architecture.

In the second step, we would introduce components for native-image native libraries for supported architectures. On amd64 hosts, people would need to install the component for aarch64 if they need to develop for aarch64 and on aarch64 hosts they would need a component for amd64.

This feature would be used as follows:

1. In development, users would do mx build; mx build --target-arch=aarch64.
2. In production, users would do gu install native-image-aarch64, or something similar. That is, in GraalVM this would be just another component related to the native image.

[araspitzu]

Any news on this?

[pejovica]

The first set of changes that add support for multiarch native projects to mx is being reviewed. Once that's merged, I'll convert the projects used by the native-image and add support for the aarch64 cross-toolchain.

[plebcity]

We would love to be able to cross compile to aarch64 using the native-image tool. Is this supported now?

[mageddo]

Still a unreleased feature, tried this on graalvm 23.3.1 JDK 19

$ native-image --no-server --no-fallback --target=linux-aarch64 -jar build/libs/graalvm-vanilla-all.jar ./build/graalvm-vanilla
Warning: Ignoring server-mode native-image argument --no-server.
Error: Invalid Path entry /home/typer/.sdkman/candidates/java/22.3.1.r19-grl/lib/svm/clibraries/linux-aarch64
Caused by: java.nio.file.NoSuchFileException: /home/typer/.sdkman/candidates/java/22.3.1.r19-grl/lib/svm/clibraries/linux-aarch64

But if a try to install the native image module for a cross OS, I will the another error

$ gu -u install "https://github.com/graalvm/graalvm-ce-builds/releases/download/vm-22.3.1/llvm-installable-svm-java19-linux-aarch64-22.3.1.jar"
Processing Component: https://github.com/graalvm/graalvm-ce-builds/releases/download/vm-22.3.1/llvm-installable-svm-java19-linux-aarch64-22.3.1.jar
Downloading: Component https://github.com/graalvm/graalvm-ce-builds/releases/download/vm-22.3.1/llvm-installable-svm-java19-linux-aarch64-22.3.1.jar: https://github.com/graalvm/graalvm-ce-builds/releases/download/vm-22.3.1/llvm-installable-svm-java19-linux-aarch64-22.3.1.jar from github.com
Installation of https://github.com/graalvm/graalvm-ce-builds/releases/download/vm-22.3.1/llvm-installable-svm-java19-linux-aarch64-22.3.1.jar failed: Component LLVM Runtime Core could not be installed. It requires Architecture aarch64, but the GraalVM provides amd64
Error: Component LLVM Runtime Core could not be installed. It requires Architecture aarch64, but the GraalVM provides amd64

[mageddo]

Related issue #407