native-image parameters are not compatible after upgrade to the patch version 22.0.1

[davinkevin]

Describe the issue

We have the following dockerfile, using the official image with version 22.0.0:

FROM scratch as external-dependencies

ADD distDockerTar.tar /

FROM ghcr.io/graalvm/native-image-community:22.0.0@sha256:0e7b3ee194aadc80845a29cd4e6501b9ed4422e6b998267926df31d63caf4770 as native-build

COPY --from=external-dependencies / /

RUN native-image -jar /opt/app/lib/foo.jar --static -march=compatibility --no-fallback

The build was working perfectly, with the following report:

#10 [native-build 3/3] RUN native-image -jar /opt/app/lib/foo.jar --static -march=compatibility --no-fallback
#10 1.072 ========================================================================================================================
#10 1.073 GraalVM Native Image: Generating 'foo' (static executable)...
#10 1.073 ========================================================================================================================
#10 1.073 For detailed information and explanations on the build output, visit:
#10 1.073 https://github.com/oracle/graal/blob/master/docs/reference-manual/native-image/BuildOutput.md
#10 1.073 ------------------------------------------------------------------------------------------------------------------------
#10 2.526 [1/8] Initializing...                                                                                    (2.0s @ 0.10GB)
#10 2.526  Java version: 22+36, vendor version: GraalVM CE 22+36.1
#10 2.527  Graal compiler: optimization level: 2, target machine: compatibility
#10 2.527  C compiler: gcc (redhat, aarch64, 11.4.1)
#10 2.527  Garbage collector: Serial GC (max heap size: 80% of RAM)
#10 2.527  1 user-specific feature(s):
#10 2.527  - com.oracle.svm.thirdparty.gson.GsonFeature
#10 2.528 ------------------------------------------------------------------------------------------------------------------------
#10 2.528 Build resources:
#10 2.528  - 17.71GB of memory (75.6% of 23.44GB system memory, determined at start)
#10 2.528  - 12 thread(s) (100.0% of 12 available processor(s), determined at start)
#10 10.82 [2/8] Performing analysis...  [****]                                                                     (8.1s @ 0.38GB)
#10 10.82     4,183 reachable types   (75.3% of    5,556 total)
#10 10.83     5,158 reachable fields  (45.8% of   11,266 total)
#10 10.84    19,274 reachable methods (44.9% of   42,945 total)
#10 10.84     1,412 types,    63 fields, and   715 methods registered for reflection
#10 10.84        58 types,    57 fields, and    53 methods registered for JNI access
#10 10.84         4 native libraries: dl, pthread, rt, z
#10 12.32 [3/8] Building universe...                                                                               (1.5s @ 0.36GB)
#10 13.45 [4/8] Parsing methods...      [*]                                                                        (1.1s @ 0.43GB)
#10 14.40 [5/8] Inlining methods...     [***]                                                                      (0.8s @ 0.43GB)
#10 21.72 [6/8] Compiling methods...    [***]                                                                      (7.3s @ 0.35GB)
#10 22.67 [7/8] Laying out methods...   [*]                                                                        (1.0s @ 0.44GB)
#10 24.47 [8/8] Creating image...       [*]                                                                        (1.5s @ 0.57GB)
#10 24.47    6.59MB (37.77%) for code area:    11,194 compilation units
#10 24.47    9.44MB (54.07%) for image heap:  118,865 objects and 47 resources
#10 24.47    1.42MB ( 8.16%) for other data
#10 24.47   17.45MB in total
#10 24.47 ------------------------------------------------------------------------------------------------------------------------
#10 24.47 Top 10 origins of code area:                                Top 10 object types in image heap:
#10 24.47    5.13MB java.base                                            2.08MB byte[] for code metadata
#10 24.47  932.39kB svm.jar (Native Image)                               1.68MB byte[] for java.lang.String
#10 24.47  125.44kB jdk.httpserver                                       1.16MB java.lang.String
#10 24.47  114.32kB java.logging                                       993.41kB java.lang.Class
#10 24.47   57.42kB org.graalvm.nativeimage.base                       359.48kB com.oracle.svm.core.hub.DynamicHubCompanion
#10 24.47   42.26kB jdk.proxy3                                         293.91kB byte[] for general heap data
#10 24.47   40.96kB jdk.proxy1                                         264.42kB java.util.HashMap$Node
#10 24.47   22.03kB org.graalvm.collections                            241.59kB heap alignment
#10 24.47   21.33kB jdk.proxy4                                         235.45kB java.lang.String[]
#10 24.47   20.62kB foo.jar             220.12kB java.lang.Object[]
#10 24.47   54.03kB for 6 more packages                                  1.98MB for 1059 more object types
#10 24.47 ------------------------------------------------------------------------------------------------------------------------
#10 24.47 Recommendations:
#10 24.47  HEAP: Set max heap for improved and more predictable memory usage.
#10 24.47 ------------------------------------------------------------------------------------------------------------------------
#10 24.47                         1.6s (6.6% of total time) in 200 GCs | Peak RSS: 1.09GB | CPU load: 8.55
#10 24.47 ------------------------------------------------------------------------------------------------------------------------
#10 24.47 Produced artifacts:
#10 24.47  /app/foo (executable)
#10 24.47 ========================================================================================================================
#10 24.48 Finished generating 'foo' in 24.0s.
#10 DONE 24.5s

Then, we have upgraded to the patch version 22.0.1:

FROM scratch as external-dependencies

ADD distDockerTar.tar /

FROM ghcr.io/graalvm/native-image-community:22.0.1@sha256:bed26bb73a090382d35cb5e8b6efaff76dcb1c974c3de0e690957028aceb7d87 as native-build

COPY --from=external-dependencies / /

RUN native-image -jar /opt/app/lib/foo.jar --static --libc=musl -march=compatibility --no-fallback

And the result is different:

#10 [native-build 3/3] RUN native-image -jar /opt/app/lib/foo.jar --static -march=compatibility --no-fallback
#10 1.028 ========================================================================================================================
#10 1.028 GraalVM Native Image: Generating 'foo' (static executable)...
#10 1.028 ========================================================================================================================
#10 1.028 For detailed information and explanations on the build output, visit:
#10 1.028 https://github.com/oracle/graal/blob/master/docs/reference-manual/native-image/BuildOutput.md
#10 1.028 ------------------------------------------------------------------------------------------------------------------------
#10 2.383 [1/8] Initializing...                                                                                    (1.9s @ 0.10GB)
#10 2.383  Java version: 22.0.1+8, vendor version: GraalVM CE 22.0.1+8.1
#10 2.383  Graal compiler: optimization level: 2, target machine: compatibility
#10 2.383  C compiler: gcc (redhat, aarch64, 11.4.1)
#10 2.384  Garbage collector: Serial GC (max heap size: 80% of RAM)
#10 2.384  1 user-specific feature(s):
#10 2.384  - com.oracle.svm.thirdparty.gson.GsonFeature
#10 2.385 ------------------------------------------------------------------------------------------------------------------------
#10 2.385 Build resources:
#10 2.385  - 17.71GB of memory (75.6% of 23.44GB system memory, determined at start)
#10 2.385  - 12 thread(s) (100.0% of 12 available processor(s), determined at start)
#10 2.403
#10 2.403 Error: Invalid option '--static'. Building static executable images is only supported with musl libc. Remove the '--static' option or add the '--libc=musl' option..
#10 2.403 ------------------------------------------------------------------------------------------------------------------------
#10 2.404                         0.1s (6.1% of total time) in 10 GCs | Peak RSS: 0.48GB | CPU load: 3.41
#10 2.404 ========================================================================================================================
#10 2.404 Failed generating 'foo' after 1.9s.
#10 ERROR: process "/bin/sh -c native-image -jar /opt/app/lib/foo.jar --static -march=compatibility --no-fallback" did not complete successfully: exit code: 1
------
 > [native-build 3/3] RUN native-image -jar /opt/app/lib/foo.jar --static -march=compatibility --no-fallback:
2.385 ------------------------------------------------------------------------------------------------------------------------
2.385 Build resources:
2.385  - 17.71GB of memory (75.6% of 23.44GB system memory, determined at start)
2.385  - 12 thread(s) (100.0% of 12 available processor(s), determined at start)
2.403
2.403 Error: Invalid option '--static'. Building static executable images is only supported with musl libc. Remove the '--static' option or add the '--libc=musl' option..
2.403 ------------------------------------------------------------------------------------------------------------------------
2.404                         0.1s (6.1% of total time) in 10 GCs | Peak RSS: 0.48GB | CPU load: 3.41
2.404 ========================================================================================================================
2.404 Failed generating 'foo' after 1.9s.

If we follow the error recommendation, "Remove the '--static' option or add the '--libc=musl'", we have the following error:

#10 [native-build 3/3] RUN native-image -jar /opt/app/lib/foo.jar --static --libc=musl -march=compatibility --no-fallback
#10 1.377 ========================================================================================================================
#10 1.377 GraalVM Native Image: Generating 'foo' (static executable)...
#10 1.378 ========================================================================================================================
#10 1.378 For detailed information and explanations on the build output, visit:
#10 1.378 https://github.com/oracle/graal/blob/master/docs/reference-manual/native-image/BuildOutput.md
#10 1.378 ------------------------------------------------------------------------------------------------------------------------
#10 1.848
#10 1.849 [1/8] Initializing...                                                                                    (0.0s @ 0.11GB)
#10 1.849 Error: Default native-compiler executable 'x86_64-linux-musl-gcc' not found via environment variable PATH
#10 1.850 Error: To prevent native-toolchain checking provide command-line option -H:-CheckToolchain
#10 1.850 ------------------------------------------------------------------------------------------------------------------------
#10 1.852                          0.1s (5.8% of total time) in 7 GCs | Peak RSS: 0.41GB | CPU load: 3.91
#10 1.852 ========================================================================================================================
#10 1.852 Failed generating 'foo' after 1.1s.
#10 ERROR: process "/bin/sh -c native-image -jar /opt/app/lib/foo.jar --static --libc=musl -march=compatibility --no-fallback" did not complete successfully: exit code: 1
------
 > [native-build 3/3] RUN native-image -jar /opt/app/lib/foo.jar --static --libc=musl -march=compatibility --no-fallback:
1.378 https://github.com/oracle/graal/blob/master/docs/reference-manual/native-image/BuildOutput.md
1.378 ------------------------------------------------------------------------------------------------------------------------
1.848
1.849 [1/8] Initializing...                                                                                    (0.0s @ 0.11GB)
1.849 Error: Default native-compiler executable 'x86_64-linux-musl-gcc' not found via environment variable PATH
1.850 Error: To prevent native-toolchain checking provide command-line option -H:-CheckToolchain
1.850 ------------------------------------------------------------------------------------------------------------------------
1.852                          0.1s (5.8% of total time) in 7 GCs | Peak RSS: 0.41GB | CPU load: 3.91
1.852 ========================================================================================================================
1.852 Failed generating 'foo' after 1.1s.
------

My expectation was to have everything setup and ready in the native-image docker image, it's the main reason we use it 😇.

So the bug is mainly about the CLI parameters breaking change in a patch version, but other element can be considered (no x86_64-linux-musl-gcc in the image
for example).

Describe GraalVM and your environment:

• GraalVM version:
  • docker image "ghcr.io/graalvm/native-image-community:22.0.1@sha256:bed26bb73a090382d35cb5e8b6efaff76dcb1c974c3de0e690957028aceb7d87",
  • native-image 22.0.1 2024-04-16
  • GraalVM Runtime Environment GraalVM CE 22.0.1+8.1 (build 22.0.1+8-jvmci-b01)
  • Substrate VM GraalVM CE 22.0.1+8.1 (build 22.0.1+8, serial gc)
• JDK major version: [e.g.:8] 21
• OS: [e.g. macOS Catalina] macOS, Linux
• Architecture: [e.g.: AMD64] arm64, amd64

I tried to add as much information as possible, because everything is based on official images and code we build is just Java (no dependency nor framework),
I think it's not required to provide more data… but let me know if you need something else.

[ZynoZin]

Hello @davinkevin

For this version: 22.0.1, to be able to use --static you need to install musl libc. Here is the change log, I suggest you check it out. Also you can find instructions on installing musl here.

[davinkevin]

@ZynoZin , I know, but should this be included in the docker image provided by the project?

[ZynoZin]

Hello @davinkevin , thank you for reaching out. Can you please check this image: ghcr.io/graalvm/native-image-community:22.0.1-muslib it should have the needed library.

[davinkevin]

Thank you for your answer. I just discover this new image, good to know.

Using this image, and with --static --libc=musl, I have now the correct result, similar to what I had with 22.0.0.

PS: There is a strange error if --libc=musl is not provided, maybe something to improve.

Finally, you answered part of my request, you still think this breaking change in the cli "api" is not an issue for a patch release?

[ZynoZin]

No, this change was necessary because there was a bug when musl is not used in this case, so it became required. Thus, it throws an error during build time if not used. And it was mentioned in the patch release change log.

[davinkevin]

Thank you for the extra details. 👍