Native image size increased substantially with version 20.1

[ChadKillingsworth]

Describe the issue
👋 We've been using native-image generation for quite some time with Closure-Compiler. I recently upgraded from version 19.0.0 to 20.1.0. The images generated by Graal increased in size by an order of magnitude when I did that.

19.0.0 - Java 8:
compiler.jar: 8,607,558 bytes
native-image on MacOS: 36,693,476 bytes (other OSs are similar size)

20.1.0 - Java 8:
compiler.jar: 8,607,558 bytes
native-image on MacOS: over 180 MB (other OSs are similar size)

Now I'm upgrading from Java 8 to Java 11 and getting even larger images:

20.1.0 - Java 11:
native-image on MacOS: over 270 MB (other OSs are similar size)

I tried looking through compiler options, but nothing stood out that would contribute to the size increase so significantly

Steps to reproduce the issue
Please include both build steps as well as run steps

1. Obtain compiler.jar file: https://mvnrepository.com/artifact/com.google.javascript/closure-compiler/v20200719
2. Utilize the reflect-config file: https://github.com/google/closure-compiler-npm/blob/master/build-scripts/reflection-config.json
3. Build with native-image tool:

native-image --no-server --no-fallback --initialize-at-build-time \
  -H:IncludeResourceBundles=com.google.javascript.rhino.Messages \
  -H:IncludeResourceBundles=org.kohsuke.args4j.Messages \
  -H:IncludeResourceBundles=org.kohsuke.args4j.spi.Messages \
  -H:IncludeResourceBundles=com.google.javascript.jscomp.parsing.ParserConfig \
  -H:ReflectionConfigurationFiles=reflection-config.json \
  -H:IncludeResources=(externs.zip)|(.*(js|txt)) \
  -jar compiler.jar

Describe GraalVM and your environment:

• 20.1.0
• JDK major version: 8 or 11
• OS: macOS Cataline, Windows 10, Linux
• Architecture: 64bit on each

More details

/Users/chad/git/closure-compiler-npm/temp/graalvm-ce-java11-20.1.0/Contents/Home/bin/native-image --no-server --no-fallback --initialize-at-build-time -H:IncludeResourceBundles=com.google.javascript.rhino.Messages -H:IncludeResourceBundles=org.kohsuke.args4j.Messages -H:IncludeResourceBundles=org.kohsuke.args4j.spi.Messages -H:IncludeResourceBundles=com.google.javascript.jscomp.parsing.ParserConfig -H:ReflectionConfigurationFiles=/Users/chad/git/closure-compiler-npm/build-scripts/reflection-config.json -H:IncludeResources=|externs.zip|||.*|js|txt|| -H:Optimize=2 -R:-TruffleCompilation --native-image-info --verbose -jar /Users/chad/git/closure-compiler-npm/packages/google-closure-compiler-osx/compiler.jar
Downloading: Component catalog from www.graalvm.org
Processing Component: Native Image
Component Native Image (org.graalvm.native-image) is already installed.
Executing [
/Users/chad/git/closure-compiler-npm/temp/graalvm-ce-java11-20.1.0/Contents/Home/bin/java \
-XX:+UseParallelGC \
-XX:+UnlockExperimentalVMOptions \
-XX:+EnableJVMCI \
-Dtruffle.TrustAllTruffleRuntimeProviders=true \
-Dtruffle.TruffleRuntime=com.oracle.truffle.api.impl.DefaultTruffleRuntime \
-Dgraalvm.ForcePolyglotInvalid=true \
-Dgraalvm.locatorDisabled=true \
-Dsubstratevm.IgnoreGraalVersionCheck=true \
-Djava.lang.invoke.stringConcat=BC_SB \
--add-exports \
jdk.internal.vm.ci/jdk.vm.ci.runtime=ALL-UNNAMED \
--add-exports \
jdk.internal.vm.ci/jdk.vm.ci.code=ALL-UNNAMED \
--add-exports \
jdk.internal.vm.ci/jdk.vm.ci.aarch64=ALL-UNNAMED \
--add-exports \
jdk.internal.vm.ci/jdk.vm.ci.amd64=ALL-UNNAMED \
--add-exports \
jdk.internal.vm.ci/jdk.vm.ci.meta=ALL-UNNAMED \
--add-exports \
jdk.internal.vm.ci/jdk.vm.ci.hotspot=ALL-UNNAMED \
--add-exports \
jdk.internal.vm.ci/jdk.vm.ci.services=ALL-UNNAMED \
--add-exports \
jdk.internal.vm.ci/jdk.vm.ci.common=ALL-UNNAMED \
--add-exports \
jdk.internal.vm.ci/jdk.vm.ci.code.site=ALL-UNNAMED \
--add-exports \
jdk.internal.vm.ci/jdk.vm.ci.code.stack=ALL-UNNAMED \
--add-opens \
jdk.internal.vm.compiler/org.graalvm.compiler.debug=ALL-UNNAMED \
--add-opens \
jdk.internal.vm.compiler/org.graalvm.compiler.nodes=ALL-UNNAMED \
--add-opens \
jdk.unsupported/sun.reflect=ALL-UNNAMED \
--add-opens \
java.base/jdk.internal.module=ALL-UNNAMED \
--add-opens \
java.base/jdk.internal.ref=ALL-UNNAMED \
--add-opens \
java.base/jdk.internal.reflect=ALL-UNNAMED \
--add-opens \
java.base/java.io=ALL-UNNAMED \
--add-opens \
java.base/java.lang=ALL-UNNAMED \
--add-opens \
java.base/java.lang.reflect=ALL-UNNAMED \
--add-opens \
java.base/java.lang.invoke=ALL-UNNAMED \
--add-opens \
java.base/java.lang.ref=ALL-UNNAMED \
--add-opens \
java.base/java.net=ALL-UNNAMED \
--add-opens \
java.base/java.nio=ALL-UNNAMED \
--add-opens \
java.base/java.nio.file=ALL-UNNAMED \
--add-opens \
java.base/java.security=ALL-UNNAMED \
--add-opens \
java.base/javax.crypto=ALL-UNNAMED \
--add-opens \
java.base/java.util=ALL-UNNAMED \
--add-opens \
java.base/java.util.concurrent.atomic=ALL-UNNAMED \
--add-opens \
java.base/sun.security.x509=ALL-UNNAMED \
--add-opens \
java.base/jdk.internal.logger=ALL-UNNAMED \
--add-opens \
org.graalvm.sdk/org.graalvm.nativeimage.impl=ALL-UNNAMED \
--add-opens \
org.graalvm.sdk/org.graalvm.polyglot=ALL-UNNAMED \
--add-opens \
org.graalvm.truffle/com.oracle.truffle.polyglot=ALL-UNNAMED \
--add-opens \
org.graalvm.truffle/com.oracle.truffle.api.impl=ALL-UNNAMED \
-XX:+UseJVMCINativeLibrary \
-Xss10m \
-Xms1g \
-Xmx13743895344 \
-Duser.country=US \
-Duser.language=en \
-Djava.awt.headless=true \
-Dorg.graalvm.version=20.1.0 \
-Dorg.graalvm.config= \
-Dcom.oracle.graalvm.isaot=true \
-Djava.system.class.loader=com.oracle.svm.hosted.NativeImageSystemClassLoader \
-Xshare:off \
--module-path \
/Users/chad/git/closure-compiler-npm/temp/graalvm-ce-java11-20.1.0/Contents/Home/lib/truffle/truffle-api.jar \
-javaagent:/Users/chad/git/closure-compiler-npm/temp/graalvm-ce-java11-20.1.0/Contents/Home/lib/svm/builder/svm.jar \
-Djdk.internal.lambda.disableEagerInitialization=true \
-Djdk.internal.lambda.eagerlyInitialize=false \
-Djava.lang.invoke.InnerClassLambdaMetafactory.initializeLambdas=false \
-cp \
/Users/chad/git/closure-compiler-npm/temp/graalvm-ce-java11-20.1.0/Contents/Home/lib/svm/builder/llvm-wrapper-shadowed.jar:/Users/chad/git/closure-compiler-npm/temp/graalvm-ce-java11-20.1.0/Contents/Home/lib/svm/builder/svm.jar:/Users/chad/git/closure-compiler-npm/temp/graalvm-ce-java11-20.1.0/Contents/Home/lib/svm/builder/objectfile.jar:/Users/chad/git/closure-compiler-npm/temp/graalvm-ce-java11-20.1.0/Contents/Home/lib/svm/builder/llvm-platform-specific-shadowed.jar:/Users/chad/git/closure-compiler-npm/temp/graalvm-ce-java11-20.1.0/Contents/Home/lib/svm/builder/javacpp-shadowed.jar:/Users/chad/git/closure-compiler-npm/temp/graalvm-ce-java11-20.1.0/Contents/Home/lib/svm/builder/svm-llvm.jar:/Users/chad/git/closure-compiler-npm/temp/graalvm-ce-java11-20.1.0/Contents/Home/lib/svm/builder/pointsto.jar \
'com.oracle.svm.hosted.NativeImageGeneratorRunner$JDK9Plus' \
-imagecp \
/Users/chad/git/closure-compiler-npm/temp/graalvm-ce-java11-20.1.0/Contents/Home/lib/svm/builder/llvm-wrapper-shadowed.jar:/Users/chad/git/closure-compiler-npm/temp/graalvm-ce-java11-20.1.0/Contents/Home/lib/svm/builder/svm.jar:/Users/chad/git/closure-compiler-npm/temp/graalvm-ce-java11-20.1.0/Contents/Home/lib/svm/builder/objectfile.jar:/Users/chad/git/closure-compiler-npm/temp/graalvm-ce-java11-20.1.0/Contents/Home/lib/svm/builder/llvm-platform-specific-shadowed.jar:/Users/chad/git/closure-compiler-npm/temp/graalvm-ce-java11-20.1.0/Contents/Home/lib/svm/builder/javacpp-shadowed.jar:/Users/chad/git/closure-compiler-npm/temp/graalvm-ce-java11-20.1.0/Contents/Home/lib/svm/builder/svm-llvm.jar:/Users/chad/git/closure-compiler-npm/temp/graalvm-ce-java11-20.1.0/Contents/Home/lib/svm/builder/pointsto.jar:/Users/chad/git/closure-compiler-npm/temp/graalvm-ce-java11-20.1.0/Contents/Home/lib/svm/library-support.jar:/Users/chad/git/closure-compiler-npm/packages/google-closure-compiler-osx/compiler.jar \
-H:Path=/Users/chad/git/closure-compiler-npm/packages/google-closure-compiler-osx \
-H:FallbackThreshold=0 \
-H:ClassInitialization=:build_time \
-H:IncludeResourceBundles=com.google.javascript.rhino.Messages \
-H:IncludeResourceBundles=org.kohsuke.args4j.Messages \
-H:IncludeResourceBundles=org.kohsuke.args4j.spi.Messages \
-H:IncludeResourceBundles=com.google.javascript.jscomp.parsing.ParserConfig \
'-H:IncludeResources=|externs.zip|||.*|js|txt||' \
-H:Optimize=2 \
-R:-TruffleCompilation \
-H:+DumpTargetInfo \
-H:Class=com.google.javascript.jscomp.CommandLineRunner \
-H:Name=compiler \
-H:CLibraryPath=/Users/chad/git/closure-compiler-npm/temp/graalvm-ce-java11-20.1.0/Contents/Home/lib/svm/clibraries/darwin-amd64 \
-H:ReflectionConfigurationFiles=/Users/chad/git/closure-compiler-npm/build-scripts/reflection-config.json \

]
[compiler:36775]    classlist:   3,510.02 ms,  0.96 GB
[compiler:36775]        (cap):   3,492.65 ms,  0.96 GB
[compiler:36775]        setup:   6,103.76 ms,  0.96 GB
# Building image for target platform: org.graalvm.nativeimage.Platform$DARWIN_AMD64
# Using native toolchain:
#   Name: LLVM (clang)
#   Vendor: apple
#   Version: 11.0.3
#   Target architecture: x86_64
#   Path: /usr/bin/cc
# Using CLibrary: com.oracle.svm.core.c.libc.GLibc
[compiler:36775]     (clinit):   1,051.52 ms,  3.89 GB
# Static libraries:
#   ../../temp/graalvm-ce-java11-20.1.0/Contents/Home/lib/svm/clibraries/darwin-amd64/liblibchelper.a
#   ../../temp/graalvm-ce-java11-20.1.0/Contents/Home/lib/libnet.a
#   ../../temp/graalvm-ce-java11-20.1.0/Contents/Home/lib/svm/clibraries/darwin-amd64/libstrictmath.a
#   ../../temp/graalvm-ce-java11-20.1.0/Contents/Home/lib/svm/clibraries/darwin-amd64/libffi.a
#   ../../temp/graalvm-ce-java11-20.1.0/Contents/Home/lib/svm/clibraries/darwin-amd64/libdarwin.a
#   ../../temp/graalvm-ce-java11-20.1.0/Contents/Home/lib/libnio.a
#   ../../temp/graalvm-ce-java11-20.1.0/Contents/Home/lib/libjava.a
#   ../../temp/graalvm-ce-java11-20.1.0/Contents/Home/lib/svm/clibraries/darwin-amd64/libjvm.a
#   ../../temp/graalvm-ce-java11-20.1.0/Contents/Home/lib/libzip.a
# Other libraries: -framework Foundation,pthread,dl,z
[compiler:36775]   (typeflow):  22,066.44 ms,  3.89 GB
[compiler:36775]    (objects):  23,668.89 ms,  3.89 GB
[compiler:36775]   (features):   3,788.24 ms,  3.89 GB
[compiler:36775]     analysis:  52,679.34 ms,  3.89 GB
[compiler:36775]     universe:   2,138.00 ms,  3.89 GB
[compiler:36775]      (parse):  29,979.19 ms,  5.23 GB
[compiler:36775]     (inline):   7,883.12 ms,  5.23 GB
[compiler:36775]    (compile):  41,466.11 ms,  5.78 GB
[compiler:36775]      compile:  81,952.25 ms,  5.78 GB
[compiler:36775]        image:  47,181.08 ms,  5.33 GB
[compiler:36775]        write:   3,283.08 ms,  5.33 GB
[compiler:36775]      [total]: 197,244.43 ms,  5.33 GB

[ChadKillingsworth]

One clarification: The windows image size is still only 39MB. This appears to be a Linux/MacOS issue.

[concavelenz]

Can we provide any additional information or otherwise help with resolve this? npm download size is a significant concern when we start talking about 100s of MB for each supported platform.

[ChadKillingsworth]

This problem still exists with version 20.2.0

[concavelenz]

Is it possible to get confirmation that this is expected for Graal Native image 20+?

[cstancu]

I built an image from closure-compiler-v20210505.jar with the latest GraalVM built from master and I cannot reproduce the issue. I used -H:+AllowIncompleteClasspath since latest reflection-config.json contains some missing classes. The resulting image is 43MB, an expected increase resulting from improving Native Image compatibility with Java code, but we are continuously working to reduce the footprint of GraalVM native images, so it will likely get smaller in the long run.

[ChadKillingsworth]

@cstancu I just build GraalVM from master/source on my Mac. The native image produced was 216MB. What JDK were you using?

[ChadKillingsworth]

Or perhaps there are default compilation options or environment variables getting you much better results? Using the GraalVM Dashboard the size inflation is all coming from the heap - specifically *.[B (its 184 MB).