Putting the native GraalVM configs in the main jar?

[nielsbasjes]

Hi @ben-manes,

I was looking into an issue raised against my library (Yauaa which uses caffeine) about using it in a GraalVM native application.

Some of the comments mentioned that they have to do specific things also for Caffeine to get it running.

I'm wondering on how I should handle this in my library.

I was thinking about simply including all the configs needed to make this possible in the main jar file, but that requires me to also include these configs for your library, which to me is a bit strange.

Now I did notice that you have a clear example on how to use caffeine in a graalvm native application
https://github.com/ben-manes/caffeine/tree/master/examples/graal-native/src/main/resources/META-INF/native-image

Why did you choose to keep this a separate example and not simply include these configs in the main jar distribution?

[ben-manes]

I asked that question to Quarkus when they temporarily wanted me to include reachability metadata. That was before there was a standard format and they used jandex for the manifest. Afterwards they decided against it because it led to larger binaries since it no longer prunes to just what the application uses. For anything configuration-driven the AOT processor cannot know what path to take, causing it to include the universe of classes instead of the relevant subset.

Caffeine uses code generation to optimize the memory usage to only the fields in use (e.g. expiration, weak keys). This is ideal on the JVM since there is no cost for unloaded classes and a slightly larger jar file is preferable to wasted memory usage. If AOT is precompiling all variants then it does increase the binary size unnecessarily. I think that is why the JDK team didn't adopt Graal AOT as is and are working on in language pre-computations, precompiled code caches, etc. Currently AOT is more of an expert mode option with common reachability metadata provided by the frameworks (Quarkus', Oracle's graalvm-reachability-metadata and manual fine tuning.

AOT explicitly breaks encapsulation, requires expert level tradeoff decisions, and is non-standard Java. Therefore, I don't think its appropriate for users to expect libraries to provide direct support. Taking their needs into consideration and integration testing is already above average. If they are experts then they'll want to tune the reachability metadata anyway, and if they are not then they shouldn't be using AOT. I think in both of our projects having integration testing to ensure it works is good enough and then we can direct those users back to their community for support. So for your next steps, maybe add an example to squash question, serve as an integration test, and redirect users to the graalvm-reachability-metadata for further library support.

[Stephan202]

And IIUC step 2 can even be skipped/automated using Google AutoService:

@AutoService(Feature.class)
public final class CaffeineFeature implements Feature {
    ...
}

[nielsbasjes]

In my mind the main reason for going native is runtime performance.
For me a static few MiB in unused classes is not important in the multi GiB world we live in.
I consider 5 or even 14 MiB as an irrelevant value. (Note, I did my CS degree in the computers had 1MiB era)
Apparently the Quarkus people think differently about this point.

I did some tests and I was not able to reduce the list of classes by providing an override. Seems like all configs are added together. So your fear of people being unable to override it seems to be correct.

So I was thinking about this approach:

You generate the full graalvm native config files for all of your generated classes and turn that into a separate jar you publish: caffeine-graalvm-native-3.2.3.jar.

This new jar is only the config files like META-INF/native-image/com.github.ben-manes.caffeine/caffeine/reflect-config.json and the indication that it depends on caffeine-3.2.3.jar

Intended effects:

• non-native users of caffeine are completely unaffected.
• Lazy native users (like me) can simply depend on caffeine-graalvm-native, get a +14MiB native executable and full speed without any hassle.
• Size conscious native users (i.e. who want to tweak the config like Quarkus) can unzip this jar and (manually?) prune from there.

For reference what I created for my tests: https://github.com/nielsbasjes/yauaa/blob/NativeGraalVM/examples/graalvm-native/src/main/resources/META-INF/native-image/com.github.ben-manes.caffeine/caffeine/reflect-config.json

[nielsbasjes]

I'm going to try the Feature thing you showed!

[nielsbasjes]

I got the Feature running ... but no white smoke right now ...

Error: Classes that should be initialized at run time got initialized during image building:
 com.github.benmanes.caffeine.cache.SSMS was unintentionally initialized at build time. To see why com.github.benmanes.caffeine.cache.SSMS got initialized use --trace-class-initialization=com.github.benmanes.caffeine.cache.SSMS
com.github.benmanes.caffeine.cache.BoundedBuffer was unintentionally initialized at build time. To see why com.github.benmanes.caffeine.cache.BoundedBuffer got initialized use --trace-class-initialization=com.github.benmanes.caffeine.cache.BoundedBuffer
com.github.benmanes.caffeine.cache.SSMSW was unintentionally initialized at build time. To see why com.github.benmanes.caffeine.cache.SSMSW got initialized use --trace-class-initialization=com.github.benmanes.caffeine.cache.SSMSW
com.github.benmanes.caffeine.cache.PSW was unintentionally initialized at build time. To see why com.github.benmanes.caffeine.cache.PSW got initialized use --trace-class-initialization=com.github.benmanes.caffeine.cache.PSW
com.github.benmanes.caffeine.cache.StripedBuffer was unintentionally initialized at build time. To see why com.github.benmanes.caffeine.cache.StripedBuffer got initialized use --trace-class-initialization=com.github.benmanes.caffeine.cache.StripedBuffer
com.github.benmanes.caffeine.cache.Caffeine was unintentionally initialized at build time. To see why com.github.benmanes.caffeine.cache.Caffeine got initialized use --trace-class-initialization=com.github.benmanes.caffeine.cache.Caffeine
com.github.benmanes.caffeine.cache.PSWMS was unintentionally initialized at build time. To see why com.github.benmanes.caffeine.cache.PSWMS got initialized use --trace-class-initialization=com.github.benmanes.caffeine.cache.PSWMS
com.github.benmanes.caffeine.cache.BoundedLocalCache was unintentionally initialized at build time. To see why com.github.benmanes.caffeine.cache.BoundedLocalCache got initialized use --trace-class-initialization=com.github.benmanes.caffeine.cache.BoundedLocalCache
com.github.benmanes.caffeine.cache.NodeFactory was unintentionally initialized at build time. To see why com.github.benmanes.caffeine.cache.NodeFactory got initialized use --trace-class-initialization=com.github.benmanes.caffeine.cache.NodeFactory
com.github.benmanes.caffeine.cache.PS was unintentionally initialized at build time. To see why com.github.benmanes.caffeine.cache.PS got initialized use --trace-class-initialization=com.github.benmanes.caffeine.cache.PS
com.github.benmanes.caffeine.cache.SS was unintentionally initialized at build time. To see why com.github.benmanes.caffeine.cache.SS got initialized use --trace-class-initialization=com.github.benmanes.caffeine.cache.SS
com.github.benmanes.caffeine.cache.LocalCacheFactory was unintentionally initialized at build time. To see why com.github.benmanes.caffeine.cache.LocalCacheFactory got initialized use --trace-class-initialization=com.github.benmanes.caffeine.cache.LocalCacheFactory
To see how the classes got initialized, use --trace-class-initialization=com.github.benmanes.caffeine.cache.SSMS,com.github.benmanes.caffeine.cache.BoundedBuffer,com.github.benmanes.caffeine.cache.SSMSW,com.github.benmanes.caffeine.cache.PSW,com.github.benmanes.caffeine.cache.StripedBuffer,com.github.benmanes.caffeine.cache.Caffeine,com.github.benmanes.caffeine.cache.PSWMS,com.github.benmanes.caffeine.cache.BoundedLocalCache,com.github.benmanes.caffeine.cache.NodeFactory,com.github.benmanes.caffeine.cache.PS,com.github.benmanes.caffeine.cache.SS,com.github.benmanes.caffeine.cache.LocalCacheFactory```

[ben-manes]

I suspect that the real reason is if the build times increase non-linearly with the number of classes because the simple example takes 4 minutes to compile on GHA. For a meaningful application then fine tuning that would be required but takes a lot of work. That's why I think the frameworks like Quarkus take ownership so they can add features by the required subset and users mostly write small services within their scaffolding. I doubt many people use Graal AOT directly like we both tried except in their own experiments, since neither of us or other projects seem to get a lot of demand (but it is interesting to experiment with). So a full config might only be practical for examples like yours and not be used in an application, making me hesitant to bother (it is all so confusing).

My guess is that Graal AOT will be an important inflection point but not be a long-term solution in the ecosystem. The work in Leyden to provide a spectrum of AOT flavors and the current ability in jlink/jpackage to bundle the jre into a runtime image gives most of the benefits. I am hoping many of the non-AOT improvements like Graal JIT to replace C1/C2 makes their way upstream. I just don't think Graal AOT will be the long-term solution and the application frameworks that take the burden to support it do so in order to evangelize their own adoption, but will abandon it as the ecosystem moves on.

I guess I just have a lot of hesitation doing anything more as it feels very murky.

[nielsbasjes]

FYI: This is the config I used for my project: https://github.com/nielsbasjes/yauaa/blob/main/analyzer/src/main/resources/META-INF/native-image/nl.basjes.parse.useragent/yauaa/reachability-metadata.json