Support for creating native image with GraalVM

[arnzel]

Did anyone tried creating native image with GraalVm ? I tried with following Dockerfile

FROM oracle/graalvm-ce:19.2.0.1
WORKDIR /opt/graalvm
RUN gu install native-image
ARG JARFILE
COPY ${JARFILE} /${JARFILE}
WORKDIR /
RUN native-image -jar ${JARFILE}

and got this error

Fatal error: java.lang.IllegalArgumentException: java.lang.IllegalArgumentException: Class io.dropwizard.jetty.ConnectorFactory cannot be instantiated reflectively. It must be a non-abst
ract instance type.

[joschi]

As far as I know, Jersey (and HK2) aren't supporting Graal native images at the moment.

[meshuga]

It looks like Dropwizard uses SPI in a way it wasn't intended to be used. When building native image, GraalVM will try to create instances of all classes that implement SPI interfaces, as described in https://github.com/oracle/graal/blob/92f0b657a5f4892f7380517a714bf93d13db2530/substratevm/src/com.oracle.svm.hosted/src/com/oracle/svm/hosted/ServiceLoaderFeature.java#L79

It means that for an SPI interface io.dropwizard.jackson.Discoverable, GraalVM will try to create instances of e.g. io.dropwizard.jetty.ConnectorFactory, since it's defined in META-INF/services directory in dropwizard-jetty project. That will fail because ConnectorFactory is an interface.

There are other interfaces that extend Discoverable, which are used by io.dropwizard.jackson.DiscoverableSubtypeResolver. To make Dropwizard work with GraalVM, I think that feature must be adjusted, so that it would drop usage of SPI for that functionality in its current form.

Jersey should be working now oracle/graal#846.

[ccleve]

This bug is definitely worth fixing. With Graal we could distribute a Dropwizard app as a native binary, eliminating the need for a shaded jar file. It would even eliminate the need to install a jvm on a target system.

[meshuga]

I tried to build a native image of Dropwizard 2.0 with GraalVM CE 19.3.1 with Java 11. The source code is available at https://github.com/meshuga/dropwizard-graal.

To overcome the error described above, I turned off ServiceLoaderFeature and it seems the issue I described above issue can be solved by writing a custom feature that would (no change of Dropwizard internals is needed):

1. Register the resources by calling Resources.registerResource(...), as done in https://github.com/oracle/graal/blob/0ab4d527916ebec006189a04692bfcb0edff6592/substratevm/src/com.oracle.svm.hosted/src/com/oracle/svm/hosted/ServiceLoaderFeature.java#L233
2. Don't initialise interfaces, that is omit RuntimeReflection.registerForReflectiveInstantiation(implementationClass); call if possible (ref https://github.com/oracle/graal/blob/92f0b657a5f4892f7380517a714bf93d13db2530/substratevm/src/com.oracle.svm.hosted/src/com/oracle/svm/hosted/ServiceLoaderFeature.java#L222)

With the feature turned off, I was able to build a native image within 2-4 minutes (on MacBook Pro 2017) with outcoming binary having 27 MB but get a startup error when initializing Hibernate Validator and Joda Time:

java.io.IOException: Resource not found: "org/joda/time/tz/data/ZoneInfoMap" ClassLoader: java.lang.ClassLoader@1083c7928
        at org.joda.time.tz.ZoneInfoProvider.openResource(ZoneInfoProvider.java:225)
        at org.joda.time.tz.ZoneInfoProvider.<init>(ZoneInfoProvider.java:138)
        at org.joda.time.tz.ZoneInfoProvider.<init>(ZoneInfoProvider.java:97)
        at org.joda.time.DateTimeZone.getDefaultProvider(DateTimeZone.java:555)
        at org.joda.time.DateTimeZone.getProvider(DateTimeZone.java:449)
        at org.joda.time.DateTimeZone.forTimeZone(DateTimeZone.java:367)
        at org.joda.time.DateTimeZone.getDefault(DateTimeZone.java:175)
        at com.fasterxml.jackson.datatype.joda.cfg.FormatConfig.<clinit>(FormatConfig.java:22)
        at com.oracle.svm.core.hub.ClassInitializationInfo.invokeClassInitializer(ClassInitializationInfo.java:350)
        at com.oracle.svm.core.hub.ClassInitializationInfo.initialize(ClassInitializationInfo.java:270)
        at java.lang.Class.ensureInitialized(DynamicHub.java:475)
        at com.fasterxml.jackson.datatype.joda.deser.DateTimeDeserializer.forType(DateTimeDeserializer.java:34)
        at com.fasterxml.jackson.datatype.joda.JodaModule.<init>(JodaModule.java:20)
        at io.dropwizard.jackson.Jackson.configure(Jackson.java:63)
        at io.dropwizard.jackson.Jackson.newObjectMapper(Jackson.java:30)
        at io.dropwizard.setup.Bootstrap.<init>(Bootstrap.java:61)
        at io.dropwizard.Application.run(Application.java:86)
        at com.github.meshuga.GraalApplication.main(GraalApplication.java:10)
Exception in thread "main" javax.validation.ValidationException: Cannot instantiate provider type: class org.hibernate.validator.HibernateValidator
        at javax.validation.Validation$NewProviderInstance.run(Validation.java:422)
        at javax.validation.Validation$NewProviderInstance.run(Validation.java:404)
        at javax.validation.Validation$ProviderSpecificBootstrapImpl.run(Validation.java:240)
        at javax.validation.Validation$ProviderSpecificBootstrapImpl.configure(Validation.java:213)
        at io.dropwizard.validation.BaseValidator.newConfiguration(BaseValidator.java:26)
        at io.dropwizard.jersey.validation.Validators.newConfiguration(Validators.java:33)
        at io.dropwizard.jersey.validation.Validators.newValidatorFactory(Validators.java:26)
        at io.dropwizard.setup.Bootstrap.<init>(Bootstrap.java:64)
        at io.dropwizard.Application.run(Application.java:86)
        at com.github.meshuga.GraalApplication.main(GraalApplication.java:10)
Caused by: java.lang.InstantiationException: Type `org.hibernate.validator.HibernateValidator` can not be instantiated reflectively as it does not have a no-parameter constructor or the no-parameter constructor has not been added explicitly to the native image.
        at java.lang.Class.newInstance(DynamicHub.java:796)
        at javax.validation.Validation$NewProviderInstance.run(Validation.java:419)
        ... 9 more

I think after all, it's possible to make Dropwizard run as native image (with narrowed configuration e.g. HTTP only). The compilation time will be biggest issue here since the framework has a lot of transitive dependencies but I think with time, the compilation time of GraalVM will be improved.

Technically, the biggest challenge will be to provide proper support for Jersey and HK2, as @joschi mentioned.

[OneCricketeer]

I see something about HK2 mentioned here.

https://developers.redhat.com/blog/2019/04/12/migrating-java-applications-to-quarkus-lessons-learned/

[OneCricketeer]

If Jersey is the limiting factor, then that just goes back to my rant on #1463

[trnl]

@OneCricketeer, @meshuga I wonder if you have any updates on it?

[meshuga]

I'm no longer working on projects that involve Dropwizard, thus will be hard to test things out on production-like systems.

I checked the timeline of the Jersey roadmap for GraalVM support and it looks like it should be supported in release 2.34 - https://github.com/eclipse-ee4j/jersey/wiki/Road-Map - fingers crossed!

Once done, it should be possible to adjust Dropwizard to create native images. JDBI seems to work with native images (jdbi/jdbi#1797), same with Hibernate (https://in.relation.to/2020/02/14/hibernate-orm-5-4-12/)

[trnl]

Graal VM support moved to Jersey 2.35.

[subash89]

@trnl Do you know any issue/jira created on jersey side related to this so that we can follow up on an ETA? I could not find any.

[kenzley]

@subash89 The jersey side issue is: eclipse-ee4j/jersey#4547.

It appears that basic graalvm support was added in Jersey 2.35. See: https://github.com/eclipse-ee4j/jersey/blob/master/docs/src/main/docbook/graalvm-native-image.xml.

[joschi]

Unfortunately, the support for Optional<T> is still broken in Jersey 2.35.
#3969 (comment)

[github-actions]

This issue is stale because it has been open 90 days with no activity. Remove the "stale" label or comment or this will be closed in 14 days.

[trnl]

@joschi, can you please reopen the issue? It is still very relevant and actually a big downside for Dropwizard in comparison with other frameworks.

[joschi]

@trnl Would you be willing to work on the issue if we reopen it?

Looking at the effort it takes for Quarkus, Micronaut, and Spring Native to work properly with GraalVM Native Image, I don't think that we (@dropwizard/committers) will work on it anytime soon.

[trnl]

@joschi can you elaborate more on the efforts or point out where I can find more information about it?

[joschi]

@trnl The existence of Spring Native itself, https://info.michael-simons.eu/2020/09/15/about-the-tooling-available-to-create-native-graalvm-images/, https://www.salvis.com/blog/2021/12/29/graalvm-native-image-first-impressions/, etc.

At least I am not willing and able to spend that much time on this.

[zUniQueX]

Short update: I'm currently testing on the dropwizard-example project, if the approach is maintainable for dropwizard. Currently I'm facing issues with Hibernate, which tends to have problems with GraalVM (see Hibernate 5.4.12 blog post).

[vk-98]

Is there any update on this particular support for GraalVM with Dropwizard?

[meshuga]

Hibernate team only focuses on supporting Hibernate with Quarkus but it looks like Hibernate 6 works with graalvm, just need to provide configurations in SPI.

https://discourse.hibernate.org/t/hibernate-6-native-image-from-scratch-no-spring-boot-no-micronaut-no-quarkus/7001