Optimize Hibernate native footprint by making ByteBuddy unreachable

[sdeleuze]

As experimented on sdeleuze@hibernate-custom-initiators, various optimizations are possible when SpringHibernateJpaPersistenceProvider configures Hibernate with a custom list of StandardServiceInitiator.

On native, it should allow to remove Bytebuddy from the reachable code path at runtime (enhancements are needed on Hibernate side for this), but likely many other optimizations should be possible for both JVM and native, potentially leveraging some AOT optimizations like entity scanning, making XML parsing not reachable as well when there is no such file, etc.

I would be interested to explore those optimizations in collaboration with the Spring Data team cc @mp911de @schauder @christophstrobl.

Note the impact on compatibility and maintainability should be evaluated as we would probably have to provide distinct variants for Hibernate 5 and Hibernate 6, or to support only Hibernate 6.

[sdeleuze]

As part of this effort, we could also collaborate with GraalVM to remove AWT and freetype dependencies, see https://twitter.com/sdeleuze/status/1593279741141868544.

[sdeleuze]

I have been able to implement this enhancement via this commit.

The impact of the removal of ByteBuddy (not used on native at runtime) from Spring Hibernate applications is the following on MacOS aarch64:

• 820 classes removed
• Executable 4.13 Mbytes smaller
• Reduction of 1.45 Mb of the RSS consumption
• 6% faster compilation time

The fact that we support both Hibernate 5.6.x and 6.x makes the advanced customization of the Hibernate StandardServiceInitiator a bit challenging, but I have something that looks like to work for both.

Such more advanced Hibernate customization could unlock more optimizations in the future, but of course could make Hibernate support more fragile, so I would need some feedback to see if we are open to consider inclusion in 6.1 or not.

[sdeleuze]

After discussing with @jhoeller, we agreed that this kind of customization is too involved and that we need a less impacting way to prevent ByteBuddy to be reachable on native images, something that does not require overriding the list of StandardServiceInitiator since that would make our Hibernate support too fragile.

A related refactoring seems to be planned for Hibernate 6.3.0 as mentioned in https://hibernate.atlassian.net/browse/HHH-16073, we should probably wait for it and implement a solution for Hibernate 6.3+ that prevents ByteBuddy to be reachable on native. I will also reach the Hibernate team to share our feedback.

[sdeleuze]

After discussing with the Hibernate team, it may be too late to have an impact on Hibernate 6.3.0, but we can propose a PR that could potentially be merged in Hibernate 6.4.0 to remove ByteBuddy reachability, the challenge being finding the an acceptable solution for both Spring and Hibernate teams.

[sdeleuze]

With the latest Hibernate 6.3.0-SNAPSHOT changes, it looks doable to remove ByteBuddy reachability with a focused substitution that should provide the same benefits for the end user than the proper fix we will try to have in Hibernate 6.4.