Declare Spring modules with JDK 9 module metadata [SPR-13501]

[spring-projects-issues]

Juergen Hoeller opened SPR-13501 and commented

JDK 9's Jigsaw initiative aims to allow for module metadata (module-info.java) that can be added to framework and library jars while still keeping them compatible with JDK 8. Let's do this for Spring Framework 5.0's modules as far as feasible. However, we might not be able to express our optional dependency arrangement that way, in which case we might have to resort to an "automatic modules" approach for the more modest purposes of #18289.

---

Issue Links:

• Stable module names for Spring Framework jars on JDK 9 module path [SPR-13716] #18289 Stable module names for Spring Framework jars on JDK 9 module path ("depends on")
• Upgrade core framework build to JDK 17 #17778 Upgrade core framework build to JDK 11 ("depends on")
• Document Spring recommendations/restrictions for Java 9 module setups [SPR-14579] #19148 Document Spring recommendations/restrictions for Java 9 module setups

1 votes, 12 watchers

[spring-projects-issues]

Sam Brannen commented

Juergen Hoeller, is this issue a duplicate of #18289?

[spring-projects-issues]

Juergen Hoeller commented

Not necessarily: This one is about shipping explicit module-info.java files in our own jars, and is rather unlikely to happen at this point since we cannot express optional dependencies that way. However, for the purposes of #18289, we could also aim for what Jigsaw calls "automatic modules", i.e. putting our regular jars onto the module path and letting them participate in a module-based arrangement that way (with implicit access to all other modules).

[spring-projects-issues]

Sam Brannen commented

OK. I see the differences between the two issues now. Thanks for the clarification!

[spring-projects-issues]

Juergen Hoeller commented

The Jigsaw page has just been updated with a proposal towards optional dependencies:
http://openjdk.java.net/projects/jigsaw/spec/issues/#CompileTimeDependences

[spring-projects-issues]

Sam Brannen commented

while still keeping them compatible with JDK 9

Is that supposed to say "JDK 8"?

[spring-projects-issues]

Grigory Kislin commented

Hello!
Are module-info.java planned to add in Spring 5.1, as was announced in video [Spring Framework 5.0 on JDK 8 & 9|https://www.youtube.com/watch?v=0-sbPBf81KA]? In recent 5.1.RC2 version they are missed.

[spring-projects-issues]

Juergen Hoeller commented

This issue is marked as "General Backlog", indicating that we won't deal with it for 5.1 (otherwise it'd be marked for 5.1 GA still) and probably not in subsequent 5.x releases either (otherwise it'd be marked as "5.x Backlog").

Specifically, we can't ship module-info files quite yet since we'd need stable module names for all of our optional dependencies... and many of those don't declare stable module names at this point (that is, they don't even include an Automatic-Module-Name manifest entry in their jar). Also, we'd need to build the entire framework on JDK 9+ for the compiler to understand the module-info.java format which is not entirely trivial either, even if the framework itself is known to work fine with JDK 9/10/11 at runtime.

All in all, my prediction about module-info files for 5.1 turned out to be too ambitious. Our current focus is on general JDK 11 compatibility (#20937) on the classpath and as automatic modules on the module path, as well as GraalVM compatibility (#21529). The use of jlink requires manual addition of module-info.class files to the framework jars for the time being... which might stay that way for several years still until we ship a JDK 11 baselined Spring Framework 6.0 against a new generation of dependencies.

[spring-projects-issues]

yboompook commented

I tried to jlink my project with moditect (maven plugin to add module-info to dependencies that don't have one), java-11. I pass throw lot's of problem and it still doen't work, however I think my work could help you to make it work in spring 6.

poc on github

stackoverflow

[jabrena]

Any progress with this issue?

[cesarbiods]

Sounds like this will happen maybe 10-15 years from now and by then we'll be on JDK 50.

[xenoterracide]

also checking in, is there anything that other people can help with on this? as I too would like to see more done than simply automatic module naming.

[lpandzic]

It seems that new Java features might be more feature rich when using module system with one example being JEP 409: Sealed classes:

The classes specified by permits must be located near the superclass: either in the same module (if the superclass is in a named module) or in the same package (if the superclass is in the unnamed module).

My SO question on this topic.

While JEP 409 most likely won't be used much until JEP 406: Pattern Matching for switch comes out of preview, I believe this sets a clear precedent of where we are going.

[dsyer]

For the sake of clarity, you can build a jlink package for a Spring Boot app. It works well with Java 11 and 17 (with 17 much leaner at runtime), but only includes JDK modules for reasons explained above - the other dependencies have to be added on the classpath. Notes here: https://github.com/dsyer/sample-docker-microservice.

[lpandzic]

Just to reemphasize my previous comment: this is first example, that I know of, of a Java language feature being semi locked behind modularization.

Split packages are biggest obstacle to modularization and split packages are everywhere.

Regarding the Sealed Classes and Pattern matching I believe the pragmatic choice today is to either:

1. put whole hierarchy inside one package
2. put whole hierarchy in a separate Maven module and then Java modularize just that Maven module

I was surprised that things like Lombok work well with modularization with some configuration changes.

The clean way forward is to modularize the whole application but the ecosystem is simply not ready for this and unless a major player like Spring or Jakarta or someone else pushes "small" library maintainers to modularise their code I don't see modularization happening any time soon - it's simply too big of an undertaking for every developer to isolate/repackage every dependency that breaks modularization rules.

[Sineaggi]

Now that Spring Framework 6 is on the horizon, have there been any thoughts towards moving to modules?

[grubeninspekteur]

Just adding some thoughts for your consideration of modules in the long term: Smaller artifacts generated by jlink are not the only benefit (cf. Ron Pressler: What modules are about). One is having an encapsulation level above package. For example, classes like Function in the org.springframework.cglib.core.internal package can be properly hidden from user code and only exported to Spring's own modules. This also allows to evolve classes like Assert without having to deprecate methods first. People tend to import these types by accident, despite the "internal" warning. As a bonus, the number of code completion suggestions for such types in IDEs is reduced.

Directly following from the encapsulation feature is more secure code, since modules must explicitly open their types for reflection. Now of course Spring relies heavily on reflection, so you would have to open most of your beans/data types to it anyway; but you could restrict access only to those parts that need it; for example, open beans to spring.core, but not spring.validation, so a vulnerability in the validation module could not be exploited to gain access to a private EntityManager, for example. Native images somewhat mitigate reflection attacks with their requirement of explicitly stating the types to reflect on, but they do not discriminate between who is doing the reflection.

However, since adding such strong encapsulation would be a breaking change, I do not expect it to arrive before the next major bump of Spring. And like you said, there are no stable automatic module names in all dependencies. Unfortunately, unlike generics, we seem to have arrived at a catch-22 situation with JPMS. No one bothers to use it until the big libraries and frameworks migrate (often taking the role of trailblazers, see AOT), and they won't migrate since there is little demand.

Edit (2023-04-24): I misunderstood the opens clause in JPMS: it is only required if you want the reflecting code to break encapsulation (setAccessible etc.). If you do proper constructor injection and only serialize/deserialize via public constructors, you wouldn't even need to open your implementations to Spring, only export them. This reduces the attack surface for reflection gadgets. Opens would probably be required for runtime AOT proxies though, as they have to be created inside the target package. Also note that even if you open just a single package, by defining a class in that package you gain reflective access to the entire module. Users of a modularized Spring would have to be aware of this, and Spring would have to be very cautious about the rights of runtime generated classes.

[xenoterracide]

My personal reason for wanting this is to avoid accidental imports as well. Unfortunately tools like Gradle and maven only go so far to prevent you from importing a jar that you didn't intentionally intend to import from. Basically trying to avoid transient hell. I've seen people not only import from internal packages but also from utility packages of libraries that we'd rather not be using directly in that way, for example.

[xenoterracide]

Has anyone done any benchmarks on startup time with large codebases? I don't fully understand the low level implications of JPMS. Isn't it supposed to know what jar's have what packages, and so if you're doing classpath scanning it wouldn't open a jar at all that doesn't allow you to opens (or whatever, my jpms is rusty)? My mental theory is that in a moderate+ sized codebase, jpms across the board for hibernate and spring could speedup startup time. Sadly I'm not working on such a project atm, but if someone has one and it's gradle they could use this to test maybe.

https://github.com/gradlex-org/extra-java-module-info

[Blackwolf291]

9 years later, is there a list of compatibility issues and libraries that prevent a strict module definition, so that it's clear what needs working on?

[bowbahdoe]

This isn't the most professional start in the world, but I wrote up a quick script to check the modularization statuses of "everything you'd get if you went to https://start.spring.io and selected all the options."

https://spring-modularization.netlify.app/

Hopefully the people paid to work on spring will prioritize this eventually, but in the meantime you'll notice a lot of "long tail" libraries that do not have any module name and many that only have an automatic module name.

For spring to add explicit descriptors, dependencies need to have at least automatic (i.e. stable) names. To be able to jlink spring, everything needs an explicit module info.

Note that this site will not show whether there is a newer version that does have a module info. Just whether the one you'd get from spring has one.

[xenoterracide]

at least the most egregious problems appear to come from 3rd party starters, and not things spring itself depends on. Like vaadin. I want to buy all the people with green a "beer". Seems like it would be a good idea to go look/poke at anything that is in the tree of an org.springframework.

[bowbahdoe]

@xenoterracide I have considered making pizza bounties. I have legitimately offered at least one person a pizza for merging a module-info PR though they didn't take me up on it. (yet, its still valid)

Last night I handled JSQLParser/JSqlParser#1999 which is a dependency of spring-data-relational. Once they make their next release we should bump the dep version there

I also made an issue on micrometer. micrometer-metrics/micrometer#4990

This is a wider ecosystem issue which is going to take a lot of work, convincing, arguments, and likely bribery. Best to start on it.

[Sineaggi]

@bowbahdoe is there a way to simplify the list down to direct dependencies of spring-framework, and not spring boot? So in this case starting with the most popular spring libraries, like spring-context, spring-web (and webmvc)? Otherwise a similar issues should be raised in the spring boot repo.

[bowbahdoe]

@Sineaggi

Yes, but keep in mind im an idiot and its a bit of a jank script. If you can give me a POM file with all the root dependencies that would be enough to run it.

Another potentially narrower place to start would be spring-petclinic. Since that is what could most easily be used to evaluate startup / perf benefits of a fully jlinked spring once we get there in however long

[bowbahdoe]

@Sineaggi This is the list from just the org.springframework deps.

The list for petclinic is longer. I'll figure out how best to present that at some point. But it was a good call to run it on all the spring framework deps. This confirms that everything with a group org.springframework is in fact good to go on adding module-infos.

The yellow transitive deps just need to be updated as well so these parts can be jlinked.

(edit: scratch that. this doesn't include optional dependencies)

[xenoterracide]

@bowbahdoe are we in any better shape after SB 3.3.0? any chance you can make your janky script auto update the site periodically? p.s. If I didn't say it already good work. p.p.s I don't see netflix DGS on there.

[bowbahdoe]

@xenoterracide A problem with the script I made is that it doesn't include optional dependencies. This is actually pretty important because they at least need stable names too all the way up the chain.

I highly encourage not relying on me to do novel work in any reasonable time frame. If you have an interest you should try to edit the script and make it more robust / usable for actual tracking rather than an info dump.

Maybe @sgammon has some intuition on how to proceed? (from another modularization thread)

[sgammon]

Heya @bowbahdoe

Happy to help, but the deadlock on JPMS is probably greater than Spring: JPMS support still hasn't quite landed in e.g. Guava. Using jlink in earnest not only requires modules, but requires full modules (and actively rejects Automatic-Module-Name JARs).

Many projects seem to have adopted Automatic-Module-Name as a stop-gap measure, thinking it was roughly equivalent to providing a full-blown module-info.class descriptor. That has never been the case.

I am trying to push the ecosystem to support JPMS, via the javamodules project, and specifically the attic repo, which provides JPMS-enabled artifacts with verifiable build provenance. It has a focus on projects that are "too important" or too slow-moving to quickly ship JPMS.

I would be happy to host experimental libs in that repo if it would help, or to help generally with JPMS, but my guess is that (1) Spring's project authors know better than me what to do, and that (2) even if Spring were fully JPMS-enabled, upstream libs would probably preclude jlink and other pure-JPMS features from being usable anyway.

I would be shocked if a Spring app of reasonable complexity didn't have Guava in the dependency graph, and that's game over.

No question it is a mess, but we're here now, JPMS isn't going away, and so the attic repo / waiting patiently are the best plans I have at this time. I'm coordinating this effort here if you want to follow or contribute or request libraries.

[sgammon]

(It's not all bad btw. Guava couldn't ship JPMS for the longest time because it also has upstream dependencies that need module descriptors. Luckily, Error Prone and j2objc have shipped support, so once Guava follows, it should unblock a lot of dependency graphs for a lot of people.)

[bowbahdoe]

@sgammon so what I think we need here (and I'm just guessing you might be more primed to provide - or at least would have an opinion on) is a more robust way for projects to evaluate their supply chains and apply downward pressure on libraries.

What I produced above does not seem to be good enough for spring's purposes.

I'm under no illusions this will go quickly but starting in earnest 10 years after modules were added in is...well better than 20.

[xenoterracide]

I highly encourage not relying on me to do novel work in any reasonable time frame.

yeah, I was mostly just hoping to see the version that had been executed updated to 3.3.0 so we could see if there was progress with any of the major versions you did list, since new major/minor would have been updated. I figure that's an easy change for you. I'm not needing perfection. Ya'll are doing good work.

It's a shame that 10 years later that we don't have automatic-modules everywhere, or that it has to result in ... arguments ... with authors because they don't use JPMS, and people aren't clamoring. I wonder how many don't ask because they hit a library that doesn't have any support and so they simply stop. Saddest thing here is the number of spring ... platform? ... libraries that don't even have automatic modules