Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Compatibility with JDK 9 Jigsaw Builds [SPR-12549] #17151

Closed
spring-issuemaster opened this issue Dec 17, 2014 · 33 comments

Comments

Projects
None yet
2 participants
@spring-issuemaster
Copy link
Collaborator

commented Dec 17, 2014

Juergen Hoeller opened SPR-12549 and commented

The Jigsaw branch recently got merged into the JDK 9 mainline snapshots: a good opportunity to do some tests against b42 or higher (https://jdk9.java.net/download/), making sure Spring 4.1.4 builds and runs on it as far as possible.

Further Resources


Affects: 4.1.3

Issue Links:

  • #17186 General compatibility with JDK 8u40
  • #17608 General compatibility with JDK 8u60

0 votes, 6 watchers

@spring-issuemaster

This comment has been minimized.

Copy link
Collaborator Author

commented Dec 22, 2014

Juergen Hoeller commented

As expected, the core framework does build and run on JDK 9. However, the usual suspects in the wider build - the spring-aspects module and the spring-oxm module - break since their build setup relies on custom compilers, none of which have been made compatible with JDK 9 yet.

To be revisited for Spring Framework 4.2 - however, we may be stuck on a JDK 8 build for some time to come. AspectJ 1.9 is unlikely to happen before 2016, and Castor / JiBX / XMLBeans might not get updated any time soon. That said, at this point, there is no need for us to build on JDK 9 either.

@spring-issuemaster

This comment has been minimized.

Copy link
Collaborator Author

commented Mar 3, 2015

Juergen Hoeller commented

An update against JDK 9 build 52: spring-oxm builds and its tests pass now. However, spring-aspects still doesn't compile due to AspectJ 1.8.5 not being able to find the standard JDK class library on a JDK 9 installation.

Andy Clement, any chance for AspectJ 1.8.x. to 'tolerate' JDK 9 in such a scenario? There is no need for -target 1.9 yet and probably won't be for years to come; all we'd need is the ability to build and run the entire test suite on JDK 9.

Juergen

@spring-issuemaster

This comment has been minimized.

Copy link
Collaborator Author

commented Mar 3, 2015

Andy Clement commented

Hey Juergen, it has been vaguely on my radar to look at but I have not yet had a forcing function to do the work. I see I have that now :) I'll take a look when I get a minute.

@spring-issuemaster

This comment has been minimized.

Copy link
Collaborator Author

commented Mar 3, 2015

Andy Clement commented

Hmmm. If it were just the weaver that needed smarts for understanding Java9 that would be one thing but it is also the JDT compiler that we depend upon - that complicates things. I'll reach out to the eclipse guys to see if anyone has already started anything on this.

@spring-issuemaster

This comment has been minimized.

Copy link
Collaborator Author

commented Mar 3, 2015

Juergen Hoeller commented

Since we don't need support for the 1.9 language level, maybe there's a way to patch the JDT compiler to just find the class libraries? Assuming there are no other incompatibilities, of course...

If it turns out to be too much hassle, we can defer this until early 2016 as well. It would just be nice to run a CI build against JDK 9 from this summer onwards, once JDK 9 enters its API freeze.

@spring-issuemaster

This comment has been minimized.

Copy link
Collaborator Author

commented Mar 4, 2015

Andy Clement commented

maybe there's a way to patch the JDT compiler to just find the class libraries?

It is a bit messy due to all the layers of abstraction in JDT. However, I found that they have a BETA_JAVA9 branch of JDT with some preliminary work in it. I grabbed that, found it was a little incomplete and added a bit more code - this got compilation working but the weaver didn't work so I did some extra hacks to get the weaver limping along too. The upshot is that there does exist an AspectJ 1.9.0.BETA-1. This has been published to repo.spring.io but that repo seems to be going slow from a refresh point of view so my gradle build isn't finding it there (I can't see it in the artifactory browser even though I know it is there) so if you use this repository (which is what repo.spring.io is the facade for):

maven { url "http://s3.amazonaws.com/maven.springframework.org/snapshot" }

and this version

1.9.0.BETA-1

you might get a bit further. Let me know how it goes.

@spring-issuemaster

This comment has been minimized.

Copy link
Collaborator Author

commented Mar 5, 2015

Juergen Hoeller commented

Andy, you're the man: The Spring Framework master build and tests pass fine with Aspectj 1.9.0.BETA-1. Thanks for that amazingly quick turnaround!

I'm trying to conditionally add it to 4.2, with our build dynamically switching to Aspectj 1.9.0.BETA-1 when executing on JDK 9... or simply always using it, even on JDK 8, but just for our use of ajc and not for the runtime libs as published in our POMs. The latter would be similar to how we dealt with AspectJ 1.8 vs 1.7 at the time when JDK 8 wasn't GA yet.

Juergen

@spring-issuemaster

This comment has been minimized.

Copy link
Collaborator Author

commented Mar 5, 2015

Juergen Hoeller commented

In yesterday's efforts, I accidentally ran the spring-oxm tests after previously executing them on JDK 8... That's what made them pass, since with the generated artifacts not present (i.e. a clean build), the compileTestJava phase fails on JDK 9 build 52, like previously.

Nevertheless, with AspectJ 1.9.0.BETA-1 used for the ajc task, a few spring-core tests fine-tuned and spring-oxm tests selectively deactivated, everything passes fine on JDK 9 with a clean build now. To be pushed once the AspectJ beta shows up at repo.spring.io.

Juergen

@spring-issuemaster

This comment has been minimized.

Copy link
Collaborator Author

commented Mar 5, 2015

Sam Brannen commented

Juergen Hoeller, have you been executing gradlew clean test against JDK 9?

I ask, because I just installed the following on my Mac:

java version "1.9.0-ea"
Java(TM) SE Runtime Environment (build 1.9.0-ea-b53)
Java HotSpot(TM) 64-Bit Server VM (build 1.9.0-ea-b53, mixed mode)

And when I execute gradlew clean test, I get the following compiler error:

:spring-core:compileJava
/Users/XXX/source/spring-framework/spring-core/src/main/java/org/springframework/util/CollectionUtils.java:356: error: incompatible types: List<CAP#1> cannot be converted to List<V>
			List<V> values = Collections.unmodifiableList(entry.getValue());
			                                             ^
  where V,K are type-variables:
    V extends Object declared in method <K,V>unmodifiableMultiValueMap(MultiValueMap<? extends K,? extends V>)
    K extends Object declared in method <K,V>unmodifiableMultiValueMap(MultiValueMap<? extends K,? extends V>)
  where CAP#1 is a fresh type-variable:
    CAP#1 extends V from capture of ? extends V
1 error
:spring-core:compileJava FAILED
@spring-issuemaster

This comment has been minimized.

Copy link
Collaborator Author

commented Mar 5, 2015

Sam Brannen commented

Made further changes to Gradle build to support JDK 9 in GitHub commit 7bc44a9:

Set Gradle Daemon JVM args compatible with JDK 9

This commit adds the following to gradle.properties in order to execute
the Gradle daemon on JDK 9, since Gradle's DaemonParameters
automatically sets the MaxPermSize JVM argument, which is no longer
supported on JDK 9.

org.gradle.jvmargs=-XX:MaxMetaspaceSize=1024m -Xmx1024m

@spring-issuemaster

This comment has been minimized.

Copy link
Collaborator Author

commented Mar 5, 2015

Sam Brannen commented

See also the "DaemonParameters does not support JDK 9 by default" bug in the Gradle Forum.

@spring-issuemaster

This comment has been minimized.

Copy link
Collaborator Author

commented Mar 6, 2015

Juergen Hoeller commented

Hmm. a clean test run does pass for me against JDK 9 build 52... including spring-core. Maybe it's a regression in JDK 9 build 53?

BTW, after those Gradle JVM settings changed, the Gradle wrapper seems to fork a new JVM for every run now. Can we avoid that somehow, even with the new settings in place?

Juergen

@spring-issuemaster

This comment has been minimized.

Copy link
Collaborator Author

commented Mar 6, 2015

Juergen Hoeller commented

Andy Clement, I can still only grab AspectJ 1.9.0.BETA-1 from the Amazon URL, not from anywhere at repo.spring.io. It does show up at http://repo.spring.io/snapshot/org/aspectj/aspectjtools/ but I can't seem to grab it from there... And shouldn't it be in the /milestone category instead of /snapshot?

Juergen

@spring-issuemaster

This comment has been minimized.

Copy link
Collaborator Author

commented Mar 6, 2015

Juergen Hoeller commented

Sam Brannen, it's indeed a regression with Java 9 build 53! Fails for me on that very same spot as well now...

So much for tracking a moving target, even within the same week :-)

Juergen

@spring-issuemaster

This comment has been minimized.

Copy link
Collaborator Author

commented Mar 6, 2015

Juergen Hoeller commented

Collections.unmodifiableMultiValueMap explicitly casts from List<? extends V> to List<V> now, which apparently is necessary as of Java 9 build 53.

@spring-issuemaster

This comment has been minimized.

Copy link
Collaborator Author

commented Mar 6, 2015

Sam Brannen commented

The aforementioned bug regarding DaemonParameters is now being tracked as Gradle issue GRADLE-3256.

@spring-issuemaster

This comment has been minimized.

Copy link
Collaborator Author

commented Mar 6, 2015

Sam Brannen commented

Thanks for fixing that regression, Juergen!

And yeah.... so much for tracking a moving target. Guess we should probably change the resolution of this task to "In Progress". ;)

@spring-issuemaster

This comment has been minimized.

Copy link
Collaborator Author

commented Mar 6, 2015

Sam Brannen commented

Juergen Hoeller,

BTW, after those Gradle JVM settings changed, the Gradle wrapper seems to fork a new JVM for every run now. Can we avoid that somehow, even with the new settings in place?

Are you certain it's forking a new process?

I believe the behavior you are experiencing is due to the fact that the daemon process is now run headless with JDK 9. At least on my Mac, the daemon still seems to be doing its job.

For example, in the spring-test folder I observe the following behavior:

[spring-test (master)]$ ../gradlew --stop
Stopping daemon(s).
Gradle daemon stopped.

[spring-test (master)]$ ../gradlew -a clean compileJava
BUILD SUCCESSFUL
Total time: 18.906 secs

[spring-test (master)]$ ../gradlew -a clean compileJava
BUILD SUCCESSFUL
Total time: 7.691 secs

spring-test (master)]$ ../gradlew -a clean compileJava
BUILD SUCCESSFUL
Total time: 6.793 secs

[spring-test (master)]$ gradle --stop
Stopping daemon(s).
Gradle daemon stopped.

[spring-test (master)]$ ../gradlew -a clean compileJava
BUILD SUCCESSFUL
Total time: 17.544 secs

So clean compileJava takes about 18 seconds for me if the daemon is not currently running and about 7 seconds if the daemon is running.

I also compared the number of active Java processes on my machine when following the above steps with JDK 8, and I see the org.gradle.launcher.daemon.bootstrap.GradleDaemon 2.2.1 process running in both scenarios.

Can you please verify on your machine using the example above?

Thanks,

Sam

@spring-issuemaster

This comment has been minimized.

Copy link
Collaborator Author

commented Mar 6, 2015

Juergen Hoeller commented

The daemon works fine. However, I mostly use Gradle without the daemon, and in that case, a message is being logged that it is forking a new JVM to apply custom settings now...

"To honour the JVM settings for this build a new JVM will be forked. Please consider using the daemon: http://gradle.org/docs/2.2.1/userguide/gradle_daemon.html."

Juergen

@spring-issuemaster

This comment has been minimized.

Copy link
Collaborator Author

commented Mar 6, 2015

Sam Brannen commented

How are you invoking Gradle when you get that warning?

@spring-issuemaster

This comment has been minimized.

Copy link
Collaborator Author

commented Mar 6, 2015

Juergen Hoeller commented

Well, I get that warning using plain gradlew now. Specifying --daemon makes it go away (that's what I meant above), but it is a regression from previous non-daemon behavior where it was simply reusing the bootstrap JVM. Not a showstopper, of course; I just wonder whether that's avoidable.

@spring-issuemaster

This comment has been minimized.

Copy link
Collaborator Author

commented Mar 6, 2015

Sam Brannen commented

If you kill the daemon with gradlew \-\-stop and add org.gradle.daemon=true to gradle.properties, does that solve your problem (i.e., keep you from having to specify \-\-daemon)?

@spring-issuemaster

This comment has been minimized.

Copy link
Collaborator Author

commented Mar 6, 2015

Andy Clement commented

You are right Juergen, it probably should be in /milestone, i was just in a hurry. I republished it this morning into there and I can immediately consume it from repo.spring.io/milestone. Wonder if that snapshot area doesn't like a non standard naming scheme (it obviously wasn't called BUILD-SNAPSHOT).

@spring-issuemaster

This comment has been minimized.

Copy link
Collaborator Author

commented Mar 6, 2015

Juergen Hoeller commented

It does, but it's not the --daemon that I mind. I just wonder why it's not reusing the bootstrap JVM anymore if there is no daemon. I suppose we have a difference in JVM settings that makes it fork a different JVM now? Can we apply them to the bootstrap JVM as well, allowing it to be reused again? That would be preferable to enforcing the daemon IMO.

@spring-issuemaster

This comment has been minimized.

Copy link
Collaborator Author

commented Mar 6, 2015

Sam Brannen commented

Juergen Hoeller, do you by any chance have GRADLE_OPTS or JAVA_OPTS set as environment variables, and if so what are the values?

@spring-issuemaster

This comment has been minimized.

Copy link
Collaborator Author

commented Mar 6, 2015

Juergen Hoeller commented

Sam Brannen, neither of the two. However, in our gradlew.bat in the repo, there is a set GRADLE_OPTS entry which I assume will be picked up here: Changing that entry to -XX:MaxMetaspaceSize=1024m -Xmx1024m, like we have it in gradle.properties now, doesn't seem to help though. Maybe Gradle simply forks a new JVM based on the plain presence of a org.gradle.jvmargs entry...

@spring-issuemaster

This comment has been minimized.

Copy link
Collaborator Author

commented Mar 6, 2015

Juergen Hoeller commented

Andy Clement, cool, works for me as well now! I'm going to commit our revised Gradle build against repo.spring.io/milestone in a few min, using AspectJ 1.9.0.BETA-1 for ajc then but preserving AspectJ 1.8.5 for our runtime declarations (which is the only part exposed in our POMs). Together with a few other tweaks, that allows our build to run on either JDK 8 or 9!

@spring-issuemaster

This comment has been minimized.

Copy link
Collaborator Author

commented Mar 6, 2015

Stéphane Nicoll commented

Sam Brannen for the record I am affected by this as well.

@spring-issuemaster

This comment has been minimized.

Copy link
Collaborator Author

commented Mar 6, 2015

Sam Brannen commented

Stéphane Nicoll,

What OS are you using?

And how are you invoking Gradle?

@spring-issuemaster

This comment has been minimized.

Copy link
Collaborator Author

commented Mar 6, 2015

Sam Brannen commented

Juergen Hoeller,

However, in our gradlew.bat in the repo, there is a set GRADLE_OPTS entry which I assume will be picked up here: Changing that entry to -XX:MaxMetaspaceSize=1024m -Xmx1024m, like we have it in gradle.properties now, doesn't seem to help though.

Yes, that should be picked up if you're executing gradlew.bat on Windows.

Note, however, that those JVM settings are overwritten whenever you execute gradlew wrapper since we configure this in build.gradle (search for gradleBatOpts). So make sure you update gradleBatOpts if you make any changes.

Maybe Gradle simply forks a new JVM based on the plain presence of a org.gradle.jvmargs entry...

The "20.1.1. Forked Java processes" section of the Gradle reference manual states the following:

Many settings (like the Java version and maximum heap size) can only be specified when launching a new JVM for the build process. This means that Gradle must launch a separate JVM process to execute the build after parsing the various gradle.properties files. When running with the daemon, a JVM with the correct parameters is started once and reused for each daemon build execution. When Gradle is executed without the daemon, then a new JVM must be launched for every build execution, unless the JVM launched by the Gradle start script happens to have the same parameters.

This launching of an extra JVM on every build execution is quite expensive, which is why if you are setting either org.gradle.java.home or org.gradle.jvmargs we highly recommend that you use the Gradle Daemon. See Chapter 19, The Gradle Daemon for more details.

The explanation in the last paragraph is why I recommended adding org.gradle.daemon=true to gradle.properties since we are now setting org.gradle.jvmargs.

Thoughts?

@spring-issuemaster

This comment has been minimized.

Copy link
Collaborator Author

commented Mar 6, 2015

Stéphane Nicoll commented

Sam, OSX and I am using the gradle wrapper.

@spring-issuemaster

This comment has been minimized.

Copy link
Collaborator Author

commented Mar 7, 2015

Sam Brannen commented

Update: I have reverted the troublesome changes as described in GitHub commit b1fdb4a:

Remove Gradle Daemon JVM args compatible with JDK 9

This commit reverts the changes made in 7bc44a9 so that developers who
do not use the Gradle daemon are not adversely affected by the explicit
JVM args that were introduced in that commit.

Developers who wish to run the build against JDK 9 with the Gradle
daemon can add the following to the gradle.properties file in their
'gradle user home' directory (e.g., ~/.gradle/gradle.properties):

org.gradle.daemon=true
org.gradle.jvmargs=-XX:MaxMetaspaceSize=1024m -Xmx1024m

See also: https://issues.gradle.org/browse/GRADLE-3256

@spring-issuemaster

This comment has been minimized.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.