Require Java 11 for building Pekko

[raboof]

The discussion on whether or not to require Java 11 for building Pekko is at https://lists.apache.org/thread/ro3rz6s8mvj7j9dp39oszf7rnr6hn19d

This ticket is meant to collect the tasks/opportunities that are unlocked once we make that jump:

• Remove notOnJdk8 and isJdk8 from project/JdkOptions.scala, and anything that depends on it
• Update scalatest and make sure its dependency modules are consistent with 'their' scalatest version again (Update scalatest to 3.2.18 #1116)
• Update sbt-paradox (Update sbt-paradox to 0.10.6 #1061)
• Create a safeguard to make sure we don't inadvertently update compile-time dependencies to require Java 9 or later (how?)

[raboof]

Summarizing this thread, the main remaining roadblock is the risk that we'd unintentionally accept an update of a runtime dependency that drops support for Java 8.

Any ideas on how we could avoid that risk?

[pjfanning]

It feels like that at least some of our tests should be run with pre-compiled snapshot jars. At the moment, we run full builds with multiple JDKs but when we release, we build with a specific JDK. It might be better to have our tests work this way too.

• build just one set of snapshot jars with our preferred JDK
• test these snapshot jars with multiple JDKs - including the JDK that we note to be minimum supported

[pjfanning]

We have a similar issue with the Pekko modules (eg Pekko HTTP) - we want to support having Pekko HTTP run with Pekko Core 1.0 and Pekko Core 1.1. There is a risk that Pekko HTTP jars could only work for Pekko Core 1.1 if the Pekko HTTP jars are compiled with Pekko Core 1.1. But we release jars that are built only with Pekko Core 1.0. A possibly better test pipeline would be

• build just one set of Pekko HTTP snapshot jars with our preferred JDK and Pekko Core 1.0
• test these snapshot jars with multiple JDKs and Pekko Core versions.

[mdedetrich]

Packaging tests into a jar (if thats what is being asked) is a PITA, I tried to do this at a previous company where I worked for different reasons using sbt-native-packager and it wasn't an easy/obvious/trivial thing to do

For me, in order to not blow out the matrix the best compromise is to test with multiple JDK's locally in the project (i.e. pekko-http would test for JDK 8/11/17/21 etc etc) but do cross testing (i.e. testing pekko-http with pekko core 1.0.x and 1.1.x) with JDK 8. We have to be wary about blowing out our gh actions, another Apache project got called out for it

Another thing I want to add is that the issue regarding building with JDK 1.8 has largely been solved, its abstracted away in pekko-sbt-paradox and so aside from sbt-paradox-site, we don't need to do any workarounds in any of the pekko modules. I don't want people to mis-interpret this as stonewalling this idea, just pointing out that the issue is less pressing now compared to before.

[raboof]

the issue regarding building with JDK 1.8 has largely been solved

I think it could still simplify our build quite a bit (e.g. #1194). Given how complicated our build is I think that would be valuable, even if it's just an incremental improvement and does not solve 'everything'.

Packaging tests into a jar (if thats what is being asked) is a PITA

Of course this simplification shouldn't come at the cost of severely complicating other parts of the build.

My interpretation of PJ's proposal was that we'd run the build and the unit tests with the latest JDK, and then do a separate 'integration test' against the published snapshot jars. That sounds attractive to me, as such 'full' integration tests would be useful in its own right. Perhaps they wouldn't need to be ran on PR validation at all, even, but on a nightly/weekly schedule (and span multiple projects, too).

[pjfanning]

the issue regarding building with JDK 1.8 has largely been solved

I think it could still simplify our build quite a bit (e.g. #1194). Given how complicated our build is I think that would be valuable, even if it's just an incremental improvement and does not solve 'everything'.

Packaging tests into a jar (if thats what is being asked) is a PITA

Of course this simplification shouldn't come at the cost of severely complicating other parts of the build.

My interpretation of PJ's proposal was that we'd run the build and the unit tests with the latest JDK, and then do a separate 'integration test' against the published snapshot jars. That sounds attractive to me, as such 'full' integration tests would be useful in its own right. Perhaps they wouldn't need to be ran on PR validation at all, even, but on a nightly/weekly schedule (and span multiple projects, too).

@raboof's interpretation is basically what I was getting at.

I think we can refactor some of the unit tests subprojects (eg actor-tests) so that it doesn't depend on the sbt module but depends on a jar in mavenLocal repo or possibly Apache Nexus snapshots repo.

If we do it properly, I think we can actually save some time in the build (no compiling with multiple JDKs - no building docs with multiple JDKs). We don't necessarily need to end up with extra GitHub Action total build time.

[raboof]

I think we can refactor some of the unit tests subprojects (eg actor-tests) so that it doesn't depend on the sbt module but depends on a jar in mavenLocal repo or possibly Apache Nexus snapshots repo.

hmm, those are 'unit tests' more than 'integration tests' though - for those my gut feeling is it'd be more convenient to keep them in their regular location (keeping the build simple and test/modify roundtrips quick), and only do separate 'integration tests' against the jars.

[pjfanning]

When you run actor-tests for Java 21, why do you need to use actor code built with Java 21 ? It is actually inaccurate because we have no intent right now to release jars with code built with Java 21. I see the value in running actor-tests with Java 21 just that it should use actor module classes built with Java 11 (the version we use to build releases).

[raboof]

When you run actor-tests for Java 21, why do you need to use actor code built with Java 21 ?

You don't ;). Running the unittests from the same sbt session with which you're building the code (in a single JVM) seems typical, though?

It is actually inaccurate because we have no intent right now to release jars with code built with Java 21. I see the value in running actor-tests with Java 21 just that it should use actor module classes built with Java 11 (the version we use to build releases).

I don't feel strongly on whether we should build with Java 11 or with Java 21 (though we should likely continue to allow both) - I'm mainly trying to get away from 8

[laglangyue]

As a volunteer, I strongly agree to use Java 11 and more latest java version. It can push user to update their java verison.
My problems is:
Is the integration testing of Java 8 sufficient to avoid bugs when unit testing is missing ?
Is it necessary for us to enrich our integration testing ?

[pjfanning]

Without some form of Java 8 integration testing, we can't merge this. Or maybe we get Pekko 1.1 released and then start working towards Pekko 2.0 where Java 8 is no longer supported.