Migrate to Java 21 #5421

leonardehrenfried · 2023-10-12T07:21:58Z

Summary

This migrates the build and container image to use Java 21, which is the latest LTS version with a lot of nice features.

Heads up, @miles-grant-ibigroup .

codecov · 2023-10-12T07:30:58Z

Codecov Report

All modified and coverable lines are covered by tests ✅

Comparison is base (a9b0151) 66.80% compared to head (ef8e807) 66.80%.
Report is 3 commits behind head on dev-2.x.

Additional details and impacted files

@@              Coverage Diff              @@
##             dev-2.x    #5421      +/-   ##
=============================================
- Coverage      66.80%   66.80%   -0.01%     
+ Complexity     15557    15556       -1     
=============================================
  Files           1806     1806              
  Lines          69821    69821              
  Branches        7357     7357              
=============================================
- Hits           46646    46642       -4     
- Misses         20718    20721       +3     
- Partials        2457     2458       +1

see 2 files with indirect coverage changes

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

vpaturet · 2023-11-06T08:17:14Z

There is a transitive dependency on Netty, which has a compatibility issue with JDK 21 (https://netty.io/news/2023/09/21/4-1-99-Final.html)
The dependency comes from:

azure service bus --> patch available
asynchttpclient --> no patch available as of today, netty should be updated explicitly in the OTP pom file.

@leonardehrenfried we should probably review the dependencies that are currently ignored in the Renovate configuration, some might need to be patched for JDK 21.

leonardehrenfried · 2023-11-06T09:15:34Z

It seems that it's a problem on Mac OS only. Did you hit that bug on your machine?

leonardehrenfried · 2023-11-06T09:35:21Z

Also, it seems that this is no longer needed as netex-java-model uses a newer version:

OpenTripPlanner/pom.xml

Lines 559 to 569 in 322a457

    
                       <!-- 
        
                           The jaxb-api provides annotations like XmlElement indicating how classes should be (de)serialized. 
        
                           The jaxb-runtime pulls in the jaxb-api of the same version transitively. But we specify both api and 
        
                           runtime dependencies anyway here at the top level to prevent transitive dependencies from pulling in 
        
                           mismatched versions. This is overriding an older version of jaxb-api pulled in by the netex-java-model. 
        
                       --> 
        
                       <dependency> 
        
                           <groupId>jakarta.xml.bind</groupId> 
        
                           <artifactId>jakarta.xml.bind-api</artifactId> 
        
                           <version>3.0.1</version> 
        
                       </dependency>

leonardehrenfried · 2023-11-06T09:58:54Z

I have updated all Azure dependencies so that the netty dependencies look like this now:

❯ mvn dependency:list|grep netty
[INFO]    io.netty:netty-tcnative-boringssl-static:jar:2.0.62.Final:compile
[INFO]    io.netty:netty-resolver-dns-native-macos:jar:osx-x86_64:4.1.100.Final:compile
[INFO]    io.netty:netty-tcnative-boringssl-static:jar:linux-aarch_64:2.0.62.Final:compile
[INFO]    io.netty:netty-codec-http2:jar:4.1.100.Final:compile
[INFO]    io.netty:netty-codec-socks:jar:4.1.60.Final:compile
[INFO]    io.netty:netty-transport:jar:4.1.60.Final:compile
[INFO]    io.netty:netty-handler-proxy:jar:4.1.60.Final:compile
[INFO]    io.netty:netty-transport-native-unix-common:jar:4.1.60.Final:compile
[INFO]    io.projectreactor.netty:reactor-netty-http:jar:1.0.38:compile
[INFO]    io.netty:netty-tcnative-boringssl-static:jar:osx-x86_64:2.0.62.Final:compile
[INFO]    com.typesafe.netty:netty-reactive-streams:jar:2.0.4:compile
[INFO]    io.netty:netty-handler:jar:4.1.60.Final:compile
[INFO]    io.netty:netty-codec:jar:4.1.60.Final:compile
[INFO]    com.azure:azure-core-http-netty:jar:1.13.9:compile
[INFO]    io.netty:netty-tcnative-classes:jar:2.0.62.Final:compile
[INFO]    io.netty:netty-codec-dns:jar:4.1.100.Final:compile
[INFO]    io.netty:netty-codec-http:jar:4.1.60.Final:compile
[INFO]    io.netty:netty-resolver-dns-classes-macos:jar:4.1.100.Final:compile
[INFO]    io.netty:netty-tcnative-boringssl-static:jar:windows-x86_64:2.0.62.Final:compile
[INFO]    io.netty:netty-common:jar:4.1.60.Final:compile
[INFO]    io.netty:netty-resolver-dns:jar:4.1.100.Final:compile
[INFO]    io.netty:netty-tcnative-boringssl-static:jar:osx-aarch_64:2.0.62.Final:compile
[INFO]    io.netty:netty-tcnative-boringssl-static:jar:linux-x86_64:2.0.62.Final:compile
[INFO]    io.netty:netty-buffer:jar:4.1.60.Final:compile
[INFO]    io.netty:netty-transport-native-epoll:jar:linux-x86_64:4.1.60.Final:compile
[INFO]    io.projectreactor.netty:reactor-netty-core:jar:1.0.38:compile
[INFO]    io.grpc:grpc-netty-shaded:jar:1.58.0:compile
[INFO]    org.asynchttpclient:async-http-client-netty-utils:jar:2.12.3:compile
[INFO]    io.netty:netty-transport-native-kqueue:jar:osx-x86_64:4.1.60.Final:compile
[INFO]    io.netty:netty-resolver:jar:4.1.60.Final:compile

It's a mixture of 4.1.60 and 4.1.100.

AHC hasn't had a proper release in a while but there are beta versions of AHC 3 available. Do you know what version 3 is about?

leonardehrenfried · 2023-11-06T10:25:39Z

AHC 3.0.0.Beta3 seems compatible with version 2 and uses the newest version of Netty: I build graph and ran some updaters and everything worked fine in Java 21.

vpaturet · 2023-11-06T10:52:56Z

Did you try to pin the Netty version to 4.1.100? This may be cleaner than using a beta version.

vpaturet · 2023-11-06T10:55:36Z

It seems that it's a problem on Mac OS only. Did you hit that bug on your machine?

Nope, this works fine on my Linux machine and in our cloud deployment.

leonardehrenfried · 2023-11-06T10:57:10Z

Did you try to pin the Netty version to 4.1.100? This may be cleaner than using a beta version.

I tried the Netty BOM which didn't have the desired effect. Is this what you suggested?

vpaturet · 2023-11-06T11:10:13Z

Adding:

        <dependency>
            <groupId>io.netty</groupId>
            <artifactId>netty-bom</artifactId>
            <version>4.1.100.Final</version>
            <type>pom</type>
            <scope>import</scope>
        </dependency>

under <dependencyManagement> will pin the dependency.
Do you mean the updater failed with this config?

vpaturet · 2023-11-06T11:25:49Z

Also, it seems that this is no longer needed as netex-java-model uses a newer version:

OpenTripPlanner/pom.xml

Lines 559 to 569 in 322a457



<dependency>

<groupId>jakarta.xml.bind</groupId>

<artifactId>jakarta.xml.bind-api</artifactId>

<version>3.0.1</version>

</dependency>

It is recommended to explicitly set all direct dependencies (i.e. the one that are visible in java imports) in the pom.
See https://maven.apache.org/guides/introduction/introduction-to-dependency-mechanism.html

Although transitive dependencies can implicitly include desired dependencies, it is a good practice to explicitly specify the dependencies your source code uses directly. This best practice proves its value especially when the dependencies of your project change their dependencies.

However in this particular case this is an API, not an implementation, so incompatibilities would be detected at compile time.
This can be removed when we update NeTEx Java model, I'll update the corresponding PR.

leonardehrenfried · 2023-11-06T12:46:31Z

Adding:

        <dependency>
            <groupId>io.netty</groupId>
            <artifactId>netty-bom</artifactId>
            <version>4.1.100.Final</version>
            <type>pom</type>
            <scope>import</scope>
        </dependency>

under <dependencyManagement> will pin the dependency. Do you mean the updater failed with this config?

Thanks for that. I forgot about the scope definition and now it does what we expect it to do:

❯ mvn dependency:list|grep netty
[INFO]    io.netty:netty-resolver-dns-native-macos:jar:osx-x86_64:4.1.100.Final:compile
[INFO]    io.netty:netty-tcnative-boringssl-static:jar:windows-x86_64:2.0.61.Final:compile
[INFO]    io.netty:netty-tcnative-boringssl-static:jar:osx-aarch_64:2.0.61.Final:compile
[INFO]    io.netty:netty-transport-classes-epoll:jar:4.1.100.Final:compile
[INFO]    io.netty:netty-codec-http2:jar:4.1.100.Final:compile
[INFO]    io.projectreactor.netty:reactor-netty-http:jar:1.0.38:compile
[INFO]    com.typesafe.netty:netty-reactive-streams:jar:2.0.4:compile
[INFO]    io.netty:netty-transport:jar:4.1.100.Final:compile
[INFO]    io.netty:netty-tcnative-boringssl-static:jar:osx-x86_64:2.0.61.Final:compile
[INFO]    io.netty:netty-tcnative-boringssl-static:jar:linux-aarch_64:2.0.61.Final:compile
[INFO]    com.azure:azure-core-http-netty:jar:1.13.9:compile
[INFO]    io.netty:netty-codec:jar:4.1.100.Final:compile
[INFO]    io.netty:netty-codec-dns:jar:4.1.100.Final:compile
[INFO]    io.netty:netty-transport-native-kqueue:jar:osx-x86_64:4.1.100.Final:compile
[INFO]    io.netty:netty-resolver:jar:4.1.100.Final:compile
[INFO]    io.netty:netty-transport-classes-kqueue:jar:4.1.100.Final:compile
[INFO]    io.netty:netty-buffer:jar:4.1.100.Final:compile
[INFO]    io.netty:netty-resolver-dns-classes-macos:jar:4.1.100.Final:compile
[INFO]    io.netty:netty-common:jar:4.1.100.Final:compile
[INFO]    io.netty:netty-tcnative-boringssl-static:jar:2.0.61.Final:compile
[INFO]    io.netty:netty-transport-native-epoll:jar:linux-x86_64:4.1.100.Final:compile
[INFO]    io.netty:netty-codec-socks:jar:4.1.100.Final:compile
[INFO]    io.netty:netty-handler-proxy:jar:4.1.100.Final:compile
[INFO]    io.netty:netty-resolver-dns:jar:4.1.100.Final:compile
[INFO]    io.netty:netty-transport-native-unix-common:jar:4.1.100.Final:compile
[INFO]    io.netty:netty-codec-http:jar:4.1.100.Final:compile
[INFO]    io.projectreactor.netty:reactor-netty-core:jar:1.0.38:compile
[INFO]    io.netty:netty-tcnative-classes:jar:2.0.61.Final:compile
[INFO]    io.netty:netty-handler:jar:4.1.100.Final:compile
[INFO]    io.grpc:grpc-netty-shaded:jar:1.58.0:compile
[INFO]    org.asynchttpclient:async-http-client-netty-utils:jar:2.12.3:compile
[INFO]    io.netty:netty-tcnative-boringssl-static:jar:linux-x86_64:2.0.61.Final:compile

vpaturet · 2023-11-06T14:15:29Z

I simplified Jakarta dependency management in 650193c

leonardehrenfried added the improvement label Oct 12, 2023

leonardehrenfried requested a review from a team as a code owner October 12, 2023 07:21

leonardehrenfried force-pushed the java-21 branch from 15ca789 to d272160 Compare October 12, 2023 07:31

leonardehrenfried requested review from vpaturet and jtorin October 12, 2023 14:18

t2gran added this to the 2.5 (next release) milestone Oct 12, 2023

jtorin previously approved these changes Oct 20, 2023

View reviewed changes

leonardehrenfried marked this pull request as draft October 24, 2023 08:19

leonardehrenfried marked this pull request as ready for review November 2, 2023 18:32

Migrate to Java 21

322a457

leonardehrenfried force-pushed the java-21 branch from d272160 to 322a457 Compare November 6, 2023 09:16

Use the newest version of the Azure dependencies

23ed4ca

leonardehrenfried dismissed jtorin’s stale review via 23ed4ca November 6, 2023 09:36

Add BOM for netty

ef8e807

leonardehrenfried requested a review from jtorin November 7, 2023 08:32

jtorin approved these changes Nov 7, 2023

View reviewed changes

vpaturet approved these changes Nov 9, 2023

View reviewed changes

leonardehrenfried merged commit 36fa295 into opentripplanner:dev-2.x Nov 9, 2023
5 checks passed

t2gran pushed a commit that referenced this pull request Nov 9, 2023

Add changelog entry for #5421 [ci skip]

1bd4aaf

leonardehrenfried deleted the java-21 branch November 9, 2023 09:36

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Migrate to Java 21 #5421

Migrate to Java 21 #5421

leonardehrenfried commented Oct 12, 2023

codecov bot commented Oct 12, 2023 •

edited

vpaturet commented Nov 6, 2023 •

edited

leonardehrenfried commented Nov 6, 2023

leonardehrenfried commented Nov 6, 2023

leonardehrenfried commented Nov 6, 2023

leonardehrenfried commented Nov 6, 2023

vpaturet commented Nov 6, 2023

vpaturet commented Nov 6, 2023

leonardehrenfried commented Nov 6, 2023

vpaturet commented Nov 6, 2023

vpaturet commented Nov 6, 2023

leonardehrenfried commented Nov 6, 2023

vpaturet commented Nov 6, 2023

Migrate to Java 21 #5421

Migrate to Java 21 #5421

Conversation

leonardehrenfried commented Oct 12, 2023

Summary

codecov bot commented Oct 12, 2023 • edited

Codecov Report

vpaturet commented Nov 6, 2023 • edited

leonardehrenfried commented Nov 6, 2023

leonardehrenfried commented Nov 6, 2023

leonardehrenfried commented Nov 6, 2023

leonardehrenfried commented Nov 6, 2023

vpaturet commented Nov 6, 2023

vpaturet commented Nov 6, 2023

leonardehrenfried commented Nov 6, 2023

vpaturet commented Nov 6, 2023

vpaturet commented Nov 6, 2023

leonardehrenfried commented Nov 6, 2023

vpaturet commented Nov 6, 2023

codecov bot commented Oct 12, 2023 •

edited

vpaturet commented Nov 6, 2023 •

edited