Add support for running openHAB with Java 21

[wborn]

Today Java 21 will be launched. 🚀 This is a LTS release, so we should also support using it with openHAB.

We can use this issue to keep track of what still needs to be fixed or investigated (similar to #1316).

There's also a launch live stream: https://dev.java/community/java-21-launch/

TODO:

• Upgrade Netty to 4.1.99 or newer to prevent JVM crashes (see 4.1.99 release notes) (Upgrade hivemq-mqtt-client to 1.3.3, Netty to 4.1.99.Final openhab-core#3824)
• Update Java compatibility check in karaf and karaf.bat

[wborn]

It seems that official Java 21 releases are delayed (see Temurin blog).

Nevertheless I was able to use the Temurin 21-beta+34-202308082331 nightly build and got OH 4.1.0-SNAPSHOT running without issues so far after disabling the recently added Java compatibility check.

It seems the recent Karaf 4.4.4 upgrade helped a lot because OH 4.0.3 does not start with Java 21:

2023-10-02 11:59:11.900 [SEVERE] [org.apache.karaf.main.Main] - Could not launch framework
java.lang.IllegalStateException: Error initializing storage for Equinox container.
	at org.eclipse.osgi.internal.framework.EquinoxContainer.<init>(EquinoxContainer.java:110)
	at org.eclipse.osgi.launch.Equinox.<init>(Equinox.java:53)
	at org.eclipse.osgi.launch.EquinoxFactory.newFramework(EquinoxFactory.java:35)
	at org.eclipse.osgi.launch.EquinoxFactory.newFramework(EquinoxFactory.java:30)
	at org.apache.karaf.main.Main.launch(Main.java:291)
	at org.apache.karaf.main.Main.main(Main.java:183)
Caused by: java.lang.RuntimeException: Error occurred while checking the system module.
	at org.eclipse.osgi.storage.Storage.checkSystemBundle(Storage.java:467)
	at org.eclipse.osgi.storage.Storage.createStorage(Storage.java:187)
	at org.eclipse.osgi.internal.framework.EquinoxContainer.<init>(EquinoxContainer.java:108)
	... 5 more
Caused by: org.osgi.framework.BundleException: Invalid manifest header Export-Package: "org.osgi.dto;version="1.1",org.osgi.resource;version="1.0",org.osgi.resource.dto;version="1.0";uses:="org.osgi.dto",org.osgi.framework;version="1.10",org.osgi.framework.dto;version="1.10";uses:="org.osgi.dto",org.osgi.framework.hooks.bundle;version="1.1";uses:="org.osgi.framework",org.osgi.framework.hooks.resolver;version="1.0";uses:="org.osgi.framework.wiring",org.osgi.framework.hooks.service;version="1.1";uses:="org.osgi.framework",org.osgi.framework.hooks.weaving;version="1.1";uses:="org.osgi.framework.wiring",org.osgi.framework.launch;version="1.2";uses:="org.osgi.framework",org.osgi.framework.namespace;version="1.1";uses:="org.osgi.resource",org.osgi.framework.startlevel;version="1.0";uses:="org.osgi.framework",org.osgi.framework.startlevel.dto;version="1.0";uses:="org.osgi.dto",org.osgi.framework.wiring;version="1.2";uses:="org.osgi.framework,org.osgi.resource",org.osgi.framework.wiring.dto;version="1.3";uses:="org.osgi.dto,org.osgi.resource.dto",org.osgi.service.condpermadmin;version="1.1.1";uses:="org.osgi.framework,org.osgi.service.permissionadmin",org.osgi.service.packageadmin;version="1.2";uses:="org.osgi.framework",org.osgi.service.permissionadmin;version="1.2",org.osgi.service.resolver;version="1.1";uses:="org.osgi.resource",org.osgi.service.startlevel;version="1.1";uses:="org.osgi.framework",org.osgi.service.url;version="1.0",org.osgi.util.tracker;version="1.5.2";uses:="org.osgi.framework",org.apache.karaf.version;version="4.4.3",org.apache.karaf.jaas.boot.principal;uses:=javax.security.auth;version="4.4.3",org.apache.karaf.jaas.boot;uses:="javax.security.auth,javax.security.auth.callback,javax.security.auth.login,javax.security.auth.spi,org.osgi.framework";version="4.4.3",org.apache.karaf.info;version="4.4.3",, org.apache.karaf.branding, sun.misc, com.sun.jmx.remote.protocol, com.sun.jmx.remote.protocol.jmxmp, org.apache.karaf.jaas.boot.principal;uses:=javax.security.auth;version=4.4.3, org.apache.karaf.jaas.boot;uses:="javax.security.auth,javax.security.auth.callback,javax.security.auth.login,javax.security.auth.spi,org.osgi.framework";version=4.4.3, org.apache.karaf.diagnostic.core;uses:=org.osgi.framework;version=4.4.3, org.apache.karaf.diagnostic.core.common;uses:=org.apache.karaf.diagnostic.core;version=4.4.3"
	at org.eclipse.osgi.util.ManifestElement.parseHeader(ManifestElement.java:355)
	at org.eclipse.osgi.container.builders.OSGiManifestBuilderFactory.createBuilder(OSGiManifestBuilderFactory.java:109)
	at org.eclipse.osgi.storage.Storage.getBuilder(Storage.java:835)
	at org.eclipse.osgi.storage.Storage.checkSystemBundle(Storage.java:415)
	... 7 more

We might also want to upgrade Netty to 4.1.99.Final or newer because it can crash the JVM, see:

https://netty.io/news/2023/09/21/4-1-99-Final.html

[openhab-bot]

This issue has been mentioned on openHAB Community. There might be relevant details there:

https://community.openhab.org/t/openhab-4-1-release-discussion/152252/44

[jlaur]

@wborn - it seems like the effort to upgrade from Java 17 to 21 is not that big? But I guess this would require a major new version of openHAB, i.e. 5.0 rather than 4.2?

[J-N-K]

"Running with" and "Requiring" are two different things. "Requiring" would indeed be something for a major release, "Running with" is backward compatible and therefore nice to have even for a minor version.

[holgerfriedrich]

To allow compiling and running with Java21 as well, it looks like we need to upgrade a few more things,

• ecj to >= 3.36.0 (available)
• xtend and related libraries for code generation (support libraries seem to break compiling on Java21, and are not released in a version compatible with Java21 yet). This seems a blocker right now.
• SAT plugin needs to be updated as 0.15.0 does not support spotbugs analysis for Java 21 files

I have also seen warnings about zip validation during compilation which could not be suppressed via -Djdk.util.zip.disableZip64ExtraFieldValidation=true in JAVA_TOOL_OPTIONS.

[holgerfriedrich]

@wborn I finally managed to create a distro zip compiled with Java 21 and running on Java 21 🥳
xtend is still a blocker for Java 21, so I had to fix 12 directories (many of the core.model.x folders) to Java 17.

I will open the necessary PRs the next days. A few prerequisites are not yet merged, i.e. the SAT plugin and Karaf 4.4.5 patches.
The remaining stuff is basically ecj and a few maven plugins.

[holgerfriedrich]

@kaikreuzer has released SAT 0.16.0 - thanks a lot!

So here comes the first PR for Java 21: openhab/openhab-core#4161.

The question to be decided is how we want to select the target version.

• For core, I propose to select automatically based on the JDK release. JDK21 generates Java21 target code (though ~50% of the class files will be Java17 or older, due to embedded dependencies or forced downgrades for xtend), JDK17 generates Java17 code. A profile can be used to manually force JDK21 down to Java17 code.
• For addons, I would stick with Java17 unless explicitly selected to avoid problems when dropping plugins into a standard OH release. go with the same approach (changed my mind after I had seem some strange issues during verify). Anyway, there is probably still a lot of work to be done....

WDYT?

And how should we deal with the CI? Shall we add J21 (matrix build)?

[holgerfriedrich]

After some discussion in openhab/openhab-core#4161 (comment), the new proposed approach is as follows:

• Allow compilation with JDK 17 and 21, both resulting in Java 17 class files to maintain compatibility with current releases.
• Add a profile j21 which can be explicitly activated to generate Java 21 class files.

All PRs have been adapted. @wborn FYI.

[holgerfriedrich]

To update the status here as well:
With the PRs mentioned above (openhab/openhab-addons#16612, openhab/openhab-addons#16615, openhab/openhab-core#4171) , everything should be fine:

See my CI builds here: 🥳
https://github.com/holgerfriedrich/openhab-core/actions
https://github.com/holgerfriedrich/openhab-addons/action
The builds include Java 17, Java 21 compiler to generate Java 17 classes, and Java 21 compiler generating Java 21 classes.

What it still open, is to update the CI to a matrix build. I have not yet opened PRs, but it is there already on private repo above.

[wborn]

Adding Java 21 to the GHA build matrix of the distro and webui repos would be nice as well.

[holgerfriedrich]

Adding Java 21 to the GHA build matrix of the distro and webui repos would be nice as well.

I will add it, thanks for testing the J21 builds.

[holgerfriedrich]

Just for reference: openhab/openhabian#1861 introduced Zulu JDK 21 installation to openHABian