Permalink
Switch branches/tags
Find file Copy path
Fetching contributors…
Cannot retrieve contributors at this time
544 lines (418 sloc) 27.2 KB

JEP-211: Java 11 support in Jenkins

Abstract

Java 11 was released on September 25, 2018. It is an LTS version with a long support timeline. This Jenkins Enhancement Proposal describes actions required to make Jenkins releases for Java 11 publicly available in Weekly and LTS releases.

Specification

The specification has been created according to the results of the Jenkins and Java 10+ hackathon.

All stories in this JEP have tickets created in JIRA. Stories are aggregated in JENKINS-40689, JENKINS-52012 and JENKINS-52284.

If the tickets are not linked in the description, they can be found there. JENKINS-52012 and JENKINS-52284 EPICs are considered as mandatory in this JEP, JENKINS-40689 - nice to have.

Goals and non-goals

Goals:

  • Jenkins WAR packages run on Java 8 and Java 11

    • Running on Java 11 may require extra options, e.g. for loading detached modules. Although we have a plan for some cases (java.xml.bind for JAXB), there may be other modules requiring updates

  • Jenkins on Java 11 is fully supported

  • No big bang

    • The most of core and plugin patches will be delivered in Weekly releases ahead of the Java 11 general availability announcements

    • We will be landing these patches using the existing code review process and test automation flows

Non-goals:

  • Jenkins is fully clean from “Illegal Reflective Access” warnings

    • There are many known places where Jenkins prints warnings about Illegal reflective access (see JENKINS-40689)

    • We acknowledge the problem, but we do not consider it as a blocker for Java 11 support. By default OpenJDK 11 just prints a warning once on the startup, there is no short-term plan to change this behavior in OpenJDK

  • All plugins are operational with Java 11

  • Building all components with OpenJDK 11

    • Some Jenkins components may be updated to support features offered in Java 9+, but there is no plan to update all tools

  • Full multi-release JARs support in Development tools

  • Cleanup of removed/deprecated features

    • During Jenkins & Java 10+ Hackathon we have discovered several stories, which may impact Jenkins behavior on future versions:

      • Signal API is deprecated (JENKINS-51995)

      • Java Web Start is removed from Java 10+ (JENKINS-50301)

  • There is no plan to remove all functionality as a part of this JEP, but it may be done and included into the release

Scope of changes

Out of the scope for this JEP:

  • Packaging in subprojects: Jenkins X, Jenkins Evergreen, Jenkinsfile Runner, etc. They will be handled in follow-up JEPs if needed.

  • Gradle build flow, as well as other

  • Windows Installers Their rework and Java removal is a separate project in Platform SIG.

  • Docker packaging for Alpine/Slim See the reasoning below

Jenkins core patches

Must-have stories are defined in JENKINS-52012. All stories in this epic need to be completed.

Library updates

  • The JENKINS-52012 epic includes a number of library updates in the core we know about: Groovy, ASM, etc.

  • Some updates may require downstream plugin updates.

    • For Example, Groovy update requires cleanup of the Metaspace leak memory in Script Security and Pipeline plugins

Core patches

  • Jenkins JNLPLauncher built-in documentation will be updated to indicate that Java Web Start feature is not available in Java 10+

  • https://github.com/jenkinsci/docker/tree/java11 is merged into master and deleted

  • Extras Executable WAR patch to permit running with Java 11 is permitted without the “--enable-future-java” flag (JENKINS-52285)

Build flow updates (JENKINS-51903)

  • Jenkinsfile is updated to run tests with JDK 11

    • It includes Unit tests, JTH and ATH smoke tests

  • It is possible to build Jenkins Core with the release profile on JDK 8

Plan for other Java 11 patches

There is a number of pending patches and tickets (e.g. detaching of JNA/JNR API, Lib Process Utils Patch, etc.), which cleanup Illegal Reflective Access attempts in Jenkins.

  • These patches will be reviewed and integrated into weekly releases once ready

  • These patches do not block the Java 11 GA release

The patches will be tracked in the JENKINS-40689 EPIC.

Jenkins Docker packaging

Jenkins Master Images (JENKINS-51985)

  • Official jenkins/jenkins image is available for Java 11. Suggested labels:

    • jdk11

    • lts-jdk11

    • VERSION-jdk11

  • Automatic build flow on Trusted CI is updated to build and release images. Weekly and LTS releases are performed automatically

  • https://github.com/jenkinsci/docker/tree/java11 experimental branch is integrated into the master branch and deleted to avoid confusion.

Jenkins Agent Images (JENKINS-52279, JENKINS-51986)

BlueOcean Docker Image (JENKINS-52280)

  • BlueOcean build for Java 11 should be made a part of the build/release flow for the component

  • It can be done after the GA release

Plugins

JENKINS-52012 tracks updates required in plugins. There are the following conditions for the GA release:

  • All plugins pass ATH with JDK 11

  • All known issues are documented in the Java 11 Compatibility Issues Wiki page (see below)

  • Plugin updates are mentioned in upgrade guidelines

Currently we know about 2 plugins which will need to be updated: “Pipeline: Support” plugin (JENKINS-52187), Monitoring Plugin (JENKINS-52092). More plugin compatibility issues may be discovered during testing.

New policy: Building Jenkins components/plugins with JDK 11

The following policy is suggested:

  • All Jenkins core components are required to retain compatibility with Java 8 so that Jenkins can run with it

  • All Jenkins plugins are required to retain Java 8 compatibility in GA releases without alpha/beta prefix

    • Plugins may require Java 11 for alpha/beta releases only, and these versions can be made accessible in the experimental update center for Java 11 (INFRA-1870)

    • In order to support Java11-only alpha/beta releases, the plugins must use Plugin POM with JENKINS-20679 patches (3.29 on Dec 06, 2018)

    • The policy may be reconsidered once JENKINS-55048 is integrated and widely adopted in Jenkins LTS

  • Individual Jenkins components may choose to require JDK 11 to build

    • It includes Jenkins core libs, plugins and potentially the core itself

    • It is up to maintainers to decide when they are ready to accept such requirement in components they maintain

    • Components that choose to require JDK 11 for building must have Jenkinsfiles running tests on Java 8 and Java 11

  • Be explicit that all Java 11 support is available in the experimental mode until Jenkins officially supports it (currently we consider Java 11 support as a preview mode - docs)

  • If a downstream component includes Java 9+ bits (e.g. lib-process-utils), downstream components (e.g. Jenkins core for lib-process-utils) must be still buildable and testable with JDK8

This policy may require patches in parent POMs:

  • 2 Parent POMs should be updated: Jenkins POM and Plugin POM

  • For known issues Maven plugin versions should be updated to versions compatible with JDK10+. Support of JDK 8 is a must (see “Building with JDK 11”)

  • If builds on Java 11 work correctly after the patches, support for JDK 11 can be released for tools

  • JENKINS-20679 - Plugin POM should be updated to support Minimum-Java-Version metadata injection

Rollout plan

The rollout procedure should be coordinated within the Platform SIG.

Timeline

  • Experimental Java 11 Support is available in Jenkins 2.127+

    • Announced in this blogpost

    • We have started integrating some patches starting from 2.127 when the “--enable-future-java” flag was introduced

    • There is no official preview announcement for weekly releases at this stage

  • JENKINS-52012 - Preview in weekly releases

  • JENKINS-51805 - GA in weekly releases

  • JENKINS-52284 - GA in LTS

    • Java 11 support will be available in LTS once the LTS baseline updates to the Weekly release

    • No special timeline set, optimistic ETA is February 2018

  • JENKINS-40689 - other non-blocker issues

The referenced EPICs contain the detailed plan for what is included into the each milestone.

Website

  • Java Support Page is updated to indicate that Java 11 is supported

  • “Running Jenkins with Java 10 and 11” blogpost is updated to refer the new guidelines

  • For Java 11 the website should be updated only after the official release of OpenJDK 11

  • There is an announcement blogpost for Java 11 support general availability in weekly

    • The blogpost will include upgrade guidelines, “make a backup” will one of the required steps there

  • There is an announcement blogpost for Java 11 support general availability in LTS

Wiki

Post-release support

After the release of Java 11 support, there may be a number of defects reported by early adopters. It may cause additional workload on plugin and core maintainers who will need assistance with triage of issues after the release.

After the weekly release availability the Java 11 Support Team (@jenkinsci/java11-support) will be responsible to provide an extra support for the issues:

  • Java 11 Support Team will periodically review open defects and triage them (e.g. once per week)

  • Java 11 Support Team may request additional information from the reporter. Finally, they are expected to communicate the triage outcome.

  • Possible triage outcomes:

    • Accepted by Java 11 Support Team. In such case one of maintainers assigns the issue to himself and delivers the fix

    • Rejected by Java 11 Support Team - functional defect in the plugin (e.g. reliance on Java version or private fields in Reflections) or lack of justification for a fix

    • Issue is closed - Not a defect, Duplicate, etc.

  • For accepted issues maintainers will prioritize and schedule the fix

    • Java 11 support is considered as a “Feature” with an obvious workaround: “Downgrade to Java 8”

    • Fixes for Java 11 will be prioritized by the team, but incompatibilities won’t be considered as Blocker issues if downgrade is possible

  • Issues rejected by Java 11 maintainers will be assigned to component leads in JIRA (if any).

The proposed support model will be in place until “Availability in LTS + 2 months”. After this period Jenkins component maintainers will be responsible for triaging and fixing issues in their components. SECURITY reports will be triaged by the Jenkins Security Team.

Java 11 Support Team is responsible to report the project status at Platform SIG meetings and, if needed, at Jenkins Governance meetings.

LTS Backporting

All backporting will be done according to the LTS Backporting Process.

There is no plan to backport changes required for Java 11 support to previous LTS baselines. Particular compatibility fixes may be backported on-demand, but major updates will not be considered due to the serious risk of regressions.

Motivation

In September 2018 we expect Java 11 to be released. It is an LTS version with a long support timeline. Over last year the Jenkins project has received many issue reports about Java 9 and then Java 10 compatibility.

  • During Jenkins World 2017 hackfest Mark Waite and Baptiste Mathus invested some time to explore Jenkins compatibility with Java 9

  • In Jenkins 2.111 we had to prevent Jenkins from starting up on unsupported Java versions toprevent false expectations from users.

  • In Jenkins 2.127 we partially re-enabled the behavior by offering a new --enable-future-java which allowed running with Java 9 and above

  • Before the Jenkins & Java 10+ Hackathon we offered preview versions of Jenkins on Java 10 and 11 (run guidelines)

  • During the hackathon we were able to get major Jenkins features running with Java 10 and 11. See the summary here

  • We made progress with regards to Java 11 during the DevOps World | Jenkins World 2018 hackathon, key issues like Pipeline metaspace leaks are addressed

Taking the success of the Jenkins and Java 10+ hackathon, there is an interest to continue working on these stories towards making Java 11 support available in Jenkins releases (weekly and then LTS).

Reasoning

“Goals and non-goals” section in the specification lists design decisions taken to ensure it can be delivered by a small team. Non-goals in the specification are defined to limit the scope of work. The main objective is to get Jenkins running with Java 11, there will be follow-up tasks to cleanup Illegal Reflective Access warnings and to adopt new features.

Support of Java 10 and 12+

Originally this JEP was targeting support of Java 10 and Java 11 early access. Java 10 is in the End of Life starting from 25.09.2018 when Java 11 was officially released. To limit the scope, Java 10 support was excluded from this JEP. We will neither be testing new patches with Java 10 nor providing Docker packages with this version. It may be possible to run Jenkins WAR files with Java 10 as it is described here, but Java 11 Support Team will not provide support for Java 10 issue reports unless the issues are reproducible on Java 11.

Java 12 is in the early access state at the time this JEP is accepted (Dec 12, 2018). With accelerated pace of Java releases, Java 13 and newer versions are also expected soon. Although we are interested to invest into adopting new Java versions, it is also out of the scope of this JEP. We will neither be testing new patches with Java 12+ nor providing Docker packages with these versions. Issue reports from early adopters may be triaged, but Java versions 12 and above won’t be officially supported as a part of this JEP.

Docker packaging for Alpine/Slim

Jenkins offers official slim and Alpine packages for Java 11. These images are based on the standard jenkins/docker image. Unfortunately there is no packaging provided for Alpine now. Hence there is a decision to not add Alpine images to the scope of this JEP.

Current JEP does NOT consider migrating to another base image. There is a well-known issue with OpenJDK distributions by Oracle, but it is not clear how it is going to impact the Docker images we use. The provider may just start building Java on its own.

Postponing of Alpine/Slim packaging also postpones the question of multi-classifier images (like alpine-jdk11).

Docker image labeling for JDK8

As a part of this JEP, we do not change labeling for JDK8. These images will be posted as is without explicit reference to Java version in these images (latest, lts, slim, alpine). Changing of the image label may be reconsidered once there is a JEP proposed for changing default labels to Java 11 or another base image. It is not in the scope for this JEP.

Experimental Update Center for Java 11

During the preview availability preparation it was discovered that the Pipeline: Support Plugin compatibility fix (JENKINS-51998) is being delayed due to the discovered compatibility-breaking changes. We discussed an option to ship and alpha/beta release to the Experimental Update Center, but this option would cause additional risk for Java 8 users who use this update center.

It was decided to create a new temporary update center so that custom Java 11 patches can be quickly deployed and used by Java 11 adopters. It will complement the timestamped snapshots and Incremental releases JEP-305 which can be used by developers.

In order to deliver the patches, the Maven HPI Plugin should inject Minimum-Java-Version manifest entry which will be later processed by the Update Center generator. There is a JENKINS-20679 for it and some patches proposed by Daniel Beck.

The update center deployment is tracked as INFRA-1870, see the specification below.

Backwards Compatibility

The following backward compatibility requirements are defined:

  • Jenkins Core and Updated plugins should fully support JDK 8

  • In the case of compatibility issues, it is possible to migrate from Java 11 to Java 8 by replacing Java in PATH or by replacing the official Docker image

    • Java 8 and Java 11 XML formats are similar

Security

Process

Only Java 11 with the latest security fixes will be supported at the moment of the public release.

Starting from the Java 11 preview availability, we adopt a partial security process for Java 11 issues. Approach:

  • Security issues for Java 11 should be always reported according to the standard vulnerability reporting process. It is mandatory, because it is required potential impact on Java 8 users

  • Security issues for Java 11 will be triaged by the Java 11 Support Team, the team should include Security team members

  • Security fixes for Java 11 Preview that do not apply to instances with a Java 8 runtime, may be delivered directly to the Core and Plugins in public, the releases will not be coordinated by the Jenkins Security Team in such case

    • Conversely, this means that any security issue targetting both Java 11 and Java 8 will obviously be processed in private and as usual by the Jenkins Security Team

  • Security advisories will not be published unless deemed necessary by the Java 11 Support Team and the Jenkins Security Officer

Starting from the Java 11 general availability, Jenkins security issues on Java 11 will be processed according to the standard Jenkins Security Process.

Security risks

  • In particular cases Java 11 may introduce new security defects (e.g. Groovy Sandbox escaping in Script Security plugin)

    • As a part of the Preview availability, we acknowledge the potential risk of security regressions when running Jenkins with Java 11. Preview versions of Java 11 are supposed to be used for evaluation only

    • In order to mitigate the Groovy update risk, Groovy will not be updated to 3.x in the incoming GA release. It means that Java 9+-alike features will not be available in Groovy DSLs within Jenkins

    • If a security issue is reported, is will be reviewed with a high priority by the Java 11 Support Team

Infrastructure Requirements

ci.jenkins.io

  • Tool Infrastructure should offer the latest version of OpenJDK 11 - INFRA-1688

  • OpenJDK 11 should be added to ci.jenkins.io (jdk11)

Jenkins Pipeline Library

  • buildPlugin(), runATH(), and runPCT() should support running tests with JDK 11 ( INFRA-1690, INFRA-1691, INFRA-1692)

  • It is possible to do fine-grain configurations in buildPlugin(), so we do not run Java 11 tests on core versions which do not support it (INFRA-1687)

  • essentialsTest() should support defining Java version matrix for testing (INFRA-1693)

DockerHub

  • CD Flow for JDK 11 images is updated to support the Master branch with Java 11 packages (INFRA-1694)

  • If necessary, DockerHub configuration should be adjusted to host official master and agent images for Java 11

Temporary Experimental Update Center for Java 11

  • A new temporary update center should be deployed (INFRA-1870)

  • The patches must not impact output of the latest and stable update centers

  • The patch must ensure that Java11-only updates do not get to the standard Experimental Update Center

  • Name of the update center: temporary-experimental-java11

  • This update center will stay available until May 2019 at least. After that the Jenkins infrastructure team may remove it after coordinating the change with the Java 11 Support Team

Testing

Java 11 support in Jenkins requires a serious amount of testing. During Jenkins and Java 10+ hackathon there was a significant amount of exploratory testing performed, and after several patches there was no major issues discovered. More tests should be performed.

In order to track the testing effort, a status Google doc has been created: here. Testers are welcome to report their results there.

Tests to be performed:

In order to perform such testing, ATH and PCT tools should be updated to support Java 11.