Switch branches/tags
Nothing to show
Find file Copy path
372 lines (287 sloc) 15.6 KB

JEP-300: Jenkins Evergreen

Table 1. Metadata




Jenkins Evergreen Overview


R. Tyler Croy


Accepted 👌






Jenkins Evergreen was originally referred to as "Jenkins Essentials" and has since been renamed, see this mailing list thread.


We often describe Jenkins using the metaphor of lego bricks which can be pieced together by the end-user in order to solve their problems. Borrowing that metaphor, Jenkins Evergreen provides the end-user with a pre-assembled collection of legos that can be immediately used to implement CI [1] and CD [2] workloads. At the same time, this focus on end-users success in these well-defined scenarios will help the Jenkins project develop new features, and fix bugs, more rapidly than before.


Jenkins Evergreen covers an ambitious scope which is more thoroughly detailed in subsequent JEP [3] documents in this series. This specification describes the problem, and overall proposed solution, as it relates to the Jenkins project and existing structures.

Target Audience

Jenkins Evergreen is targeted at existing and new Jenkins users who are not necessarily interested in becoming Jenkins experts, but rather wish to use Jenkins as a means to an end. That end being: building, testing, and delivering their projects, with as little overhead as possible.

Jenkins Evergreen is not going to be an ideal solution for everybody who is interested in Jenkins. It is expected that some number of users will continue to "build their own" Jenkins environment, cobbling together a core .war and the specific set of plugins which meet their desired goals.


The pillars to Jenkins Evergreen will be described in more detail in future documents but this section provides an overview of the four core aspects of Jenkins Evergreen which, taken together, will address the problems facing the Target Audience.

Automatically Updated Distribution

In order to provide an easier-to-use and easier-to-manage Jenkins environment, Jenkins Evergreen will be distributed as an automatically self-updating distribution, containing Jenkins core and a version-locked set of plugins considered "essential." Rather than attempting to mirror the existing Weekly and LTS release lines for core, plus some plugin version matrix, Jenkins Evergreen will update in a manner similar to Google Chrome. [4]

For Jenkins end-users, this automatically updating distribution will mean that Jenkins Evergreen will require significantly less overhead to manage, receiving improvements and bug fixes without any user involvement. From the user perspective, their Jenkins will appear to automatically improve over time.

For Jenkins project developers, this approach introduces multiple improvements:

  1. Greatly reduced time between core and "foundational" plugin changes landing, and being adoptable by downstream components.

  2. Greatly reduced version matrix for testing core/plugin changes against one another.

  3. Small-batch changes, automatically distributed to Jenkins instances which are inherently more Connected, can can automatically report success/error telemetry, providing rapid feedback for changes. This feedback mechanism, rather than relying on end-users to manually upgrade, notice potential issues, and file tickets in JIRA, when coupled with an automatically updating distribution allows for rapodily backing out changes which may cause problems for users.

The components responsible for this automatically self-updating distribution will be referred to as the "Evergreen" distribution system, and will be described further in subsequent JEP documents.

Automatic Sane Defaults

Providing a core along with "essential" plugins is a good first step to helping Jenkins users successfully automate their CI/CD workloads, but requires additional "smoothing" over some of the numerous options and configurations plugins.

This will take numerous forms, including but not limited to:

  1. Refactoring in "essential" plugins to identify shared configuration opportunities. For example, numerous plugins benefit from the use of a GitHub authentication credential, yet Blue Ocean, GitHub Branch Source, and GitHub OAuth all have slightly different views on where and how these credentials should be stored.

  2. Scripting (init.groovy.d/) environment-driven defaults, such as detecting Docker or Cloud support which can be automatically configured, etc.

For the various cloud platforms, rather than support all the environment-driven defaults in a single package, Jenkins Evergreen should provide a "flavor" specific to that environment. For example, Azure users should click a "Deploy to Azure" button which launches an Azure-specific flavor of Jenkins Evergreen, whereas Kubernetes users should deploy a [Helm]( chart.

The specifics of each flavor will be subject to different documents describing their design.


In order to provide a more seamless experience for end-users, and ensure that Jenkins project developers receive useful error and usage telemetry to drive further improvements in Jenkins, Jenkins Evergreen must necessarily be viewed as a "Connected" application. This entails some yet-to-be-specified number of server-side applications to coordinate updates, receive and process telemetry, broker 3rd-party service authentications, relay webhooks, etc.

Obvious Path to User Success

The final pillar in Jenkins Evergreen, is to ensure that Jenkins provides an obvious path for a user to configure and use it successfully. This largely entails in-application documentation, examples, and disabling legacy functionality within the application. All with the end goal of preventing users from inadvertently choosing legacy, or poorly supported, options when configuring their CI/CD workloads.


The "bucket of legos" approach is one of the numerous reasons why Jenkins has reached its current level of popularity and adoption, thereby allowing savvy users to create an solution customized to their needs. This approach is not productive or useful for end-users [5] who are weighing their options between running Jenkins, or using a CI-as-a-Service offering such as Travis CI or Circle CI.

Additionally, existing processes around "Suggested Plugins", or any others for that matter, result in many "fiefdoms" of development rather than a shared understanding of problems and solutions which should be addressed to make new, and existing, users successful with Jenkins.


For novice-to-intermediate users, the time necessary to prepare a Jenkins environment "from scratch" into something productive for common CI/CD workloads, can span from hours to days, depending on their understanding of Jenkins and it’s related technologies. The preparation of the environment can also be very error prone and require significant on-going maintenance overhead in order to continue to stay up-to-date, secure, and productive.

Additionally, many Jenkins users suffer from a paradox of choice [6] when it comes to deciding which plugins should be combined, in which ways, and how they should be configured, in order to construct a suitable CI/CD environment for their projects. While this is related to the problem which JEP-2 [7] attempted to address in the "Setup Wizard" introduced in Jenkins 2.0, Jenkins Evergreen aims to address the broader problem of providing users with a low-overhead, easily maintained, and solid distribution of common features (provided by a set of existing plugins) which will help the user focus on building, testing, and delivering their projects rather than maintaining Jenkins.

Inside the Project

Within the Jenkins project there is little organizational overlap between those common installed, arguably "required", plugins and their developers. This same gap exists, to a certain extent, between core and the extended plugin developer community. This results in:

  • Configuration redundancies, such as the Configure System page entries for "GitHub" (provided by GitHub) and "GitHub Enterprise Servers" (provided by GitHub Branch Source).

  • User-success isolation, that is to say, dependent plugins which are many layers away how users are interacting with Jenkins, may never get useful feedback on how their code is being exercised. This can lead to costly rewrites and refactoring, such as that done with SCM API v2. [8]

  • Slow adoption of new, or planned, core changes in plugins, causing significant amounts of testing workloads in order to successfully make significant improvements to Jenkins core. [9]


Mission and Priority Specification

This document does not propose a specific design or implementation, as other JEPs might. This is intentional. Later JEPs in this series will will address design and implementation. This document specifies the concept of "Jenkins Evergreen" and the mission and priorities of the Jenkins Evergreen project. Limiting the scope of this JEP to high-level goals will let us build a solid consensus for that direction before we begin working on individual features. Having a unified plan as we begin is vital to addressing the "User-success isolation" mentioned in the Motivation section.

Alternative Approaches

The first foray into this problem domain with "Suggested Plugins" [7] resulted in a tremendous increase in installations of plugins which had been deemed "useful." The "Suggested Plugins" approach has failed to make significant changes in user success.

For example, Jenkins 2.0 significantly emphasized Jenkins Pipeline however, at the time of this document’s writing the anonymous usage statistics indicate over 10 million Freestyle Jobs, compared to just over 2 million Pipelines [10] after 20 months since 2.0 has been released, and over 3 years since Pipeline was first introduced. [11] Simply providing an end-user a smaller set of choices (plugins) still requires the end-user to configure and utilize them correctly to solve their problems.

The underlying problem remains unsolved, to this end the pillars of Automatic Sane Defaults and Obvious Path to User Success aim to ensure that users are able to rapidly adopt features provided by "essential" plugins to successfully solve their problems.

By channeling this work through the Automatically Updated Distribution, Jenkins Evergreen aims to cross some of the organizational boundaries currently present in the Jenkins project, as is necessary to provide a more cohesive solution for end-users.

Plainly stated, Jenkins Evergreen means: the whole is greater than the sum of its parts.

Suggested Plugins

As mentioned earlier, the "Suggested Plugins" functionality in the "Setup Wizard" might be considered an alternate approach which has been implemented in Jenkins. Expanding on this functionality had been considered, but without the support from an Automatically Updated Distribution it’s highly unlikely that this approach would result significant benefits to developers or end-users that would help cross some of the existing organizational boundaries.

CloudBees Assurance Program

Another approach suggested by colleagues at CloudBees was to emulate the "CloudBees Assurance Program." [12] This would entail creating a form of "LTS for Plugins" which meet some quality criteria bar. This idea was swiftly rejected as the maintenance burden on a core LTS alone is significant. Additionally, an "LTS for Plugins" would not necessarily be useful for the Target Audience of users who wish to "get stuff done" but would rather provide more benefit to long-lived, slower moving, Jenkins environments which are likely already being serviced by experienced administrators or solutions providers.

Backwards Compatibility

There are no anticipated backwards compatibility issues to discuss in this document. Future enhancements to plugins under the guise of Jenkins Evergreen may make backwards incompatible changes, and thereby require new core version dependencies in their pom.xml.


There are no inherent security impacts to discuss in this document, another document related to Jenkins Evergreen will discuss a proposed process for handling Security (CERT) advisories and updates.

Infrastructure Requirements

There are no infrastructure requirements to discuss in this document, additional documents which propose new service-oriented functionality will however have infrastructure requirements documented in each proposal.


There are no testing issues related to this proposal.


  • Chrome release channels which provide the most analogous release process to what’s proposed in this document.

  • openSUSE Tumbleweed: "The Tumbleweed distribution is a pure rolling release version of openSUSE containing the latest stable versions of all software instead of relying on rigid periodic release cycles"

  • Chakra Linux’s "half-rolling release model": "aims to provide a stable core of software, and rolling applications on top of it, and it is one of the keys of the success of the distribution. "