Building continuous delivery pipelines and similarly complex tasks in Jenkins using freestyle projects and existing plugins is very awkward. You need to mix Parameterized Trigger, Copy Artifact, Promoted Builds, Conditional Build Step, and more just to express what should be a simple script. This project attempts to make it possible to directly write that script, what people often call a workflow (sometimes abbreviated flow), while integrating with Jenkins features like slaves and publishers.
Not all implemented yet, of course; see below for status.
Your whole workflow is a single Groovy script using an embedded DSL, possibly quite short and legible; there is no need to jump between multiple job configuration screens to see what is going on. Conditions, loops, variables, parallel tasks, and so on are defined using regular language constructs. At any point you can insert a shell script to do “real work” (compilation, etc.).
Standard DSL functions let you run external processes, grab slave nodes, allocate workspaces, build “legacy” (freestyle) jobs, and so on.
If Jenkins is restarted (intentionally, or because of a crash) while your workflow is running, when it comes back up, execution is resumed where it left off. This applies to external processes (shell scripts) so long as the slave can be reattached, and losing the slave connection temporarily is not fatal either.
Flows can pause in the middle and wait for a human to approve something, or enter some information. Executors are not consumed while the flow is waiting.
See here for details on using version control from a workflow.
node('linux') { // grab a slave and allocate a workspace
git url: '…' // clone/checkout
sh 'mvn verify' // run your build
}
If you do not want to build from sources, some early releases (as of this writing 0.1-beta-1) are available on the Jenkins experimental update center.
You need to be running a recent Jenkins weekly release, currently 1.568 or newer.
For OSS Jenkins users, follow these instructions and install Workflow: Aggregator (its dependencies will be pulled in automatically).
Jenkins Enterprise by CloudBees users can click Enable access under Access to experimental plugin releases in the main Jenkins configuration screen, and then install both Workflow: Aggregator and the proprietary add-on Workflow: Groovy CPS Execution Checkpoint.
There is a demo using Docker available if you want to try a complete setup quickly.
- Changelog
- Source repository
- JIRA
- Trello board tracking active and proposed tasks.
- CI job with validated merge support.
- Video tutorial on implementing a Step API
While the implementation is divided into a number of plugins, for ease of prototyping they are all kept in one repository using snapshot dependencies.
step-apidefines a generic build step interface (not specific to flows) that many plugins could in the future depend on.basic-stepsadd some generic step implementations.apidefines the essential aspects of flows and their executions. In particular, the engine running a flow is extensible and so could in the future support visual orchestration languages.supportadds general implementations of some internals needed by flows, such as storing state.jobprovides the actual job type and top-level UI for defining and running flows.durable-task-stepuses thedurable-taskplugin to define a shell script step that can survive restarts.scm-stepadds SCM-related steps. There is more documentation there.cpsis the flow engine implementation based on the Groovy language, and supporting long-running flows using a continuation passing style transformation of the script.stmis a simple engine implementation using a state transition machine, less intended for end users than as a reference for how engines can work.aggregatoris a placeholder plugin allowing you tomvn hpi:runand see everything working together.