-
Notifications
You must be signed in to change notification settings - Fork 582
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
- Loading branch information
Showing
9 changed files
with
325 additions
and
0 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,75 @@ | ||
# pipelineRestartSteps | ||
|
||
## Description | ||
Support of restarting failed stages or steps in a pipeline is limited in Jenkins. | ||
|
||
This has been documented in the [Jenkins Jira issue JENKINS-33846](https://issues.jenkins-ci.org/browse/JENKINS-33846). | ||
|
||
For declarative pipelines there is a solution available which partially addresses this topic: | ||
https://jenkins.io/doc/book/pipeline/running-pipelines/#restart-from-a-stage. | ||
|
||
Nonetheless, still features are missing, so it can't be used in all cases. | ||
The more complex Piper pipelines which share a state via [`commonPipelineEnvironment`](commonPipelineEnvironment.md) will for example not work with the standard _restart-from-stage_. | ||
|
||
The step `pipelineRestartSteps` aims to address this gap and allows individual parts of a pipeline (e.g. a failed deployment) to be restarted. | ||
|
||
This is done in a way that the pipeline waits for user input to restart the pipeline in case of a failure. In case this user input is not provided the pipeline stops after a timeout which can be configured. | ||
|
||
## Prerequisites | ||
none | ||
|
||
|
||
## Example | ||
|
||
Usage of pipeline step: | ||
|
||
```groovy | ||
pipelineRestartSteps (script: this) { | ||
node { | ||
//your steps ... | ||
} | ||
} | ||
``` | ||
|
||
!!! caution | ||
Use `node` inside the step. If a `node` exists outside the step context, the `input` step which is triggered in the process will block a Jenkins executor. | ||
|
||
In case you cannot use `node` inside this step, please choose the parameter `timeoutInSeconds` carefully! | ||
|
||
|
||
## Parameters | ||
|
||
| parameter | mandatory | default | possible values | | ||
| ----------|-----------|---------|-----------------| | ||
|script|yes||| | ||
|sendMail|no|`true`|| | ||
|timeoutInSeconds|no|`900`|| | ||
|
||
### Details: | ||
|
||
* `script` defines the global script environment of the Jenkinsfile run. Typically `this` is passed to this parameter. This allows the function to access the [`commonPipelineEnvironment`](commonPipelineEnvironment.md) for storing the measured duration. | ||
* If `sendMail: true` the step `mailSendNotification` will be triggered in case of an error | ||
* `timeoutInSeconds` defines the time period where the job waits for input. Default is 15 minutes. Once this time is passed the job enters state FAILED. | ||
|
||
|
||
## Step configuration | ||
|
||
We recommend to define values of step parameters via [config.yml file](../configuration.md). | ||
|
||
In following sections the configuration is possible: | ||
|
||
| parameter | general | step | stage | | ||
| ----------|-----------|---------|-----------------| | ||
|script|||| | ||
|sendMail|X|X|X| | ||
|timeoutInSeconds|X|X|X| | ||
|
||
## Return value | ||
none | ||
|
||
## Side effects | ||
none | ||
|
||
## Exceptions | ||
none | ||
|
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,126 @@ | ||
#!groovy | ||
import org.jenkinsci.plugins.workflow.steps.FlowInterruptedException | ||
import org.junit.Rule | ||
import org.junit.Test | ||
import org.junit.rules.RuleChain | ||
import util.* | ||
|
||
import static org.hamcrest.CoreMatchers.containsString | ||
import static org.hamcrest.CoreMatchers.is | ||
import static org.junit.Assert.assertThat | ||
|
||
class PipelineRestartStepsTest extends BasePiperTest { | ||
|
||
private JenkinsErrorRule jer = new JenkinsErrorRule(this) | ||
private JenkinsLoggingRule jlr = new JenkinsLoggingRule(this) | ||
private JenkinsStepRule jsr = new JenkinsStepRule(this) | ||
|
||
@Rule | ||
public RuleChain chain = Rules.getCommonRules(this) | ||
.around(new JenkinsReadYamlRule(this)) | ||
.around(jer) | ||
.around(jlr) | ||
.around(jsr) | ||
|
||
@Test | ||
void testError() throws Exception { | ||
|
||
def mailBuildResult = '' | ||
helper.registerAllowedMethod('mailSendNotification', [Map.class], { m -> | ||
mailBuildResult = m.buildResult | ||
return null | ||
}) | ||
|
||
helper.registerAllowedMethod('timeout', [Map.class, Closure.class], { m, closure -> | ||
assertThat(m.time, is(1)) | ||
assertThat(m.unit, is('SECONDS')) | ||
return closure() | ||
}) | ||
|
||
def iterations = 0 | ||
helper.registerAllowedMethod('input', [Map.class], { m -> | ||
iterations ++ | ||
assertThat(m.message, is('Do you want to restart?')) | ||
assertThat(m.ok, is('Restart')) | ||
if (iterations > 1) { | ||
throw new FlowInterruptedException() | ||
} else { | ||
return null | ||
} | ||
}) | ||
|
||
try { | ||
jsr.step.pipelineRestartSteps ([ | ||
script: nullScript, | ||
jenkinsUtilsStub: jenkinsUtils, | ||
sendMail: true, | ||
timeoutInSeconds: 1 | ||
|
||
]) { | ||
throw new hudson.AbortException('I just created an error') | ||
} | ||
} catch(err) { | ||
assertThat(jlr.log, containsString('ERROR occured: hudson.AbortException: I just created an error')) | ||
assertThat(mailBuildResult, is('UNSTABLE')) | ||
} | ||
} | ||
|
||
@Test | ||
void testErrorNoMail() throws Exception { | ||
|
||
def mailBuildResult = '' | ||
helper.registerAllowedMethod('mailSendNotification', [Map.class], { m -> | ||
mailBuildResult = m.buildResult | ||
return null | ||
}) | ||
|
||
helper.registerAllowedMethod('timeout', [Map.class, Closure.class], { m, closure -> | ||
assertThat(m.time, is(1)) | ||
assertThat(m.unit, is('SECONDS')) | ||
return closure() | ||
}) | ||
|
||
def iterations = 0 | ||
helper.registerAllowedMethod('input', [Map.class], { m -> | ||
iterations ++ | ||
assertThat(m.message, is('Do you want to restart?')) | ||
assertThat(m.ok, is('Restart')) | ||
if (iterations > 1) { | ||
throw new FlowInterruptedException() | ||
} else { | ||
return null | ||
} | ||
}) | ||
|
||
try { | ||
jsr.step.pipelineRestartSteps ([ | ||
script: nullScript, | ||
jenkinsUtilsStub: jenkinsUtils, | ||
sendMail: false, | ||
timeoutInSeconds: 1 | ||
|
||
]) { | ||
throw new hudson.AbortException('I just created an error') | ||
} | ||
} catch(err) { | ||
assertThat(jlr.log, containsString('ERROR occured: hudson.AbortException: I just created an error')) | ||
assertThat(mailBuildResult, is('')) | ||
} | ||
} | ||
|
||
@Test | ||
void testSuccess() throws Exception { | ||
|
||
jsr.step.pipelineRestartSteps ([ | ||
script: nullScript, | ||
jenkinsUtilsStub: jenkinsUtils, | ||
sendMail: false, | ||
timeoutInSeconds: 1 | ||
|
||
]) { | ||
nullScript.echo 'This is a test' | ||
} | ||
|
||
assertThat(jlr.log, containsString('This is a test')) | ||
} | ||
} |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,45 @@ | ||
package com.sap.piper | ||
|
||
import org.jenkinsci.plugins.workflow.steps.MissingContextVariableException | ||
import org.junit.Before | ||
import org.junit.Rule | ||
import org.junit.Test | ||
import org.junit.rules.ExpectedException | ||
import org.junit.rules.RuleChain | ||
import util.BasePiperTest | ||
import util.JenkinsLoggingRule | ||
import util.JenkinsShellCallRule | ||
import util.Rules | ||
|
||
import static org.hamcrest.Matchers.* | ||
import static org.junit.Assert.assertThat | ||
|
||
class JenkinsUtilsTest extends BasePiperTest { | ||
private JenkinsLoggingRule jlr = new JenkinsLoggingRule(this) | ||
private JenkinsShellCallRule jscr = new JenkinsShellCallRule(this) | ||
|
||
@Rule | ||
public RuleChain rules = Rules | ||
.getCommonRules(this) | ||
.around(jscr) | ||
.around(jlr) | ||
|
||
@Test | ||
void testNodeAvailable() { | ||
def result = jenkinsUtils.nodeAvailable() | ||
assertThat(jscr.shell, contains("echo 'Node is available!'")) | ||
assertThat(result, is(true)) | ||
} | ||
|
||
@Test | ||
void testNoNodeAvailable() { | ||
helper.registerAllowedMethod('sh', [String.class], {s -> | ||
throw new MissingContextVariableException(String.class) | ||
}) | ||
|
||
def result = jenkinsUtils.nodeAvailable() | ||
assertThat(jlr.log, containsString('No node context available.')) | ||
assertThat(result, is(false)) | ||
} | ||
|
||
} |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,52 @@ | ||
import com.sap.piper.JenkinsUtils | ||
import com.sap.piper.ConfigurationHelper | ||
import groovy.transform.Field | ||
|
||
@Field String STEP_NAME = 'pipelineRestartSteps' | ||
@Field Set STEP_CONFIG_KEYS = [ | ||
'sendMail', | ||
'timeoutInSeconds' | ||
] | ||
@Field Set PARAMETER_KEYS = STEP_CONFIG_KEYS | ||
|
||
def call(Map parameters = [:], body) { | ||
handlePipelineStepErrors (stepName: STEP_NAME, stepParameters: parameters) { | ||
def script = parameters.script ?: [commonPipelineEnvironment: commonPipelineEnvironment] | ||
def jenkinsUtils = parameters.jenkinsUtilsStub ?: new JenkinsUtils() | ||
// load default & individual configuration | ||
Map config = ConfigurationHelper | ||
.loadStepDefaults(this) | ||
.mixinGeneralConfig(script.commonPipelineEnvironment, STEP_CONFIG_KEYS) | ||
.mixinStepConfig(script.commonPipelineEnvironment, STEP_CONFIG_KEYS) | ||
.mixinStageConfig(script.commonPipelineEnvironment, parameters.stageName?:env.STAGE_NAME, STEP_CONFIG_KEYS) | ||
.mixin(parameters, PARAMETER_KEYS) | ||
.use() | ||
|
||
def restart = true | ||
while (restart) { | ||
try { | ||
body() | ||
restart = false | ||
} catch (Throwable err) { | ||
echo "ERROR occured: ${err}" | ||
if (config.sendMail) | ||
if (jenkinsUtils.nodeAvailable()) { | ||
mailSendNotification script: script, buildResult: 'UNSTABLE' | ||
} else { | ||
node { | ||
mailSendNotification script: script, buildResult: 'UNSTABLE' | ||
} | ||
} | ||
|
||
try { | ||
timeout(time: config.timeoutInSeconds, unit: 'SECONDS') { | ||
input message: 'Do you want to restart?', ok: 'Restart' | ||
} | ||
} catch(e) { | ||
restart = false | ||
throw err | ||
} | ||
} | ||
} | ||
} | ||
} |