fortifyupload step: unable to reliably determine step status in pipeline

[maroschutte]

Jenkins and plugins versions report

Jenkins: 2.332.2
OS: Linux - 4.14.273-207.502.amzn2.x86_64

ace-editor:1.1
ant:1.13
antisamy-markup-formatter:2.7
apache-httpcomponents-client-4-api:4.5.13-1.0
authentication-tokens:1.4
authorize-project:1.4.0
aws-credentials:191.vcb_f183ce58b_9
aws-java-sdk-ec2:1.12.163-315.v2b_716ec8e4df
aws-java-sdk-minimal:1.12.163-315.v2b_716ec8e4df
bootstrap4-api:4.6.0-3
bootstrap5-api:5.1.3-6
bouncycastle-api:2.25
branch-api:2.1044.v2c007e51b_87f
caffeine-api:2.9.2-29.v717aac953ff3
checks-api:1.7.2
cloudbees-folder:6.714.v79e858ef76a_2
command-launcher:1.6
config-file-provider:3.9.0
configuration-as-code:1414.v878271fc496f
credentials:1087.v16065d268466
credentials-binding:1.27.1
cucumber-reports:5.7.0
dark-theme:156.v6cf16af6f9ef
dashboard-view:2.426.vb_93dcdb_b_0eb_0
display-url-api:2.3.6
docker-build-step:2.8
docker-commons:1.19
docker-workflow:1.28
dropdown-viewstabbar-plugin:1.7
durable-task:495.v29cd95ec10f2
ec2:1.68
echarts-api:5.3.0-2
extended-read-permission:3.2
extensible-choice-parameter:1.8.0
external-monitor-job:191.v363d0d1efdf8
font-awesome-api:6.0.0-1
fortify:21.2.37
gatling:1.3.0
git:4.11.0
git-client:3.11.0
git-server:1.10
gitlab-api:1.0.6
gitlab-branch-source:1.5.9
gradle:1.38
groovy:2.4
handlebars:3.0.8
handy-uri-templates-2-api:2.1.8-1.0
htmlpublisher:1.29
hudson-wsclean-plugin:1.0.8
jackson2-api:2.13.2-260.v43d711474c77
javadoc:217.v905b_86277a_2a_
javax-activation-api:1.2.0-2
javax-mail-api:1.6.2-5
jaxb:2.3.0.1
jdk-tool:1.5
jnr-posix-api:3.1.7-3
jquery:1.12.4-1
jquery-detached:1.2.1
jquery3-api:3.6.0-2
jsch:0.1.55.2
junit:1.58
lockable-resources:2.14
m2release:0.16.2
mailer:408.vd726a_1130320
matrix-auth:3.1
matrix-project:758.v7a_ea_491852f3
maven-plugin:3.18
momentjs:1.1.1
nested-view:1.24
node-iterator-api:1.5.1
nodejs:1.5.1
pam-auth:1.7
percentage-du-node-column:0.1.0
pipeline-build-step:2.16
pipeline-graph-analysis:188.v3a01e7973f2c
pipeline-input-step:446.vf27b_0b_83500e
pipeline-milestone-step:100.v60a_03cd446e1
pipeline-model-api:2.2075.vce74e77b_ce40
pipeline-model-definition:2.2075.vce74e77b_ce40
pipeline-model-extensions:2.2075.vce74e77b_ce40
pipeline-rest-api:2.23
pipeline-stage-step:291.vf0a8a7aeeb50
pipeline-stage-tags-metadata:2.2075.vce74e77b_ce40
pipeline-stage-view:2.23
plain-credentials:1.8
plugin-usage-plugin:2.2
plugin-util-api:2.16.0
popper-api:1.16.1-2
popper2-api:2.11.4-1
resource-disposer:0.18
role-strategy:3.2.0
saml:2.296.v0016349946db_
scm-api:595.vd5a_df5eb_0e39
script-security:1145.vb_cf6cf6ed960
snakeyaml-api:1.29.1
snyk-security-scanner:3.2.4
sonar:2.14
splunk-devops:1.9.9
splunk-devops-extend:1.9.9
ssh-credentials:1.19
sshd:3.226.vb_1769a_7fb_b_a_6
startup-trigger-plugin:2.9.3
structs:308.v852b473a2b8c
theme-manager:0.6
timestamper:1.17
token-macro:285.vff7645a_56ff0
trilead-api:1.57.v6e90e07157e1
update-sites-manager:2.0.0
variant:1.4
versioncolumn:2.2
view-job-filters:2.3
windows-slaves:1.8
workflow-aggregator:2.7
workflow-api:1143.v2d42f1e9dea_5
workflow-basic-steps:941.vdfe1b_a_132c64
workflow-cps:2686.v7c37e0578401
workflow-cps-global-lib:564.ve62a_4eb_b_e039
workflow-durable-task-step:1128.v8c259d125340
workflow-job:1174.vdcb_d054cf74a_
workflow-multibranch:711.vdfef37cda_816
workflow-scm-step:2.13
workflow-step-api:622.vb_8e7c15b_c95a_
workflow-support:817.v58126df57338
ws-cleanup:0.41

What Operating System are you using (both controller, and any agents involved in the problem)?

amazon linux

Reproduction steps

In a pipeline with multiple stages, let 'stage A' set the stage and build result to UNSTABLE (for instance because unstable() is called). When in a later 'stage B' a fortify analysis is done which ends with a call to the fortifyUpload step, the only way to query the outcome of this step currently is checking currentBuild.result == 'UNSTABLE' (see

fortify-plugin/src/main/java/com/fortify/plugin/jenkins/steps/FortifyUpload.java

Line 241 in d6d6b5a

run.setResult(Result.UNSTABLE);

)

This however will not do, because it is impossible to know whether it is stage A that set the status to UNSTABLE and the fortifyupload step didn't find any issues, or it actually was the fortifyUpload step that found issues and (again) set the status to UNSTABLE.

I have prepared a pull request introducing an extra flag to optionally make the fortifyUpload step throw an exception when issues are found. This exception can be caught and the pipeline author can react to it however he may seem fit.

Expected Results

My expected result is that as a pipeline author I can always determine whether there were fortify issues found.

Actual Results

My fortify analysis stage incorrectly reports there were issues when stage A already reported an UNSTABLE state.

Anything else?

No response

[maroschutte]

A different solution to the one in my patch would be to have the fortifyUpload step return the status. An example of this approach can be found in the waitForQualityGate step in the sonarqube plugin

[akaryakina]

We're now going to return a number of issues satisfying the failure criteria, so you can use it to determine if that was the reason why the build has been marked as unstable. Here's my pipeline example:

pipeline {
    agent { label 'agent1' }
    stages {
        stage('FailIfCriticals') {
            steps {
                script {
                    def criticalCount = fortifyUpload(appName: 'test', appVersion: '1.0', failureCriteria: '[fortify priority order]:Critical', filterSet: '', pollingInterval: '', resultsFile: '', timeout: '')
                    if (criticalCount > 0) {
                        echo "Found critical vulnerabilities: ${criticalCount}"
                    }
                }
            }
        }
    }
}

[maroschutte]

Thanks for implementing this. Now the final obvious question: when will this be released ?

[akaryakina]

Well, we're dev complete and starting the testing. We cannot communicate any ETAs, but it's possible that it's going to happen at the end of August.