A build result of UNSTABLE should not necessarily result in a failure

[justinmchase]

This line here:

checks-api-plugin/src/main/java/io/jenkins/plugins/checks/status/BuildStatusChecksPublisher.java

Line 146 in fa7f8da

else if (result.isBetterOrEqualTo(Result.UNSTABLE) || result.isBetterOrEqualTo(Result.FAILURE)) {

I wish it would return state NEUTRAL or success, or perhaps be configurable.

I have a step which is currently producing errors which I want to know about but cannot block the build for. For example linter errors. We introduced a linter and it generates warnings which we want to fix over time and then eventually flip to a hard FAILURE but for now I want it to show as UNSTABLE.

I would create a PR if I had some guidance how best to implement such a change.

[XiongKezhi]

We marked the UNSTABLE as a failure because some users found that NEUTRAL may shadow the test failures: #11.

In Jenkins, plugins usually treat UNSTABLE as a failure, for example, the JUnit plugin will report UNSTABLE if one of the tests fails, and the Warnings-NG plugin uses UNSTABLE or FAILURE to conclude a build if it fails the quality gate.

It makes sense to make it configurable, you can implement it although it's a little complicated since it involves changes both on this API and the GitHub implementation:

1. StatusChecksProperties is the extension point for SCM platform implementations to provide the status checks properties, it would retrieve properties from implementations according to the job, current we only provide name and skip-bility.
2. GitHubStatusChecksProperties implements the above extension point for GitHub. Since we have both Git and GitHub project or SCM which target at GitHub, this class will get the user configurations from the proper GitHubStatusChecksConfigurations according to the SCM.
3. And finally, GitHubSCMSourceStatusChecksTrait and GitSCMStatusChecksExtension implement GitHubStatusChecksConfigurations and provided configurations for the end users.

So, for this status, maybe define a getStatusMap?

[justinmchase]

I follow your logic, if you think its fine to make it configurable I will try to follow up with a PR. It seems like this should be able to default to the current behavior and would be backwards compatible and would be a pretty reasonable thing to have configurable.

[agaudreault]

Not sure if that can help for your usecase, but we sometime use the UNSTABLE stageResult instead of the buildResult to display unstable behaviour that should not fail the build.

With that, the stage will show in yellow in the UI while the pipeline will still be successful.

catchError(message: "WARNING: The operation failed", buildResult: 'SUCCESS', stageResult: 'UNSTABLE', catchInterruptions: false) {
   // steps that will throw an error, but not fail the build if they do
}

[XiongKezhi]

Solved by #78 and jenkinsci/github-checks-plugin#124

[tperka-catalogicsoftware]

@XiongKezhi does it still work? I've checked the Publish unstable builds as neutral status checks checkbox but still getting failures in Github. Does even Github have neutral check result? I only see passing, pending and failing in their docs.

[uhafner]

See https://docs.github.com/en/rest/checks/runs#create-a-check-run:

conclusion
Required if you provide completed_at or a status of completed. The final conclusion of the check. Note: Providing conclusion will automatically set the status parameter to completed. You cannot change a check run conclusion to stale, only GitHub can set this.
Can be one of: action_required, cancelled, failure, neutral, success, skipped, stale, timed_out