Added ignoreFailure attribute to deployHubOutput goal.

[vjkoskela]

No description provided.

[psantos1113]

Hello,

Thanks for submitting a pull request.
What was the exception that you were getting when trying to deploy to the hub such that you wanted to ignore it and not fail your maven build?

[vjkoskela]

We experienced two exceptions; first, locally trying to deploy to the hub when it was not serving on port 8080 and second, after restarting the hub it had lost the credentials we had created. In the latter case it was from Jenkins so I have the logs; the exception was:

[INFO] --- hub-maven-plugin:1.0.4:deployHubOutput (deploy-black-duck-manifest) @ parent-pom ---
[info] BlackDuck Software parent-pom_bdio.jsonld file deployment starting...
Sep 28, 2016 11:47:54 PM org.restlet.engine.connector.HttpClientHelper start
INFO: Starting the internal HTTP client
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 11.944 s
[INFO] Finished at: 2016-09-28T23:47:54+00:00
[INFO] Final Memory: 45M/349M
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal com.blackducksoftware.integration:hub-maven-plugin:1.0.4:deployHubOutput (deploy-black-duck-manifest) on project parent-pom: Cannot communicate with hub server. Unauthorized (401) - Unauthorized -> [Help 1]
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException
Build step 'Execute shell' marked build as failure
Sending e-mails to: villek@qualtrics.com
Finished: FAILURE

Essentially, we would like to see the publish be entirely exception neutral.

[psantos1113]

There are multiple options on how to configure the deploy goal in maven. We want the build to fail if the deploy task has an error to avoid running the risk of succeeding the build, but not deploying the latest data to the hub. Such that the hub may display stale data which may impact the business especially if security vulnerability information is stale. By logging the error but not failing the build there is no indication in the CI system that there was a problem deploying and that it should be investigated further.

If you're using Jenkins, to work around the issue without a change to the plugin you could create a downstream job that uses the same workspace that only invokes the deploy to the hub maven goal. You invoke the job that deploys the file as the last job so that other jobs or steps will not fail and complete as needed. You could also look into Jenkins pipeline and invoke the maven command with the deploy goal. The pipeline can pass the build if it completes N number of steps. If you put the the deploy goal as the N+1 step then your build will be considered a success. This way the entire build does not fail and you can invoke your additional jobs.

[vjkoskela]

I understand that there are several work around. However, being able to control this at the plugin level works best for our build and deploy pipeline.

While I would like to require the analysis succeed on every build, we have had issues keeping your product up and running; so I am not comfortable gating production builds including hot fixes on the success of this step.

Several other plugins allow for such modes and the failure is recorded in the build log. I'm sorry to see you close this pull request as it would be useful to us and is backwards compatible. Can you elaborate on why this cannot be accepted?

[stavvy-akamen]

Hi Ville,

First, thank you so much for the pull request we always appreciate that. The reluctance for us to accept this particular piece of functionality stems from our concern that we are breaking the life-cycle. Typically we find that when a plugin is implemented and relies on an external system, any time the external system is down the build fails while relying on the goals/phases. For example we use SonarQube and it functions exactly the same, if SQ is down the build fails.

We would however really like to see other examples of where these override switches exist. Can you please link us to these implementations?

thanks!

[vjkoskela]

Thanks for your reply @akamen. Just to be clear my pull request does not break anything, it is fully backwards compatible where the default behavior is that any failure to interact with Black Duck Hub causes the build to fail.

Our concern is exactly as you describe. The Black Duck Hub is an external system to our build process and as such can fail for a variety of reasons including: 1) Hub software failure, 2) Hub hardware/vm/docker failure, 3) Network failure. In our case, we do not wish to block builds including critical hot-fixes on the failure of a non-critical external system.

I can see how some customers may wish to include Black Duck in their critical build path, but in our opinion that is not, at least at this time, an option for us. Can you see how the ability to ignore failures in a plugin that is non-critical to our build process and subject to distributed system failure modes is desirable?

Numerous plugins in Maven include the ability to disregard failures. Many of these plugins are far more critical than either Sonarqube or Black Duck. For example, you can disregard test failures with Surefire and Failsafe. Findbugs (byte-code analysis), Checkstyle (code analysis) and Jacoco (code coverage) all allow you to ignore their results. These plugins are all arguably more critical to the correctness of a build than either Sonarqube or Black Duck and none are prone to transient failures from being external (since they all run local to the build). Yet, the results of their execution can be ignored and the build allowed to complete successfully.

Surefire:
http://maven.apache.org/surefire/maven-surefire-plugin/test-mojo.html#testFailureIgnore

Failsafe:
http://maven.apache.org/surefire/maven-failsafe-plugin/verify-mojo.html#testFailureIgnore

Checkstyle:
http://maven.apache.org/plugins-archives/maven-checkstyle-plugin-2.16/check-mojo.html#failOnViolation

Findbugs:
http://gleclaire.github.io/findbugs-maven-plugin/check-mojo.html#failOnError

Jacoco:
http://www.eclemma.org/jacoco/trunk/doc/check-mojo.html#haltOnFailure

You are correct that Sonarqube does not permit ignoring communication failures. However, if you take a look at the Sonarqube Maven Plugin goal configuration you will see that it only has one option (skip).

http://www.mojohaus.org/sonar-maven-plugin/sonar-mojo.html

I have used Sonarqube before, and would probably use it again, but in my opinion it leaves a lot to be desired. Simply put as a provider of commercial software the bar for Black Duck and how it can be integrated into customer build systems is already much higher than for a free open-source project like Sonarqube. Now SonarSource provides additional plugins and commercial support for SonarQube and were I a customer of theirs I would have a similar ask.

I'm happy to discuss any of this further and your coworker Kaila Gervais has my contact information. Additionally, I am happy to change the PR in any way to suite the goals of your plugin; however, I do feel strongly that this is an important and non-intrusive feature inline with many other plugins in the Maven ecosystem.

Thank you,
Ville Koskela

[stavvy-akamen]

@vjkoskela appreciate the thoughtful response, we will consider the points you raised and get back to you shortly.

[stavvy-akamen]

@vjkoskela after discussing this with the team, your feature request is accepted. Due to significant code changes and new features to the maven plugin we will close down this pull request and open up a new ticket.

[vjkoskela]

Glad to hear that. I'm happy to expand my PR to include to all existing mojos. Obviously, I can't apply it to features that are not yet available. Would that enable us to move forward with it? Incorporating these changes into your refactoring should be trivial.