This repository has been archived by the owner on Sep 23, 2024. It is now read-only.
Handle Jenkins errors #107
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
This is not tested yet, WIP. |
Pull Request Test Coverage Report for Build 376
💛 - Coveralls |
Remove detecting baseline failures during testing patches, as it's confusing and difficult to support in the Jenkins pipeline. It is also not skt's job to be operating on baselines, as they're out of its control or knowledge. Moreover, we have no baseline failures recorded in our database, at all.
Remove the PUBLISH_FAILURE sktm.tresult value, since it would be just an infrastructure failure (an error), unrelated to patches themselves, no code actually generates it, and there are no records of it occuring in our database.
Fetch and log the data used both by baseline and by patchwork Jenkins build handling, beforehand, in a single place, instead of for each case separately.
Add another sktm.tresult: ERROR. It signifies that the test encountered an error and has no valid result. This value (sktm.tresult.ERROR) should not be stored in the database, or posted as a "check" in patchwork, since it's not a result, but an absence of.
Consider any result coming from Jenkins, except straight SUCCESS or UNSTABLE (with matching skt results), an error. Ignore builds with such results, leaving patch tests in "pendingpatches", and baseline untested, both to be retried later. UNSTABLE is a Jenkins build result signifying some errors, which were not fatal for the completion of the build, while FAILURE is a fatal build error. We can consider "build" to be our test run, and test failures not fatal for the run. It's awkward naming, but it seems to be the best mapping to Jenkins build results. See also http://javadoc.jenkins-ci.org/hudson/model/Result.html
spbnick
force-pushed
the
handle_jenkins_errors
branch
from
2376057
to
d1e0e33
Compare
|
While I didn't test it either and just checked the code, most of this is removal of unused code and cleanups and should be good to go 👍 the Jenkins part should be properly tested though |
|
I tested this mostly, still waiting for one job to finish, then we can merge this, and I'll deliver this to staging, along with the required pipeline changes. |
|
OK, tested with the new pipeline, success and infrastructure failure (cancelled Beaker jobs), baseline and patchwork commands worked fine. @veruu, do you think this is ready to go in? |
|
I think so. And if there is something that breaks in a way we didn't expect we can fix it afterwards. |
veruu
approved these changes
Jul 18, 2018
|
Thank you, Veronika :) |
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This adds detection of test errors (infrastructure failures) and makes sktm ignore Jenkins builds which have them, so that baseline is not degraded and patches are held in
pendingpatches, both to be rechecked on subsequent runs.