I think ideally the parser should notice whenever it's got to the end of the output and mark and outstanding tests as failed, which would catch this situation in the more general case. Unfortunately there's no clean way for the parser to know when it's reached the end (close() is specifically blocked although it's not clear why).
However for the specific case of exiting with an exception, this patch makes the output 'unstable' rather than succeeding or failing outright (depending on whether the 'allow failing build results' is enabled).
Make the test output parser set tests to failed if tests exit with an…
Thank you for a pull request! Please check this document for how the Jenkins project handles pull requests
Could the scenario happen even if we have no testsuite as well ? (e.g. internal failure)
JUnit supports the concept of errors. I think we should really have an error here instead.
If this is the official XSD for JUnit http://windyroad.com.au/dl/Open%20Source/JUnit.xsd, then maybe we are missing the timestamp (which could be parsed from the line)
Output 'error' elements to the JUnit instead of 'failure' when a test…
You're right, this should be an 'error'. I've added support for 'error' elements to the JUnit outputter and made it output them. I'm not sure about parsing the time: the rest of the the time parsers just grab the time the test took, whereas we only have the wallclock time of the box, so we could probably work it out but since test crashed anyway, I think it's unlikely to ever be that useful.
Are we missing the updated XCodeBuildOutputParser in your latest commit/push ?
As for the time, I agree that it's not critical. I am not even sure this schema is official and I suspect parsers have learned to be more resistant :) Jenkins doesn't seem to complain without it today.
Note: make sure that the TestSuite is probably updated to count the errors. Ideally they would 'bubble up' all the way to the 'All tests' one I suspect.
Oops, commit modified files too (second part of adding 'error' test/J…
Oops, yes, sorry: I forgot to git add -u. I added addError() to the TestSuite to match addFailure() and called it when it detects an exception so hopefully this should be okay.
Merge pull request #54 from dbkr/master
XCTest output parsing: handle tests suites exiting with an error
Thanks for the Pull Request. Merged.