-
Notifications
You must be signed in to change notification settings - Fork 1.2k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
junit reporter format? #92
Comments
Hi @mattaylor, this sounds like a great idea. We'll work on this and release an update. Will keep you posted! 👍 |
+1 on this! |
FYI heres a brain dead livescript module to convert the json output to xunit...
|
+1 on junit/xunit reporter to be built into newman |
This would be awesome to have junit/xunit built in. |
This would be fantastic since most CI tools can easily parse xunit style reporting. |
+1 one this. Any update? would be very nice to integrate with jenkins |
+1 |
Newman 1.2.0 now supports junit-style output for tests. (Thanks @pal-thomassen) Sample response:
|
The syntax of the report file does not seem to correspond to the junit supported syntax found at http://llg.cubic.org/docs/junit/ successes is not recognised. |
Jenkins seems to expect the optional failure inside the testcase.
|
A In case of multiple iterations, each test is run multiple times. So having a single The next newman version (v1.2.5) will have |
just tested 1.2.6. successes and failures are in the testsute and testcase node. The testsuite node is aggregating the test case node. However Jenkins still isn't parsing that format it seems. Jenkins reads that junit output, it shows them all as successful, even when there are failures in the junit xml. |
To add some additional detail to this, the Jenkins xsd for jUnit is located here: https://svn.jenkins-ci.org/trunk/hudson/dtkit/dtkit-format/dtkit-junit-model/src/main/resources/com/thalesgroup/dtkit/junit/model/xsd/junit-4.xsd So, an example file would like like below, if there were an error. Seems like the current version that newman creates is missing the failures sumed up in the testsuites node, and its also missing the failure node within the testcase (if it had an error). For example: <testsuite failures="1" tests="3">
<testcase classname="foo1" name="ASuccessfulTest"/>
<testcase classname="foo2" name="AnotherSuccessfulTest"/>
<testcase classname="foo3" name="AFailingTest" failures="1">
<failure type="NotEnoughFoo"> details about failure </failure>
</testcase>
</testsuite> |
+10 |
XML output updated to adhere to the Jenkins JUnit xsd document as posted below, in response to issue postmanlabs#92. While we only list the child failure once per test case, the message notes how many of the iterations actually failed. Using the XML output as generated by these changes allows the "failure" count to display in Jenkins test output and trend charts. https://svn.jenkins-ci.org/trunk/hudson/dtkit/dtkit-format/dtkit-junit-model/src/main/resources/com/thalesgroup/dtkit/junit/model/xsd/junit-4.xsd
Tested the patch from commit #184 and it worked. Thanks! |
I don't understand the latest state of this issue. |
Any chance of adding support for custom templates for rendering test reports? A Junit test reporter would be particularly usefull for example to allow jenkins and similar CI tools process and render test results effectively .
The text was updated successfully, but these errors were encountered: