The entirety of JUnit XML files is skipped if they are minified #1132

underyx · 2015-05-15T13:39:25Z

I'm using py.test and its built-in JUnit XML generator to make reports of my tests. Go.cd always fails to display any of the contents, logging this:

13:16:44.907 Unable to publish test properties. Error was javax.xml.transform.TransformerException: com.sun.org.apache.xml.internal.utils.WrappedRuntimeException: The element type "all-results" must be terminated by the matching end-tag "</all-results>".
13:16:44.912 Could not publish property org.xml.sax.SAXParseException; lineNumber: 1; columnNumber: 1; Premature end of file.
13:16:44.913 Could not publish property org.xml.sax.SAXParseException; lineNumber: 1; columnNumber: 1; Premature end of file.
13:16:44.914 Could not publish property org.xml.sax.SAXParseException; lineNumber: 1; columnNumber: 1; Premature end of file.
13:16:44.915 Could not publish property org.xml.sax.SAXParseException; lineNumber: 1; columnNumber: 1; Premature end of file.

Now, bear with me here, cause I have zero knowledge of Java, but skimming through the source I think the issue is that the py.test JUnit files have no newlines in them, so this line skips the complete file, with its 200 test cases. ~~If that's not the problem, please let me know and I'll edit the title of this issue.~~ Edit: I tried reformatting the same XML file and it works now. This means that there's a workaround:

Workaround

Install the xmllib2-utils package on your agents
Change your py.test config to output to junit.min.xml instead of junit.xml
Add the following step to the end of the pipeline:

<exec command="/bin/bash">
  <arg>-c</arg>
  <arg>xmllint --format junit.min.xml > junit.xml</arg>
  <runif status="passed" />
</exec>

The text was updated successfully, but these errors were encountered:

srinivasupadhya · 2015-05-16T05:27:52Z

can you post sample XML file so someone can reproduce the issue?

underyx · 2015-05-19T09:41:21Z

Sure, @srinivasupadhya, something like this:

<?xml version="1.0" encoding="utf-8"?><testsuite errors="0" failures="0" name="pytest" skips="0" tests="2" time="0.001"><testcase classname="test.test_api_app" name="test_get_context_from_headers_generic[header_dict0-expected_diff0]" time="0.003357410430908203"/><testcase classname="test.test_api_app" name="test_get_context_from_headers_generic[header_dict1-expected_diff1]" time="0.00024437904357910156"/></testsuite>

Instead of ignoring the first line of a report if it contains "<?xml", replace the prolog pattern[1] in the first line with an empty string. If the processed first line is empty, ignore it. See GitHub Issue gocd#1132 [1]: http://www.w3.org/TR/REC-xml/#NT-prolog

gocd#1132

ketan · 2015-12-11T04:39:45Z

@rajiesh - reopening for you to look at

…ed (#1132)"

…eport Revert "The entirety of JUnit XML files is skipped if they are minified (#1132)"

…#1132) This reverts commit 84f1458.

The entirety of JUnit XML files is skipped if they are minified (#1132)

rajiesh · 2015-12-14T14:03:40Z

verified it on 15.3.0-2753(7aa885f) and is working as expected.

underyx · 2015-12-14T15:38:31Z

Yay 🎉

…le things.

Reverting all changes related to #1132 as this is breaking multiple things

arvindsv · 2015-12-23T19:33:08Z

Re-opening again. Causes issues with big (say 12MB) XML files.

2015-12-23 15:58:21,169 [loopThread] INFO  thoughtworks.go.work.DefaultGoPublisher:90 - Stopping Transmission for Build [build-windows/632/build-server-windows/2/server-split-ru\
nInstance-6/140899]
2015-12-23 15:58:21,185 [loopThread] ERROR springframework.scheduling.timer.MethodInvokingTimerTaskFactoryBean:68 - Invocation of method 'loop' on target class [class com.though\
tworks.go.agent.AgentController] failed
java.lang.OutOfMemoryError: Java heap space
  at com.sun.org.apache.xerces.internal.util.XMLStringBuffer.append(XMLStringBuffer.java:208)
  at com.sun.org.apache.xerces.internal.impl.XMLEntityScanner.scanData(XMLEntityScanner.java:1370)
  at com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl.scanCDATASection(XMLDocumentFragmentScannerImpl.java:1654)
  at com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl$FragmentContentDriver.next(XMLDocumentFragmentScannerImpl.java:3020)
  at com.sun.org.apache.xerces.internal.impl.XMLDocumentScannerImpl.next(XMLDocumentScannerImpl.java:606)
  at com.sun.org.apache.xerces.internal.impl.XMLNSDocumentScannerImpl.next(XMLNSDocumentScannerImpl.java:117)
  at com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl.scanDocument(XMLDocumentFragmentScannerImpl.java:510)
  at com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:848)
  at com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:777)
  at com.sun.org.apache.xerces.internal.parsers.XMLParser.parse(XMLParser.java:141)
  at com.sun.org.apache.xerces.internal.parsers.AbstractSAXParser.parse(AbstractSAXParser.java:1213)
  at com.sun.org.apache.xerces.internal.jaxp.SAXParserImpl$JAXPSAXParser.parse(SAXParserImpl.java:648)
  at org.jdom.input.SAXBuilder.build(SAXBuilder.java:453)
  at org.jdom.input.SAXBuilder.build(SAXBuilder.java:851)
  at com.thoughtworks.go.util.XmlUtils.buildXmlDocument(XmlUtils.java:88)
  at com.thoughtworks.go.domain.UnitTestReportGenerator.pumpFileContent(UnitTestReportGenerator.java:158)
  at com.thoughtworks.go.domain.UnitTestReportGenerator.pumpFileContentIfValid(UnitTestReportGenerator.java:152)
  at com.thoughtworks.go.domain.UnitTestReportGenerator.merge(UnitTestReportGenerator.java:142)
  at com.thoughtworks.go.domain.UnitTestReportGenerator.mergeAllTestResultToSingleFile(UnitTestReportGenerator.java:112)
  at com.thoughtworks.go.domain.UnitTestReportGenerator.generate(UnitTestReportGenerator.java:63)
  at com.thoughtworks.go.config.TestArtifactPlan.mergeAndUploadTestResult(TestArtifactPlan.java:101)
  at com.thoughtworks.go.config.TestArtifactPlan.publish(TestArtifactPlan.java:66)
  at com.thoughtworks.go.domain.DefaultJobPlan.publishArtifacts(DefaultJobPlan.java:105)
  at com.thoughtworks.go.remote.work.BuildWork.completeJob(BuildWork.java:198)
  at com.thoughtworks.go.remote.work.BuildWork.build(BuildWork.java:146)
  at com.thoughtworks.go.remote.work.BuildWork.doWork(BuildWork.java:91)
  at com.thoughtworks.go.agent.JobRunner.run(JobRunner.java:53)
  at com.thoughtworks.go.agent.AgentController.retrieveWork(AgentController.java:186)
  at com.thoughtworks.go.agent.AgentController.loop(AgentController.java:132)
  at sun.reflect.GeneratedMethodAccessor10.invoke(Unknown Source)
  at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
  at java.lang.reflect.Method.invoke(Method.java:606)

darioblanco · 2016-07-07T12:09:35Z

I have the same problem with go-server 16.6.0-3590, for the very same case. The issue is still there.

06:30:23.178 Unable to publish test properties. Error was javax.xml.transform.TransformerException: com.sun.org.apache.xml.internal.utils.WrappedRuntimeException: The element type "all-results" must be terminated by the matching end-tag "</all-results>".

The xml file is generated by py-test, and is one single line indeed.

After running @underyx solution with xmllint --format -o junit-pytest.xml junit-pytest.xml as another task command (will rewrite that file), I could get rid of the error. However, it will only show test results in the Tests tab from the job view, when I go into the Tests tab from the stage view, I get a There are tests configured in this stage but could not compute results.. So basically two problems in one.

kmugrage · 2016-08-09T21:12:44Z

I'm seeing this with 16.7.0 (3819-b0b9921bdea58101121cc181d697355177d2f197)

  - unit_test:
      jobs:
        unit_test:
          tasks:
            - exec:
                command: py.test
                arguments:
                  - --junitxml
                  - testoutput.xml
                  - "./unit_test/python_test.py"
          artifacts:
            - test:
                source: testoutput.xml

Contents of the XML

<?xml version="1.0" encoding="utf-8"?><testsuite errors="0" failures="1" name="pytest" skips="1" tests="2" time="0.018"><testcase classname="unit_test.python_test.SimpleTest" name="test_fail" time="0.000195026397705"><failure message="test failure">self = &lt;python_test.SimpleTest testMethod=test_fail&gt;

    def test_fail(self):
&gt;       self.assertEqual(11, 7 + 3)
E       AssertionError: 11 != 10

unit_test/python_test.py:18: AssertionError</failure></testcase><testcase classname="unit_test.python_test.SimpleTest" name="test_pass" time="7.41481781006e-05"/><testcase classname="unit_test.python_test.SimpleTest" name="test_skipped" time="9.79900360107e-05"><skipped message="demonstrating skipping" type="pytest.skip">/usr/lib/python2.7/dist-packages/_pytest/unittest.py:113: Skipped: demonstrating skipping</skipped></testcase></testsuite>

Error in GoCD

19:39:59.343 [go] Uploading artifacts from /var/lib/go-agent/pipelines/Application/testoutput.xml to [defaultRoot]
19:39:59.357 Unable to publish test properties. Error was javax.xml.transform.TransformerException: com.sun.org.apache.xml.internal.utils.WrappedRuntimeException: The element type "all-results" must be terminated by the matching end-tag "</all-results>".
19:39:59.358 Could not publish property org.xml.sax.SAXParseException; lineNumber: 1; columnNumber: 1; Premature end of file.
19:39:59.359 Could not publish property org.xml.sax.SAXParseException; lineNumber: 1; columnNumber: 1; Premature end of file.
19:39:59.360 Could not publish property org.xml.sax.SAXParseException; lineNumber: 1; columnNumber: 1; Premature end of file.
19:39:59.360 Could not publish property org.xml.sax.SAXParseException; lineNumber: 1; columnNumber: 1; Premature end of file.

kmugrage · 2016-08-09T21:31:03Z

Echoing @darioblanco - adding xmllint works for the job but not at the stage level

bjorn-spire · 2017-09-25T11:09:32Z

and seeing the same problem with 17.10.0, running the xml file through xmllint --format allowed it to get picked up by Go. :)

ketan · 2018-02-07T11:29:52Z

Closing as dup of #983

zabil mentioned this issue Jul 28, 2015

Release 15.3 proposal #1104

Closed

37 tasks

jyotisingh added this to the Release 15.3 milestone Jul 28, 2015

sghill mentioned this issue Oct 17, 2015

Parsing minified JUnit XML #1570

Closed

zabil modified the milestones: Release 15.4, Release 15.3 Dec 3, 2015

xli added a commit to xli/gocd that referenced this issue Dec 8, 2015

fix bug The entirety of JUnit XML files is skipped if they are minified

26ec3c8

gocd#1132

xli mentioned this issue Dec 8, 2015

The entirety of JUnit XML files is skipped if they are minified (#1132) #1685

Merged

xli added a commit to xli/gocd that referenced this issue Dec 9, 2015

fix bug The entirety of JUnit XML files is skipped if they are minified

6e7c1f7

gocd#1132

xli added a commit to xli/gocd that referenced this issue Dec 9, 2015

fix bug The entirety of JUnit XML files is skipped if they are minified

29d8c04

gocd#1132

ketan closed this as completed in #1685 Dec 11, 2015

ketan reopened this Dec 11, 2015

ketan added a commit that referenced this issue Dec 11, 2015

Revert "The entirety of JUnit XML files is skipped if they are minifi…

84f1458

…ed (#1132)"

ketan added a commit that referenced this issue Dec 11, 2015

Merge pull request #1693 from gocd/revert-1685-parse_minified_junit_r…

5422c89

…eport Revert "The entirety of JUnit XML files is skipped if they are minified (#1132)"

xli added a commit to xli/gocd that referenced this issue Dec 11, 2015

The entirety of JUnit XML files is skipped if they are minified (gocd…

a27a024

…#1132) This reverts commit 84f1458.

zabil added a commit that referenced this issue Dec 14, 2015

Merge pull request #1697 from xli/parse_minified_junit_report

d12861f

The entirety of JUnit XML files is skipped if they are minified (#1132)

zabil closed this as completed Dec 15, 2015

jyotisingh added a commit to jyotisingh/gocd that referenced this issue Dec 23, 2015

Reverting all changes related to gocd#1132 as this is breaking multip…

62cac38

…le things.

jyotisingh added a commit to jyotisingh/gocd that referenced this issue Dec 23, 2015

Reverting all changes related to gocd#1132 as this is breaking multip…

18ee89a

…le things.

arvindsv added a commit that referenced this issue Dec 23, 2015

Merge pull request #1747 from jyotisingh/revert_all_changes_for_1132

c0f91b2

Reverting all changes related to #1132 as this is breaking multiple things

arvindsv reopened this Dec 23, 2015

arvindsv modified the milestones: Release 16.1, One of the upcoming releases Jan 4, 2016

arvindsv modified the milestones: One of the upcoming releases, Release 16.1 Jan 4, 2016

arvindsv added the [stream] other label Jan 4, 2016

arvindsv mentioned this issue Jun 4, 2016

Agent OOMException when preparing JUnit xml report #2160

Closed

ketan closed this as completed Feb 7, 2018

eladtayar mentioned this issue May 12, 2018

Junit xml is missing from the Properties and Test tab #4732

Closed

This was referenced Jan 6, 2023

GoCD doesn't parse a JUnit XML produced by pytest #11159

Closed

Correct aggregation of test results with minified XML #11160

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

The entirety of JUnit XML files is skipped if they are minified #1132

The entirety of JUnit XML files is skipped if they are minified #1132

underyx commented May 15, 2015

srinivasupadhya commented May 16, 2015 via email

underyx commented May 19, 2015

ketan commented Dec 11, 2015

rajiesh commented Dec 14, 2015

underyx commented Dec 14, 2015

arvindsv commented Dec 23, 2015

darioblanco commented Jul 7, 2016

kmugrage commented Aug 9, 2016

kmugrage commented Aug 9, 2016

bjorn-spire commented Sep 25, 2017

ketan commented Feb 7, 2018

The entirety of JUnit XML files is skipped if they are minified #1132

The entirety of JUnit XML files is skipped if they are minified #1132

Comments

underyx commented May 15, 2015

Workaround

srinivasupadhya commented May 16, 2015 via email

underyx commented May 19, 2015

ketan commented Dec 11, 2015

rajiesh commented Dec 14, 2015

underyx commented Dec 14, 2015

arvindsv commented Dec 23, 2015

darioblanco commented Jul 7, 2016

kmugrage commented Aug 9, 2016

kmugrage commented Aug 9, 2016

bjorn-spire commented Sep 25, 2017

ketan commented Feb 7, 2018