Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

JUnit reporter reports passing tests using an element that is not in the JUnit XML schema #768

Closed
dlipovetsky opened this issue Feb 10, 2021 · 1 comment

Comments

@dlipovetsky
Copy link
Contributor

dlipovetsky commented Feb 10, 2021

In #586, the JUnit reporter was extended to capture the output of passing tests when the -reportPassing flag is used.

For a passing test, the reporter writes:

<?xml version="1.0" encoding="UTF-8"?>
  <testsuite name="example suite" tests="1" failures="0" errors="0" time="3.026">
      <testcase name="example test" classname="example suite" time="2.000321685">
          <passed>Sample output</passed>
      </testcase>
  </testsuite>

However, the passed element is not in the JUnit XML schema. While that alone might a theoretical problem, it is also a practical problem, because some parsers, e.g., the TeamCity XML Reporting Plugin JUnit parser, conform to that schema, ignore the element, and show no output for passing tests.

The fix seems to be to use the system-out element instead. I have confirmed that, when using the system-out attribute, the TeamCity XML Reporting Plugin Junit parser shows output for a passing test.

<?xml version="1.0" encoding="UTF-8"?>
  <testsuite name="example suite" tests="1" failures="0" errors="0" time="3.026">
      <testcase name="example test" classname="example suite" time="2.000321685">
          <system-out>Sample output</system-out>
      </testcase>
  </testsuite>

(For anyone asking why the system-err element is not used: Ginkgo intercepts both stdout and stderr together, and the default reporter writes that output to stdout).

To be clear, I am not a JUnit expert, and I am looking for feedback. I assume the passed element was working for @manosnoam, who authored #586.

@manosnoam
Copy link
Contributor

manosnoam commented Feb 11, 2021

Thanks for fixing it @dlipovetsky , indeed <system-out> should be used instead of <passed>.
Originally I added it, assuming it's the contra for <failed> tag, which is supported.

<passed> is not supported in other apps schemas that parses Junit, for example Polarion.
(in ReportPortal id did go through though).

@onsi onsi closed this as completed Mar 7, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants