Define an offical XSD for JUnit XML test reports

[Siphonophora]

I maintain a project used to output tests reports for .Net code into the junit format. In trying to provide this capability, we keep bumping up against the question of what, exactly, is the junit format. For example spekt/junit.testlogger#41 asks for system-out to recorded per test case, and not at the end of the file. The user who opened that issue pointed to the jenkins schema you used in this repo for some unit testing. But the standard we adopted for our logger was the Ant unit schema (which I picked b/c of comments suggesting it was the de-facto standard which I found a while ago in this or the junit4 repo/docs).

Among the .Net community, the junit format is used extensively with CICD tools like gitlab, circleCI... And if you look at the gitlab docs https://docs.gitlab.com/ee/ci/unit_test_reports.html you will see quite a lot of tooling exists to produce junit reports. So, I'm sure i'm sure mine is not the only project that has run into this question.

The Jenkins schema looks like an improvement over the ant one, at least when it comes to system-out, but swapping schemas on my end will be a breaking change for some of my users whose CICD systems are doing strict validation against an XSD. I'm reluctant to start adding feature flags to let users choose which schema they want.

Note, I also saw #373 is looking at creating a new format, but i'm really interested in what the current format is, and probably having a version number or something on the format. I also saw that the discussions on that issue are talking about switching to json, yml or other formats. Please keep in mind what a big ecosystem of users there who are using this format.

Deliverables

Please produce or select an XSD as the official one. Since it looks like you are doing all you internal unit tests against the jenkins format, it seems like that might be the answer. But a definite answer would be great one way or the other.

[sbrannen]

Tentatively slated for 5.8 M2 for team discussion

[marcphilipp]

We can only set a standard for the XML reports written by JUnit (LegacyXmlReportGeneratingListener which is used by ConsoleLauncher). For those, we use Jenkins' XSD for validation: https://github.com/junit-team/junit5/blob/main/platform-tests/src/test/resources/jenkins-junit.xsd. We're currently working on a new XML-based format for which we'll provide an XSD from day 1.