New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
mesonbuild/mtest.py: filter more invalid googletest JUnit4 attributes #10634
Conversation
googletest 1.12.1 generates new JUnit4 invalid attributes file and line. Maybe all gtest "invalid" attributes are actually valid JUnit5 attributes, and maybe schema should be upgraded to JUni5. Signed-off-by: Dimitri John Ledkov <dimitri.ledkov@canonical.com>
Codecov Report
@@ Coverage Diff @@
## master #10634 +/- ##
===========================================
- Coverage 68.82% 50.30% -18.53%
===========================================
Files 406 203 -203
Lines 88062 44034 -44028
Branches 19560 9764 -9796
===========================================
- Hits 60612 22151 -38461
+ Misses 22871 19879 -2992
+ Partials 4579 2004 -2575
Continue to review full report at Codecov.
|
This is the same as |
This seems a bit suspicious. The correct fix would seem to be either to fix Google test to produce valid XML output or update our schema file to allow those entries. Deleting entries from XML like this does not seem right, especially if the file and line entries are valid and we remove them. That is a different bug report waiting to happen. |
As far as I can see, Meson is already deleting some of the entries from the googletest output to get it to match the required format, with comment
It's Meson that is asserting in its test suite that googletest will produce XML output that conforms to a particular schema, not googletest. If googletest doesn't promise to produce XML output according to a required schema, and if Meson promises that its output will conform to that schema, then Meson presumably shouldn't trust googletest to do that, either by postprocessing its XML output, doing a parse-and-reserialize to take only the parts that are understood, or (as a last resort) not picking up this extra information from googletest at all.
If https://github.com/junit-team/junit5/blob/main/platform-tests/src/test/resources/jenkins-junit.xsd is accurate, then However, https://junit.org/junit5/docs/snapshot/user-guide/#advanced-topics says:
The format Meson is producing seems to be the "de facto standard for JUnit 4 based test reports", so perhaps Meson shouldn't be promising that it won't contain additional attributes, because it's a de facto standard rather than a formal standard? I don't really know XML Schema, but perhaps inserting some At the same time, producing this de facto standard seems more useful than Open Test Reporting, because it's the JUnit-reporting de facto standard that interoperates with things like Gitlab-CI test result collection, and Open Test Reporting doesn't. |
googletest 1.12.1 generates new JUnit4 invalid attributes file and line.
Maybe all gtest "invalid" attributes are actually valid JUnit5 attributes, and maybe schema should be upgraded to JUni5.