New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
LegacyXmlReportGeneratingListener should create a Test-nnn.xml fiel for each class #1989
Comments
My workaround right now is to replace
Which is an ugly workaround, btw. |
One more thing: org.junit.platform.reporting.legacy.xml.XmlReportWriter.writeSuiteAttributes(TestIdentifier, List, NumberFormat, XMLStreamWriter) needs to use testIdentifier.getLegacyReportingName() instead of testIdentifier.getDisplayName() to makes it similar to the old junit4 report. old code: new code: |
The reason there's a report per test engine is that the JUnit Platform does not require test engines to use classes. However, the resulting XML files contain the class name in the Thinking out loud, I think we could add a reporting granularity level option to the console launcher with 0 being the current level of engines, 1 being their immediate children and so forth. However, since we have a new reporting format in the making (see #373), it doesn't feel like the right time to do so.
I assume you mean if we changed the granularity it would start making a difference, right? Currently it's only used to output the test engine's display name. |
Tentatively slated for 5.6 Backlog for team discussion. |
Sounds like a good idea to make it configurable via console launcher option. That's what I was looking for first. I'm just saying that the "data" that is generated by the LegacyXmlReportGeneratingListener is not 100% compatible with what we get when we use JUnit4, and thats a stopper for us to migrate to JUnit5. Unfortunately, the JUnit5 ant task JUnitLauncher does not support @ParameterizedTest and has some other problems, too (e.g. https://bz.apache.org/bugzilla/show_bug.cgi?id=63680 / https://bz.apache.org/bugzilla/show_bug.cgi?id=63446). So this is also no way to go. I follow the conversation about a new reporting format and would be happy about a universal format. |
PR #1981 could be used as the basis of a solution to this (see #1980 (comment) as an example). |
This issue has been automatically marked as stale because it has not had recent activity. Given the limited bandwidth of the team, it will be automatically closed if no further activity occurs. Thank you for your contribution. |
+1 for this, please |
This issue has been automatically marked as stale because it has not had recent activity. Given the limited bandwidth of the team, it will be automatically closed if no further activity occurs. Thank you for your contribution. |
This version changes the way JUnit tests are run internally. No modifications need to be made to a project that already uses version 1.8. Switching from JUnit 4 to 5 in Ant is more complex than just switching version numbers and requires [multiple adaptations](https://www.selikoff.net/2018/07/29/ant-and-junit-5). Since the switch to support JUnit 5 in AntRun version 1.7, some elements of the process left something to be desired in comparison to version 4. - First, the `jacoco` Ant task [cannot include the `junitlauncher` task](jacoco/jacoco#673) (which is required to run JUnit 5). The workaround that was implemented in version 1.7.1 is to manually run the JUnit console launcher. - Unfortunately, the report format produced by this launcher incorrectly merges all test cases into a single category instead of separating tests by classes as is done in JUnit 4. After much searching and debugging, I discovered that [this is intended](junit-team/junit5#1989) and that the JUnit team [does not plan to change the way tests reports are produced to fix this](junit-team/junit5#1980 (comment)). - The only other option to get a proper HTML report is to use the `junitlauncher` task --back to square one. However, versions of Ant starting from 1.10.6 have a new option that [allows using JaCoCo with the task](jacoco/jacoco#673 (comment)). - It looked like the problem was solved, but implementing the solution resulted in a bunch of errors (in a nutshell, `junitlauncher` does not exist). It took some more searching to discover that the `ant-optional` package in Debian-based Linux distributions [is missing a required JAR file](https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=967933;msg=22), which was causing the issue. The issue is fixed in Ant 1.10.10, which is not available for my Linux distribution (Ubuntu 20.04). - I thus wrote extra code in the build script that downloads the missing file into the local project and defines the `junitlauncher` task manually by referring to this local file. One must be careful, however, to download the version of the file that matches the version of Ant that runs the script. Total time for all this: about 6 hours over two days.
We are migrating to JUnit 5 (from JUnit 4) and using the console launcher in combination with ant like described in https://github.com/junit-team/junit5-samples/blob/master/junit5-jupiter-starter-ant/build.xml
We cannot use junitlauncher-task because the output supports no @ParameterizedTest and so on.
After running the tests we generate a junitreport (with junitreport ant task) for the test run and a
unitth report (http://junitth.sourceforge.net/) for the last 100 test runs.
The problem is that the LegacyXmlReportGeneratingListener generates one BIG TEST-nnn.xml for a few hundred test cases. This results in a junitreport without any classes & packages. Just one big list of all tests.
The "OLD" JUnit 4 Ant Task generated a TEST-nnn.xml for each test class. With this, junitreport & unitth can create readable reports. The report generated by LegacyXmlReportGeneratingListener is not readable and useless because it exists of one big "TEST" (it even looses the test class names).
Steps to reproduce
Run
with multiple test classes. You just get one big TEST-junit-jupiter.xml instead of a TEST-nnn.xml for each class.
Context
The text was updated successfully, but these errors were encountered: