Generate JUnit/XML test report

[fchauvel]

Hi,

I added an option -j (or --junit-report) that serializes a GreenTestResult into an JUnit/XML file. I think this addresses Issue #104, a feature I also need.

I added a new module reporting.py, which contains an JUnitXML class. I then import this class in the cmdline.py module, once tests are run. A few preliminary unit tests are in the module test.test_reporting.py. Note that I had to comment out the deletion of captured output, once they have been displayed (see result.py) otherwise they are no longer available and I cannot include them in the report. The tests seems to pass (on my machine), but there may be consequences I do not foresee.

This implies a new dependency to lxml 4.3.3, which I use to generate the XML tree.

Does it sounds like a good approach? I can refine or adjust as needed.

[CleanCut]

Thank you, what a great feature! Take a look at my requested changes and let me know what you think.

[fchauvel]

Thanks for looking at my code. I implemented the changes you suggested. There are other features I'd like to implement, including:

• Reporting the time individually spent on each test. I haven't understood whether Green measures this.
• Reporting the time spent per test suites.
• Reporting the environment used to tests (python version, os, etc.)
• Adding count of failing, skipped, erroneous tests per test suite. Seems straightforward.

What do you think?

[CleanCut]

I'm certainly open to your ideas (though I suggest they should each come in separate pull requests!). Given that I have never had anyone request these features, I would definitely want to focus on keeping the code fairly clean and making the new output optional in most cases. You did a good job on both of those in this PR. 😄

Green does not currently do any time measurements other than the one from start-to-finish of the entire run. You could add more timing measurements and output as a new option. Same with more fine-grained count summaries (I assume you mean per-module, because suites are a messy catch-all recursive construct and don't map one-to-one to modules at all). Where would you put the output? Are you thinking in-line with verbose output, or as a supplemental report after the run?

Oh, you should also be aware that Green runs tests simultaneously in multiple subprocesses. You'll want to take that into account if you start timing individual parts, because the times of tests from different processes will overlap. It would be really nice to get a sum of all processor time and combine that with the total time to produce a "speedup" measurement.

Some of the environment is output already if you do --version or lots of verbosity -vvv. That could certainly be expanded to include things like Operating System, etc.

[CleanCut]

✅ This shipped in 2.15.0, which was just released.