Skip to content
This repository has been archived by the owner on Aug 12, 2021. It is now read-only.

Added very basic JUnit output #16

Open
wants to merge 1 commit into
base: master
Choose a base branch
from
Open

Conversation

Gilnaa
Copy link
Contributor

@Gilnaa Gilnaa commented Apr 25, 2019

This PR adds a very basic JUnit formatter, and should probably not be merged without further discussion.

This output format is widely used, but sadly is not standardized.
The best summary I could find is was in Catch's website.

Some talk points:

  • Some data and attributes are currently stubbed, such as the test-suite and class names.
  • It is assumed only one test suite exists. (I'm not sure this concept is relevant to libtest's tests).
  • Tests are given the run time "0" as they are currently not measured by the framework (benchmarks are given real time).
  • It seems as if std-out/err are owned by the test-suite as far as JUnit is concerned, which is not the case in libtest. Should we concat the output of the [Failing?] testcases?
  • We should probably give command-line flags to set the test-suite name, or somehow infer it from cargo. (or both)
  • The chrono crate is used to format the timestamp in ISO8601, but if the dependency is unwanted, I can always rewrite this one function.
  • Ideally I would want to test this feature either by using the junitparser python package or by validating the output using a JUnit crate, but I'm not sure if that's the best solution. Maybe I should just compare it to a known good output.

Thoughts?

@calebcartwright
Copy link
Member

Just wanted to say that I'm excited about JUnit format potentially being added!

FWIW, In my experience the JUnit result files emitted by cargo-junit seem to be "correct", at least in the sense that various CI engines are able to correctly parse/understand those JUnit files (albeit with missing timestamps). Sharing in case that's of any use as a comparison point

@Gilnaa
Copy link
Contributor Author

Gilnaa commented May 8, 2019

Thank you! I'll make sure to compare those

@Alxandr
Copy link

Alxandr commented Jul 26, 2019

I'm curious as to why the timings for the tests aren't recorded? Most test frameworks I've worked with reports duration of tests OOTB, and I don't see a good reason for libtest to not do so.

@gnzlbg
Copy link
Contributor

gnzlbg commented Jul 27, 2019 via email

@iostat
Copy link

iostat commented Jan 25, 2020

+1 on this! I'd love to see this PR merged, especially since we have --report-time which seems sufficient to report test times and TrResult captures stdout (maybe expandable to stderr as well?). I was gearing up to write yet another own cargo subcommand (as cargo-junit seems to just hang with my nightly, and cargo-suity doesn't handled ignored on the crates.io published version), but this is a much cleaner way to go, and would go far in integrating w/ Jenkins and other established CI tooling. Any way I can help this PR move through?

@Gilnaa
Copy link
Contributor Author

Gilnaa commented Jan 25, 2020

Testing the output against some CI framework will be nice, but I think that this is repo is semi-dead for some reason. (no commits or merges since april)

We should probably find the maintainer somewhere else.

Edit: In another PR it is mentioned that this repo is actually not the crate that is distributed with Rust.
I have no idea why this isn't mentioned in the README

@iostat It's probably needed to retarget this PR against the rustc codebase, but unfortunately I no longer have leisure to do so. If you could do it, I think it would be great.

@Progdrasil
Copy link

+1 on this as well. I'm eagerly awaiting a built in exporter to junit rather than an ad-hoc one.

@svenstaro
Copy link

Hey guys, any chance we can get this effort going again? I think it'd be quite convenient to have this in.

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

7 participants