Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add CTRF report format #2942

Closed
bradwilson opened this issue Jun 3, 2024 · 2 comments
Closed

Add CTRF report format #2942

bradwilson opened this issue Jun 3, 2024 · 2 comments

Comments

@bradwilson
Copy link
Member

Requested here: #1052 (comment)

CTRF documentation: https://ctrf.io/docs/intro

@bradwilson
Copy link
Member Author

This adds the CTRF report format to the v3 console runner (via -ctrf switch) and v3 MSBuild runner (via Ctrf attribute).

This is currently not being considered for v2 back-port because it heavily depends on the JSON serializer that's part of xUnit.net v3 (and the v3 runners can run v2 tests, thereby creating CTRF reports).

Everything from XML v2 mapped fairly well into CTRF, with some additions:

  • Under extra for environment, I added the computer name and user name that ran the tests, and then listed each individual assembly as a "suite" (to use the CTRF name); under each assembly, I listed the test collections as well.
  • Under extra for tests, I added test ID,the test collection ID the test belongs to, the CLR type and method names of the test, and all the traits.
  • I also special-cased traits named Category and placed them into the official tags element for the test, as this seemed like the most common term of art for categorizing xUnit.net tests (as it matches up with the fact that both NUnit and MSTest include a [Category] attribute). So if you have [Trait("Category", "Foo")] and [Trait("Category", "Bar")] on a test, the tags element is rendered as tags: ["Foo", "Bar"].

From a file size perspective, the existing XML v2 report is pretty much always smallest. The HTML and CTRF seem very similar in size, which varies from 0-50% larger than the XML v2 report. The TRX file size is generally 200%+ larger than the XML v2 report, so the CTRF is most definitely a more compact test framework independent format:

image

@bradwilson
Copy link
Member Author

Available in v3 0.1.1-pre.474 https://xunit.net/docs/using-ci-builds

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

1 participant