Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add JUnit XML output #1748

Merged
merged 15 commits into from
Nov 1, 2020
Merged

Add JUnit XML output #1748

merged 15 commits into from
Nov 1, 2020

Conversation

nohwnd
Copy link
Member

@nohwnd nohwnd commented Oct 30, 2020

PR Summary

Fix #1515

Adding JUnit option to XML export.

PR Checklist

  • PR has meaningful title

  • Summary describes changes

  • PR is ready to be merged

    • If not, use the arrow next to Create Pull Request to mark it as a draft. PR can be marked Ready for review when it's ready.
  • All tests pass

    • Run ./build.ps1 -Clean; ./test.ps1 -NoBuild. Use a new PowerShell process when C# code is changed.
  • Tests are added/update (if required)

  • Documentation is updated/added (if required)

    • Configuration object + Invoke-Pester help has typos. They list possible (and default) values as NUnit2.5 and not NUnitXml (and soon JUnitXml) which are the only one actually values allowed.
  • Add Export-JUnitReport function

  • Exercise the Export-JUnit report via Invoke-Pester with simple interface

  • Exercise the Export-JUnit report via Invoke-Pester with config interface

  • (maybe) Update parameter names (action and node -> block, testresult) to match nunit better <- I give up on this

  • Copy the path-switch from nunit to get path/name for scriptblocks

  • Use ExpandedPath where needed

  • Since it doesn't care about contexts (nested test-suite are not allowed), maybe use Fold-Block to simplify current testcases-lookup.

  • Add stack trace as inner text at failure-element

@nohwnd nohwnd marked this pull request as draft October 30, 2020 12:30
@nohwnd nohwnd changed the title Add JUnit output on par with Pester 4 Add JUnit XML output Oct 31, 2020
@nohwnd nohwnd marked this pull request as ready for review October 31, 2020 11:58
@nohwnd
Copy link
Member Author

nohwnd commented Oct 31, 2020

@fflaten wanna review?

@fflaten
Copy link
Collaborator

fflaten commented Oct 31, 2020

@fflaten wanna review?

Sure. Will look through it later today.

src/Pester.ps1 Outdated Show resolved Hide resolved
src/functions/TestResults.ps1 Outdated Show resolved Hide resolved
src/functions/TestResults.ps1 Outdated Show resolved Hide resolved
src/functions/TestResults.ps1 Outdated Show resolved Hide resolved
src/functions/TestResults.ps1 Outdated Show resolved Hide resolved
@fflaten
Copy link
Collaborator

fflaten commented Oct 31, 2020

Testsuitelement (container) is not showing skipped tests correctly. Returns skipped="0" disabled="" in a run with 2 tests filtered out using -ExcludeTagFilter "SomeContextTag". Parent-element (testsuites) showed name="Pester" tests="7" errors="0" failures="3" disabled="2" time="1.117"> as expected.

Write-JUnitTestSuiteAttributes to $XmlWriter.WriteAttributeString('disabled', $Result.NotRunCount + $Result.SkippedCount) ?

In what scenario is SkippedCount increased btw? NotRunCount was the only one with 2 in my result-object.

@fflaten
Copy link
Collaborator

fflaten commented Oct 31, 2020

From issue:

Also noticed that nunitxml uses the DisplayedErrorMessage-property for failuremessage. That causes file:line to be shown both in failuremessage and stack trace. Maybe strip out "at item:line" to avoid duplicate info the in test report?

This isn't addressed. Should it be? Currently returns stacktrace twice, see #1515 (comment)

@nohwnd
Copy link
Member Author

nohwnd commented Nov 1, 2020

From issue:

Also noticed that nunitxml uses the DisplayedErrorMessage-property for failuremessage. That causes file:line to be shown both in failuremessage and stack trace. Maybe strip out "at item:line" to avoid duplicate info the in test report?

This isn't addressed. Should it be? Currently returns stacktrace twice, see #1515 (comment)

I saw that, and did not address it because I would have to fix the nunit output as well. At the moment I would like to avoid goldplating the functionality. IIRC it won't return stack trace twice, the display message contains the first line of the stack trace, but nothing more. Stacktrace then contains the whole stack trace.

@fflaten
Copy link
Collaborator

fflaten commented Nov 1, 2020

Good job. What about the missing "disabled" attribute in testsuite?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Add JUnit back in
2 participants