Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add support for NUnit3 XML-reports #2208

Merged
merged 22 commits into from May 6, 2023
Merged

Add support for NUnit3 XML-reports #2208

merged 22 commits into from May 6, 2023

Conversation

fflaten
Copy link
Collaborator

@fflaten fflaten commented Jul 11, 2022

PR Summary

Add initial support for NUnit3-compatible XML-reports for test results.

Features:

  • Support standard output
  • Use UTC time (string format 'o')
  • Distinguish Context vs Describe using a _TYPE property
    • Note: Only available during Invoke-Pester, not in Export/ConvertTo-NUnitReport
  • Add tags as properties
  • Add Data as properties (when Data/TestCases/ForEach is dictionary)
  • Support site-attribute to identify BeforeAll/AfterAll vs Test-failure
  • Support Assembly test-suite per testfile. Enables filter on test file in Azure DevOps etc.
    • Add duplicate environemnt-elements (one per assembly), but worth it
  • Support auto-parameters for both parameterized blocks and tests
    • Ex. name="TestBlock(1,2,3)" when block or test name doesn't use variables
    • Note: Only supported when -ForEach/TestCases are used with dictionary-values

Limitations:

  • name is still full path for test-case-elements.
    • Workaround because Reports/CIs like Azure DevOps doesn't use fullname-attribute as intended for NUnit3. See comment

TODO:

  • Tested with Azure DevOps Publish Task (most/all features)
  • Tested with ReportUnit
  • Tested with other CIs? Which? Who can test?

PR Checklist

  • PR has meaningful title
  • Summary describes changes
  • PR is ready to be merged
    • If not, use the arrow next to Create Pull Request to mark it as a draft. PR can be marked Ready for review when it's ready.
  • Tests are added/update (if required)
  • Documentation is updated/added (if required)

Fix #1087
Fix #1879
Fix #1284
Related #2039
Related #1530
Related #301
Related #2143

src/functions/TestResults.NUnit3.ps1 Fixed Show fixed Hide fixed
src/functions/TestResults.NUnit3.ps1 Fixed Show fixed Hide fixed
src/functions/TestResults.NUnit3.ps1 Fixed Show fixed Hide fixed
src/functions/TestResults.NUnit3.ps1 Fixed Show fixed Hide fixed
src/functions/TestResults.NUnit3.ps1 Fixed Show fixed Hide fixed
@fflaten fflaten added this to the NUnit 3 milestone Mar 18, 2023
@fflaten
Copy link
Collaborator Author

fflaten commented Mar 20, 2023

/azp run

@azure-pipelines
Copy link
Contributor

Azure Pipelines successfully started running 1 pipeline(s).

Reverts commit 96908c3 . Assembly-suite are shown as test file
in reports (ex. Azure DevOps) which are usefull.
Caching result of Get-RunTimeEnvironment to reduce perf impact
of duplicated environment-elements.
@fflaten
Copy link
Collaborator Author

fflaten commented Mar 22, 2023

/azp run

@azure-pipelines
Copy link
Contributor

Azure Pipelines successfully started running 1 pipeline(s).

also extracts suite site-attribute calculation to own function
@fflaten
Copy link
Collaborator Author

fflaten commented Mar 22, 2023

I think this is finally ready for review and a experimental release.

I regret self-assigning this task. The only thing worse than no official schema is schemas that doesn't match docs nor the original product's output. 😄

@fflaten fflaten marked this pull request as ready for review March 22, 2023 22:12
@fflaten
Copy link
Collaborator Author

fflaten commented Mar 22, 2023

ReportUnit 1.5 seems to use name on suites also, so might need to use fullname as workaround there as well. Unlike test-case there isn't another attribute we can place the name is for those who need it, so waiting on feedback/requests before changing it.

@fflaten fflaten changed the title WIP: Add support for NUnit3 XML-reports Add support for NUnit3 XML-reports Mar 22, 2023
@nohwnd nohwnd merged commit d318bdd into pester:main May 6, 2023
14 checks passed
@fflaten fflaten deleted the nunit3 branch May 6, 2023 23:43
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
2 participants