Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

PowerShell - Added coverage collector #59009

Draft
wants to merge 3 commits into
base: devel
from

Conversation

Projects
None yet
3 participants
@jborean93
Copy link
Contributor

commented Jul 12, 2019

SUMMARY

Added a wrapper that will collect code coverage of modules and module_utils.

TODO:

  • Integration into ansible-test to set the _ANSIBLE_COVERAGE_OUTPUT env var and retrieve the coverage files once complete
  • Look into improving performance in any way possible.
ISSUE TYPE
  • Feature Pull Request
COMPONENT NAME

ansible-test
powershell

@ansibot

This comment has been minimized.

Copy link
Contributor

commented Jul 12, 2019

@ansibot ansibot added core_review and removed needs_revision labels Jul 15, 2019

@jborean93

This comment has been minimized.

Copy link
Contributor Author

commented Jul 15, 2019

I've had a look into the performance problems and have been able to improve the time it takes for a module to run. Still not the same as a non coverage run but the majority of the extra time is now taken up by internal functions we have no control over.

I have also looked into Set-PSDebug to see if line tracing is a better candidate for coverage collection but it is not feasible because;

  • The scope of the trace levels are set on a script level, we cannot set it to the pipeline/runspace globally and therefore would have to edit the module code to enable it
  • The output is just a string, it is easily parsible but extra work and not sure if the format is guaranteed in future versions
  • I could not find a way to collect coverage for PowerShell module util code (.psm1) so this would only be good for our Windows modules
  • After tweaking the performance a bit for the breakpoint method, the biggest obstacle is no longer there

Now that I am somewhat happy with the actual coverage collection process, I need to figure out the best way to integrate that into ansible-test and generate the code coverage xml report to be sent to Code Cov. The following questions need to be answered;

  • How can we integrate the PowerShell coverage info into the xml generation process
  • Can we somehow report on line branches coverage (if statement evaluation)
  • Should we even use the .xml or just use the basic CodeCov .json format
  • When fetching the coverage results back should that be merged into the 1 file or kept separate like we do with test/reulsts/coverage/*

@bcoca bcoca removed the needs_triage label Jul 16, 2019

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.