Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Windows build produces much higher coverage results than macOS. #239

Closed
parekhkb opened this issue Nov 15, 2018 · 2 comments
Closed

Windows build produces much higher coverage results than macOS. #239

parekhkb opened this issue Nov 15, 2018 · 2 comments
Labels
needs more info More details are needed

Comments

@parekhkb
Copy link

parekhkb commented Nov 15, 2018

I am running dotnet test on multiple test projects and attempting to merge the output.

dotnet test "/Users/parekhkb/UnitTests/ProjectA.Test/Xxx.ProjectA.Test.csproj" --configuration Debug --no-build --no-restore /property:CollectCoverage=true /property:CoverletOutput=/Users/parekhkb/out/CodeCoverageResult/CoverageReport /property:CoverletOutputFormat=\"json,opencover\" /property:Exclude=\"[Xxx*.Test*],[Xxx.Data.Database.*],[Xxx.*Data*.Migrations.*]\" 

dotnet test "/Users/parekhkb/UnitTests/ProjectB.Test/Xxx.ProjectB.Test.csproj" --configuration Debug --no-build --no-restore /property:CollectCoverage=true /property:CoverletOutput=/Users/parekhkb/out/CodeCoverageResult/CoverageReport /property:CoverletOutputFormat=\"json,opencover\" /property:Exclude=\"[Xxx*.Test*],[Xxx.Data.Database.*],[Xxx.*Data*.Migrations.*]\" 
/property:MergeWith=/Users/parekhkb/out/CodeCoverageResult/CoverageReport.json

On macOS I get a result of 22.4% line coverage. But the same run on windows produces 72.5% line coverage.

UPDATE:
When I run the same set of runs without the MergeWith, I get drastically lower coverage results, which gives me new information. My final test run in the list is our Integration test suite. In that suite we setup our async worker background process, we then make web requests against a TestServer (in-process) and verify data directly in the DB. This is where a bulk of our test coverage comes from.

On windows, instrumentation of these the web server and the background worker seems to work fine, although they produce slightly less coverage than DotCover or OpenCover. But on macOS these tests result in 0% coverage.

@parekhkb parekhkb changed the title MergeWith on Windows works, but on macOS it seems to overwrite the output. Windows build produces much higher coverage results than macOS. Nov 15, 2018
@MarcoRossignoli MarcoRossignoli added the needs more info More details are needed label Sep 12, 2019
@MarcoRossignoli
Copy link
Collaborator

But on macOS these tests result in 0% coverage.

Could be a dup of #110 can you try with collectors?

@MarcoRossignoli
Copy link
Collaborator

Close for stale, feel free to re-open if needed.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
needs more info More details are needed
Projects
None yet
Development

No branches or pull requests

2 participants