Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Code coverage threshold does not fail if no coverage generated #1083

Closed
martincostello opened this issue Feb 10, 2021 · 3 comments · Fixed by #1115
Closed

Code coverage threshold does not fail if no coverage generated #1083

martincostello opened this issue Feb 10, 2021 · 3 comments · Fixed by #1115
Assignees
Labels
bug Something isn't working driver-console Issue related to dotnet net tool driver driver-msbuild Issue related to msbuild driver Priority:0 Critical to the release untriaged To be investigated

Comments

@martincostello
Copy link
Contributor

If coverlet.msbuild is configured with a Threshold value for a minimum code coverage required by tests, but then no coverage is output for coverlet for some reason, then rather than fail the build, it instead passes because the total coverage is given as 100%.

+--------+------+--------+--------+
| Module | Line | Branch | Method |
+--------+------+--------+--------+

+---------+------+--------+--------+
|         | Line | Branch | Method |
+---------+------+--------+--------+
| Total   | 100% | 100%   | 100%   |
+---------+------+--------+--------+
| Average | NaN% | NaN%   | NaN%   |
+---------+------+--------+--------+

You can also see the average is reported as NaN%.

See reactiveui/refit#1073 for a scenario where this behaviour masked an issue with code coverage generation because tests still passed.

Coverlet should fail the build and draw the developer's attention to the issue, rather than give a false impression that code coverage is being maintained above previous thresholds configured by the repository.

@MarcoRossignoli
Copy link
Collaborator

Thanks for reporting this.

@MarcoRossignoli MarcoRossignoli added the untriaged To be investigated label Feb 10, 2021
@MarcoRossignoli MarcoRossignoli added Priority:1 Very important to release, not a ship blocker Priority:0 Critical to the release and removed Priority:1 Very important to release, not a ship blocker labels Feb 21, 2021
@daveMueller
Copy link
Collaborator

I start to work on this.

@daveMueller daveMueller self-assigned this Mar 6, 2021
@MarcoRossignoli
Copy link
Collaborator

Thx Dave!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working driver-console Issue related to dotnet net tool driver driver-msbuild Issue related to msbuild driver Priority:0 Critical to the release untriaged To be investigated
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants