per file coverage threshold

[graingert]

use cases:

• fail on unexecuted files
• starting to do unit tests in a big project, overall coverage is low - but we can increase per file coverage more easily

[nedbat]

Can you say more about what you want this to do? What would the user experience be?

[graingert]

--fail-under-file 0 would return exit code 2 for any unexecuted files

[graingert]

--fail-under-file 10 would return exit code 2 for any files with individual coverage less than 10%

[nedbat]

Thanks, that makes it clear.

I'm not sure how this would help for your second case ("starting to do unit tests in a big project"): coverage would fail for a very long time, until you managed to get at least 10% (or whatever) coverage in every single file. That seems like it would be discouraging, and push you toward the wrong metric.

[cbernecker]

Is there any ongoing action to implement --fail-under-file 10 ? I guess this would be a big benefit for the most projects. Because with that feature you can check which developer doesn't did his test homework.

[wshaikh]

Let's say I have 10 files, of which 9 of them are 100%, and one of them is 0%, and I set my limit to 90% coverage. Currently, it will pass because it's taking the average, but I don't want it to pass. That one file has below my coverage percentage of 90% so it should fail.

When can we expect this feature?

[nedbat]

See also #717, which is similar.

[nedbat]

One option while waiting for coverage.py to add this as a feature: implement it as a separate tool. You can get a JSON report from coverage.py, and then check the totals for each file. This would be a way to experiment with different styles of rules also ("tests/" must have 100%, "project/" must have 90%, or whatever).

[nedbat]

I've written a proof-of-concept using the JSON report: https://github.com/nedbat/coveragepy/blob/master/lab/goals.py

Try it and let me know what you think.