Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add --fail-under-line and --fail-under-branch options to return exit status 2/4/6 under a coverage threshold #173

Merged
merged 1 commit into from
Jan 9, 2018

Conversation

mayeut
Copy link
Contributor

@mayeut mayeut commented Jul 22, 2017

This might be useful to fail tests if coverage is not high enough.

@denniswjackson
Copy link

Making this --fail-under-lines allows an option for branch coverage as well.

@mayeut mayeut changed the title Add --fail-under option to return exit status 2 under a coverage threshold Add --fail-under-line and --fail-under-branch options to return exit status 2 under a coverage threshold Jul 25, 2017
mayeut added a commit to mayeut/websockets that referenced this pull request Aug 2, 2017
tox.ini being unix-centric since the inclusion of the C extension,
we can do C extension coverage but without validation for now
c.f. gcovr/gcovr#173
mayeut added a commit to mayeut/websockets that referenced this pull request Aug 2, 2017
tox.ini being unix-centric since the inclusion of the C extension,
we can do C extension coverage but without validation for now
c.f. gcovr/gcovr#173
mayeut added a commit to mayeut/websockets that referenced this pull request Aug 15, 2017
tox.ini being unix-centric since the inclusion of the C extension,
we can do C extension coverage but without validation for now
c.f. gcovr/gcovr#173
@latk
Copy link
Member

latk commented Jan 7, 2018

I like this enhancement very much. Only two things are missing:

  1. Please split the unrelated whitespace changes into a separate commit. Such improvements are good, but not when intermingled with important changes. To do so:

    • switch to your branch: git checkout fail-under
    • soft-reset to the state before this commit: git reset HEAD^. Your workdir is now dirty with the changes from this commit.
    • add the whitespace changes interactively: git add -p --all. You will be asked for each change whether you want to stage it.
    • verify that all whitespace changes are staged and no other changes were staged by running git diff and git diff --staged. To restart the staging, run git reset.
    • commit the staged whitespace changes: git commit
    • commit the actual changes: git commit --all
    • force-push the changes to update this PR: git push --force-with-lease
  2. Test cases. You can copy any suitable existing test case and adapt the copy to your needs. We'll have to add three new cases:

    • one where the coverage for both criteria is exceeded. This test case needs no changes other than adding the options to the gcovr invocations in the Makefile.
    • one where the branch coverage fails. Relative to the success-case, we need to raise the coverage limit and then test the exit status after completion: gcovr ... ; test $? = 2 or something like that.
    • one where the line coverage fails. Equivalent changes to the previous case.

Please let me know if you need any help, e.g. with running the test suite.

Related PRs:

  • your Use pypy3.5-5.8.0 for testing #175 (merged) fixes the Travis tests. Consider rebasing your changes onto master so that the automated build for this PR succeeds.
  • my Fix HTML tests #185 (pending) also affects the tests. I'll have to add the necessary changes for whatever PR is merged second.

@latk latk mentioned this pull request Jan 8, 2018
@mayeut mayeut force-pushed the fail-under branch 2 times, most recently from 21a6770 to 3cc1f71 Compare January 8, 2018 22:32
@mayeut
Copy link
Contributor Author

mayeut commented Jan 8, 2018

  1. addressed by rebasing on master after Fix most PEP 8 warnings #187
  2. Done in simple1 txt test

Exit code now depends on which test fails:
2 => line coverage failed
4 => branch coverage failed
6 => both failed

@mayeut mayeut changed the title Add --fail-under-line and --fail-under-branch options to return exit status 2 under a coverage threshold Add --fail-under-line and --fail-under-branch options to return exit status 2/4/6 under a coverage threshold Jan 8, 2018
@latk latk merged commit 3cf2cd4 into gcovr:master Jan 9, 2018
@mayeut mayeut deleted the fail-under branch January 9, 2018 20:54
@latk latk mentioned this pull request Jan 10, 2018
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants