New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add --fail-under-line and --fail-under-branch options to return exit status 2/4/6 under a coverage threshold #173

Merged
merged 1 commit into from Jan 9, 2018

Conversation

Projects
None yet
3 participants
@mayeut
Contributor

mayeut commented Jul 22, 2017

This might be useful to fail tests if coverage is not high enough.

@denniswjackson

This comment has been minimized.

denniswjackson commented Jul 24, 2017

Making this --fail-under-lines allows an option for branch coverage as well.

@mayeut mayeut force-pushed the mayeut:fail-under branch from 0630135 to 0ddcde6 Jul 25, 2017

@mayeut mayeut changed the title from Add --fail-under option to return exit status 2 under a coverage threshold to Add --fail-under-line and --fail-under-branch options to return exit status 2 under a coverage threshold Jul 25, 2017

mayeut added a commit to mayeut/websockets that referenced this pull request Aug 2, 2017

C extension coverage with tox
tox.ini being unix-centric since the inclusion of the C extension,
we can do C extension coverage but without validation for now
c.f. gcovr/gcovr#173

mayeut added a commit to mayeut/websockets that referenced this pull request Aug 2, 2017

C extension coverage with tox
tox.ini being unix-centric since the inclusion of the C extension,
we can do C extension coverage but without validation for now
c.f. gcovr/gcovr#173

mayeut added a commit to mayeut/websockets that referenced this pull request Aug 15, 2017

C extension coverage with tox
tox.ini being unix-centric since the inclusion of the C extension,
we can do C extension coverage but without validation for now
c.f. gcovr/gcovr#173
@latk

This comment has been minimized.

Member

latk commented Jan 7, 2018

I like this enhancement very much. Only two things are missing:

  1. Please split the unrelated whitespace changes into a separate commit. Such improvements are good, but not when intermingled with important changes. To do so:

    • switch to your branch: git checkout fail-under
    • soft-reset to the state before this commit: git reset HEAD^. Your workdir is now dirty with the changes from this commit.
    • add the whitespace changes interactively: git add -p --all. You will be asked for each change whether you want to stage it.
    • verify that all whitespace changes are staged and no other changes were staged by running git diff and git diff --staged. To restart the staging, run git reset.
    • commit the staged whitespace changes: git commit
    • commit the actual changes: git commit --all
    • force-push the changes to update this PR: git push --force-with-lease
  2. Test cases. You can copy any suitable existing test case and adapt the copy to your needs. We'll have to add three new cases:

    • one where the coverage for both criteria is exceeded. This test case needs no changes other than adding the options to the gcovr invocations in the Makefile.
    • one where the branch coverage fails. Relative to the success-case, we need to raise the coverage limit and then test the exit status after completion: gcovr ... ; test $? = 2 or something like that.
    • one where the line coverage fails. Equivalent changes to the previous case.

Please let me know if you need any help, e.g. with running the test suite.

Related PRs:

  • your #175 (merged) fixes the Travis tests. Consider rebasing your changes onto master so that the automated build for this PR succeeds.
  • my #185 (pending) also affects the tests. I'll have to add the necessary changes for whatever PR is merged second.

@latk latk referenced this pull request Jan 8, 2018

Closed

added coverage_min argument #116

@mayeut mayeut force-pushed the mayeut:fail-under branch 2 times, most recently from 21a6770 to 3cc1f71 Jan 8, 2018

@mayeut mayeut force-pushed the mayeut:fail-under branch from 3cc1f71 to df9fc47 Jan 8, 2018

@mayeut

This comment has been minimized.

Contributor

mayeut commented Jan 8, 2018

  1. addressed by rebasing on master after #187
  2. Done in simple1 txt test

Exit code now depends on which test fails:
2 => line coverage failed
4 => branch coverage failed
6 => both failed

@mayeut mayeut changed the title from Add --fail-under-line and --fail-under-branch options to return exit status 2 under a coverage threshold to Add --fail-under-line and --fail-under-branch options to return exit status 2/4/6 under a coverage threshold Jan 8, 2018

@latk

latk approved these changes Jan 9, 2018

@latk latk merged commit 3cf2cd4 into gcovr:master Jan 9, 2018

1 check passed

continuous-integration/travis-ci/pr The Travis CI build passed
Details

@mayeut mayeut deleted the mayeut:fail-under branch Jan 9, 2018

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment