Goal Zero: Turn on branch coverage for pytest-cov

[sztamas]

Using --cov-branch will reveal a few branches that aren't reached when pytesting the project.

Just to state the obvious, having a less than 100% coverage shows that the tests don't cover all the code-base for sure, but the reverse isn't true - having 100% coverage doesn't mean that the test coverage is good enough. Covering all the lines and branches, doesn't mean that your tests are verifying all the possible outcomes.

To illustrate just for fun :-)

Given this function:

def rest(xs: list):
    if xs is None or len(xs) <= 1:
        return []
    return xs[1:]

Running this test will report a 100% branch coverage:

def test_rest():
    assert rest([]) == []
    assert rest([1, 2]) == [2]

Although there is no test for passing in None, a list with 1 element etc.

Or to make the example more extreme this is 100% as well:

def test_rest():
    rest([]) == []
    rest([1, 2]) == [2]

While removing the assertions completely looks silly, it highlights the problem that the test is as good as your assertions are.

[timothycrosley]

I'm open to changing this, but only after the project achieves 100% branch coverage (which may be in reach after the other testing tickets are resolved). The reason being that the 100% coverage level (currently calculated by lines) acts as very recognizable and easily understood quality bar for all PRs and the project itself. Once the coverage goes below that, it's easier to be complacent. If we can get to 100% branch coverage, all the better! I'd be happy to raise the bar to that level.

[timothycrosley]

This is finally implemented in the main branch 🎉