Something wrong about branch coverage of class statements

[nedbat]

Originally reported by Jean-Paul Calderone (Bitbucket: exarkun, GitHub: exarkun)

---

Consider the attached module. Using "coverage run --branch" on it and then "coverage html" produces a coverage report which indicates two partially covered lines.

I would expect that all three methods in the example would be treated the same (ie, either that all three would be marked partially covered, or all three would be marked completely covered).

---

• Bitbucket: https://bitbucket.org/ned/coveragepy/issue/62
• This issue had attachments: foo.py. See the original issue for details.

[nedbat]

Branch coverage gets a bit tricky. I can explain why the coverage report looks the way it does. When analyzing code paths, coverage.py builds a list of "arcs": pairs of line numbers that are possible transitions during execution.

In your code:

class X:
	def foo(self):
		"hello"

	def bar(self):
		"world"

	def baz(self):
		"..."

these are the arcs:

[(-1, 1), (-1, 2), (-1, 5), (-1, 8), (1, -1), (1, 2), (2, -1), (2, 5), (5, -1), (5, 8), (8, -1)]

In these arcs, a -1 is used specially: as the first element in the pair, it means execution can enter at the line number given as the second element. If a -1 is the second element in the pair, it means execution can leave a context at the line number given as the first element.

So for example, (-1, 5) means that when executing the bar function, execution enters at line 5, and the (5, -1) arc means that execution also leaves at line 5.

When actually executing the code, the lines executed are 1, then 2, then 5, then 8, so the arcs actually encountered during execution are (-1, 1), (1, -1), (1, 2), (2, 5), (5, 8), (8, -1). The possible arcs not encountered were (-1, 2), (-1, 5), (-1, 8), (2, -1), (5, -1). Lines missing branches are those named in the first elements of the missing arcs, so there are missing branches on 2 and 5.

Look at the possible arcs for line 2: (2, -1) and (2, 5). The first is what would happen if you executed foo(). The second is what happens when you define foo. The problem is that for line 8, both execution and definition result in an arc of (8, -1). Execution runs from line 8 to the exit of the function, definition runs from line 8 to the exit of the class. There are really two possibilities, but they are both represented by -1, so the set of arcs doesn't include all the information it could.

When the program is run, we get an arc of (8, -1), so there are no arc possibilities for line 8 that are missing from the execution data, so 8 is not shown as missing a branch.

Now you can help me: is this just a hypothetical example, or do you really have code like this? Function defined with only docstrings are unusual.

[nedbat]

Original comment by Jean-Paul Calderone (Bitbucket: exarkun, GitHub: exarkun)

---

The real code that triggers this issue is doing interface definitions based on zope.interface. For example, http://buildbot.twistedmatrix.com/builds/twisted-coverage.py-r28849/twisted_internet_interfaces.html. I don't think this is the only case that can trigger this problem, though. Methods with only docstrings are rare outside of interface definitions, but certainly not impossible.

[nedbat]

Thanks for the real code example. I can see a way forward to make all the methods be marked as missing a branch, but I'm not sure that's what you want.

I would guess that you don't want coverage to draw your attention to these classes at all, since you know they are not executed, and are not going to be adding tests that will execute them.

You can keep coverage from mentioning them by excluding these classes from the coverage measurement. The best way would be to create a coverage.ini file, and use a regex to exclude any class derived from Interface:

# coverage.ini to control how coverage.py works
[report]
exclude_lines =
    # pragma: no cover
    class \w+\(Interface\):
    raise AssertionError

(I included "pragma: no cover" since I assume you still want that, and "raise AssertionError" since those statements, if they exist, are never meant to be executed, though you may have tests that do exercise those lines.)

If you use this coverage.ini file, then your interface classes will never be highlighted as missing execution.

[nedbat]

OK, I've changed coverage.py to record the multiple exits separately. Fixed in <<changeset cb4e94096b20 (bb)>>.

[nedbat]

Original comment by Tom Prince (Bitbucket: tomprince, GitHub: tomprince)

---

Excluding class \w+\(Interface\): will catch many but not all of the instances of this, since we have quite a few case of one interface inheriting from another (instead of from Interface directly.

Having looked at the coverage code, I don't see any obvious way to catch these instances, though.

Is there perhaps some way we could find them by inspecting the modules, and then telling coverage to ignore them?

[nedbat]

Tom: I marked this as fixed three years ago. Are you still seeing problems? If so, can you provide a code example that demonstrates the problem?

[nedbat]

Original comment by Tres Seaver (Bitbucket: tseaver, GitHub: tseaver)

---

I just stumbled across this myself today, while trying to assure that 'zope.interface' itself has 100% branch coverage. E.g.:

$ git clone https://github.com/zopefoundation/zope.interface.git
$ tox -e coverage # doesn't do branches
$ .tox/coverage/bin/nosetests --with-coverage --cover-branches

All the "branches" that get marked are empty "methods" of interfaces (defined using the 'class' statement, but with a different metaclass than 'type'): their suites consist purely of docstrings.

(Using 'coverage-4.0a1-py27').

[nedbat]

Original comment by Tristan Seligmann (Bitbucket: mithrandi, GitHub: mithrandi)

---

I'm still seeing this issue; the interface methods are marked "partially covered" since the body is never executed (and essentially can never be executed, there's no way to call it).