line coverage percentage affected by certain forms

[jasonracey]

It appears that certain forms are automatically counted as being covered. This might be because these cases do not contain any logic, but the fact that they are used to calculate lines covered % can generate a misleading result. Here's an example of how to get cloverage to report 100% line coverage when there are no tests:

(ns cloverage-check.math)
(defn add-one [arg] (+ 1 arg))

Forms covered: 33.33 %
Lines covered: 100.00 %

I've put together a couple of use cases here:
https://github.com/jasonracey/cloverage-check

[lvh]

What would the correct behavior here be? This doesn't feel wrong -- it just feels that line coverage is not a great measuring stick of coverage :)

[JacekLach]

To me the real issue here is that cloverage doesn't distinguish between code hit by the file being evaluated, and code hit by the tests.

If we assumed that all test code either lives in separate namespaces (which is common), you could work around using some kind of global flag that ignores instrumentation hits during initial eval, and then is disabled before tests are ran. This seems fragile though, as the failure mode if the convention is not upheld is unintuitive.

Another possible approach is to hook into clojures namespace loading mechanism, insturmenting them as they get requested by test namespaces, rather than preloading them. This is in theory more correct, but could be tricky to get right - it threatens to become recursive, which means the bits of state that cloverage uses to track what it's currently instrumenting get more complicated.

[lvh]

It might be useful to be able to differentiate between code running in different contexts, agreed. I'm not super sure that it's wrong for Cloverage to report this as-is though: form coverage correctly reports the miss, and line coverage correctly reports that line was looked at at least once. In particular, e.g. if you have a defn with a missing parameter declaration or something, it'd still fail, right?