testing: exclude statically unreachable code in test coverage

[santhosh-tekuri]

What version of Go are you using (go version)?

$ go version
go version go1.12.1 darwin/amd64

Does this issue reproduce with the latest release?

yes

What operating system and processor architecture are you using (go env)?

go env Output

$ go env
GOARCH="amd64"
GOBIN=""
GOCACHE="/Users/santhosh/Library/Caches/go-build"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="darwin"
GOOS="darwin"
GOPATH="/Users/santhosh/go"
GOPROXY=""
GORACE=""
GOROOT="/usr/local/go"
GOTMPDIR=""
GOTOOLDIR="/usr/local/go/pkg/tool/darwin_amd64"
GCCGO="gccgo"
CC="clang"
CXX="clang++"
CGO_ENABLED="1"
GOMOD=""
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fdebug-prefix-map=/var/folders/v5/44ncgqws3dg3pmjnktpt_l2w0000gq/T/go-build063045597=/tmp/go-build -gno-record-gcc-switches -fno-common"

i have code like this in my project:

const trace = false

func doSomething() {
    if trace {
        fmt.Println("some information")
    }
}

because trace is false, the coverage reports the tracing code as not covered
can such code be shown as "not tracked"

I enable trace if we are debugging some issue in our code using build tags
I can enable trace during coverage generation, but tracing output is large and it takes significant time.

I have seen types package from stdlib also uses if trace
so may be this can be taken as enhancement

[bcmills]

Flagging statically-unreachable code seems like a useful signal to me, and whether the compiler is able to determine statically that it is unreachable is an implementation detail.

100% code coverage is usually not an appropriate goal for Go projects. Can you give some more detail as to why this particular behavior is important?

[santhosh-tekuri]

I am not targeting for 100% coverage. the code coverage numbers does not reflect expected ones. the html report generated shows all tracing code as red. so when seeing reports, I have to mentally discard all those tracing lines of code highlighted in red.

currently I am doing a workaround of enable trace variable but do not generate any trace during tests suing build tags. this keeps code coverage track those lines.

[sam-github]

I was looking for a similar feature, not so much to reach 100% coverage, but because when reviewing coverage to determine if uncovered lines should be covered, its useful to have an annotation that says "not reachable, expected to be red". I guess I could literally put such a comment in my code, but I think it'd be pretty nice to get a report that said "92% covered, 6%uncovered, and 2% marked unreachable".

I think of uncovered code a bit like lint output or compiler warnings. Developers shouldn't have to reevaluate every time they checkout a project and run lint if the output is relevant. If its been decided that it is irrelevant, its handy to suppress it with in-line markup, and then nobody has to see the output again and then go look to see if its valid.

[maxim-ge]

@sam-github agree with your rationale, would be nice to have

[faraonc]

totally nice to have

[bcmills]

@maxim-ge, @faraonc: please see http://golang.org/wiki/NoPlusOne.

[kstenerud]

I'm not sure I understand the rationale behind "100% code coverage is usually not an appropriate goal for Go projects". What causes a project written in Go to need different coverage metrics than the same project written in another language such as Python or Java? Those languages support annotations to mark uncoverable regions (like untestable error handling), such that you can tell at a glance that 100% of your coverable code is in fact covered.

Uncovered code is code that has no evidence whatsoever that it does what it's supposed to under any circumstances, nor is it robust against regressions.

[bcmills]

@kstenerud, a project written in Go doesn't need different coverage metrics than the same project written in Java. (100% coverage is not usually an appropriate goal for Java projects either.)

However, a Go project usually does not require the same coverage as a project written in Python, unless the Python project is making extensive use of analyzers such as PyChecker or mypy. That is because you can rely more on the Go compiler itself to flag common issues on rare paths (such as misspelled variable names in panic statements). In that sense, the type-checker itself is a form of “test coverage”, and necessarily already covers 100% of lines — so if you're comparing to a Python project without an additional analyzer, the comparable Go coverage is effectively already 100%.

[jcollum]

(100% coverage is not usually an appropriate goal for Java projects either.)

100% code coverage is typically not worth the time investment required, regardless of language. I agree that higher code coverage is a good goal in non-typed languages.

All that being said, I think it would be helpful to mark some lines as ignored for code coverage purposes. It can be very difficult from a unit test perspective to get the code in to some states.

This ticket is over a year old and still open. That's a shame.

[kstenerud]

@bcmills Static type checking is not code testing. They are completely different things. If tests and coverage were only for type checking, then there'd be no need for unit tests in go at all.

The point of 100% test coverage with "unreachable" annotation support is so that you can run your coverage tool and see at a glance (or with CI rules) that 100% of the code that you WANT to have covered has in fact been covered. As it is now, you have to manually step through the entire report, checking every red line to make sure that it's a code path you actually don't intend to test, ON EVERY CHECKIN (and doing this automatically in your CI is impossible). This leads to human error, and libraries that aren't properly tested and will instead blow up in the wild on some edge case because getBaseFactor() deep in your library should have returned 5113 instead of 5118, but someone forgot to write a test covering it, and it got missed in the 10,000 line coverage report (that nobody reads anymore because they don't want to spend 20 minutes scanning it on every checkin).