cmd/cover: go test -cover counts statements in init() as covered even if there aren't any tests

[ray-harris]

What version of Go are you using (go version)?

$ go version
go version go1.13.3 darwin/amd64

Does this issue reproduce with the latest release?

Yes

What operating system and processor architecture are you using (go env)?

go env Output

$ go env
GOARCH="amd64"
GOOS="darwin"

What did you do?

$ go mod init example.com/pkg/notests
go: creating new go.mod: module example.com/pkg/notests

$ cat > notests.go <<EOF
package notests

func init() {
	var _ = 1 + 1
}
EOF

$ cat > notests_test.go <<EOF
package notests
EOF

$ go test -cover

What did you expect to see?

testing: warning: no tests to run
PASS
coverage: [no statements]
ok  	example.com/pkg/notests	0.297s

What did you see instead?

testing: warning: no tests to run
PASS
coverage: 100.0% of statements
ok  	example.com/pkg/notests	0.297s

The purpose of test coverage is to ensure that all statements in a module are exercised by tests.

Statements that are run regardless of tests are usually ignored by test coverage. This includes global variable, constant, and type declarations.

I would argue that statements in an init function should be ignored by code coverage as well since users can't directly call the init function.

[straightdave]

In the preprocessing step, the builtin tool go cover would inject some code to count coverage.
It's not that smart and just injects counters to code blocks one by one, not statements.

Say there's a simple code like:

package main

import (
	"fmt"
	"time"
)

func init() {
	var _ = 1 + 1
}

func main() {
	fmt.Println("hello world")
}

After being processed the original source code would possibly become:

//line /Users/wei.wu/repos/go/bin/dave.go:1
package main

import (
	"fmt"
	"time"
)

func init() {
	GoCover_0_313465376561633966613762.Count[0]++
	var _ = 1 + 1
}

func main() {
	GoCover_0_313465376561633966613762.Count[1]++
	fmt.Println("hello world")
}

var GoCover_0_313465376561633966613762 = struct {
	Count   [2]uint32
	Pos     [3 * 2]uint32
	NumStmt [2]uint16
}{
	Pos: [3 * 2]uint32{
		8, 11, 0x2000d, // [0]
		13, 16, 0x2000d, // [1]
	},
	NumStmt: [2]uint16{
		1, // 0
		1, // 1
	},
}

[FiloSottile]

I feel like code in init function is in fact exercised by the tests, as supposedly it generate global state that the tests might use. Coverage is only about code being exercised, not about its results actually mattering.

[rayharris-ibm]

@FiloSottile
The issue is that it doesn't make sense to me that the code coverage reported would ever be greater than 0% if there are no tests.

From https://blog.golang.org/cover

Test coverage is a term that describes how much of a package's code is exercised by running the package's tests.

To me, a coverage report of anything greater than 0 implies that at least one test was run. This isn't a huge issue, but it can be misleading when coverage data is collected into a reporting system. I came across this issue while evaluating several code coverage reporting systems.

If there is at least one test, then I have no problem with the code in an init function being counted. But if there are no tests, how can test coverage be anything other than 0?

[rayharris-ibm]

@straightdave

In src/cmd/cover/cover.go, adding this case to the switch statement in func (f *File) Visit(node ast.Node) ast.Visitor eliminates instrumentation for main() and init() functions:

case *ast.FuncDecl:
        if n.Recv == nil && (n.Name.String() == "main" || n.Name.String() == "init") {
                return nil
        }

I haven't looked yet at what would need to be changed to keep the instrumentation, but to zero out the coverage percentage if there are no tests.

[straightdave]

🤣 I am working on a project which DOES want to record coverage in both main and init functions. So please don't make this change to the official cover tool...

[rayharris-ibm]

@straightdave If there are no tests, do you want to see a code coverage greater than zero, then?

[straightdave]

In the classic view, yes, code-coverage is related to testing. If no tests to run, there's no coverage.
But in my personal opinion, "coverage" just means how much code is touched during execution. It doesn't matter it is covered (touched) by a test or just normal execution.