Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

cmd/cover: go test -cover counts statements in init() as covered even if there aren't any tests #35289

Open
ray-harris opened this issue Oct 31, 2019 · 7 comments
Labels
Milestone

Comments

@ray-harris
Copy link

@ray-harris ray-harris commented Oct 31, 2019

What version of Go are you using (go version)?

$ go version
go version go1.13.3 darwin/amd64

Does this issue reproduce with the latest release?

Yes

What operating system and processor architecture are you using (go env)?

go env Output
$ go env
GOARCH="amd64"
GOOS="darwin"

What did you do?

$ go mod init example.com/pkg/notests
go: creating new go.mod: module example.com/pkg/notests

$ cat > notests.go <<EOF
package notests

func init() {
	var _ = 1 + 1
}
EOF

$ cat > notests_test.go <<EOF
package notests
EOF

$ go test -cover

What did you expect to see?

testing: warning: no tests to run
PASS
coverage: [no statements]
ok  	example.com/pkg/notests	0.297s

What did you see instead?

testing: warning: no tests to run
PASS
coverage: 100.0% of statements
ok  	example.com/pkg/notests	0.297s

The purpose of test coverage is to ensure that all statements in a module are exercised by tests.

Statements that are run regardless of tests are usually ignored by test coverage. This includes global variable, constant, and type declarations.

I would argue that statements in an init function should be ignored by code coverage as well since users can't directly call the init function.

@ray-harris ray-harris changed the title go test -cover counts statements in init() as covered even if there aren't any tests cmd/go: go test -cover counts statements in init() as covered even if there aren't any tests Nov 1, 2019
@ray-harris ray-harris changed the title cmd/go: go test -cover counts statements in init() as covered even if there aren't any tests cmd/cover: go test -cover counts statements in init() as covered even if there aren't any tests Nov 1, 2019
@straightdave
Copy link

@straightdave straightdave commented Nov 2, 2019

In the preprocessing step, the builtin tool go cover would inject some code to count coverage.
It's not that smart and just injects counters to code blocks one by one, not statements.

Say there's a simple code like:

package main

import (
	"fmt"
	"time"
)

func init() {
	var _ = 1 + 1
}

func main() {
	fmt.Println("hello world")
}

After being processed the original source code would possibly become:

//line /Users/wei.wu/repos/go/bin/dave.go:1
package main

import (
	"fmt"
	"time"
)

func init() {
	GoCover_0_313465376561633966613762.Count[0]++
	var _ = 1 + 1
}

func main() {
	GoCover_0_313465376561633966613762.Count[1]++
	fmt.Println("hello world")
}

var GoCover_0_313465376561633966613762 = struct {
	Count   [2]uint32
	Pos     [3 * 2]uint32
	NumStmt [2]uint16
}{
	Pos: [3 * 2]uint32{
		8, 11, 0x2000d, // [0]
		13, 16, 0x2000d, // [1]
	},
	NumStmt: [2]uint16{
		1, // 0
		1, // 1
	},
}
@FiloSottile
Copy link
Member

@FiloSottile FiloSottile commented Nov 5, 2019

I feel like code in init function is in fact exercised by the tests, as supposedly it generate global state that the tests might use. Coverage is only about code being exercised, not about its results actually mattering.

@FiloSottile FiloSottile added this to the Unplanned milestone Nov 5, 2019
@rayharris-ibm
Copy link

@rayharris-ibm rayharris-ibm commented Nov 6, 2019

@FiloSottile
The issue is that it doesn't make sense to me that the code coverage reported would ever be greater than 0% if there are no tests.

From https://blog.golang.org/cover

Test coverage is a term that describes how much of a package's code is exercised by running the package's tests.

To me, a coverage report of anything greater than 0 implies that at least one test was run. This isn't a huge issue, but it can be misleading when coverage data is collected into a reporting system. I came across this issue while evaluating several code coverage reporting systems.

If there is at least one test, then I have no problem with the code in an init function being counted. But if there are no tests, how can test coverage be anything other than 0?

@rayharris-ibm
Copy link

@rayharris-ibm rayharris-ibm commented Nov 6, 2019

@straightdave

In src/cmd/cover/cover.go, adding this case to the switch statement in func (f *File) Visit(node ast.Node) ast.Visitor eliminates instrumentation for main() and init() functions:

case *ast.FuncDecl:
        if n.Recv == nil && (n.Name.String() == "main" || n.Name.String() == "init") {
                return nil
        }

I haven't looked yet at what would need to be changed to keep the instrumentation, but to zero out the coverage percentage if there are no tests.

@straightdave
Copy link

@straightdave straightdave commented Nov 7, 2019

🤣 I am working on a project which DOES want to record coverage in both main and init functions. So please don't make this change to the official cover tool...

@rayharris-ibm
Copy link

@rayharris-ibm rayharris-ibm commented Nov 7, 2019

@straightdave If there are no tests, do you want to see a code coverage greater than zero, then?

@straightdave
Copy link

@straightdave straightdave commented Nov 8, 2019

In the classic view, yes, code-coverage is related to testing. If no tests to run, there's no coverage.
But in my personal opinion, "coverage" just means how much code is touched during execution. It doesn't matter it is covered (touched) by a test or just normal execution.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
4 participants
You can’t perform that action at this time.