cmd/compile: change in initialization order when using go test -cover

[ianlancetaylor]

m.go:

package main

import (
	"flag"
)

var (
	fooFlag = flag.String("foo", "", "this should be ok")
	foo     = flag.Lookup("foo")

	barFlag = flag.String("bar", "", "this should be also ok, but is "+notOK()+".")
	bar     = flag.Lookup("bar")
)

func notOK() string {
	return "not OK"
}

m_test.go:

package main

import (
	"testing"
)

func TestFoo(t *testing.T) {
	if foo == nil {
		t.Fatal()
	}
}

func TestBar(t *testing.T) {
	if bar == nil {
		t.Fatal()
	}
}

When run with go test, this passes. When run with go test -cover it fails:

--- FAIL: TestBar (0.00s)
    m_test.go:15: 
FAIL
	main	coverage: 100.0% of statements
exit status 1
FAIL	example.com	0.003s

The variables have no initialization dependencies visible to the compiler, so the order of initialization rules say that they should be initialized in declaration order. That is what happens with go test, but it appears to not happen with go test -cover.

This test passed with go test -cover in versions of Go before Go 1.13.

CC @golang/runtime @thanm

[thanm]

Thanks. I'll take a look.

[cher11]

It looks that it is not a bug really:

Below is the instrumented source code.

barFlag now depends on the newly added variable GoCover. According to the Go spec at https://go.dev/ref/spec#Package_initialization, it changed the initialization order:

First variables with no dependencies are initialized, in particular the 'bar' one (and GoCover). And then barFlag is initialized only. Go compiler knows nothing about the dependencies through the flag library.

go tool cover --mode=set go-flags-coverage-problem/pkg.go
//line go-flags-coverage-problem/pkg.go:1
package main

import (
"google3/base/go/flag"
)

var (
fooFlag = flag.String("foo", "", "this should be ok")
foo = flag.Lookup("foo")

    barFlag = flag.String("bar", "", "this should be also ok, but is "+notOK()+".")
    bar     = flag.Lookup("bar")

)

func notOK() string {GoCover.Count[0] = 1;
return "not OK"
}

func main() {GoCover.Count[1] = 1;

}

var GoCover = struct {
Count [2]uint32
Pos [3 * 2]uint32
NumStmt [2]uint16
} {
Pos: [3 * 2]uint32{
15, 17, 0x20015, // [0]
19, 21, 0x2000e, // [1]
},
NumStmt: [2]uint16{
1, // 0
0, // 1
},
}

[cher11]

(I also commented in the internal tool)

[thanm]

@cher11 thanks, yes. looking at it from the perspective of the compiler being fed the instrumented source code, it is doing the "right" thing with respect to initialization. To the average user however (e.g. folks who are not looking "under the hood" at the instrumented source code) it will seem that the compiler is behaving incorrectly, so I think it is important that we handle this case better.

[thanm]

This test passed with go test -cover in versions of Go before Go 1.13.

Interesting here-- it looks as though Matthew completely rewrote the initorder code in the compiler for 1.13, so I suspect that prior to that point the test was working essentially by accident.

[thanm]

OK, I will send a CL shortly with a fix. This fix is specific to the new coverage implementation however (e.g. GOEXPERIMENT=coverageredesign) -- with the old implementation this would be a good deal more difficult to work around.

[gopherbot]

Change https://go.dev/cl/443715 mentions this issue: cmd/compile: special case coverage vars in pkg init order

[mdempsky]

The solution taken here (adding special cases to pkginit/initorder.go) means we can't easily switch to using types2's Info.InitOrder, unless we add the same special case logic to go/types and types2.

Can we instead change cmd/cover to avoid instrumenting the user code in ways that affects initialization order? E.g., couldn't cmd/cover just put the GoCover variable(s) first so its initialization doesn't delay initializing other variables? Perhaps in a new source file that cmd/go arranges to pass to cmd/compile first on the command line?

[thanm]

Yes, it does seem as though changing cmd/cover would be possible, although I think a little help from the Go command will be needed as well. I'll send a CL later this morning.