cmd/go: coverpkg flag should be part of cache key

[mitar]

Go version

go version go1.22.1 linux/amd64

Output of go env in your module/workspace:

GO111MODULE=''
GOARCH='amd64'
GOBIN=''
GOCACHE='/root/.cache/go-build'
GOENV='/root/.config/go/env'
GOEXE=''
GOEXPERIMENT=''
GOFLAGS=''
GOHOSTARCH='amd64'
GOHOSTOS='linux'
GOINSECURE=''
GOMODCACHE='/go/pkg/mod'
GONOPROXY=''
GONOSUMDB=''
GOOS='linux'
GOPATH='/go'
GOPRIVATE=''
GOPROXY='https://proxy.golang.org,direct'
GOROOT='/usr/local/go'
GOSUMDB='sum.golang.org'
GOTMPDIR=''
GOTOOLCHAIN='local'
GOTOOLDIR='/usr/local/go/pkg/tool/linux_amd64'
GOVCS=''
GOVERSION='go1.22.1'
GCCGO='gccgo'
GOAMD64='v1'
AR='ar'
CC='gcc'
CXX='g++'
CGO_ENABLED='0'
GOMOD='/code/go.mod'
GOWORK=''
CGO_CFLAGS='-O2 -g'
CGO_CPPFLAGS=''
CGO_CXXFLAGS='-O2 -g'
CGO_FFLAGS='-O2 -g'
CGO_LDFLAGS='-O2 -g'
PKG_CONFIG='pkg-config'
GOGCCFLAGS='-fPIC -m64 -fno-caret-diagnostics -Qunused-arguments -Wl,--no-gc-sections -fmessage-length=0 -ffile-prefix-map=/tmp/go-build3544536393=/tmp/go-build -gno-record-gcc-switches'

What did you do?

On this example repository I ran:

go test -covermode=atomic -cover -count 1 -coverpkg=./... ./...

and:

go test -covermode=atomic -cover -coverpkg=./... ./...

What did you see happen?

I expected in both cases all coverage numbers to be the same.

(I have the problem with coverage numbers themselves, but I will report that separately.)

What did you expect to see?

Different numbers:

$ go test -covermode=atomic -cover -count 1 -coverpkg=./... ./...
ok  	github.com/mitar/accurate-test-coverage	0.002s	coverage: 0.0% of statements in ./... [no tests to run]
ok  	github.com/mitar/accurate-test-coverage/otherpkg	0.002s	coverage: 75.0% of statements in ./...
ok  	github.com/mitar/accurate-test-coverage/pkg	0.003s	coverage: 25.0% of statements in ./...

$ go test -covermode=atomic -cover -coverpkg=./... ./...
ok  	github.com/mitar/accurate-test-coverage	(cached)	coverage: 0.0% of statements in ./... [no tests to run]
ok  	github.com/mitar/accurate-test-coverage/otherpkg	(cached)	coverage: 100.0% of statements in ./...
ok  	github.com/mitar/accurate-test-coverage/pkg	(cached)	coverage: 33.3% of statements in ./...

[mitar]

(attn @thanm)

[seankhliao]

I can't reproduce your result for 100% and 33% ?
It consistently reports 75% and 25%

[mitar]

Interesting. I did try before inside Docker container to make sure it is reproducible, but I was also playing a lot with contents of files. I think something got wrong with the cache. If I try now inside clean container with clean checkout without any prior cache, it works as expected.

This is now full reproduction:

$ docker run --rm -t -i golang:1.22.1-alpine3.18 sh
/go # apk add git
fetch https://dl-cdn.alpinelinux.org/alpine/v3.18/main/x86_64/APKINDEX.tar.gz
fetch https://dl-cdn.alpinelinux.org/alpine/v3.18/community/x86_64/APKINDEX.tar.gz
(1/8) Installing brotli-libs (1.0.9-r14)
(2/8) Installing libunistring (1.1-r1)
(3/8) Installing libidn2 (2.3.4-r1)
(4/8) Installing nghttp2-libs (1.57.0-r0)
(5/8) Installing libcurl (8.5.0-r0)
(6/8) Installing libexpat (2.6.2-r0)
(7/8) Installing pcre2 (10.42-r1)
(8/8) Installing git (2.40.1-r0)
Executing busybox-1.36.1-r5.trigger
OK: 18 MiB in 24 packages
/go # cd /
/ # git clone https://github.com/mitar/accurate-test-coverage
Cloning into 'accurate-test-coverage'...
remote: Enumerating objects: 35, done.
remote: Counting objects: 100% (7/7), done.
remote: Compressing objects: 100% (6/6), done.
remote: Total 35 (delta 0), reused 6 (delta 0), pack-reused 28
Receiving objects: 100% (35/35), done.
Resolving deltas: 100% (4/4), done.
/ # cd accurate-test-coverage/
/accurate-test-coverage # rm main.go 
/accurate-test-coverage # go test -covermode=atomic -cover -coverpkg=./... ./...
ok  	github.com/mitar/accurate-test-coverage	0.002s	coverage: 0.0% of statements in ./... [no tests to run]
ok  	github.com/mitar/accurate-test-coverage/otherpkg	0.002s	coverage: 100.0% of statements in ./...
ok  	github.com/mitar/accurate-test-coverage/pkg	0.002s	coverage: 33.3% of statements in ./...
/accurate-test-coverage # git checkout .
Updated 1 path from the index
/accurate-test-coverage # go test -covermode=atomic -cover -coverpkg=./... ./...
ok  	github.com/mitar/accurate-test-coverage	0.003s	coverage: 0.0% of statements in ./... [no tests to run]
ok  	github.com/mitar/accurate-test-coverage/otherpkg	(cached)	coverage: 100.0% of statements in ./...
ok  	github.com/mitar/accurate-test-coverage/pkg	(cached)	coverage: 33.3% of statements in ./...
/accurate-test-coverage # go test -covermode=atomic -cover -count 1 -coverpkg=./... ./...
ok  	github.com/mitar/accurate-test-coverage	0.003s	coverage: 0.0% of statements in ./... [no tests to run]
ok  	github.com/mitar/accurate-test-coverage/otherpkg	0.003s	coverage: 75.0% of statements in ./...
ok  	github.com/mitar/accurate-test-coverage/pkg	0.003s	coverage: 25.0% of statements in ./...

[mitar]

I think -coverpkg should force all tests to re-run. Or cache should depend on -coverpkg.

[mitar]

Hm, #74873 was opened later than this one, but OK. :-)

[seankhliao]

yeah, but it has more discussion and a linked CL 🤷