cmd/go: go 1.22.0: go test throws errors when processing folders not listed in coverpkg argument

[sonic-martin]

Go version

go version go1.22.0 linux/amd64, go version go1.22.0 darwin/amd64, go version go1.22.0 windows/amd64

Output of go env in your module/workspace:

GO111MODULE=''
GOARCH='amd64'
GOBIN='/home/martin/bin'
GOCACHE='/home/martin/.cache/go-build'
GOENV='/home/martin/.config/go/env'
GOEXE=''
GOEXPERIMENT=''
GOFLAGS=''
GOHOSTARCH='amd64'
GOHOSTOS='linux'
GOINSECURE=''
GOMODCACHE='/mnt/c/dev/git/pkg/mod'
GONOPROXY='*.mycompany.com'
GONOSUMDB='*.mycompany.com'
GOOS='linux'
GOPATH='/mnt/c/dev/git'
GOPRIVATE='*.mycompany.com'
GOPROXY='https://proxy.golang.org,direct'
GOROOT='/mnt/c/dev/git/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.linux-amd64'
GOSUMDB='sum.golang.org'
GOTMPDIR=''
GOTOOLCHAIN='auto'
GOTOOLDIR='/mnt/c/dev/git/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.linux-amd64/pkg/tool/linux_amd64'
GOVCS=''
GOVERSION='go1.22.0'
GCCGO='gccgo'
GOAMD64='v1'
AR='ar'
CC='gcc'
CXX='g++'
CGO_ENABLED='1'
GOMOD='/mnt/c/dev/git/golang-test-cover/go.mod'
GOWORK=''
CGO_CFLAGS='-O2 -g'
CGO_CPPFLAGS=''
CGO_CXXFLAGS='-O2 -g'
CGO_FFLAGS='-O2 -g'
CGO_LDFLAGS='-O2 -g'
PKG_CONFIG='pkg-config'
GOGCCFLAGS='-fPIC -m64 -pthread -Wl,--no-gc-sections -fmessage-length=0 -ffile-prefix-map=/tmp/go-build2876750124=/tmp/go-build -gno-record-gcc-switches'

What did you do?

Have a main.go and a package internal with some functions.

See reproducer: https://github.com/sonic-martin/golang-test-cover

Run go test:

go test -v ./... -coverprofile=coverage.out -coverpkg=./internal/... -covermode count
go tool cover -html=coverage.out -o coverage.html
go tool cover -func=coverage.out

What did you see happen?

$ go test -v ./... -coverprofile=coverage.out -coverpkg=./internal/... -covermode count
example.com/m: open /tmp/go-build1123786392/b002/covmeta.f6e4431d5ec1fd71f02b3ce4e56eb691a86525173d917007425576a7d9db7c72: no such file or directory
=== RUN   TestHelloer
--- PASS: TestHelloer (0.00s)
PASS
coverage: 100.0% of statements in ./internal/...
ok      example.com/m/internal  0.003s  coverage: 100.0% of statements in ./internal/...
$ echo $?
1
$ go tool cover -html=coverage.out -o coverage.html
$ go tool cover -func=coverage.out
example.com/m/internal/helloer.go:3:    Helloer         100.0%
total:                                  (statements)    100.0%

coverage.out:

mode: count
example.com/m/internal/helloer.go:3.23,5.2 1 1

The both outputs of go tool cover are fine.

What did you expect to see?

Before go 1.22.0 it worked fine and no no such file or directory errors happend.

There are two workarounds:

(1) go test with -coverpkg equal to the folder/packages processed

go test -v ./... -coverprofile=coverage.out -coverpkg=./... -covermode count
        example.com/m           coverage: 0.0% of statements
=== RUN   TestHelloer
--- PASS: TestHelloer (0.00s)
PASS
coverage: 50.0% of statements in ./...
ok      example.com/m/internal  0.003s  coverage: 50.0% of statements in ./...

But changes semantic of the coverage result.

(2) go test with GOEXPERIMENT=nocoverageredesign

$ GOEXPERIMENT=nocoverageredesign go test -v ./... -coverprofile=coverage.out -coverpkg=./internal/... -covermode count
?       example.com/m   [no test files]
=== RUN   TestHelloer
--- PASS: TestHelloer (0.00s)
PASS
coverage: 100.0% of statements in ./internal/...
ok      example.com/m/internal  0.002s  coverage: 100.0% of statements in ./internal/...

See the discussion from Brian Candler, Aldemar F, Thomas McNulty and me in the mailing list:
https://groups.google.com/g/golang-nuts/c/PjAWIdSdQHc

[aldemarf]

Thanks for opening this issue, Martin.

[thanm]

Thanks for the report. I will take a look.

[shaxbee]

Got bunch of CI pipelines broken after 1.22 upgrade with same covmeta.[hash]: no such file or directory error for packages with no tests.
Only setting GOEXPERIMENT=nocoverageredesign or reverting back to 1.21 is fixing it ATM.

Edit:
Setting GOEXPERIMENT=nocoverageredesign is disabling go build -covermode -coverpkg flags, effectively breaking build.

[zegerius]

Looks like this was introduced in 36e75f6

[fredbi]

I am also facing this problem, but the go test program returns an exit status specifically on windows only

[gopherbot]

Change https://go.dev/cl/568835 mentions this issue: cmd/go: respect -coverpkg for unselected no-test packages

[thanm]

Thanks for the reports. I have sent a patch https://go.dev/cl/568835 that should fix this issue, if anyone wants to try it out.

[zegerius]

@thanm I tested in two projects and experience no more issues after applying your patch. Thanks, looking forward to the release.

[kavu]

@thanm Hi! As this fix already flew over the 1.22.1, may we hope that it will be cherry-picked to 1.22.2? I have high hopes for it, we have several dozens of broken pipelines here. Thank you in advance!

[thanm]

cherry-picked to 1.22.2?

I will bring up this possibility with the release team, but given that there is a workaround it seems unlikely that this sort of backport would be approved. Backports are generally reserved for serious problems (compiler or runtime crash, incorrect results, etc) with no workaround. We'll see.

[kavu]

Thank you!

Yeah, we already keep in mind workaround with GOEXPERIMENT=nocoverageredesign, but just trying to keep faith that we won't have to wait till 1.23. Thank you again for your work!

[muhlemmer]

For what it's worth, applying the patch on the 1.22 release branch for my local builds works fine.

[thanm]

@gopherbot please consider this for backport to 1.22

[gopherbot]

Backport issue(s) opened: #66137 (for 1.22).

Remember to create the cherry-pick CL(s) as soon as the patch is submitted to master, according to https://go.dev/wiki/MinorReleases.

[gopherbot]

Change https://go.dev/cl/569575 mentions this issue: cmd/go: respect -coverpkg for unselected no-test packages

[thediveo]

Has this landed on gotip yet? I run into this issue using gotip just pulled a few moments ago.

[thanm]

Yes, this is landed on tip. If you are still seeing it (or something like it) I would be interested in a reproducer.

[thediveo]

Thanks! It seems it was a problem of the shell alias not carrying into a shell script.