cmd/test2json: CGO_ENABLED is set, when running tests from prebuilt binary

[cardil]

What version of Go are you using (go version)?

$ go version
go version go1.18.1 linux/amd64

Does this issue reproduce with the latest release?

Yes.

What operating system and processor architecture are you using (go env)?

go env Output

$ go env
GO111MODULE=""
GOARCH="amd64"
GOBIN=""
GOCACHE="/home/ksuszyns/.cache/go-build"
GOENV="/home/ksuszyns/.config/go/env"
GOEXE=""
GOEXPERIMENT=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GOINSECURE=""
GOMODCACHE="/home/ksuszyns/.gvm/pkgsets/go1.18.1/global/pkg/mod"
GONOPROXY=""
GONOSUMDB=""
GOOS="linux"
GOPATH="/home/ksuszyns/.gvm/pkgsets/go1.18.1/global"
GOPRIVATE=""
GOPROXY="https://proxy.golang.org,direct"
GOROOT="/home/ksuszyns/git/golang-go"
GOSUMDB="sum.golang.org"
GOTMPDIR=""
GOTOOLDIR="/home/ksuszyns/git/golang-go/pkg/tool/linux_amd64"
GOVCS=""
GOVERSION="go1.18.1"
GCCGO="gccgo"
GOAMD64="v1"
AR="ar"
CC="gcc"
CXX="g++"
CGO_ENABLED="1"
GOMOD="/dev/null"
GOWORK=""
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build2603697588=/tmp/go-build -gno-record-gcc-switches"

What did you do?

1. Checkout the reproduction repo: https://github.com/cardil/repro-cgo-drift
2. Run make repro

What did you expect to see?

I expect to see the test pass regardless if executed via:

go test -count=1 ./

or

go test -c -o ./repro.test ./
go tool test2json -t ./repro.test -test.count=1

What did you see instead?

The process fails with the following output:

go test -c -o ./repro.test ./
go tool test2json -t ./repro.test -test.count=1
{"Time":"2022-08-19T12:42:13.742590288+02:00","Action":"output","Test":"TestCgo","Output":"--- FAIL: TestCgo (0.00s)\n"}
{"Time":"2022-08-19T12:42:13.743167948+02:00","Action":"output","Test":"TestCgo","Output":"    cgo_test.go:12: Expecting CGO_ENABLED equal \"\", got \"1\"\n"}
{"Time":"2022-08-19T12:42:13.743197548+02:00","Action":"fail","Test":"TestCgo","Elapsed":0}
{"Time":"2022-08-19T12:42:13.743212794+02:00","Action":"output","Output":"FAIL\n"}
{"Time":"2022-08-19T12:42:13.743499898+02:00","Action":"fail","Elapsed":0.006}

Additional context

I stumbled on this problem while running tests via the Goland IDE. The Goland IDE uses the pair of commands to execute tests:

go test -c -o /tmp/Goland/test.bin ./package/path
go tool test2json -t /tmp/Goland/test.bin -test.v -test.paniconexit0

We are executing google/ko in our tests to built test OCI images. KO sets the CGO_ENABLED=0 by default because the default base image is distroless, which doesn't contain any system libraries.

When I execute tests from Goland IDE, I have effectively CGO_ENABLED=1 when the test starts. This overrides the KO default value, and the produced image is invalid. The faulty image contains dynamic binary, which can't run on distroless.

When the tests are executed with go test directly, all is well.

I think running tests via go tool test2json -t shouldn't change the execution environment for the test.

See also: knative-extensions/control-protocol#204

[cardil]

I think the following fragment is responsible for this discrepancy:

go/src/cmd/go/main.go

Lines 202 to 213 in 6001c04

	// Set environment (GOOS, GOARCH, etc) explicitly.
	// In theory all the commands we invoke should have
	// the same default computation of these as we do,
	// but in practice there might be skew
	// This makes sure we all agree.
	cfg.OrigEnv = os.Environ()
	cfg.CmdEnv = envcmd.MkEnv()
	for _, env := range cfg.CmdEnv {
	if os.Getenv(env.Name) != env.Value {
	os.Setenv(env.Name, env.Value)
	}
	}

[seankhliao]

Arguably the test is both fragile and wrong, it tests for an unset value (ie use the default) when it actually needs the specific value of off.

[gopherbot]

Change https://go.dev/cl/424999 mentions this issue: cmd/test2json: use original environment to mirror go test

[cardil]

The point is that the test2json uses effective env - same as go env, but go test actually uses the original env:

go/src/cmd/go/internal/test/test.go

Line 1358 in bf4e35b

env := cfg.OrigEnv[:len(cfg.OrigEnv):len(cfg.OrigEnv)]

The PR #54554 align this to be the same.

[bcmills]

go tool test2json is not a supported way to run tests. If you want precise control over the process's environment, use go tool -n test2json to obtain the path to the tool, and then run it with whatever environment you need for it.