Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

testing: odd interaction between -json flag and TestMain #31969

Open
eliben opened this issue May 10, 2019 · 2 comments

Comments

Projects
None yet
3 participants
@eliben
Copy link

commented May 10, 2019

What version of Go are you using (go version)?

$ go version
go version go1.12 linux/amd64

Does this issue reproduce with the latest release?

Yes

What operating system and processor architecture are you using (go env)?

go env Output
$ go env
GOARCH="amd64"
GOBIN=""
GOCACHE="/usr/local/google/home/eliben/.cache/go-build"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GOOS="linux"
GOPATH="/usr/local/google/home/eliben/go"
GOPROXY="https://proxy.golang.org"
GORACE=""
GOROOT="/usr/lib/google-golang"
GOTMPDIR=""
GOTOOLDIR="/usr/lib/google-golang/pkg/tool/linux_amd64"
GCCGO="gccgo"
CC="clang"
CXX="clang++"
CGO_ENABLED="1"
GOMOD=""
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build529164426=/tmp/go-build -gno-record-gcc-switches"

What did you do?

package main

import (
	"fmt"
	"os"
	"testing"
)

func TestJoe(t *testing.T) {
	fmt.Println("from joe")
}

func TestMain(m *testing.M) {
	if len(os.Getenv("XYZ")) > 0 {
		return
	}
	fmt.Println("before run")
	s := m.Run()
	fmt.Println("after run")
	os.Exit(s)
}

What did you expect to see?

When running go test with this file, the test passes. It also passes with go test -json. However, if I tickle the early return in TestMain:

$ XYZ=2 go test -json
{"Time":"2019-05-10T09:33:09.534255206-07:00","Action":"output","Package":"_testing/testmain","Output":"ok  \t_testing/testmain\t0.019s\n"}
{"Time":"2019-05-10T09:33:09.534473515-07:00","Action":"fail","Package":"_testing/testmain","Elapsed":0.019}

Note the fail status in the last line. The exit code of this go test -json run is 0, even though it (wrongly) reported failure. Regularly running go test here passes, as expected.

@andybons

This comment has been minimized.

Copy link
Member

commented May 13, 2019

@andybons andybons added this to the Unplanned milestone May 13, 2019

@hanleym

This comment has been minimized.

Copy link

commented May 14, 2019

This seems to happen when TestMain returns without m.Run() being called; regardless of os.Exit().

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.