testing: Calling t.Fatal in a subtest panics

[cznic]

Please answer these questions before submitting your issue. Thanks!

What version of Go are you using (go version)?

go version go1.7.1 linux/amd64

What operating system and processor architecture are you using (go env)?

GOARCH="amd64"
GOBIN=""
GOEXE=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GOOS="linux"
GOPATH="/home/jnml"
GORACE=""
GOROOT="/home/jnml/go"
GOTOOLDIR="/home/jnml/go/pkg/tool/linux_amd64"
CC="gcc"
GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build509817202=/tmp/go-build -gno-record-gcc-switches"
CXX="g++"
CGO_ENABLED="1"

What did you do?

jnml@r550:~$ go get -u github.com/cznic/gc
jnml@r550:~$ cd $GOPATH/src/github.com/cznic/gc
jnml@r550:~/src/github.com/cznic/gc$ git checkout 421781a  
Note: checking out '421781a'.

You are in 'detached HEAD' state. You can look around, make experimental
changes and commit them, and you can discard any commits you make in this
state without impacting any branches by performing another checkout.

If you want to create a new branch to retain commits you create, you may
do so (now or later) by using -b with the checkout command again. Example:

  git checkout -b <new-branch-name>

HEAD is now at 421781a... Subtest issue reproduction case.
jnml@r550:~/src/github.com/cznic/gc$ 

What did you expect to see?

Something like

jnml@r550:~/src/github.com/cznic/gc$ go test
...
=== RUN   TestParser/GOROOT
    --- FAIL: TestParser/GOROOT (0.00s)
        all_test.go:1084: TODO
...

What did you see instead?

jnml@r550:~/src/github.com/cznic/gc$ go test
panic: test executed panic(nil) or runtime.Goexit

goroutine 15 [running]:
panic(0x5fcc60, 0xc4202af730)
    /home/jnml/go/src/runtime/panic.go:500 +0x1a1
testing.tRunner.func1(0xc42041f440)
    /home/jnml/go/src/testing/testing.go:579 +0x25d
runtime.Goexit()
    /home/jnml/go/src/runtime/panic.go:340 +0xec
testing.(*common).FailNow(0xc42041fa40)
    /home/jnml/go/src/testing/testing.go:453 +0x39
testing.(*common).Fatal(0xc42041fa40, 0xc420067ed0, 0x1, 0x1)
    /home/jnml/go/src/testing/testing.go:490 +0x6f
github.com/cznic/gc.testParser(0xc42041fa40, 0xc420712000, 0x645, 0x800)
    /home/jnml/src/github.com/cznic/gc/all_test.go:1084 +0x2ab
github.com/cznic/gc.TestParser.func2(0xc42041f440)
    /home/jnml/src/github.com/cznic/gc/all_test.go:1096 +0x4b
testing.tRunner(0xc42041f440, 0xc42251a090)
    /home/jnml/go/src/testing/testing.go:610 +0x81
created by testing.(*T).Run
    /home/jnml/go/src/testing/testing.go:646 +0x2ec
exit status 2
FAIL    github.com/cznic/gc 6.783s
jnml@r550:~/src/github.com/cznic/gc$ 

Additional information

Running with -race produces the same output.

[spenczar]

This appears to me to be a bug in the code in github.com/cznic/gc. The test in question looks like this:

func TestParser(t *testing.T) {
  cover := append(gorootTestFiles, ycover)
  _ = t.Run("Yacc", func(*testing.T) { testParserYacc(t, cover) }) &&
    t.Run("GOROOT", func(*testing.T) { testParser(t, cover) }) &&
    t.Run("States", testParserStates)

}

The t.Run calls are ignoring their parameter, so the t in the testParser call is TestParser's *testing.T, not the one that's passed to t.Run("GOROOT". It should look like this:

func TestParser(t *testing.T) {
  cover := append(gorootTestFiles, ycover)
  _ = t.Run("Yacc", func(t *testing.T) { testParserYacc(t, cover) }) &&
    t.Run("GOROOT", func(t *testing.T) { testParser(t, cover) }) &&
    t.Run("States", testParserStates)

}

[cznic]

@spenczar So simple ;-)

--- FAIL: TestParser (4.19s)
    --- FAIL: TestParser/GOROOT (0.00s)
        all_test.go:1084: TODO
FAIL
exit status 1
FAIL    github.com/cznic/gc 6.841s

It's a trap. Thank you!

[gopherbot]

Change https://golang.org/cl/52770 mentions this issue: testing: explain how SkipNow and FailNow stop execution