Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

testing: silly typo in subtests causes hard-to-understand error message #21175

Closed
barnex opened this issue Jul 26, 2017 · 4 comments

Comments

Projects
None yet
4 participants
@barnex
Copy link

commented Jul 26, 2017

What did you do?

Run go test on this (wrong) test file. The subtest is wrongfully referring to the outer testing.T:

package main

import "testing"

func TestPanic(t *testing.T) {
        t.Run("", func(typo *testing.T) {
                t.Fatalf("...") // wrong, should be typo.Fatalf, not t.Fatalf
        })
}

What did you expect to see?

Perhaps a somewhat informative error message. At least not a panic(nil) leading to a long debug session.

What did you see instead?

A very confusing panic(nil):

> go test
panic: test executed panic(nil) or runtime.Goexit

goroutine 6 [running]:
testing.tRunner.func1(0xc4200941e0)
        /home/arne/bin/go/src/testing/testing.go:712 +0x2d2
runtime.Goexit()
        /home/arne/bin/go/src/runtime/panic.go:366 +0x147
testing.(*common).FailNow(0xc4200940f0)
        /home/arne/bin/go/src/testing/testing.go:544 +0x39
testing.(*common).Fatalf(0xc4200940f0, 0x53665f, 0xd, 0x0, 0x0, 0x0)
        /home/arne/bin/go/src/testing/testing.go:588 +0x83
_/home/arne/tmp.TestPanic.func1(0xc4200941e0)
        /home/arne/tmp/panic_test.go:7 +0x5c
testing.tRunner(0xc4200941e0, 0xc42000e400)
        /home/arne/bin/go/src/testing/testing.go:747 +0xd0
created by testing.(*T).Run
        /home/arne/bin/go/src/testing/testing.go:790 +0x2de
exit status 2
FAIL    _/home/arne/tmp 0.003s

Does this issue reproduce with the latest release (go1.8.3)?

Yes. Also with go1.9rc1.

System details

go version go1.9rc1 linux/amd64
GOARCH="amd64"
GOBIN=""
GOEXE=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GOOS="linux"
GOPATH="/home/arne"
GORACE=""
GOROOT="/home/arne/bin/go"
GOTOOLDIR="/home/arne/bin/go/pkg/tool/linux_amd64"
GCCGO="gccgo"
CC="gcc"
GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build514931481=/tmp/go-build -gno-record-gcc-switches"
CXX="g++"
CGO_ENABLED="1"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOROOT/bin/go version: go version go1.9rc1 linux/amd64
GOROOT/bin/go tool compile -V: compile version go1.9rc1
uname -sr: Linux 4.4.0-83-generic
Distributor ID:	Ubuntu
Description:	Ubuntu 16.04.2 LTS
Release:	16.04
Codename:	xenial
/lib/x86_64-linux-gnu/libc.so.6: GNU C Library (Ubuntu GLIBC 2.23-0ubuntu9) stable release version 2.23, by Roland McGrath et al.
gdb --version: GNU gdb (Ubuntu 7.11.1-0ubuntu1~16.04) 7.11.1
@ALTree

This comment has been minimized.

Copy link
Member

commented Jul 26, 2017

What did you see instead?
A very confusing panic(nil)

What did you expect to see?
At least not a panic(nil)

The full error message is not

test executed panic(nil)

it's

test executed panic(nil) or runtime.Goexit

the part after the or is the relevant one.

Your erroneous Fatalf call causes a FailNow call, which triggers a runtime.Goexit on the external goroutine. The error message is (correctly) informing you of this (the fact that runtime.Goexit was called on the external goroutine before it was done with its job).

What's really misleading here is the fact that the error message refers to a runtime.Goexit call, which the user has probably no way to trace back to t.FailNow without reading the testing.go source code.

Suggestion: amend the FailNow doc by mentioning the fact that it calls runtime.Goexit, i.e. from this:

FailNow marks the function as having failed and stops its execution.

to this:

FailNow marks the function as having failed and stops its execution by calling runtime.Goexit.

This way a user reading about a crash caused by runtime.Goexit will have a way to trace the issue back to a bad t.FailNow call without having to read the testing package source code.

@ALTree ALTree changed the title testing: Silly typo in subtests causes panic(nil) testing: silly typo in subtests causes hard-to-understand error message Jul 26, 2017

@ALTree ALTree added this to the Go1.10 milestone Jul 26, 2017

@ALTree

This comment has been minimized.

Copy link
Member

commented Jul 26, 2017

Oh, and the SkipNow doc, too. It calls runtime.Goexit, like FailNow.

@cznic

This comment has been minimized.

Copy link
Contributor

commented Jul 26, 2017

Related #17421.

@gopherbot

This comment has been minimized.

Copy link

commented Aug 2, 2017

Change https://golang.org/cl/52770 mentions this issue: testing: explain how SkipNow and FailNow stop execution

@gopherbot gopherbot closed this in bd74fd3 Aug 9, 2017

@golang golang locked and limited conversation to collaborators Aug 9, 2018

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
You can’t perform that action at this time.