Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

testing: nested t.Fatalf hides errors #22882

scubed2 opened this issue Nov 26, 2017 · 7 comments

testing: nested t.Fatalf hides errors #22882

scubed2 opened this issue Nov 26, 2017 · 7 comments
FrozenDueToAge NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one.


Copy link

scubed2 commented Nov 26, 2017

What version of Go are you using (go version)?

go version
go version go1.8 linux/amd64

Does this issue reproduce with the latest release?


What operating system and processor architecture are you using (go env)?

go env
GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build180482905=/tmp/go-build -gno-record-gcc-switches"

What did you do?

I'm not sure how to put a test in the playground.
If someone could point out how to do that, I will.

package nested
package nested

import (

func TestNested(t *testing.T) {
outer := t
t.Run("sub", func(t *testing.T) {
Note that renaming t to outer is just for emphasis.

What did you expect to see?

While it's not immediately obvious to me what should happen in this case, it should at least:

  1. Print out the string passed to Fatalf (in this case, "Nope")
  2. If this is invalid, say that it is calling Fatalf from the wrong scope, so people know what the actual problem is.

This came up when debugging a test that was using gomock incorrectly. I ended up spending time tracking down this instead of fixing the test.

What did you see instead?

It gives a not very helpful stacktrace.

go test nested_test.go
panic: test executed panic(nil) or runtime.Goexit

goroutine 6 [running]:
/nix/store/5428ij3ln77vam6cacnljc1f75f214fs-go-1.8/share/go/src/testing/testing.go:622 +0x29d
/nix/store/5428ij3ln77vam6cacnljc1f75f214fs-go-1.8/share/go/src/runtime/panic.go:366 +0x153
/nix/store/5428ij3ln77vam6cacnljc1f75f214fs-go-1.8/share/go/src/testing/testing.go:486 +0x39
testing.(*common).Fatalf(0xc4200765b0, 0x5393ee, 0x4, 0x0, 0x0, 0x0)
/nix/store/5428ij3ln77vam6cacnljc1f75f214fs-go-1.8/share/go/src/testing/testing.go:530 +0x83
/dev/shm/a/nested_test.go:10 +0x5c
testing.tRunner(0xc420076750, 0xc42000eea0)
/nix/store/5428ij3ln77vam6cacnljc1f75f214fs-go-1.8/share/go/src/testing/testing.go:657 +0x96
created by testing.(*T).Run
/nix/store/5428ij3ln77vam6cacnljc1f75f214fs-go-1.8/share/go/src/testing/testing.go:697 +0x2ca
FAIL command-line-arguments 0.004s

@ianlancetaylor ianlancetaylor changed the title Nested t.Fatalf hides errors testing: nested t.Fatalf hides errors Nov 26, 2017
@ianlancetaylor ianlancetaylor added the NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. label Nov 26, 2017
@ianlancetaylor ianlancetaylor added this to the Go1.11 milestone Nov 26, 2017
Copy link

agnivade commented Nov 28, 2017

Tested on 1.9 and master. Fails on both.

Simplified the code to use different variables. It's much more clear here -

package nested

import (

func TestNested(tt *testing.T) {
        // tt.Fatalf("Nope") // This works as expected
	tt.Run("sub", func(t *testing.T) {

Just save this to a file and run go test. It gives a panic.

Copy link

creichlin commented Dec 4, 2017

This behavior is triggered because t.Run creates a new goroutine and t.Fatalf calls t.FailNow from that new goroutine which will try to exit the current test by exiting the current goroutine. This is documented in the Run and FailNow method. It should be possible to fix because Run is blocking and while it's blocked a proper error message could be printed before panicking.
I'm not sure though if such a thing is worth fixing, i would say it's rather unlikely to happen and at least it panics so you see that there is something wrong. The added complexity might out-weight the benefit.

Copy link

namusyaka commented Jan 14, 2018

Actually, I think this is a corner case, but the current behavior is not intuitive.

For example, I guess we can fix the issue with such a change.

diff --git src/testing/testing.go src/testing/testing.go
index f56dbf8f6d..6bd03b703b 100644
--- src/testing/testing.go
+++ src/testing/testing.go
@@ -733,7 +733,7 @@ func tRunner(t *T, fn func(t *T)) {
                t.duration += time.Since(t.start)
                // If the test panicked, print any test output before dying.
                err := recover()
-               if !t.finished && err == nil {
+               if (t.parent != nil && !t.parent.finished) && !t.finished && err == nil {
                        err = fmt.Errorf("test executed panic(nil) or runtime.Goexit")
                if err != nil {

and then, got:

=== RUN   TestNested
=== RUN   TestNested/sub
--- FAIL: TestNested (0.00s)
	main_test.go:10: Nope
    --- PASS: TestNested/sub (0.00s)

What do you think? I can send a CL if you hope.

Copy link

gopherbot commented Jan 16, 2018

Change mentions this issue: testing: don't overwrite error unless parent of subtest is finished

Copy link

namusyaka commented Jan 16, 2018

I've rethink this issue, and then I strongly thought that this problem should be fixed. Anyway, I will send the CL to do this discussion productively, then I decided to wait for go-1.11 development cycle.

Copy link

gopherbot commented Feb 28, 2018

Change mentions this issue: testing: gracefully handle subtest failing parent’s T

Copy link

mpvl commented Feb 28, 2018

@namusyaka I agree that the current behavior is undesirable. Thanks for the CL. It comes close to a fix, but has some remaining issues. I've created another CL which I believe comes to a good compromise. It treats calling FailNow on a parent as an error, but will still handle it gracefully without panicking.

If we decide in the future that this is not an error, we need to handle the case of parallel subtests calling FailNow on a parent concurrently. This will be very tricky to get right.

@golang golang locked and limited conversation to collaborators Mar 1, 2019
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
FrozenDueToAge NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one.
None yet

No branches or pull requests

7 participants