New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

testing: complain loudly during concurrent use of T.FatalX and T.SkipX #15758

Open
dsnet opened this Issue May 19, 2016 · 11 comments

Comments

Projects
None yet
@dsnet
Member

dsnet commented May 19, 2016

The testing package is explicit about the following:

A test ends when its Test function returns or calls any of the methods FailNow, Fatal, Fatalf, SkipNow, Skip, or Skipf. Those methods, as well as the Parallel method, must be called only from the goroutine running the Test function.

However, this is easy to overlook and it is not immediately obvious that the code below is a bad test:

func TestFoo(t *testing.T) {
    go func() {
        t.Fatal("fatal") // Called from outside the Test function
    }()
    time.Sleep(1 * time.Second)
}

This currently outputs (what a user expects):

--- FAIL: TestFoo (1.00s)
    foo_test.go:10: fatal

However, since t.Fatal is not safe to call outside of the Test function, this is poor feedback since it makes the user think that the test is correctly written when it is actually racy. Instead it should complain loudly that this is wrong.

As I'm debugging poor tests, I've noticed that the calling of t.Fatal in a goroutine is actually a fairly common phenomenon.

Related: #15674

/cc @mpvl @bradfitz

@dsnet dsnet added this to the Go1.8 milestone May 19, 2016

@mpvl mpvl self-assigned this May 21, 2016

@quentinmit quentinmit added the NeedsFix label Oct 10, 2016

@rsc

This comment has been minimized.

Contributor

rsc commented Oct 18, 2016

Seems infeasible. FailNow/SkipNow are fundamentally built around runtime.Goexit, and there is no way to tell whether they are being called from the "right" goroutine. Bad API design maybe, but it's the way it is.

We can't change from runtime.Goexit to a panic in general; the whole point of using Goexit was that it does not look like a panic and cannot be stopped.

I don't see any way to change the current behavior nor do I see any way to make the current behavior louder.

@rsc rsc closed this Oct 18, 2016

@bradfitz

This comment has been minimized.

Member

bradfitz commented Oct 18, 2016

@rsc, the http2 package has a testing-time-only mechanism to track that certain methods only run on the expected goroutines.

See https://github.com/golang/net/blob/master/http2/gotrack.go#L22

We could do something like that.

@rsc

This comment has been minimized.

Contributor

rsc commented Oct 19, 2016

I've been wondering if we should take the goroutine ID out of the runtime.Stack output, precisely for this reason. :-(

@bradfitz

This comment has been minimized.

Member

bradfitz commented Oct 19, 2016

@rsc, that'd be fine, if you give me an as-useful debugging primitive elsewhere.

@golang golang locked and limited conversation to collaborators Oct 19, 2017

@ianlancetaylor

This comment has been minimized.

Contributor

ianlancetaylor commented Apr 11, 2018

Reopening as @balshetzer has an approach that may work, based on looking for certain functions in the stack trace. I guess we can give it a try and see if it seems reasonable.

@golang golang unlocked this conversation Apr 11, 2018

@dsnet dsnet modified the milestones: Go1.8, Go1.11 Apr 23, 2018

@odeke-em

This comment has been minimized.

Member

odeke-em commented May 12, 2018

@nhooyr

This comment has been minimized.

Contributor

nhooyr commented Jun 27, 2018

As I'm debugging poor tests, I've noticed that the calling of t.Fatal in a goroutine is actually a fairly common phenomenon.

Another option here is to remove the doc and allow it. See #25467

@bcmills

This comment has been minimized.

Member

bcmills commented Jun 27, 2018

@nhooyr, t.Fatal terminates the goroutine from which it is called (and runs any deferred function calls).

If we were to allow t.Fatal from any goroutine, how would the goroutine running the test function know to exit? It could be running any arbitrary function, so terminating it immediately would risk deadlocking future tests.

It is of course possible for the author of the test to plumb in some sort of cancellation or synchronization mechanism, but if they have done that, then they can just as easily invoke that mechanism instead of calling t.Fatal from the goroutine.

@nhooyr

This comment has been minimized.

Contributor

nhooyr commented Jun 27, 2018

@bcmills We would change t.Fatal to report a error and exit whatever goroutine its called from. I don't think anyone would expect it to exit the main goroutine too.

@gopherbot

This comment has been minimized.

gopherbot commented Sep 10, 2018

Change https://golang.org/cl/134395 mentions this issue: errgroup: rethink concurrency patterns

@bcmills

This comment has been minimized.

Member

bcmills commented Sep 10, 2018

It turns out to be possible to propagate a runtime.Goexit from one goroutine to another, as shown in https://golang.org/cl/134395. I'm no longer certain that it really should be an error to call t.Fatal from another goroutine, provided that that goroutine is appropriately linked back to the test.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment