Skip to content

testing: allow multiple async t.Fatals #25467

Closed

Description

@nhooyr

Please answer these questions before submitting your issue. Thanks!

What version of Go are you using (go version)?

go version devel +e86c26789d Sat May 19 17:38:01 2018 +0000 darwin/amd64

Does this issue reproduce with the latest release?

Yes but you encounter #24438 first so thats why I'm using a devel version.

What operating system and processor architecture are you using (go env)?

macOS 10.13.4

What did you do?

package scratch

import (
	"sync"
	"testing"
)

func TestMeow(t *testing.T) {
	var wg sync.WaitGroup
	wg.Add(2)
	go func() {
		defer wg.Done()
		t.Fatal("meow")
	}()
	go func() {
		defer wg.Done()
		t.Fatal("meow")
	}()
	wg.Wait()
}
go test -race scratch_test.go

What did you expect to see?

Two test failures with error message "meow".

What did you see instead?

Race condition report due to the two concurrent writes at

c.finished = true
due to t.Fatal.

I think we should patch up this race. Its very similar to #24438 in that the test is racy, but not in an interesting way.

Activity

  1. changed the title [-]allow multiple async t.Fatals[/-] [+]testing: allow multiple async t.Fatals[/+] on May 20, 2018
  2. added
    NeedsInvestigationSomeone must examine and confirm this is a valid issue and not a duplicate of an existing one.
    on May 20, 2018
  3. added this to the Unplanned milestone on May 20, 2018
  4. cznic commented on May 20, 2018

    @cznic
    Contributor

    Is this a proposal or a bug report? I'm asking because this is working as documented.

    A test ends when its Test function returns or calls any of the methods FailNow, Fatal, Fatalf, SkipNow, Skip, or Skipf. Those methods, as well as the Parallel method, must be called only from the goroutine running the Test function.

  5. mvdan commented on May 20, 2018

    @mvdan
    Member

    Like @cznic said, this issue is different from #24438. Error is documented to support being called by any goroutine, but Fatal is not.

    However, perhaps the testing package could panic in a more obvious way, as opposed to silently working until someone uses the race detector.

  6. nhooyr commented on May 20, 2018

    @nhooyr
    ContributorAuthor

    @cznic Its a proposal. I think it should be allowed. t.Fatal already works in another goroutine aside from main, so I think it should work with multiple goroutines as well.

  7. cznic commented on May 20, 2018

    @cznic
    Contributor

    How do you propose to solve a situation where some TestFoo function launches two goroutines and one of them calls t.Fatal immediately while the other only [N minutes] later? The go test command may have even already finished before that could happen. Or worse, when the TestFoo function returns normally, but a goroutine launched by it calls/would call t.Fatal [N minutes] later?

    IMO, the status quo is the only reasonable approach.

  8. nhooyr commented on May 20, 2018

    @nhooyr
    ContributorAuthor

    How do you propose to solve a situation where some TestFoo function launches two goroutines and one of them calls t.Fatal immediately while the other only [N minutes] later? The go test command may have even already finished before that could happen.

    The main test goroutine still needs to wait for the other two goroutines like it would with t.Error. So the go test command would not have finished before that could happen.

    Or worse, when the TestFoo function returns normally, but a goroutine launched by it calls/would call t.Fatal [N minutes] later?

    That should of course cause a panic like it does now.

  9. cznic commented on May 20, 2018

    @cznic
    Contributor

    The main test goroutine still needs to wait for the other two goroutines like it would with t.Error. So the go test command would not have finished before that could happen.

    How to do that? The goroutine that invokes TestMeow in the OP has no knowledge about it launching other goroutines that may possibly later call t.Fatal.

    That should of course cause a panic like it does now.

    This program does not panic now, nor it reports any error. The t.Fatal call is just silently lost because the program exits before calling time.After returned

    func Test Foo(t *testing.T) {
            go func() {
                    <-time.After(tooLong)
                    t.Fatal("Vary dangerous error found.")
            }()
    }
  10. nhooyr commented on May 20, 2018

    @nhooyr
    ContributorAuthor

    How to do that? The goroutine that invokes TestMeow in the OP has no knowledge about it launching other goroutines that may possibly later call t.Fatal.

    The user needs to sync by themselves. Its the same problem as with t.Error. Its not a distinct problem to t.Fatal.

    This program does not panic now, nor it reports any error. The t.Fatal call is just silently lost because the program exits before calling time.After returned

    The bug there is that you're not waiting for the main goroutine. The only way

    func TestMeow(t *testing.T) {
    	go func() {
    		time.Sleep(time.Second)
    		t.Fatal("Vary dangerous error found.")
    	}()
    	t.Log("done")
    }
    
    func TestBoo(t *testing.T) {
    	time.Sleep(time.Second*3)
    }

    That however does produce the panic I was talking about. You can change the t.Fatal to t.Error to see it also occurs with t.Error.

  11. cznic commented on May 20, 2018

    @cznic
    Contributor

    Its not a distinct problem to t.Fatal.

    It's very different in that one terminates execution of the TestXXX function while the other does not. And that also explains why t.Error is allowed from any goroutine while t.Fatal is not.

  12. nhooyr commented on May 20, 2018

    @nhooyr
    ContributorAuthor

    It's very different in that one terminates execution of the TestXXX function while the other does not. And that also explains why t.Error is allowed from any goroutine while t.Fatal is not.

    Here's my argument. t.Error is allowed from any goroutine without any race conditions. t.Fatal is allowed from any goroutine too, even though it is documented that you should only be calling it from the main test goroutine. Go cannot enforce that t.Fatal should only be called from the main test goroutine. So at most, its a bug in that you are not following convention. If however, you run t.Fatal async in multiple goroutines, it goes from convention violation to race condition. I don't think that makes sense. Its not racey in an important way.

  13. bcmills commented on May 21, 2018

    @bcmills
    Contributor

    Go cannot enforce that t.Fatal should only be called from the main test goroutine.

    That's not necessarily true. See #15758.

  14. mvdan commented on May 21, 2018

    @mvdan
    Member

    Thanks so much @bcmills - that is exactly what I was proposing in my comment :)

    @nhooyr please refer to that issue as far as making the testing package's behavior more consistent and intuitive. If you would still like to propose that concurrent Fatal calls were supported and documented, feel free to open a new issue following the proposal process: https://github.com/golang/proposal

  15. locked and limited conversation to collaborators on May 21, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    FrozenDueToAgeNeedsInvestigationSomeone must examine and confirm this is a valid issue and not a duplicate of an existing one.

    Type

    No type

    Projects

    No projects

      Milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions