Skip to content

testing: are T.Run and B.Run safe for concurrent use? #18603

Description

@tombergan

I do not see any documentation specifying if T.Run is safe for concurrent use. I see the following doc for testing.T, which does not mention T.Run:

A test ends when its Test function returns or calls any of the methods FailNow, Fatal, Fatalf, SkipNow, Skip, or Skipf. Those methods, as well as the Parallel method, must be called only from the goroutine running the Test function.

The other reporting methods, such as the variations of Log and Error, may be called simultaneously from multiple goroutines.

There is a similar doc for testing.B.

I ask because I believe T.Run was safe for concurrent use in go 1.7. Then, this line was added in go 1.8, which made T.Run no longer safe for concurrent use. I don't particularly care whether T.Run is safe for concurrent use, but I think the semantics should be clarified.

Optimistically labeling this go1.8.

/cc @dsnet

Activity

  1. added this to the Go1.8 milestone on Jan 11, 2017
  2. dsnet commented on Jan 11, 2017

    @dsnet
    Member

    If T.Run was safe concurrently in Go1.7, even if it was ambiguous whether it should be, it's worth considering whether it should continue to be concurrent safe in Go1.8.

    \cc @mpvl

    (moving to Go1.8Maybe; I don't think this is a release blocker at this point)

  3. modified the milestones: Go1.8Maybe, Go1.8 on Jan 11, 2017
  4. added
    NeedsDecisionFeedback is required from experts, contributors, and/or the community before a change can be made.
    on Jan 11, 2017
  5. rsc commented on Jan 18, 2017

    @rsc
    Contributor

    Not sure whether @mpvl is around. Sent https://go-review.googlesource.com/35354.
    /cc @bradfitz

  6. gopherbot commented on Jan 18, 2017

    @gopherbot
    Contributor

    CL https://golang.org/cl/35354 mentions this issue.

  7. added
    NeedsFixThe path to resolution is known, but the work has not been done.
    and removed
    DocumentationIssues describing a change to documentation.
    NeedsDecisionFeedback is required from experts, contributors, and/or the community before a change can be made.
    on Jan 18, 2017
  8. flowblok commented on Aug 2, 2017

    @flowblok

    CL https://golang.org/cl/35354 added this comment to B.Run():

    // Run may be called simultaneously from multiple goroutines, but all such
    // calls must happen before the outer benchmark function for b returns.
    

    Which I don't think is true at all: calling B.Run() multiple times from multiple goroutines results in benchmarkLock.Unlock() being called multiple times before benchmarkLock.Lock(), which causes a crash.

    (example code: https://play.golang.org/p/0u15AvArTK)

  9. dsnet commented on Aug 2, 2017

    @dsnet
    Member

    Is this a documentation issue or a implementation issue? cl/35354 does have a test for multiple goroutines using when using testing.T.Run, but lacks a test for testing.B.Run, which clearly seems broken.

    Are you proposing we properly making testing.B.Run work also? or just remove the comment.

  10. flowblok commented on Aug 3, 2017

    @flowblok

    I don't mind which way it gets resolved, but it's significantly less effort to just remove the comment.

  11. gopherbot commented on Dec 1, 2017

    @gopherbot
    Contributor

    Change https://golang.org/cl/80841 mentions this issue: testing: remove claim that b.Run is safe for concurrent use

  12. locked and limited conversation to collaborators on Dec 1, 2018
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    DocumentationIssues describing a change to documentation.FrozenDueToAgeNeedsFixThe path to resolution is known, but the work has not been done.release-blocker

    Type

    No type

    Projects

    No projects

      Milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions