I do not see any documentation specifying if T.Run is safe for concurrent use. I see the following doc for testing.T, which does not mention T.Run:
A test ends when its Test function returns or calls any of the methods FailNow, Fatal, Fatalf, SkipNow, Skip, or Skipf. Those methods, as well as the Parallel method, must be called only from the goroutine running the Test function.
The other reporting methods, such as the variations of Log and Error, may be called simultaneously from multiple goroutines.
I ask because I believe T.Run was safe for concurrent use in go 1.7. Then, this line was added in go 1.8, which made T.Run no longer safe for concurrent use. I don't particularly care whether T.Run is safe for concurrent use, but I think the semantics should be clarified.
// Run may be called simultaneously from multiple goroutines, but all such
// calls must happen before the outer benchmark function for b returns.
Which I don't think is true at all: calling B.Run() multiple times from multiple goroutines results in benchmarkLock.Unlock() being called multiple times before benchmarkLock.Lock(), which causes a crash.
Is this a documentation issue or a implementation issue? cl/35354 does have a test for multiple goroutines using when using testing.T.Run, but lacks a test for testing.B.Run, which clearly seems broken.
Are you proposing we properly making testing.B.Run work also? or just remove the comment.