testing: BenchmarkParallel that calls b.Fatal can get a race detector failure

[ianlancetaylor]

Running this test with go test -race -bench=. will get race detector errors. The b.Fatal should be enough to cause a failure, we shouldn't get a race detector error report on top of that.

package main_test

import (
	"math/rand"
	"testing"
	"time"
)

func BenchmarkParallel(b *testing.B) {
	rand.Seed(time.Now().UnixNano())
	b.RunParallel(func(pb *testing.PB) {
		for pb.Next() {
			if rand.Float64() < 0.1 {
				b.Fatal("fatal")
			}
		}
	})
}

Sample output:

goos: linux
goarch: amd64
cpu: Intel(R) Core(TM) i7-7820HQ CPU @ 2.90GHz
BenchmarkParallel-8   	==================
WARNING: DATA RACE
Write at 0x00c0001602e1 by goroutine 26:
  testing.(*common).FailNow()
      /home/iant/go/src/testing/testing.go:741 +0x4f
  testing.(*common).Fatal()
      /home/iant/go/src/testing/testing.go:810 +0x89
  command-line-arguments_test.BenchmarkParallel.func1()
      /home/iant/foo_test.go:14 +0xb2
  testing.(*B).RunParallel.func1()
      /home/iant/go/src/testing/benchmark.go:779 +0x1a3

Previous write at 0x00c0001602e1 by goroutine 25:
  testing.(*common).FailNow()
      /home/iant/go/src/testing/testing.go:741 +0x4f
  testing.(*common).Fatal()
      /home/iant/go/src/testing/testing.go:810 +0x89
  command-line-arguments_test.BenchmarkParallel.func1()
      /home/iant/foo_test.go:14 +0xb2
  testing.(*B).RunParallel.func1()
      /home/iant/go/src/testing/benchmark.go:779 +0x1a3

Goroutine 26 (running) created at:
  testing.(*B).RunParallel()
      /home/iant/go/src/testing/benchmark.go:772 +0x355
  command-line-arguments_test.BenchmarkParallel()
      /home/iant/foo_test.go:11 +0x177
  testing.(*B).runN()
      /home/iant/go/src/testing/benchmark.go:192 +0x1c5
  testing.(*B).launch()
      /home/iant/go/src/testing/benchmark.go:325 +0x37e

Goroutine 25 (finished) created at:
  testing.(*B).RunParallel()
      /home/iant/go/src/testing/benchmark.go:772 +0x355
  command-line-arguments_test.BenchmarkParallel()
      /home/iant/foo_test.go:11 +0x177
  testing.(*B).runN()
      /home/iant/go/src/testing/benchmark.go:192 +0x1c5
  testing.(*B).launch()
      /home/iant/go/src/testing/benchmark.go:325 +0x37e
==================
--- FAIL: BenchmarkParallel-8
    foo_test.go:14: fatal
    foo_test.go:14: fatal
    foo_test.go:14: fatal
    foo_test.go:14: fatal
    foo_test.go:14: fatal
    foo_test.go:14: fatal
    benchmark.go:198: race detected during execution of benchmark
FAIL
exit status 1
FAIL	command-line-arguments	0.019s
FAIL

[gopherbot]

Change https://golang.org/cl/309572 mentions this issue: testing: benchmark parallel tests can call Fatal and SkipNow