testing: data race with parallel fuzzing

[AlekSi]

What version of Go are you using (go version)?

go version devel go1.18-40afced8d7 Fri Jan 7 02:37:20 2022 +0000 darwin/arm64

What operating system and processor architecture are you using (go env)?

go env Output

GO111MODULE=""
GOARCH="arm64"
GOBIN=""
GOCACHE="/Users/aleksi/Library/Caches/go-build"
GOENV="/Users/aleksi/Library/Application Support/go/env"
GOEXE=""
GOEXPERIMENT=""
GOFLAGS=""
GOHOSTARCH="arm64"
GOHOSTOS="darwin"
GOINSECURE=""
GOMODCACHE="/Users/aleksi/Soft/GOPATH/pkg/mod"
GONOPROXY=""
GONOSUMDB=""
GOOS="darwin"
GOPATH="/Users/aleksi/Soft/GOPATH"
GOPRIVATE=""
GOPROXY="https://proxy.golang.org"
GOROOT="/Users/aleksi/Soft/go"
GOSUMDB="sum.golang.org"
GOTMPDIR=""
GOTOOLDIR="/Users/aleksi/Soft/go/pkg/tool/darwin_arm64"
GOVCS=""
GOVERSION="devel go1.18-40afced8d7 Fri Jan 7 02:37:20 2022 +0000"
GCCGO="gccgo"
AR="ar"
CC="clang"
CXX="clang++"
CGO_ENABLED="1"
GOMOD="/Users/aleksi/Code/My/go-bug/new/go.mod"
GOWORK=""
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -arch arm64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fdebug-prefix-map=/var/folders/8p/0cm9052d7pd0smp13_35z0440000gn/T/go-build4037385395=/tmp/go-build -gno-record-gcc-switches -fno-common"

What did you do?

https://github.com/AlekSi/go-bug/tree/master/50488

func Fuzz1(f *testing.F) {
	f.Add(42)
	f.Add(43)

	f.Fuzz(func(t *testing.T, i int) {
		t.Parallel()
	})
}

go test -fuzz=Fuzz2 -race

(note: Fuzz2, not Fuzz1 as in code)

What did you expect to see?

Error about missing Fuzz2.

What did you see instead?

==================
WARNING: DATA RACE
Write at 0x00c0000ca000 by goroutine 8:
  internal/fuzz.SnapshotCoverage()
      /Users/aleksi/Soft/go/src/internal/fuzz/coverage.go:32 +0xb8
  testing/internal/testdeps.TestDeps.SnapshotCoverage()
      /Users/aleksi/Soft/go/src/testing/internal/testdeps/deps.go:198 +0x38
  testing/internal/testdeps.(*TestDeps).SnapshotCoverage()
      <autogenerated>:1 +0x24
  testing.(*F).Fuzz.func1.1.1()
      /Users/aleksi/Soft/go/src/testing/fuzz.go:330 +0x48
  runtime.deferreturn()
      /Users/aleksi/Soft/go/src/runtime/panic.go:436 +0x34
  testing.tRunner()
      /Users/aleksi/Soft/go/src/testing/testing.go:1440 +0x18c
  testing.(*F).Fuzz.func1.2()
      /Users/aleksi/Soft/go/src/testing/fuzz.go:321 +0x44

Previous write at 0x00c0000ca000 by goroutine 9:
  internal/fuzz.SnapshotCoverage()
      /Users/aleksi/Soft/go/src/internal/fuzz/coverage.go:32 +0xb8
  testing/internal/testdeps.TestDeps.SnapshotCoverage()
      /Users/aleksi/Soft/go/src/testing/internal/testdeps/deps.go:198 +0x38
  testing/internal/testdeps.(*TestDeps).SnapshotCoverage()
      <autogenerated>:1 +0x24
  testing.(*F).Fuzz.func1.1.1()
      /Users/aleksi/Soft/go/src/testing/fuzz.go:330 +0x48
  runtime.deferreturn()
      /Users/aleksi/Soft/go/src/runtime/panic.go:436 +0x34
  testing.tRunner()
      /Users/aleksi/Soft/go/src/testing/testing.go:1440 +0x18c
  testing.(*F).Fuzz.func1.2()
      /Users/aleksi/Soft/go/src/testing/fuzz.go:321 +0x44

Goroutine 8 (running) created at:
  testing.(*F).Fuzz.func1()
      /Users/aleksi/Soft/go/src/testing/fuzz.go:321 +0x7dc
  testing.(*F).Fuzz()
      /Users/aleksi/Soft/go/src/testing/fuzz.go:396 +0xc68
  github.com/AlekSi/go-bug/new.Fuzz1()
      /Users/aleksi/Code/My/go-bug/new/bug_test.go:11 +0xb0
  testing.fRunner()
      /Users/aleksi/Soft/go/src/testing/fuzz.go:695 +0xfc
  testing.runFuzzTests.func1()
      /Users/aleksi/Soft/go/src/testing/fuzz.go:515 +0x44

Goroutine 9 (running) created at:
  testing.(*F).Fuzz.func1()
      /Users/aleksi/Soft/go/src/testing/fuzz.go:321 +0x7dc
  testing.(*F).Fuzz()
      /Users/aleksi/Soft/go/src/testing/fuzz.go:396 +0xc68
  github.com/AlekSi/go-bug/new.Fuzz1()
      /Users/aleksi/Code/My/go-bug/new/bug_test.go:11 +0xb0
  testing.fRunner()
      /Users/aleksi/Soft/go/src/testing/fuzz.go:695 +0xfc
  testing.runFuzzTests.func1()
      /Users/aleksi/Soft/go/src/testing/fuzz.go:515 +0x44
==================
exit status 66

That reproducer is reduced from a much bigger example (but with the same race stack trace) in which Fuzz2 actually exists or maybe not.

[AlekSi]

@gopherbot fuzz

[gopherbot]

Change https://golang.org/cl/376254 mentions this issue: internal/fuzz: fix racing in SnapshotCoverage

[mengzhuo]

@AlekSi I've upload a CL for this issue, PTAL.

[gopherbot]

Change https://golang.org/cl/376554 mentions this issue: testing: only snapshot coverage during fuzzing

[AlekSi]

@rolandshoemaker It looks like this CL only added a benchmark. I guess you forgot to add changed files.

[rolandshoemaker]

Whoops yup, pushed the right change.

[gopherbot]

Change https://golang.org/cl/377634 mentions this issue: cmd/go: skip TestScript/test_fuzz_test_race on !race