testing: flaky issue where crash written doesn't reproduce an error

[katiehockman]

I've been playing around with this fuzz test:

func FuzzFuzzer(f *testing.F) {
	f.Fuzz(func(t *testing.T, x int, s string) {
		b := []byte(s)
		if bytes.Contains(b, strconv.AppendInt(nil, int64(x), 10)) {
			t.Errorf("minimize this! %d, len(s)=%d", x, len(s))
		}
	})
}

I've noticed that sometimes it is writing a crash that doesn't actually reproduce an error when re-run with go test. My guess is that this is a minimization issue, since I haven't been able to reproduce this bug when I set -fuzzminimizetime=0.

Still trying to figure out where the bug was introduced and why none of our tests are catching it.

/cc @golang/fuzzing

[katiehockman]

Something very weird is going on. I'm also seeing times where minimization wasn't even attempted. Those crashes are also not reproducible when run with go test

e.g.

godev test -run Fuzz -fuzz Fuzz -v -parallel 5
=== FUZZ  FuzzFuzzer
fuzz: elapsed: 0s, gathering baseline coverage: 0/13 completed
fuzz: elapsed: 0s, gathering baseline coverage: 13/13 completed, now fuzzing with 5 workers
fuzz: elapsed: 0s, execs: 100 (1508/sec), new interesting: 0 (total: 9)
--- FAIL: FuzzFuzzer (0.07s)
        --- FAIL: FuzzFuzzer (0.00s)
            foo_test.go:20: minimize this!
    
    Crash written to testdata/fuzz/FuzzFuzzer/9bbe05c1b66fbfdbd1d075945dfe9b96d1b56a04f8edf276fd6b363a92edfb5e
    To re-run:
    go test foo -run=FuzzFuzzer/9bbe05c1b66fbfdbd1d075945dfe9b96d1b56a04f8edf276fd6b363a92edfb5e
FAIL
exit status 1
FAIL    foo     0.214s

[katiehockman]

Figured it out. If a crash occurs while minimizing an interesting input, we don't write that newly crashing input to memory, or set it to the entryOut that is read by the coordinator. I have a patch that fixes it, so I'll clean that up and add a test, and plan to send the fix tomorrow 👍

[katiehockman]

Duplicate of #48731