testing: fuzzer doesn't timeout infinite loops

[stevenjohnstone]

What version of Go are you using (go version)?

$ go version
go version devel go1.18-d4139083204 Fri Sep 24 07:22:13 2021 +0000 linux/amd64

Does this issue reproduce with the latest release?

Yes

What operating system and processor architecture are you using (go env)?

go env Output

$ go env
GO111MODULE=""
GOARCH="amd64"
GOBIN=""
GOCACHE="/home/stevie/.cache/go-build"
GOENV="/home/stevie/.config/go/env"
GOEXE=""
GOEXPERIMENT=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GOINSECURE=""
GOMODCACHE="/home/stevie/go/pkg/mod"
GONOPROXY=""
GONOSUMDB=""
GOOS="linux"
GOPATH="/home/stevie/go"
GOPRIVATE=""
GOPROXY="https://proxy.golang.org,direct"
GOROOT="/home/stevie/sdk/gotip"
GOSUMDB="sum.golang.org"
GOTMPDIR=""
GOTOOLDIR="/home/stevie/sdk/gotip/pkg/tool/linux_amd64"
GOVCS=""
GOVERSION="devel go1.18-d4139083204 Fri Sep 24 07:22:13 2021 +0000"
GCCGO="gccgo"
GOAMD64="v1"
AR="ar"
CC="gcc"
CXX="g++"
CGO_ENABLED="1"
GOMOD="/home/stevie/code/go/go.mod"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build2423746336=/tmp/go-build -gno-record-gcc-switches"

What did you do?

func FuzzInfinite(f *testing.F) {
	f.Fuzz(func(t *testing.T, a int) {
		if a > 1000 {
			for {
			}
		}
	})

}

gotip test -fuzz=FuzzInfinite
warning: starting with empty corpus
fuzz: elapsed: 0s, execs: 0 (0/sec), interesting: 0
fuzz: elapsed: 3s, execs: 24 (8/sec), interesting: 1
fuzz: elapsed: 6s, execs: 24 (4/sec), interesting: 1
fuzz: elapsed: 9s, execs: 24 (3/sec), interesting: 1
fuzz: elapsed: 12s, execs: 24 (2/sec), interesting: 1
fuzz: elapsed: 15s, execs: 24 (2/sec), interesting: 1
fuzz: elapsed: 18s, execs: 24 (1/sec), interesting: 1
fuzz: elapsed: 21s, execs: 24 (1/sec), interesting: 1
fuzz: elapsed: 24s, execs: 24 (1/sec), interesting: 1
fuzz: elapsed: 27s, execs: 24 (1/sec), interesting: 1

...
fuzz: elapsed: 1h11m48s, execs: 24 (0/sec), interesting: 1
fuzz: elapsed: 1h11m51s, execs: 24 (0/sec), interesting: 1
fuzz: elapsed: 1h11m54s, execs: 24 (0/sec), interesting: 1
fuzz: elapsed: 1h11m57s, execs: 24 (0/sec), interesting: 1
fuzz: elapsed: 1h12m0s, execs: 24 (0/sec), interesting: 1
fuzz: elapsed: 1h12m3s, execs: 24 (0/sec), interesting: 1
fuzz: elapsed: 1h12m6s, execs: 24 (0/sec), interesting: 1
fuzz: elapsed: 1h12m9s, execs: 24 (0/sec), interesting: 1

What did you expect to see?

I'd expect to see a limit on how long the function under test is allowed to run. Finding accidental infinite loops (or even accidentally quadratic functions) is expected of a fuzzer e.g. these from the go-fuzz trophies:

lytics/confl#6
rasky/go-lzo@22d79fd
buger/jsonparser#179
cronokirby/saferith@d39f5a2

Ideally, the fuzzer would timeout the worker and store the input as a crasher in the corpus. FWIW go-fuzz seems to set the timeout at 10s.

What did you see instead?

Runs for over an hour without timing out.

[jayconrod]

Duplicate of #48591. We definitely need this though.

The main blocker is deciding the API/CLI to set the timeout. #48157 is a proposal for per-test timeouts, and the interface would be shared between tests and fuzzing.