Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

testing: fuzzer doesn't timeout infinite loops #48611

Closed
stevenjohnstone opened this issue Sep 24, 2021 · 1 comment
Closed

testing: fuzzer doesn't timeout infinite loops #48611

stevenjohnstone opened this issue Sep 24, 2021 · 1 comment

Comments

@stevenjohnstone
Copy link

@stevenjohnstone stevenjohnstone commented Sep 24, 2021

What version of Go are you using (go version)?

$ go version
go version devel go1.18-d4139083204 Fri Sep 24 07:22:13 2021 +0000 linux/amd64

Does this issue reproduce with the latest release?

Yes

What operating system and processor architecture are you using (go env)?

go env Output
$ go env
GO111MODULE=""
GOARCH="amd64"
GOBIN=""
GOCACHE="/home/stevie/.cache/go-build"
GOENV="/home/stevie/.config/go/env"
GOEXE=""
GOEXPERIMENT=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GOINSECURE=""
GOMODCACHE="/home/stevie/go/pkg/mod"
GONOPROXY=""
GONOSUMDB=""
GOOS="linux"
GOPATH="/home/stevie/go"
GOPRIVATE=""
GOPROXY="https://proxy.golang.org,direct"
GOROOT="/home/stevie/sdk/gotip"
GOSUMDB="sum.golang.org"
GOTMPDIR=""
GOTOOLDIR="/home/stevie/sdk/gotip/pkg/tool/linux_amd64"
GOVCS=""
GOVERSION="devel go1.18-d4139083204 Fri Sep 24 07:22:13 2021 +0000"
GCCGO="gccgo"
GOAMD64="v1"
AR="ar"
CC="gcc"
CXX="g++"
CGO_ENABLED="1"
GOMOD="/home/stevie/code/go/go.mod"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build2423746336=/tmp/go-build -gno-record-gcc-switches"

What did you do?

func FuzzInfinite(f *testing.F) {
	f.Fuzz(func(t *testing.T, a int) {
		if a > 1000 {
			for {
			}
		}
	})

}
gotip test -fuzz=FuzzInfinite
warning: starting with empty corpus
fuzz: elapsed: 0s, execs: 0 (0/sec), interesting: 0
fuzz: elapsed: 3s, execs: 24 (8/sec), interesting: 1
fuzz: elapsed: 6s, execs: 24 (4/sec), interesting: 1
fuzz: elapsed: 9s, execs: 24 (3/sec), interesting: 1
fuzz: elapsed: 12s, execs: 24 (2/sec), interesting: 1
fuzz: elapsed: 15s, execs: 24 (2/sec), interesting: 1
fuzz: elapsed: 18s, execs: 24 (1/sec), interesting: 1
fuzz: elapsed: 21s, execs: 24 (1/sec), interesting: 1
fuzz: elapsed: 24s, execs: 24 (1/sec), interesting: 1
fuzz: elapsed: 27s, execs: 24 (1/sec), interesting: 1

...
fuzz: elapsed: 1h11m48s, execs: 24 (0/sec), interesting: 1
fuzz: elapsed: 1h11m51s, execs: 24 (0/sec), interesting: 1
fuzz: elapsed: 1h11m54s, execs: 24 (0/sec), interesting: 1
fuzz: elapsed: 1h11m57s, execs: 24 (0/sec), interesting: 1
fuzz: elapsed: 1h12m0s, execs: 24 (0/sec), interesting: 1
fuzz: elapsed: 1h12m3s, execs: 24 (0/sec), interesting: 1
fuzz: elapsed: 1h12m6s, execs: 24 (0/sec), interesting: 1
fuzz: elapsed: 1h12m9s, execs: 24 (0/sec), interesting: 1

What did you expect to see?

I'd expect to see a limit on how long the function under test is allowed to run. Finding accidental infinite loops (or even accidentally quadratic functions) is expected of a fuzzer e.g. these from the go-fuzz trophies:

lytics/confl#6
rasky/go-lzo@22d79fd
buger/jsonparser#179
cronokirby/saferith@d39f5a2

Ideally, the fuzzer would timeout the worker and store the input as a crasher in the corpus. FWIW go-fuzz seems to set the timeout at 10s.

What did you see instead?

Runs for over an hour without timing out.

@jayconrod
Copy link
Contributor

@jayconrod jayconrod commented Sep 24, 2021

Duplicate of #48591. We definitely need this though.

The main blocker is deciding the API/CLI to set the timeout. #48157 is a proposal for per-test timeouts, and the interface would be shared between tests and fuzzing.

Loading

@jayconrod jayconrod closed this Sep 24, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
2 participants