internal/fuzz,cmd/go: fuzz coordinator quickly runs out of address space on linux/386

[bcmills]

Go version

go version go1.22rc2 linux/amd64

Output of go env in your module/workspace:

GO111MODULE=''
GOARCH='amd64'
GOBIN=''
GOCACHE='/usr/local/google/home/bcmills/.cache/go-build'
GOENV='/usr/local/google/home/bcmills/.config/go/env'
GOEXE=''
GOEXPERIMENT=''
GOFLAGS=''
GOHOSTARCH='amd64'
GOHOSTOS='linux'
GOINSECURE=''
GOMODCACHE='/usr/local/google/home/bcmills/pkg/mod'
GONOPROXY=''
GONOSUMDB=''
GOOS='linux'
GOPATH='/usr/local/google/home/bcmills'
GOPRIVATE=''
GOPROXY='https://proxy.golang.org,direct'
GOROOT='/usr/local/google/home/bcmills/sdk/go1.22rc2'
GOSUMDB='sum.golang.org'
GOTMPDIR=''
GOTOOLCHAIN='local'
GOTOOLDIR='/usr/local/google/home/bcmills/sdk/go1.22rc2/pkg/tool/linux_amd64'
GOVCS=''
GOVERSION='go1.22rc2'
GCCGO='/usr/bin/gccgo'
GOAMD64='v1'
AR='ar'
CC='gcc'
CXX='c++'
CGO_ENABLED='1'
GOMOD='/dev/null'
GOWORK=''
CGO_CFLAGS='-O2 -g'
CGO_CPPFLAGS=''
CGO_CXXFLAGS='-O2 -g'
CGO_FFLAGS='-O2 -g'
CGO_LDFLAGS='-O2 -g'
PKG_CONFIG='pkg-config'
GOGCCFLAGS='-fPIC -m64 -pthread -Wl,--no-gc-sections -fmessage-length=0 -ffile-prefix-map=/tmp/go-build4207537818=/tmp/go-build -gno-record-gcc-switches'

What did you do?

GOARCH=386 go1.22rc2 test cmd/go -run=TestScript/test_fuzz_fuzztime

What did you see happen?

On my workstation (a 24 vCPU Xeon VM), the 32-bit test coordinator process crashes before its 5s fuzztime expires. It appears to run out of address space.

vcs-test.golang.org rerouted to http://127.0.0.1:40697
https://vcs-test.golang.org rerouted to https://127.0.0.1:37583
go test proxy running at GOPROXY=http://127.0.0.1:37953/mod
--- FAIL: TestScript (0.03s)
    --- FAIL: TestScript/test_fuzz_fuzztime (11.18s)
        script_test.go:132: 2024-02-01T18:21:37Z
        script_test.go:134: $WORK=/tmp/cmd-go-test-3219792428/tmpdir1553243795/test_fuzz_fuzztime2893856645
        script_test.go:156: 
            > [!fuzz] skip
            [condition not met]
            > [short] skip
            [condition not met]
            > env GOCACHE=$WORK/cache
            # There are no seed values, so 'go test' should finish quickly. (5.689s)
            # Fuzzing should exit 0 after fuzztime, even if timeout is short. (5.444s)
            > go test -timeout=3s -fuzz=FuzzFast -fuzztime=5s
            [stdout]
            warning: the test binary was not built with coverage instrumentation, so fuzzing will run without coverage guidance and may be inefficient
            warning: starting with empty corpus
            fuzz: elapsed: 0s, execs: 0 (0/sec)
            fuzz: elapsed: 3s, execs: 2296729 (765565/sec)
            runtime: out of memory: cannot allocate 104857600-byte block (1586888704 in use)
            runtime: out of memory: cannot allocate 104857600-byte block (1586888704 in use)
            fatal error: out of memory
            runtime: out of memory: cannot allocate 104857600-byte block (1586888704 in use)
            fatal error: out of memory
            runtime: out of memory: cannot allocate 104857600-byte block (1586888704 in use)
            fatal error: out of memory
            runtime: out of memory: cannot allocate 104857600-byte block (1586888704 in use)
            fatal error: out of memory
            runtime: out of memory: cannot allocate 104857600-byte block (1586888704 in use)
            fatal error: out of memory
            fatal error: out of memory
            
            goroutine 52 gp=0x9da9688 m=33 mp=0x9c81608 [running]:
            runtime.throw({0x81ff4fb, 0xd})
            	/usr/local/google/home/bcmills/sdk/go1.22rc2/src/runtime/panic.go:1023 +0x4d fp=0xa4a772c sp=0xa4a7718 pc=0x808423d
            runtime.(*mcache).allocLarge(0xf7f2ee98, 0x63fff9c, 0x1)
            	/usr/local/google/home/bcmills/sdk/go1.22rc2/src/runtime/mcache.go:236 +0x1c7 fp=0xa4a7758 sp=0xa4a772c pc=0x805d267
            runtime.mallocgc(0x63fff9c, 0x81d5b40, 0x1)
            	/usr/local/google/home/bcmills/sdk/go1.22rc2/src/runtime/malloc.go:1165 +0x640 fp=0xa4a77c0 sp=0xa4a7758 pc=0x80553f0
            runtime.makeslice(0x81d5b40, 0x0, 0x63fff9c)
            	/usr/local/google/home/bcmills/sdk/go1.22rc2/src/runtime/slice.go:107 +0x4f fp=0xa4a77d4 sp=0xa4a77c0 pc=0x809e0ef
            internal/fuzz.(*mutator).mutate(0xa6a6018, {0x9f8f030, 0x1, 0x1}, 0x6400000)
            	/usr/local/google/home/bcmills/sdk/go1.22rc2/src/internal/fuzz/mutator.go:107 +0x4e3 fp=0xa4a7838 sp=0xa4a77d4 pc=0x81846f3
            internal/fuzz.(*workerClient).fuzz(0xa6a6030, {0x823e7ac, 0x9d90030}, {{0x0, 0x0}, {0x9d88010, 0x8}, {0x9d8c000, 0x1b, 0x20}, ...}, ...)
            	/usr/local/google/home/bcmills/sdk/go1.22rc2/src/internal/fuzz/worker.go:1118 +0xc4f fp=0xa4a7a80 sp=0xa4a7838 pc=0x818e44f
            internal/fuzz.(*worker).coordinate(0x9d926e0, {0x823e7ac, 0x9d90030})
            	/usr/local/google/home/bcmills/sdk/go1.22rc2/src/internal/fuzz/worker.go:156 +0x4dc fp=0xa4a7fb0 sp=0xa4a7a80 pc=0x81884fc
            internal/fuzz.CoordinateFuzzing.func3()
            	/usr/local/google/home/bcmills/sdk/go1.22rc2/src/internal/fuzz/fuzz.go:185 +0x45 fp=0xa4a7ff0 sp=0xa4a7fb0 pc=0x81800b5
            runtime.goexit({})
            	/usr/local/google/home/bcmills/sdk/go1.22rc2/src/runtime/asm_386.s:1363 +0x1 fp=0xa4a7ff4 sp=0xa4a7ff0 pc=0x80bd911
            created by internal/fuzz.CoordinateFuzzing in goroutine 9
            	/usr/local/google/home/bcmills/sdk/go1.22rc2/src/internal/fuzz/fuzz.go:184 +0x6d0
            
[…]
            exit status 2
            FAIL	fuzz	5.106s
        script_test.go:156: FAIL: testdata/script/test_fuzz_fuzztime.txt:9: go test -timeout=3s -fuzz=FuzzFast -fuzztime=5s: exit status 1
FAIL
FAIL	cmd/go	11.310s
FAIL

What did you expect to see?

All tests passing. Ideally, go test -fuzz should work reliably on 32-bit systems: it should avoid excessively large inputs, and should store existing inputs with an efficient summary/index and evict data out to temporary files on disk as needed to work within the program's available address space.

[bcmills]

(attn @golang/fuzzing)

[gopherbot]

Change https://go.dev/cl/560515 mentions this issue: cmd/go: limit GOMAXPROCS in TestScript/test_fuzz_fuzztime