testing: fuzzing failed to run if test run with open TCP connections

[shuLhan]

Go version

go version devel go1.25-cc874072f3 Mon Feb 3 08:25:31 2025 -0800 linux/amd64

Output of go env in your module/workspace:

AR='ar'                                                                                                                 
CC='gcc'
CGO_CFLAGS='-O2 -g'
CGO_CPPFLAGS=''
CGO_CXXFLAGS='-O2 -g'
CGO_ENABLED='0'
CGO_FFLAGS='-O2 -g'
CGO_LDFLAGS='-O2 -g'
CXX='g++'
GCCGO='gccgo'
GO111MODULE=''
GOAMD64='v1'
GOARCH='amd64'
GOAUTH='netrc'
GOBIN='/home/ms/go/bin'
GOCACHE='/home/ms/.cache/go-build'
GOCACHEPROG=''
GODEBUG=''
GOENV='/home/ms/.config/go/env'
GOEXE=''
GOEXPERIMENT=''
GOFIPS140='off'
GOFLAGS=''
GOGCCFLAGS='-fPIC -m64 -Wl,--no-gc-sections -fmessage-length=0 -ffile-prefix-map=/tmp/go-build2786547649=/tmp/go-build -gno-record-gcc-switches'
GOHOSTARCH='amd64'
GOHOSTOS='linux'
GOINSECURE=''
GOMOD='/home/ms/src/go/src/go.mod'
GOMODCACHE='/home/ms/go/pkg/mod'
GONOPROXY='<REDACTED>'
GONOSUMDB='<REDACTED>'
GOOS='linux'
GOPATH='/home/ms/go'
GOPRIVATE='<REDACTED>'
GOPROXY='https://proxy.golang.org,direct'
GOROOT='/home/ms/local/share/go'
GOSUMDB='sum.golang.org'
GOTELEMETRY='on'
GOTELEMETRYDIR='/home/ms/.config/go/telemetry'
GOTMPDIR=''

What did you do?

Given the following code and fuzzing test based on Fuzzing tutorial [1]

package main

import (
	"fmt"
	"html"
	"log"
	"net"
	"net/http"
	"testing"
	"unicode/utf8"
)

func FuzzReverse(f *testing.F) {
	// Uncomment either runHTTPServer or runTCPServer to make test fuzz
	// fail.
	go runHTTPServer()
	//go runTCPServer()
	testcases := []string{"Hello, world", " ", "!12345"}
	for _, tc := range testcases {
		f.Add(tc) // Use f.Add to provide a seed corpus
	}
	f.Fuzz(func(t *testing.T, orig string) {
		rev, err1 := Reverse(orig)
		if err1 != nil {
			return
		}
		doubleRev, err2 := Reverse(rev)
		if err2 != nil {
			return
		}
		if orig != doubleRev {
			t.Errorf("Before: %q, after: %q", orig, doubleRev)
		}
		if utf8.ValidString(orig) && !utf8.ValidString(rev) {
			t.Errorf("Reverse produced invalid UTF-8 string %q", rev)
		}
	})
}

func runHTTPServer() {
	http.HandleFunc("/bar", func(w http.ResponseWriter, r *http.Request) {
		fmt.Fprintf(w, "Hello, %q", html.EscapeString(r.URL.Path))
	})

	log.Fatal(http.ListenAndServe(":18080", nil))
}

func runTCPServer() {
	ln, err := net.Listen("tcp", ":18081")
	if err != nil {
		log.Fatal(`runTCPServer:`, err)
	}
	for {
		_, err := ln.Accept()
		if err != nil {
			log.Fatal(`runTCPServer: Accept`, err)
		}
	}
}

I believe the same issues has been reported several times,

• cmd/go: fuzzing doesn't work with environment startup #52133
• fuzzing process terminated without fuzzing: EOF #61742

There are other issues but I am not sure if its related,

• testing: fuzzer aborts with "fuzzing process hung or terminated unexpectedly while minimizing: EOF" #52569

The only error that I can trace is there is a broken pipe during ping [2].

[1] https://go.dev/doc/tutorial/fuzz
[2]

go/src/internal/fuzz/worker.go

Line 312 in b07b20f

if err := w.client.ping(ctx); err != nil {

What did you see happen?

The fuzz process terminated with an error EOF,

$ go test -fuzz=. -run=none -fuzztime=10s -v
=== RUN   FuzzReverse
fuzz: elapsed: 0s, gathering baseline coverage: 0/46 completed
fuzz: elapsed: 0s, gathering baseline coverage: 0/46 completed
--- FAIL: FuzzReverse (0.02s)
    fuzzing process terminated without fuzzing: EOF
=== NAME  
FAIL
exit status 1
FAIL    git.sr.ht/~shulhan/sandbox.go/fuzz      0.021s

What did you expect to see?

Uncomment both of the lines in FuzzReverse,

	//go runHTTPServer()
	//go runTCPServer()

and run the test again,

$ go test -fuzz=. -run=none -fuzztime=10s -v                                                                      
=== RUN   FuzzReverse                                                                                                   
fuzz: elapsed: 0s, gathering baseline coverage: 0/46 completed
fuzz: elapsed: 0s, gathering baseline coverage: 46/46 completed, now fuzzing with 4 workers
fuzz: elapsed: 3s, execs: 130236 (43416/sec), new interesting: 0 (total: 46)
fuzz: elapsed: 6s, execs: 255613 (41785/sec), new interesting: 0 (total: 46)
fuzz: elapsed: 9s, execs: 387455 (43941/sec), new interesting: 0 (total: 46)
fuzz: elapsed: 10s, execs: 431389 (40198/sec), new interesting: 0 (total: 46)
--- PASS: FuzzReverse (10.10s) 
=== NAME  
PASS
ok      git.sr.ht/~shulhan/sandbox.go/fuzz      10.107s

[gabyhelp]

Related Issues

• [dev.fuzz] Crash data written to corpus is incorrect #47587 (closed)
• cmd/go: fuzzing doesn't work with environment startup #52133 (closed)
• testing,internal/fuzz: Fuzzer reports 0 execs/sec and keeps running #51484 (closed)
• internal/fuzz: many data races when fuzzing trivial string input #48307 (closed)
• cmd/test2json: test2json doesn't print the final "pass" action if fuzz test is interrupted #53563 (closed)
• testing: fuzzed function can corrupt internal state of fuzzer #48606
• [dev.fuzz] cmd/go: hangs in "gathering baseline coverage" #46633 (closed)
• testing: fuzzer aborts with "fuzzing process hung or terminated unexpectedly while minimizing: EOF" #52569
• testing: fuzzer doesn't timeout infinite loops #48611 (closed)
• testing: reflect panic when the fuzz function converts between string and []byte #48320 (closed)

_{(Emoji vote if this was helpful or unhelpful; more detailed feedback welcome in this discussion.)}

[seankhliao]

fuzzing uses multiple subprocesses (since the processes can crash). that is incompatible with using a fixed port.

https://go.dev/doc/security/fuzz/

closing as working as intended.

[shuLhan]

---------------------------------------- 5 Feb 2025 01:00:42 Sean Liao ***@***.***>:

fuzzing uses multiple subprocesses (since the processes can crash). that is incompatible with using a fixed port. https://go.dev/doc/security/fuzz/ closing as working as intended. — Reply to this email directly, view it on GitHub[#71560 (comment)], or unsubscribe[https://github.com/notifications/unsubscribe-auth/AAAKSKSISEGDX6E3KP22YTD2OD54LAVCNFSM6AAAAABWPCWDW2VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDMMZUGY4TENZRGI]. You are receiving this because you authored the thread. [Tracking image][https://github.com/notifications/beacon/AAAKSKV3G7KPGS2QESDW2GL2OD54LA5CNFSM6AAAAABWPCWDW2WGG33NNVSW45C7OR4XAZNMJFZXG5LFINXW23LFNZ2KUY3PNVWWK3TUL5UWJTU5BI4GQ.gif] fuzzing uses multiple subprocesses (since the processes can crash). that is incompatible with using a fixed port.

? I am not sure this issue have to do with fixed port. Multiple subprocess, maybe.

https://go.dev/doc/security/fuzz/

I believe I have read the doc. None in the page mention about fixed port nor sub process, and its consequences.

closing as working as intended.

?? I am sorry, what is going on here?

[ianlancetaylor]

The error means that the program being fuzzed crashed unexpectedly.

In your case it most likely crashed because you are using fixed ports, and crashing if opening the port fails.

[shuLhan]

Thank you @seankhliao and @ianlancetaylor for the answers.

I keep thinking about this issue and I think there are some underlying issues with current fuzzing.
I take notes here so someone that new to Go fuzzing, like me, can read it when stumbling with the same problem in the future. Please correct me if I am wrong here.

1. The behaviour of running go test with fuzzing is different with the rest of test (TestXxx function and BenchXxx function). Unlike test function, go test with fuzzing run the test binary multiple times. This behaviour as far as I know is undocumented.

2. Any error from the test binary does not printed to the (?) stderr. This caused further confusion when inspecting the fuzzing output. For example, in my case above, it should at least print an error, something like, "listen tcp :18080: bind: address already in use" instead of only "fuzzing process terminated without fuzzing: EOF"