Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

testing: minimized crasher is unrelated to original #48326

Open
stevenjohnstone opened this issue Sep 10, 2021 · 3 comments
Open

testing: minimized crasher is unrelated to original #48326

stevenjohnstone opened this issue Sep 10, 2021 · 3 comments

Comments

@stevenjohnstone
Copy link

@stevenjohnstone stevenjohnstone commented Sep 10, 2021

What version of Go are you using (go version)?

$ go version
go version devel go1.18-7c648e2acb3 Thu Sep 9 17:28:03 2021 +0000 linux/amd64

Does this issue reproduce with the latest release?

n/a

What operating system and processor architecture are you using (go env)?

go env Output
$ go env
GO111MODULE=""
GOARCH="amd64"
GOBIN=""
GOCACHE="/home/stevie/.cache/go-build"
GOENV="/home/stevie/.config/go/env"
GOEXE=""
GOEXPERIMENT=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GOINSECURE=""
GOMODCACHE="/home/stevie/go/pkg/mod"
GONOPROXY=""
GONOSUMDB=""
GOOS="linux"
GOPATH="/home/stevie/go"
GOPRIVATE=""
GOPROXY="https://proxy.golang.org,direct"
GOROOT="/home/stevie/sdk/gotip"
GOSUMDB="sum.golang.org"
GOTMPDIR=""
GOTOOLDIR="/home/stevie/sdk/gotip/pkg/tool/linux_amd64"
GOVCS=""
GOVERSION="devel go1.18-7c648e2acb3 Thu Sep 9 17:28:03 2021 +0000"
GCCGO="gccgo"
AR="ar"
CC="gcc"
CXX="g++"
CGO_ENABLED="1"
GOMOD="/home/stevie/minimize/go.mod"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build2657391484=/tmp/go-build -gno-record-gcc-switches"

What did you do?

Here's the simplest case I can find which reproduces the problem:

package minimize

import (
	"fmt"
	"testing"
)

//go:noinline
func branch1(i int) {
	b := []byte{1, 2, 3}
	fmt.Printf("%d\n", b[4])
}

//go:noinline
func branch2(i int) {
	panic(i)
}

func FuzzMinimizeToWrongCrash(f *testing.F) {
	f.Add(-20000) // this will force the fuzzer to crash in the the first branch
	f.Fuzz(func(t *testing.T, b int) {
		if b < -20000 {
			branch1(b)
			return
		}

		if b > 10000 {
			// the corpus will contain a crasher for this branch
			branch2(b)
			return
		}
	})
}

The test is rigged so it'll crash in branch1. Minimization kicks in and it ends up crashing in branch2:

$ ./minimize.test -test.fuzz=Fuzz -test.fuzzcachedir=./cache
found a crash, minimizing...
fuzzing, elapsed: 0.0s, execs: 85 (7363/sec), workers: 8, interesting: 0
--- FAIL: FuzzMinimizeToWrongCrash (0.01s)
        panic: %!s(int=18446)
        goroutine 57 [running]:
        runtime/debug.Stack()
        	/home/stevie/sdk/gotip/src/runtime/debug/stack.go:24 +0x90
        testing.tRunner.func1.2({0x58c480, 0xc000118a68})
        	/home/stevie/sdk/gotip/src/testing/testing.go:1288 +0x265
        testing.tRunner.func1()
        	/home/stevie/sdk/gotip/src/testing/testing.go:1295 +0x225
        panic({0x58c480, 0xc000118a68})
        	/home/stevie/sdk/gotip/src/runtime/panic.go:814 +0x207
        minimize.branch2(0x0)
        	/home/stevie/minimize/fuzz_test.go:16 +0x37
        minimize.FuzzMinimizeToWrongCrash.func1(0x0, 0x480e)
        	/home/stevie/minimize/fuzz_test.go:29 +0x86
        reflect.Value.call({0x58e720, 0x5c7c78, 0x13}, {0x5ba083, 0x4}, {0xc000116b10, 0x2, 0x2})
        	/home/stevie/sdk/gotip/src/reflect/value.go:542 +0x814
        reflect.Value.Call({0x58e720, 0x5c7c78, 0xc000113a00}, {0xc000116b10, 0x2, 0x2})
        	/home/stevie/sdk/gotip/src/reflect/value.go:338 +0xc5
        testing.(*F).Fuzz.func1.1(0x0)
        	/home/stevie/sdk/gotip/src/testing/fuzz.go:389 +0x1c6
        testing.tRunner(0xc000194680, 0xc00010c980)
        	/home/stevie/sdk/gotip/src/testing/testing.go:1342 +0x102
        created by testing.(*F).Fuzz.func1
        	/home/stevie/sdk/gotip/src/testing/fuzz.go:378 +0x4e5
        
        --- FAIL: FuzzMinimizeToWrongCrash (0.00s)
    
    Crash written to testdata/corpus/FuzzMinimizeToWrongCrash/7219f755b8af33d79397c74214a3126d25f0664bbdac3dcd511325de4cc54063
    To re-run:
    go test minimize -run=FuzzMinimizeToWrongCrash/7219f755b8af33d79397c74214a3126d25f0664bbdac3dcd511325de4cc54063
FAIL
$ cat testdata/corpus/FuzzMinimizeToWrongCrash/7219f755b8af33d79397c74214a3126d25f0664bbdac3dcd511325de4cc54063 
go test fuzz v1
int(18446)

Instrumenting with this bpftrace script

uprobe:./minimize.test:"minimize.branch1" {
  printf("%d: branch 1 taken with value %d\n", tid, reg("ax"));
}
uprobe:./minimize.test:"minimize.branch2" {
  printf("%d: branch 2 taken with value %d\n", tid, reg("ax"));
}

I see that all the worker processes run branch1 first:

81977: branch 1 taken with value -20123
81965: branch 1 taken with value -20052
81959: branch 1 taken with value -20093
81952: branch 1 taken with value -20004
81959: branch 1 taken with value -20093
81959: branch 1 taken with value -20093
81959: branch 2 taken with value -1717988928
81959: branch 2 taken with value -171798893
81959: branch 2 taken with value 1271310299
81959: branch 2 taken with value -1161359159
81959: branch 2 taken with value -1404626105
81959: branch 2 taken with value -140462611
81959: branch 2 taken with value 2133437386
81959: branch 2 taken with value -216152991
81959: branch 2 taken with value 1266874889
81959: branch 2 taken with value 1844674407
81959: branch 2 taken with value 184467440
81959: branch 2 taken with value 18446744
81959: branch 2 taken with value 1844674
81959: branch 2 taken with value 184467
81959: branch 2 taken with value 18446
81968: branch 1 taken with value -20002
81989: branch 1 taken with value -20031
81948: branch 1 taken with value -20049

The tail of the log shows what looks like the negative value -20123 being cast to a large uint and then divided by 10 repeatedly to reach the value 18446 in the corpus. This may be another issue if the intention was to preserve the sign of integers on minimization 🤷

What did you expect to see?

I expected to see the original crash recorded. The first and second crashes have different coverage and different panic messages/backtraces. There's enough information to label these as different crashes.

In a more complicated example, reducing one type of crash to another and then discarding the original wastes a lot of work. The fuzzer would need to be run again to find the original crash. It'd be really useful if the fuzzer retained the original crasher and provided a minimized version as a feature.

What did you see instead?

The original crasher is lost.

@thanm
Copy link
Contributor

@thanm thanm commented Sep 10, 2021

@katiehockman katiehockman added this to the Go1.18 milestone Sep 14, 2021
@rsc rsc changed the title [dev.fuzz] minimized crasher is unrelated to original testing: minimized crasher is unrelated to original Sep 21, 2021
@toothrot
Copy link
Contributor

@toothrot toothrot commented Sep 22, 2021

Checking in on this issue as it's labeled a release blocker for Go 1.18. Is there any update?

@jayconrod
Copy link
Contributor

@jayconrod jayconrod commented Sep 22, 2021

Not yet. There are a lot of fuzzing issues. We'll get to them as soon as we can.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
5 participants