Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[dev.fuzz] cmd/go: simple off-by-one example takes longer than expected to complete #47090

Open
picatz opened this issue Jul 7, 2021 · 3 comments

Comments

@picatz
Copy link

@picatz picatz commented Jul 7, 2021

What version of Go are you using (go version)?

$ gotip version
go version devel go1.17-542e8c74e7 Fri Jun 4 15:59:32 2021 +0000 linux/amd64

What did you do?

I ported a simple example I've used to test go-fuzz in the past:

// +build gofuzz

package fuzz

func Example(data []byte) bool {
	if len(data) == 9 {
		if data[0] == 'G' && data[1] == 'O' && data[2] == 'P' && data[3] == 'H' && data[4] == 'E' && data[5] == 'R' && data[6] == 'S' && data[7] == '!' && data[8] == '!' && data[9] == '!' {
			return true
		}
	}
	return false
}

func Fuzz(data []byte) int {
	Example(data)
	return 0
}

⬇️

// +build gofuzzbeta

package fuzz

import (
	"testing"
)

// Example is a common fuzzing example that demonstrates an off-by-one/out-of-bounds 
// error which causes the program to crash. 
//
// Instead of checking that len(data) == 9 the correct code should be len(data) == 10.
func Example(data []byte) bool {
	if len(data) == 9 {
		if data[0] == 'G' && data[1] == 'O' && data[2] == 'P' && data[3] == 'H' && data[4] == 'E' && data[5] == 'R' && data[6] == 'S' && data[7] == '!' && data[8] == '!' && data[9] == '!' {
			return true
		}
	}
	return false
}

func FuzzOffByOne(f *testing.F) {
	f.Fuzz(func(t *testing.T, input []byte) {
		Example(input)
	})
}
$ gotip test -fuzz=FuzzOffByOne

What did you expect to see?

I've run the example several times before, and would generally expect a fuzzer to find a crash fairly fast, even without a corpus -- somewhere between a few seconds to a few minutes.

Using the same CPU and RAM configuration as the native fuzzer test, I was able to find it within seconds using go-fuzz:

Screen Shot 2021-07-07 at 5 26 22 PM

What did you see instead?

With the new native fuzzer, it took ~24 hours to find a crash using 2GB of RAM and 1 CPU (from an n1-standard-2 instance):

Screen Shot 2021-07-07 at 5 05 41 PM

@mknyszek
Copy link
Contributor

@mknyszek mknyszek commented Jul 7, 2021

CC @katiehockman

@mknyszek mknyszek added this to the Go1.17 milestone Jul 7, 2021
@mknyszek mknyszek removed this from the Go1.17 milestone Jul 7, 2021
@mknyszek mknyszek added this to the Go1.18 milestone Jul 7, 2021
@katiehockman
Copy link
Member

@katiehockman katiehockman commented Jul 8, 2021

CC @golang/fuzzing

@rolandshoemaker
Copy link
Member

@rolandshoemaker rolandshoemaker commented Jul 9, 2021

This is an interesting comparison since the target expects a very specific input size. It seems plausible that go-fuzz is much more conservative about increasing the size of inputs during mutation (perhaps in relation to the initial size of the input), so when starting from an empty input you're more likely to hit an input which is 9 bytes long, whereas our mutators don't really take this into account, making it much more of a lottery.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
4 participants