Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

testing: document file format for fuzz corpus files #48717

Open
rogpeppe opened this issue Oct 1, 2021 · 4 comments
Open

testing: document file format for fuzz corpus files #48717

rogpeppe opened this issue Oct 1, 2021 · 4 comments

Comments

@rogpeppe
Copy link
Contributor

@rogpeppe rogpeppe commented Oct 1, 2021

commit 8ac5cbe

The fuzz docs inside the testing package say:

Seed inputs may be registered by calling F.Add or by storing files in the directory testdata/fuzz/ (where is the name of the fuzz target) within the package containing the fuzz target.

but the docs don't say what format those files are expected to be in. I was expecting to be able to just copy a load of corpus data from the original fuzz tool (where the format is just exactly the data for the input), but I see errors of the form:

 "testdata/fuzz/FuzzDecoder/27f4f596adb49c6967f41e8f82d3855f5ae9375e": unmarshal: must include version and at least one value

The testing docs should say what format should be used for these files. Ideally there would be a some kind of migration tool to migrate from the old format to the new.

@rogpeppe rogpeppe changed the title testing: document file format for corpus files testing: document file format for fuzz corpus files Oct 1, 2021
@mvdan mvdan added the fuzz label Oct 1, 2021
@mvdan
Copy link
Member

@mvdan mvdan commented Oct 1, 2021

I'm also interested in this for a slightly different reason. Imagine I change my fuzzer to add an extra input parameter: presumably all existing fuzz corpus files won't work out of the box anymore, as the number of inputs mismatches.

It would be nice to be able to write a small program (or use a tool) to rewrite the corpus files accordingly, e.g. by setting the new input parameter to its default value.

@mknyszek mknyszek added this to the Go1.18 milestone Oct 4, 2021
@mknyszek
Copy link
Contributor

@mknyszek mknyszek commented Oct 4, 2021

@katiehockman
Copy link
Member

@katiehockman katiehockman commented Oct 4, 2021

You're in luck! @rolandshoemaker already built a tool for this: https://github.com/golang/tools/tree/master/cmd/file2fuzz
We just need to add documentation about the existence of this tool :)

/cc @golang/fuzzing

@rogpeppe
Copy link
Contributor Author

@rogpeppe rogpeppe commented Oct 4, 2021

@katiehockman Thanks! I actually built a little tool myself, but after doing that, I speculated somewhat on what a more general tool might look like. This is what I came up with:

# add an entry to the corpus; the arguments are parsed using the types
# of the function passed f.Fuzz function. If there's no -f argument
# and there's only one Fuzz* function, it'll be implied from the name of
# that. All go-fuzz-corpus commands accept a -f argument.
go-fuzz-corpus -f FuzzFoo add foo bar 1234

# add -file adds all the files as []byte entries. If there
# are any other arguments to the fuzz function, they'll be zero.
# If there's not not exactly one []byte argument to the
# fuzz function, it'll error.
go-fuzz-corpus add -file oldcorpus/*

# migrate migrates old corpus data to new. The argument is a tuple
# expression where "zero" implies the zero value for a type.
# If there aren't enough members of the tuple, additional zero
# members are implied, so an extra argument can be added
# by running migrate with no arguments.
go-fuzz-corpus migrate 'MyStruct{X: old[0], Y: old[1]}, int64(old[2])'

# list lists all items in the corpus, one per line
go-fuzz-corpus list -n

# type prints the type of arguments stored in the corpus.
go-fuzz-corpus type

# check checks that all the entries in the corpus match the expected
# fuzz function. It also warns when there are files that look like corpus
# entries but that have no corresponding Fuzz entry point.
go-fuzz-corpus check

ISTM it might be useful to add something like this to the go command.
I think that it's OK to assume without significant loss of generality that the fuzz types can be inferred directly from the code, which simplifies the design quite a bit. It wouldn't be too hard to add support for an explicit -types flag when people are using reflection or dynamically determined types.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
5 participants