Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

testing: document file format for fuzz corpus files #48717

Closed
rogpeppe opened this issue Oct 1, 2021 · 5 comments
Closed

testing: document file format for fuzz corpus files #48717

rogpeppe opened this issue Oct 1, 2021 · 5 comments
Labels
Documentation fuzz Issues related to native fuzzing support NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one.
Milestone

Comments

@rogpeppe
Copy link
Contributor

rogpeppe commented Oct 1, 2021

commit 8ac5cbe

The fuzz docs inside the testing package say:

Seed inputs may be registered by calling F.Add or by storing files in the directory testdata/fuzz/ (where is the name of the fuzz target) within the package containing the fuzz target.

but the docs don't say what format those files are expected to be in. I was expecting to be able to just copy a load of corpus data from the original fuzz tool (where the format is just exactly the data for the input), but I see errors of the form:

 "testdata/fuzz/FuzzDecoder/27f4f596adb49c6967f41e8f82d3855f5ae9375e": unmarshal: must include version and at least one value

The testing docs should say what format should be used for these files. Ideally there would be a some kind of migration tool to migrate from the old format to the new.

@rogpeppe rogpeppe changed the title testing: document file format for corpus files testing: document file format for fuzz corpus files Oct 1, 2021
@mvdan mvdan added the fuzz Issues related to native fuzzing support label Oct 1, 2021
@mvdan
Copy link
Member

mvdan commented Oct 1, 2021

I'm also interested in this for a slightly different reason. Imagine I change my fuzzer to add an extra input parameter: presumably all existing fuzz corpus files won't work out of the box anymore, as the number of inputs mismatches.

It would be nice to be able to write a small program (or use a tool) to rewrite the corpus files accordingly, e.g. by setting the new input parameter to its default value.

@mknyszek mknyszek added this to the Go1.18 milestone Oct 4, 2021
@mknyszek
Copy link
Contributor

mknyszek commented Oct 4, 2021

CC @katiehockman

@katiehockman
Copy link
Contributor

katiehockman commented Oct 4, 2021

You're in luck! @rolandshoemaker already built a tool for this: https://github.com/golang/tools/tree/master/cmd/file2fuzz
We just need to add documentation about the existence of this tool :)

/cc @golang/fuzzing

@mknyszek mknyszek added the NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. label Oct 4, 2021
@rogpeppe
Copy link
Contributor Author

rogpeppe commented Oct 4, 2021

@katiehockman Thanks! I actually built a little tool myself, but after doing that, I speculated somewhat on what a more general tool might look like. This is what I came up with:

# add an entry to the corpus; the arguments are parsed using the types
# of the function passed f.Fuzz function. If there's no -f argument
# and there's only one Fuzz* function, it'll be implied from the name of
# that. All go-fuzz-corpus commands accept a -f argument.
go-fuzz-corpus -f FuzzFoo add foo bar 1234

# add -file adds all the files as []byte entries. If there
# are any other arguments to the fuzz function, they'll be zero.
# If there's not not exactly one []byte argument to the
# fuzz function, it'll error.
go-fuzz-corpus add -file oldcorpus/*

# migrate migrates old corpus data to new. The argument is a tuple
# expression where "zero" implies the zero value for a type.
# If there aren't enough members of the tuple, additional zero
# members are implied, so an extra argument can be added
# by running migrate with no arguments.
go-fuzz-corpus migrate 'MyStruct{X: old[0], Y: old[1]}, int64(old[2])'

# list lists all items in the corpus, one per line
go-fuzz-corpus list -n

# type prints the type of arguments stored in the corpus.
go-fuzz-corpus type

# check checks that all the entries in the corpus match the expected
# fuzz function. It also warns when there are files that look like corpus
# entries but that have no corresponding Fuzz entry point.
go-fuzz-corpus check

ISTM it might be useful to add something like this to the go command.
I think that it's OK to assume without significant loss of generality that the fuzz types can be inferred directly from the code, which simplifies the design quite a bit. It wouldn't be too hard to add support for an explicit -types flag when people are using reflection or dynamically determined types.

@gopherbot
Copy link

gopherbot commented Dec 15, 2021

Change https://golang.org/cl/372434 mentions this issue: _content/doc/fuzz: document corpus file format

passionSeven added a commit to passionSeven/website that referenced this issue Oct 18, 2022
For golang/go#48255
Fixes golang/go#48717

Change-Id: Ie87c09b9e031a2221ca77d33c4e8140d787a33d2
Reviewed-on: https://go-review.googlesource.com/c/website/+/372434
Run-TryBot: DO NOT USE <katiehockman@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Julie Qiu <julieqiu@google.com>
Trust: DO NOT USE <katiehockman@google.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Documentation fuzz Issues related to native fuzzing support NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one.
Projects
Status: No status
Development

No branches or pull requests

5 participants