Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

testing: custom mutator support for fuzzing #48815

Open
s3nt3 opened this issue Oct 6, 2021 · 5 comments
Open

testing: custom mutator support for fuzzing #48815

s3nt3 opened this issue Oct 6, 2021 · 5 comments
Labels
FeatureRequest fuzz NeedsInvestigation
Milestone

Comments

@s3nt3
Copy link

@s3nt3 s3nt3 commented Oct 6, 2021

As the official fuzzer implementation provided by golang, the native fuzzer should be well suited for various usage scenarios. However, currently native fuzzers only support general mutation algorithms for built-in types. Therefore, in many cases, the native fuzzer cannot efficiently generate test inputs. For example, when testing a DSL parser, the mutator will generate a large amount of output that cannot pass the syntax or semantic check. A possible solution is to provide support for custom mutators, so that users can implement custom mutators for various fuzz targets and reuse other parts of the native fuzzer.

I tried on the existing code and designed the following interface:

 type CustomMutator interface {
     Marshal() ([]byte, error)
     Unmarshal([]byte) error
     Mutate() error
 }

The object that implements the above interface can be passed as an argument to the testing.F.Fuzz method, and it will call the Mutate method to use the custom mutation algorithm. The Marshal and Unmarshal methods ensure that it can be imported/exported to a corpus file.

I think supporting custom mutator will bring the following benefits:

  • the usage scenarios of native fuzzer are expanded
  • it is more convenient for the community to test and optimize mutation algorithms
  • the interface of custom mutator is very similar to the struct mutation interface mentioned in the draft proposal, so implementing a custom mutator may be a way to support struct mutation

In addition, custom mutator will also bring some side effects, such as the custom mutator code will be instrumented, which may affect performance and the accuracy of coverage statistics.

/cc @jayconrod

@jayconrod jayconrod changed the title [dev.fuzz] mutator: Add custom mutator support testing: custom mutator support for fuzzing Oct 6, 2021
@jayconrod jayconrod added fuzz NeedsInvestigation labels Oct 6, 2021
@jayconrod jayconrod added this to the Backlog milestone Oct 6, 2021
@jayconrod
Copy link
Contributor

@jayconrod jayconrod commented Oct 6, 2021

cc @golang/fuzzing

@s3nt3
Copy link
Author

@s3nt3 s3nt3 commented Dec 1, 2021

A friendly ping: any update?

@rolandshoemaker
Copy link
Member

@rolandshoemaker rolandshoemaker commented Dec 1, 2021

This is unlikely to be implemented for 1.18, but might be considered for 1.19.

@s3nt3
Copy link
Author

@s3nt3 s3nt3 commented Dec 2, 2021

Thanks for your reply, I will maintain an implementation in my local branch(https://github.com/s3nt3/go/tree/dev.fuzz.custom_mutator). Looking forward to supporting this feature in the new development cycle.

@icholy
Copy link

@icholy icholy commented May 28, 2022

The interface should probably satisfy encoding.{TextMarshaler,TextUnmarshaler}.

 type CustomMutator interface {
     Mutate() error
     MarshalText() ([]byte, error)
     UnmarshalText([]byte) error
 }

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
FeatureRequest fuzz NeedsInvestigation
Projects
None yet
Development

No branches or pull requests

5 participants