Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

testing: visualize fuzz corpus coverage #46765

Open
rolandshoemaker opened this issue Jun 15, 2021 · 3 comments
Open

testing: visualize fuzz corpus coverage #46765

rolandshoemaker opened this issue Jun 15, 2021 · 3 comments

Comments

@rolandshoemaker
Copy link
Member

@rolandshoemaker rolandshoemaker commented Jun 15, 2021

It would be nice to be able to visualize the source coverage provided by an input corpus, in a similar manner to the coverage information provided for unit tests.

Fuzzing counters are implemented in a slightly different fashion to the (testing) coverage counters, but probably a similar method can be used to produce compatible coverage profiles for use with go tool cover.

@rolandshoemaker
Copy link
Member Author

@rolandshoemaker rolandshoemaker commented Jun 15, 2021

This may be more complicated than it seemed on the surface.

Since we add the libfuzzer-style counters during compilation, while doing other things to the source (like in-lining), the code we are instrumenting does not necessarily look like the source on disk, making mapping from counter positions to source positions complicated.

One other approach would be to just use the existing go tool cover coverage instrumentation, in addition to the libfuzzer instrumentation, since it already exists. There may be downsides I cannot think of at the moment, but it seems like a rather simple approach.

@timothy-king
Copy link
Contributor

@timothy-king timothy-king commented Jun 15, 2021

One other approach would be to just use the existing go tool cover coverage instrumentation, in addition to the libfuzzer instrumentation, since it already exists. There may be downsides I cannot think of at the moment, but it seems like a rather simple approach.

IIUC the suggestion is to instrument the program with two different coverage metrics, produce a single binary where both coverage metrics are aggregated during execution, but the two different metrics are post processed differently. So go tool cover is used for producing reports and the libfuzzer style counters are used for determining if coverage increased. Is this an accurate understanding of the approach?

Another approach could be to re-run the fuzzer generated tests under a go tool cover instrumented binary to produce the coverage information. FWIW other fuzzing systems have accepted a similar division of labor at the cost of needing to compile 2 binaries.

@cagedmantis cagedmantis added this to the Backlog milestone Jun 17, 2021
@cagedmantis
Copy link
Contributor

@cagedmantis cagedmantis commented Jun 17, 2021

@rsc rsc changed the title [dev.fuzz] testing: visualize corpus coverage testing: visualize fuzz corpus fuzz Sep 21, 2021
@rsc rsc changed the title testing: visualize fuzz corpus fuzz testing: visualize fuzz corpus coverage Sep 21, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
4 participants