Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

cmd/go/internal/cache: in Cache.Trim, evict fuzz files #48526

Open
jayconrod opened this issue Sep 21, 2021 · 2 comments
Open

cmd/go/internal/cache: in Cache.Trim, evict fuzz files #48526

jayconrod opened this issue Sep 21, 2021 · 2 comments
Labels
fuzz NeedsFix
Milestone

Comments

@jayconrod
Copy link
Contributor

@jayconrod jayconrod commented Sep 21, 2021

The fuzzing engine started with go test -fuzz stores inputs that expand code coverage in a subdirectory within the build cache. The subdirectory name is $GOCACHE/fuzz/$pkgpath/$target where $pkgpath is the import path of the package containing the fuzz target, and $target is the name of the fuzz target. The files are named after the SHA-256 sum of their contents, but the name doesn't matter, and we don't expect any other files in these directories.

Currently, the fuzzing cache is unlimited in both time and space. The user can clear the entire cache with go clean -fuzzcache, but that may reduce effectiveness of fuzzing until new inputs are discovered that provide equivalent coverage.

Before 1.18, we should implement at least a minimal cache eviction algorithm. It's probably fine to do the same thing we do for the build cache:

  • When we fuzz a target with compatible cached files, set the modification time on those files to the current time if the previous time is more than 1 hour in the past.
  • When Cache.Trim is called (i.e., when any go command the uses the cache runs, at least 24 hours after the last Cache.Trim), delete files with modification times more than 5 days in the past.

5 days was a time limit that worked well for builds. We may want to choose a longer limit like 30 days for cached fuzz inputs.

cc @golang/Fuzzing

@jayconrod jayconrod added NeedsFix release-blocker fuzz labels Sep 21, 2021
@jayconrod jayconrod added this to the Go1.18 milestone Sep 21, 2021
@katiehockman
Copy link
Contributor

@katiehockman katiehockman commented Oct 6, 2021

Removed the release-blocker label after some internal discussion.

I have a few concerns with this. I'm imaging a scenario where someone ran fuzzing for several days, and built up a sizable cache of interesting values, then puts it down for a while (e.g. they went on vacation, this is a side project, etc). They come back to find that their entire fuzzing cache has been evicted without their knowledge. That could be a pretty frustrating experience. It makes sense for the build cache to auto-evict after a period of time, but the fuzz "cache" isn't really a cache at all in the sense that the files may be long lived.

My preference would be hold off on doing this for now, and add some very clear documentation for folks should they run into an issue where evicting their cache would be helpful. We can gain more insight on whether or not this would be a good feature to implement, and if so, how long we should let old fuzz cache entries sit around (30 days? 90 days? longer?) after the release, and re-visit this for 1.19.

@ianlancetaylor
Copy link
Contributor

@ianlancetaylor ianlancetaylor commented Jun 24, 2022

CC @golang/fuzzing

Looks like this didn't make 1.19. Moving to backlog. Please recategorize as appropriate. Thanks.

@ianlancetaylor ianlancetaylor removed this from the Go1.19 milestone Jun 24, 2022
@ianlancetaylor ianlancetaylor added this to the Backlog milestone Jun 24, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
fuzz NeedsFix
Projects
Status: No status
Development

No branches or pull requests

3 participants