Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

cmd/go/internal/cache: in Cache.Trim, evict fuzz files #48526

Open
jayconrod opened this issue Sep 21, 2021 · 1 comment
Open

cmd/go/internal/cache: in Cache.Trim, evict fuzz files #48526

jayconrod opened this issue Sep 21, 2021 · 1 comment

Comments

@jayconrod
Copy link
Contributor

@jayconrod jayconrod commented Sep 21, 2021

The fuzzing engine started with go test -fuzz stores inputs that expand code coverage in a subdirectory within the build cache. The subdirectory name is $GOCACHE/fuzz/$pkgpath/$target where $pkgpath is the import path of the package containing the fuzz target, and $target is the name of the fuzz target. The files are named after the SHA-256 sum of their contents, but the name doesn't matter, and we don't expect any other files in these directories.

Currently, the fuzzing cache is unlimited in both time and space. The user can clear the entire cache with go clean -fuzzcache, but that may reduce effectiveness of fuzzing until new inputs are discovered that provide equivalent coverage.

Before 1.18, we should implement at least a minimal cache eviction algorithm. It's probably fine to do the same thing we do for the build cache:

  • When we fuzz a target with compatible cached files, set the modification time on those files to the current time if the previous time is more than 1 hour in the past.
  • When Cache.Trim is called (i.e., when any go command the uses the cache runs, at least 24 hours after the last Cache.Trim), delete files with modification times more than 5 days in the past.

5 days was a time limit that worked well for builds. We may want to choose a longer limit like 30 days for cached fuzz inputs.

cc @golang/fuzzing

@katiehockman
Copy link
Member

@katiehockman katiehockman commented Oct 6, 2021

Removed the release-blocker label after some internal discussion.

I have a few concerns with this. I'm imaging a scenario where someone ran fuzzing for several days, and built up a sizable cache of interesting values, then puts it down for a while (e.g. they went on vacation, this is a side project, etc). They come back to find that their entire fuzzing cache has been evicted without their knowledge. That could be a pretty frustrating experience. It makes sense for the build cache to auto-evict after a period of time, but the fuzz "cache" isn't really a cache at all in the sense that the files may be long lived.

My preference would be hold off on doing this for now, and add some very clear documentation for folks should they run into an issue where evicting their cache would be helpful. We can gain more insight on whether or not this would be a good feature to implement, and if so, how long we should let old fuzz cache entries sit around (30 days? 90 days? longer?) after the release, and re-visit this for 1.19.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
2 participants