Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

internal/fuzz: revisit use of shared memory-mapped files for marshaled inputs #48163

Open
jayconrod opened this issue Sep 2, 2021 · 1 comment
Open

Comments

@jayconrod
Copy link
Contributor

@jayconrod jayconrod commented Sep 2, 2021

Currently, the fuzzing coordinator process creates a 100MB temporary file for each worker process. Both the coordinator and worker read and write the file via a shared memory map. The file is currently used for 1) a few "header" fields such as iteration count and PRNG state, and 2) passing marshaled inputs for fuzzing in both directions.

We still need to use shared memory for the header fields. If a worker process terminates unexpectedly, the coordinator needs to be able to reconstruct the input that caused the crash using the initial input (sent from the coordinator), the call count, and the PRNG state.

However, it's not strictly necessary to write marshaled inputs to shared memory. It would be simpler to pass these through the pipes we use for RPCs. If we only supported unmarshaled byte slices, there would be a performance advantage to writing and mutating those directly in shared memory without incurring the cost of marshaling and pipe I/O. Since we need to marshal inputs anyway, it's not clear the extra complexity is worthwhile.

We should investigate the performance difference and pass inputs through pipes if it's not too bad. This would simplify our implementation, and would let us use much smaller shared memory files.

Additionally, for inputs that can be read from files, such as those in testdata or the cache, we can pass file names over pipes instead of reading, and sending that data over pipes. The coordinator doesn't need to hold that data in virtual memory at all.

@jayconrod jayconrod added this to the Go1.18 milestone Sep 2, 2021
@rsc rsc changed the title [dev.fuzz] internal/fuzz: revisit use of shared memory-mapped files for marshaled inputs internal/fuzz: revisit use of shared memory-mapped files for marshaled inputs Sep 21, 2021
@toothrot
Copy link
Contributor

@toothrot toothrot commented Sep 22, 2021

Checking in on this issue as it's labeled a release blocker for Go 1.18. Is there any update?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
2 participants