Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[dev.fuzz] cmd/go: add fuzz timeout flag #46220

Closed
katiehockman opened this issue May 18, 2021 · 5 comments
Closed

[dev.fuzz] cmd/go: add fuzz timeout flag #46220

katiehockman opened this issue May 18, 2021 · 5 comments

Comments

@katiehockman
Copy link
Member

@katiehockman katiehockman commented May 18, 2021

We may want to consider adding a -fuzztimeout flag, which acts similar to a -timeout flag, but for a run of a single input to the f.Fuzz function.

If -fuzztimeout=100ms, then we give each execution of the fuzz function 100 milliseconds to run, and consider it a crasher if that execution takes longer than 100ms to complete.

Note that libFuzzer does this with a -timeout flag.

We could also consider retrofitting the existing -timeout flag, but that might cause confusion. The description for timeout is "If a test binary runs longer than duration d, panic." If someone sets timeout while fuzzing, we don't want to panic when we hit that timeout (which we will if there is no crash).

Related to #44483.

@katiehockman katiehockman added this to the Backlog milestone May 18, 2021
@katiehockman katiehockman changed the title [dev.fuzz]: add fuzz timeout flag [dev.fuzz] cmd/go: add fuzz timeout flag May 18, 2021
@AlekSi
Copy link
Contributor

@AlekSi AlekSi commented May 19, 2021

@jayconrod
Copy link
Contributor

@jayconrod jayconrod commented Jun 23, 2021

This is done, in the form of the -fuzztime flag.

We should wire that into T.Deadline though as @AlekSi points out.

EDIT: -fuzztime is a global time limit, like -timeout. This issue is about a per-input limit.

@gopherbot
Copy link

@gopherbot gopherbot commented Jun 23, 2021

Change https://golang.org/cl/330509 mentions this issue: [dev.fuzz] testing: report T.Deadline when running seed values

@katiehockman
Copy link
Member Author

@katiehockman katiehockman commented Sep 15, 2021

It would be nice to support this for 1.18, but it's not a release blocker. We'll see if we get to it.

@jayconrod
Copy link
Contributor

@jayconrod jayconrod commented Sep 15, 2021

#48157 is a proposal to add a flag and a T method for per-test timeouts. We'll wait for that to be resolved. If approved, we'll use the same flag and method.

@gopherbot gopherbot closed this in cc04ab4 Sep 20, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
4 participants