Maybe a dumb question, but can "the f.Fuzz function is missing a *testing.T as its first parameter" be enforced by the typechecker/compiler instead?
@timothy-king Not a dumb question! I don't have an answer to this question though. Possibly? The use of generics might make things easier here as well, but I haven't looked too deeply into this yet.
However, in general, the more issues like this that we can detect at compile-time the better.
Could you perhaps elaborate on this one? Do you have a specific pattern in mind?
In the case above, f.Add is attempting to add a value to the corpus which is a single string. However, the f.Fuzz function accepts an int as its type. It would be nice if this can fail at compile time, but a vet check would be helpful if that's not feasible. That code should be corrected to something like this:
All these four checks can be covered by a simple vet checker. My main question is whether this checker mets the frequency requirement. After all, running a malformed target will result in panic, so there may be few malformed targets in the tested code.
I am more interested in the bugs that fail "silently", e.g. the expected mutations are not performed. Are there some patterns that are malformed but the related tests won't crash?
changed the title
[dev.fuzz] cmd/vet: checks for malformed fuzz targetsSep 21, 2021
For anyone looking at this later, golang.org/x/tools/go/analysis/passes/tests is a vet analysis that detects problems with tests, examples, and benchmarks, for example, a missing *T parameter, or an // Output comment in the wrong place.
It's probably a good place to check fuzz tests here, rather than defining a new pass.