proposal: testing: add testing.F.Corpus to help with cross tests conversion #65253
Description
Proposal Details
I have two fuzz tests which test the same code, however one of them accepts binary data and the second one takes strings (my code has an ASCII and binary representation).
I was able to find new issues by seeding the corpus from the other test by converting the representation.
So I propose this to be added:
// Corpus let you fetch the corpus from an other test, callback will be called with all the entry in the other test corpus.
// source must be a function pointer to the actual other test.
// callback must be a function pointer which has the same signature as the other test [F.Add] argument.
// Corpus must not be called once [F.Fuzz] has been called.
func (*F) Corpus(source, callback any)Expected usage:
func FuzzString(f *testing.F) {
for _, v := range testcase {
_, err := decodeString(v)
if err != nil { f.Fatal(err) }
f.Add(v)
}
f.Corpus(FuzzBinary, func(_ *testing.T, b []byte) {
v, err := decodeBinary(b)
if err != nil { return }
str, err := v.String()
if err != nil { return }
f.Add(str)
})
f.Fuzz(func(_ *testing.T, s string) {
v, err := decodeString(s)
if err != nil { return }
testStuff(v)
})
}
func FuzzBinary(f *testing.F) {
for _, v := range testcase {
b, err := decodeString(v)
if err != nil { f.Fatal(err) }
f.Add(b)
}
f.Corpus(FuzzString, func(_ *testing.T, s string) {
v, err := decodeString(s)
if err != nil { return }
f.Add(v)
})
f.Fuzz(func(_ *testing.T, b []byte) {
v, err := decodeBinary(b)
if err != nil { return }
testStuff(v)
})
}It's unclear to me if callback should be retained, if it is retained we could run multiple fuzz tests in parallel and convert entries from one into one from the other on the fly (it would require some tricky-ish synchronisation and validation internally).
I think for now this is not needed but we can add a comment hinting this might be a thing in the future. I don't think this would break anyone since you can't currently run more than one fuzz test together.