Fuzz tests for Arrow/Parquet

[emkornfield]

I'm generally new to the code base but it looks like existing fuzz tests might be generating random data and making sure it can be read back, but we don't have any fuzz tests for malformed data. I think in the context of Rust the goal would be to avoid panics?

If this is accurate, I'd propose creating fuzz tests that check succeed as long as there is no panic. A beginning corpus arrow-testing repo 2 for each file type.

A second step would be integrate with oss-fuzz (Arrow C++ already does so).

If this is of interest and i'm correct this isn't already done, I can try to see if I can prototype something.

[tustvold]

It isn't an issue IMO if the reader panics on malformed data, this is a perfectly safe and well-defined behaviour. We should try to avoid it, but its not like UB where it would indicate a bug. Panics are just exceptions.

The bigger issue with untrusted/malicious inputs is avoiding the reader getting stuck in infinite loops or exploding the memory usage. I'm not sure how easy such things are to catch using a fuzz testing framework.

With regards to parquet, I can't help feeling the format is sufficiently complex that supporting untrusted input is essentially a fools errand though...

That's all to say adding fuzzing support would be a nice add. I'm not too familiar with the Rust ecosystem's support for it, but @crepererum may know more.

.

[crepererum]

Fuzzing is a good thing, even when you accept panics as an outcome. The fuzzer then has two wrap the method call accordingly.

Regarding the toolchain: We should use cargo-fuzz. That gives us the option to use multiple fuzzers. Then you need to choose a fuzzer. I would suggest you use libFuzzer which comes w/ LLVM, since it is the least invasive one, however it has entered maintenance mode. I see the following options:

• don't care about maintainance mode and just use libFuzzer
• use afl.rs/cargo afl¹
• use libAFL -- which is a modern Rust redesign of AFL(++) -- and also provides a libFuzzer shim

That said, the choice can easily be changed later, since the fuzzer is effectively just a "run some code on this blackbox [u8] input" (like "parse parquet from bytes").

Footnotes

1. Note that AFL was abandoned for a while, but development is now open and active under the AFL++ project. ↩