Add Fuzzer

[bcoles]

This PR adds a single-threaded dumb fuzzer for pdf-reader.

A test run through all the documents in spec/data generated 14,232 errors and took approximately 85 minutes to complete on a single core.

It's worth noting that the errors included 274 instances of execution expired indicating that 274 tests caused pdf-reader to hang for 15 seconds before being forcibly terminated. This accounts for more than 65 minutes of the run.

The fuzz duration could be decreased significantly by decreasing the timeout or resolving the underlying issue(s) in pdf-reader.

real	86m18.686s
user	85m11.488s
sys	0m53.024s

Usage

Multiple input files are supported:

./fuzz.rb <FILE1> [FILE2] [FILE3] [...]

If your shell supports file globbing:

./tools/fuzz.rb spec/data/**.pdf

Output

Each test case is written to 'fuzz.pdf' in the current working directory.

Crashes and the associated backtrace are saved in the 'crashes' directory in the current working directory.

Analysis

For working with the crashes, I typically use:

# list all error messages
head -n 1 crashes/*.trace

# list all error messages with first line of backtrace
head -n 1 crashes/*.trace -A 1

# list of unique error messages
head -n 1 crashes/*.trace | fgrep -v "==>" | sort -u

[ndbroadbent]

This is awesome, and also very scary! I'll try to fix a couple of these.

[ndbroadbent]

@bcoles - Would you also have time to take a look at https://github.com/gdelugre/origami? Origami is actually a tool for security researchers, but it was the only Ruby library I could find with enough support for AcroForms and XFA.

[bcoles]

Hi @ndbroadbent

I've augmented the fuzzer for Origami. Initial test cases resulted in 100% CPU usage.

I'm not greatly familiar with Origami, so it will take some time to flesh out the fuzzer and provide the appropriate details on their issue tracker. I'll tag you in the fuzzer PR. Let's move the discussion there.

[ndbroadbent]

Awesome, thanks for doing that!