Driller not generating valid inputs

[ekilmer]

I have been playing with Driller for a bit, and I noticed something that looks weird when drilling certain inputs. I wrote a toy program to illustrate this (below).

Basically, I ran driller using AFL to fuzz it first with the seed fuzzmesoftly here. AFL ran for a while and sent a few inputs to drill. However, the drilled input in question is Aset option\x00. Since AFL had already explored some paths, it modified its bitmap, so I had to capture both the bitmap and input to reproduce the result.

What I was expecting was driller to generate an output that makes the program crash. However, none of the generated output will make the toy program crash, and AFL wasn't smart enough to combine the results from driller and its own findings.

The generated output 00bada552100000000000000 from transition 4006d8 -> 4006dd contains the crash string, but it lacks the 'A' magic number prefix that would make the program crash during a real run.

$ echo -e "\x00\xba\xda\x55!" | ./listing_stdin
Bad magic number

vs.

$ echo -e "A\xba\xda\x55!" | ./listing_stdin
Segmentation fault

I am wondering how I could fix this? I looked a bit, and I think it comes down to this code in the tracer when it removes the preconstraints. It looks like it removes the 'A' because it was preconstrained when the tracer was initialized, but that constraint is necessary to even check for the crash string.

Thank you for the help!! Let me know if anything isn't clear.

Resources

Here are the resources: test script, input file, program, program source and binary, and output log. I'm pretty sure I tested on this commit (unless I've messed something up in my workspace...)

Test Script

import driller
import listing_stdin_c13fd238a6ed9a0311be4cd5426bb42f as in_data

import logging
l = logging.getLogger("driller.Driller")
l.setLevel(logging.DEBUG)

binary = "./listing_stdin"
d = driller.Driller(binary, in_data.input, in_data.fuzz_bitmap, "whatever~")

new_inputs = d.drill()

Input File

Here: listing_stdin_c13fd238a6ed9a0311be4cd5426bb42f.py.zip. It contains the binary path, input used, time run, and fuzz bitmap.

Program Source and Binary

I compiled this on Ubuntu 16.04 with gcc listing_stdin.c -o listing_stdin. Here is the compiled binary in case the fuzzing bitmap causes problems with a recompilation... listing_stdin.zip

#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <unistd.h>

#define MAGICNUMBER 'A'
#define OPTION_STR "set option"

int main(int argc, char *argv[]) {
    int* null_ptr = NULL;
    char config[20];
    read(0, config, sizeof(config));

    if (!config) {
        puts("Configuration syntax error");
        return 1;
    }
    if (config[0] != MAGICNUMBER) {
        puts("Bad magic number");
        return 2;
    }

    // Increment to next character
    char *directive = config + 1;
    if (strstr(directive, "\xba\xda\x55!")) {
    //if (strstr(directive, "badass!")) {
        int k = *null_ptr;
    }
    else if (!memcmp(directive, OPTION_STR, strlen(OPTION_STR))) {
        char *option = directive + strlen(OPTION_STR);
        puts(option);
    }
    else {
        puts("Unknown");
    }
    return 0;
}

Output

[23:55:06] $ python test_proof.py
INFO    | 2017-03-08 23:55:12,177 | driller.Driller | [listing_stdin] drilling started on Wed Mar  8 23:55:12 2017
DEBUG   | 2017-03-08 23:55:18,033 | driller.Driller | drilling into 'Aset option\x00'
DEBUG   | 2017-03-08 23:55:18,033 | driller.Driller | input is 'Aset option\x00'
DEBUG   | 2017-03-08 23:55:18,657 | driller.Driller | found 4004d0 -> 4004e0 transition
DEBUG   | 2017-03-08 23:55:18,657 | driller.Driller | 4004d0 -> 4004e0 has already been encountered
DEBUG   | 2017-03-08 23:55:18,677 | driller.Driller | found 400781 -> 4007a6 transition
DEBUG   | 2017-03-08 23:55:18,677 | driller.Driller | 400781 -> 4007a6 has already been encountered
DEBUG   | 2017-03-08 23:55:18,705 | driller.Driller | found 400640 -> 400650 transition
DEBUG   | 2017-03-08 23:55:18,705 | driller.Driller | 400640 -> 400650 has already been encountered
DEBUG   | 2017-03-08 23:55:18,727 | driller.Driller | found 4005e0 -> 400603 transition
DEBUG   | 2017-03-08 23:55:18,727 | driller.Driller | 4005e0 -> 400603 has already been encountered
DEBUG   | 2017-03-08 23:55:18,748 | driller.Driller | found 40079d -> 400790 transition
DEBUG   | 2017-03-08 23:55:18,748 | driller.Driller | 40079d -> 400790 has already been encountered
DEBUG   | 2017-03-08 23:55:18,832 | driller.Driller | found 4006a2 -> 4006aa transition
INFO    | 2017-03-08 23:55:18,849 | driller.Driller | found a completely new transition, exploring to some extent
INFO    | 2017-03-08 23:55:18,851 | driller.Driller | [listing_stdin] dumping input for 4006a2 -> 4006aa
INFO    | 2017-03-08 23:55:18,851 | driller.Driller | generated: be0000000000000000000000
INFO    | 2017-03-08 23:55:20,509 | driller.Driller | [listing_stdin] started symbolic exploration at Wed Mar  8 23:55:20 2017
INFO    | 2017-03-08 23:55:20,658 | driller.Driller | [listing_stdin] symbolic exploration stopped at Wed Mar  8 23:55:20 2017
INFO    | 2017-03-08 23:55:20,659 | driller.Driller | [listing_stdin] dumping input for 4000050 -> 4000050
INFO    | 2017-03-08 23:55:20,660 | driller.Driller | generated: be0000000000000000000000
DEBUG   | 2017-03-08 23:55:20,723 | driller.Driller | found 4006d8 -> 4006dd transition
INFO    | 2017-03-08 23:55:20,782 | driller.Driller | found a completely new transition, exploring to some extent
INFO    | 2017-03-08 23:55:20,785 | driller.Driller | [listing_stdin] dumping input for 4006d8 -> 4006dd
INFO    | 2017-03-08 23:55:20,785 | driller.Driller | generated: 00bada552100000000000000
INFO    | 2017-03-08 23:55:21,821 | driller.Driller | [listing_stdin] started symbolic exploration at Wed Mar  8 23:55:21 2017
INFO    | 2017-03-08 23:55:21,858 | driller.Driller | [listing_stdin] symbolic exploration stopped at Wed Mar  8 23:55:21 2017
INFO    | 2017-03-08 23:55:21,860 | driller.Driller | [listing_stdin] dumping input for 4000050 -> 4000050
INFO    | 2017-03-08 23:55:21,860 | driller.Driller | generated: 00bada552100000000000000
DEBUG   | 2017-03-08 23:55:21,907 | driller.Driller | found 4006fe -> 40071c transition
INFO    | 2017-03-08 23:55:21,945 | driller.Driller | found a completely new transition, exploring to some extent
INFO    | 2017-03-08 23:55:21,948 | driller.Driller | [listing_stdin] dumping input for 4006fe -> 40071c
INFO    | 2017-03-08 23:55:21,948 | driller.Driller | generated: 008c9a89df808f8b96909100
INFO    | 2017-03-08 23:55:22,978 | driller.Driller | [listing_stdin] started symbolic exploration at Wed Mar  8 23:55:22 2017
INFO    | 2017-03-08 23:55:23,028 | driller.Driller | [listing_stdin] symbolic exploration stopped at Wed Mar  8 23:55:23 2017
INFO    | 2017-03-08 23:55:23,030 | driller.Driller | [listing_stdin] dumping input for 4000050 -> 4000050
INFO    | 2017-03-08 23:55:23,030 | driller.Driller | generated: 008c9a89df808f8b96909100
WARNING | 2017-03-08 23:55:23,061 | simuvex.plugins.symbolic_memory | Concretizing symbolic length. Much sad; think about implementing.
DEBUG   | 2017-03-08 23:55:23,094 | driller.Driller | found 400726 -> 40073a transition
DEBUG   | 2017-03-08 23:55:23,095 | driller.Driller | 400726 -> 40073a has already been encountered

[salls]

I think the fix is to not remove the constraints that match preconstraints. In line 591 of tracer/tracer.py change new_constraints = filter(lambda x: x.cache_key not in precon_cache_keys, path.state.se.constraints) to new_constraints = path.state.se.constraints.

I believe this was necessary in the old style of preconstraining before we switched to the replacement frontend. If so, and the filter isn't necessary, making this change should not break any existing tests and will fix this issue.

[salls]

angr/tracer#18 should fix it. Waiting for some tests before I merge it

[salls]

Merged, this issue should be fixed

[ekilmer]

Looks to be fixed on my end too! Thank you!