AFL integration with multiple AFL instances running and a SymCC instance with only a single AFL process.

[DavidKorczynski]

This integrates a version where two AFL processes are started.

The reason for this integration is that some integrations (eclipser and symcc) rely on multiple processes as part of their workflow. This potentially gives an unfair total amount of CPU time or an unfair amount of time spent context switching in comparison to the other integrations, which in turn may twist the truth of the results. This integration is meant to be used as a comparison to such integrations. This came up from a discussion with @vanhauser-thc over here #1165 (comment)

[DavidKorczynski]

As an extra note on this, I assume this is something that should also be considered in other integrations? For example, Van Hauser mentions another example of comparing Eclipser and afl++ versus afl++ and afl++. Down the line it may also be that more multi-process tools are coming in.

[inferno-chromium]

Can you request an experiment as well in this pr

[DavidKorczynski]

Can you request an experiment as well in this pr

Yeah.

I will also a second flavour of SymCC that relies on a single AFL instance rather than one (the documentation suggests two https://github.com/eurecom-s3/symcc/blob/9b20609adab02279c181010c8b1e61a9a9acac62/docs/Fuzzing.txt#L91 but also says It is possible to run SymCC with only an AFL master or only a secondary AFLinstance; see the AFL docs for the implications. https://github.com/eurecom-s3/symcc/blob/9b20609adab02279c181010c8b1e61a9a9acac62/docs/Fuzzing.txt#L112)

@sebastianpoeplau am not entirely sure what implications this refers to - if you have the time then I would be happy to hear about those.

[DavidKorczynski]

I have run the three modified fuzzers from this pr symcc_afl, symcc_afl_single and afl_two_instances on the libhtp_fuzz_htp benchmark for a short period of time. All three work on that so this one should be good to go.

[sebastianpoeplau]

I will also a second flavour of SymCC that relies on a single AFL instance rather than one (the documentation suggests two https://github.com/eurecom-s3/symcc/blob/9b20609adab02279c181010c8b1e61a9a9acac62/docs/Fuzzing.txt#L91 but also says It is possible to run SymCC with only an AFL master or only a secondary AFLinstance; see the AFL docs for the implications. https://github.com/eurecom-s3/symcc/blob/9b20609adab02279c181010c8b1e61a9a9acac62/docs/Fuzzing.txt#L112)

@sebastianpoeplau am not entirely sure what implications this refers to - if you have the time then I would be happy to hear about those.

My understanding from the AFL docs is that master instances perform a set of deterministic mutations before proceeding to randomly mutating inputs (called "havoc" in the AFL UI); secondary instances skip the deterministic stage and start with "havoc" immediately. The consequences for SymCC, I think, are that running with only a secondary instance will lose any benefit that the deterministic mutations might have; running with only a master doesn't have that problem, but I believe then it takes longer until AFL reads inputs from SymCC for the first time.

[vanhauser-thc]

I will also a second flavour of SymCC that relies on a single AFL instance rather than one (the documentation suggests two https://github.com/eurecom-s3/symcc/blob/9b20609adab02279c181010c8b1e61a9a9acac62/docs/Fuzzing.txt#L91 but also says It is possible to run SymCC with only an AFL master or only a secondary AFLinstance; see the AFL docs for the implications. https://github.com/eurecom-s3/symcc/blob/9b20609adab02279c181010c8b1e61a9a9acac62/docs/Fuzzing.txt#L112)
@sebastianpoeplau am not entirely sure what implications this refers to - if you have the time then I would be happy to hear about those.

My understanding from the AFL docs is that master instances perform a set of deterministic mutations before proceeding to randomly mutating inputs (called "havoc" in the AFL UI); secondary instances skip the deterministic stage and start with "havoc" immediately. The consequences for SymCC, I think, are that running with only a secondary instance will lose any benefit that the deterministic mutations might have; running with only a master doesn't have that problem, but I believe then it takes longer until AFL reads inputs from SymCC for the first time.

In afl++ we just changed this - -M does not do deterministic anymore because it is pretty pointless (as analyzed from several experiments I made).
Also -M was never required, for neither vanilla afl nor afl++.
In my experience for symcc/Eclipers/etc. a single vanilla afl-fuzz -S is better than -M or having -M and -S. (and for afl++ I would recommend the same).

[sebastianpoeplau]

Sounds good! In vanilla afl -M has the effect of changing the layout of the output directory compared to the behavior with neither -M nor -S, effectively introducing a subdirectory per instance. Is that the default in afl++ already?

[vanhauser-thc]

@sebastianpoeplau yes. basically if neither -M or -S is given -S default is set