[fastcgs] Update commit

[alifahmed]

Changed base fuzzer to original MOpt

[jonathanmetzman]

LGTM

[alifahmed]

@jonathanmetzman The last report does not show any fastcgs results. I am wondering what could be the reason?

[jonathanmetzman]

Sorry which report are you referring to?

[alifahmed]

Sorry which report are you referring to?

Sorry, this one: https://www.fuzzbench.com/reports/2020-03-19/index.html

[jonathanmetzman]

Looking into it, but those results are still being processed and aren't complete yet.

[jonathanmetzman]

Looking into it, but those results are still being processed and aren't complete yet.

It also looks like fastcgs failed because we were using multiple -d options. Sorry about that.
I think this PR fixes that issue, so we can rerun fastcgs with newest version.

[alifahmed]

Looking into it, but those results are still being processed and aren't complete yet.

It also looks like fastcgs failed because we were using multiple -d options. Sorry about that.
I think this PR fixes that issue, so we can rerun fastcgs with newest version.

No problem :) Eagerly waiting to see how the newest commit does.

[alifahmed]

Looking into it, but those results are still being processed and aren't complete yet.

It also looks like fastcgs failed because we were using multiple -d options. Sorry about that.
I think this PR fixes that issue, so we can rerun fastcgs with newest version.

@jonathanmetzman Just saw that the Mar 19 report is online: https://www.fuzzbench.com/reports/2020-03-19/index.html
However, fastcgs results are not shown in the graphs. Curious if the benchmarks are ran with this commit, or the previous commit with the duplicate '-d' options?

Also, if it actually failed to run, I am wondering why it got higher rank than aflsmart, libfuzzer-asan, and libfuzzer in the critical difference plot?

[jonathanmetzman]

The 2020-03-19 report is indeed the one where fastcgs was broken because of duplicate -d instructions.
As for why it got a higher rank, I think it's because fastcgs shows up in the openssl results I think because of a slight bug in how we zip corpora (after fuzzing stops (usually when time expires) we do one last zip of the corpus, but in this case the fuzzing stops immediately).
Our aggregate ranking algorithm doesn't seem to penalize fuzzers for not appearing on a particular benchmark, so that is why fastcgs does better than some fuzzers that consistently end in last.

The results from the latest experiment where fastcgs runs properly should be available soon. I'll try to produce a report where fuzzers run in the latest experiment replace results from 2020-03-19

[alifahmed]

The 2020-03-19 report is indeed the one where fastcgs was broken because of duplicate -d instructions.
As for why it got a higher rank, I think it's because fastcgs shows up in the openssl results I think because of a slight bug in how we zip corpora (after fuzzing stops (usually when time expires) we do one last zip of the corpus, but in this case the fuzzing stops immediately).
Our aggregate ranking algorithm doesn't seem to penalize fuzzers for not appearing on a particular benchmark, so that is why fastcgs does better than some fuzzers that consistently end in last.

The results from the latest experiment where fastcgs runs properly should be available soon. I'll try to produce a report where fuzzers run in the latest experiment replace results from 2020-03-19

@jonathanmetzman Thanks for the clarification! Since the fuzzers are updated regularly, do you think it would be helpful to tag the reports with particular commits? (EDIT: Nevermind, #94 addresses this)