issue replicating crashes with fuzzilli and v8

[tregua87]

Fuzzilli: fuzzilli-0.9.3
V8: 8560c85e
depot_tools: a73a2497
clang version: 13.0.0
Ubuntu 20.04

I compile v8 using standard V8/Target/fuzzbuild.sh from repository.

I try a 12h campaign running:

timeout 12h swift run FuzzilliCli --profile=v8 ${V8_DIR}/out/fuzzbuild/d8 --storagePath=${V8_LOG}/v8_campaign --overwrite --exportStatistics

After the campaign, I am not able to replicate any of the crashes found.
The /crash folder contains 5 deterministic and 616 flaky. I know flaky crashes are not reproducible, the problem is that I can't even reproduce the deterministic ones.

To replicate the crashes, I tried like this:

for f in $CRASH_DIR/*; do
    if [[ "$f" == *.js  ]]
    then
        $OUT_V8 $f
    fi
done

I also tried to replay the crashes by using the suggested flags from the .js seeds.

What am I missing?

[saelo]

Hi! Is $OUT_V8 the fuzzbuild/d8 or another d8 binary? If they are two binaries, were they built from the same V8 version? Most crashes should reproduce with a debug build of d8, but some may require a different build that's closer to the fuzzbuild (e.g. with optimization).
In general, you'll typically need (at least some of) the d8 flags to reproduce crashes. It's rare for crashes to reproduce without these flags. It does happen that a bug that Fuzzilli was able to reproduce doesn't reproduce again later on, but it's not super common, so not sure what else could be the problem here.

[tregua87]

Yep, $OUT_V8 points to the same d8 binary used for fuzzing.

For instance, taking the seed program_20230411203204_33EB3953-7A98-4DB3-9574-4B0AF775F45C_deterministic.js:

function f0(a1, a2) {
    function f3() {
        return a2;
    }
    const o6 = {
        "type": "function",
    };
    const v7 = new Worker(f3, o6);
}
const v9 = new Promise(f0);
// CRASH INFO
// ==========
// TERMSIG: 72
// STDERR:
// 
// STDOUT:
// 
// ARGS: .../v8/out/fuzzbuild/d8 --expose-gc --omit-quit --future --harmony --assert-types --harmony-struct --allow-natives-syntax --interrupt-budget=1000 --fuzzing
// EXECUTION TIME: 16ms

I run

.../v8/out/fuzzbuild/d8 --expose-gc --omit-quit --future --harmony --assert-types --harmony-struct --allow-natives-syntax --interrupt-budget=1000 --fuzzing program_20230411203204_33EB3953-7A98-4DB3-9574-4B0AF775F45C_deterministic.js
[COV] no shared memory bitmap available, skipping
[COV] edge counters initialized. Shared memory: (null) with 648238 edges
Warning: unknown flag --harmony-struct.
Try --help for options
[COV] Additional 0 edges for builtins initialized.
[COV] Additional 0 edges for builtins initialized.
unnamed:25: Uncaught ReferenceError: a2 is not defined

Which is not what I expect. First why --harmony is not a flag? :D Then I don't expect that error.

I am running it in a docker container, but this should not be a problem I think.

Btw, d8 should be with debug info, according to this:

file .../v8/out/fuzzbuild/d8          
.../v8/out/fuzzbuild/d8: ELF 64-bit LSB shared object, x86-64, version 1 (SYSV), dynamically linked, interpreter /lib64/ld-linux-x86-64.so.2, for GNU/Linux 3.2.0, BuildID[xxHash]=5bac1475d4866903, with debug_info, not stripped

NOTE: I replaced actual path with ...

[saelo]

The V8 version you're fuzzing is ~ 2 years old from what I can tell, so --harmony-struct didn't exist back then. But that shouldn't prevent the testcase from reproducing.
The other odd thing is // TERMSIG: 72, that signal number doesn't exist. There were some issues with the Fuzzilli integration in V8 that have recently (last couple of months) been fixed, maybe you're running into one of these issues. Do you want to be fuzzing such an old version of V8?

[tregua87]

I understand.
We are conducting an academic research that involves V8 and fuzzing, which constraints our usage to an old V8.
Since you said this, we will try an updated version of V8! In the meanwhile, I guess I better close this issue.
Thanks for the quick reply!

[saelo]

This might be the issue you're running into since the testcase included some Worker stuff: https://chromium.googlesource.com/v8/v8.git/+/e0399e4394c4179fc24eae038f43f4ac5e3c1b62