Use AddVectoredExceptionHandler to register exception handlers

[tokatoka]

resolves #372
We should not use SetUnhandledExceptionFilter, it searches through the stack for exception handler, but possibly frida Stalker breaks it.
this pr uses AddVectoredExceptionHandler instead

f58e6a0

[tokatoka]

This one depends on changes included in #391
so I made a branch from it

[tokatoka]

Does the commit 4d082cd on this branch solves the issue for you too? @expend20 🙂
I tested this with cpp code, and also tested the original source code you pasted in the issue #372 with pageheap

#include <stdio.h>
#include <stdlib.h>
#include <stdint.h>
#include <string.h>

extern "C" __declspec(dllexport) size_t
    FuzzMeOOBR(const char *data, unsigned int len)
{

    char sdata[1024];
    memcpy(sdata, data, sizeof(sdata) < len ? sizeof(sdata) : len);

    if (len >= 8 
            && *(uint8_t *)(data + 0) == '1'
            && *(uint8_t *)(data + 1) == '3'
            && *(uint8_t *)(data + 2) == '3'
            && *(uint8_t *)(data + 3) == '7'
            ) {
                char *p = (char *)0;
                *p = 0;
    }
    return 0;
}

[tokatoka]

The stackoverflow issue is not fixed yet.
The RIP is transfered our exception handlers, which is good, but

overflow!Stack overflow!Stack overflow!Stack overflow!Stack overflow!Stack overflow!Stack overflow!Stack overflow!Stack overflow!Stack overflow!Stack overflow!Stack overflow!Stack overflow!Stack overflow!Stack overflow!Stack overflow!Stack overflow!Stack overflow!Stack overflow!Stack overflow!Stack overflow!Stack overflow!Stack overflow!Stack overflow!Stack overflow!Stack overflow!Stack overflow!Stack overflow!Stack overflow!Stack overflow!Stack overflow!Stack overflow!Stack overflow!Stack overflow!Crashed with STATUS_STACK_OVERFLOW
Child crashed!
Crashed with STATUS_ACCESS_VIOLATION

I observe double crash.
STATUS_STACK_OVERFLOW and STATUS_ACCESS_VIOLATION
I feel this is what is happening here https://peteronprogramming.wordpress.com/2016/08/10/crashes-you-cant-handle-easily-2-stack-overflows-on-windows/

[tokatoka]

I feel we need to handle this special case like here @domenukk for windows

LibAFL/libafl/src/events/llmp.rs

Line 881 in fd869ba

if child_status == 137 {

My dump file says the second exception STATUS_ACCESS_VIOLATION is triggered here

LibAFL/libafl/src/executors/inprocess.rs

Line 837 in 6229165

.fire(

possibly it happens because we need to allocate stacks even after the stack mem is close to zero.

but still we have no way to salvage the input that triggered the stackoverflow, unless we save as much stack memory as possible inside the crash handler.

[tokatoka]

My ida says this is where fire() jumps to...
so it's clear we run out of stack again inside the exception handler

[tokatoka]

they allocate this size of buffer on the stack
https://github.com/Frommi/miniz_oxide/blob/6da05dd599b438fff2a05e3c2552cbdcba4cc5a7/miniz_oxide/src/deflate/buffer.rs#L8
and that's why we need 0x10000 bytes (64 * 1024)
I think it's fine for us to reserve 0x20000 bytes here.

[domenukk]

Good work