perf-fuzz virtual machine

[andreafioraldi]

The speedup of the snapshot() syscall is so high that would be a pity to not use it, especially after having AFL++ scaling on all the threads of the CPU.
Reding the code, it cannot be ported to a LKM due to the patch to the page fault handler.
We don't need the code to share the testcases in memory, just the snapshot() syscall.

My proposal is to port https://github.com/andreafioraldi/perf-fuzz-linux-4.8.10 to the default kernel in Ubuntu 20.04 (the upcoming LTS) and distribute a VM for QEMU-KVM with Ubuntu 20.04 with the precompiled kernel for fuzzing.

[vanhauser-thc]

I know the patch and that is what I thought that is cant be put into a lkm - but I am not a kernel expert, just a novice.

would the speedup be so beneficial that is still performing better in a qemu-kvm vm compared to a stock native kernel?

[andreafioraldi]

Yes, and a native kernel performs almost like a kernel in a modern hypervisor.

[vanhauser-thc]

Copying Dominik's link here about memory snapshots for Docker: https://dl.acm.org/doi/10.1145/3357526.3357542
(no clue how performant or production ready or even code availability is)

[kallsyms]

FWIW I have a (seemingly mostly working*) LKM port of perf-fuzz here: https://github.com/kallsyms/snapshot-lkm. It's pretty rough code (weekend project) but wanted to share in case someone else wanted to take it further - e.g. making it work on other kernels besides whatever version I happened to have built :)

* It seems to yield identical output to the test/example program in the perf-fuzz repo, so I'm 99% sure it's working, but it hangs the system after just a few seconds of AFL, so I'm guessing there's some locks or something that I'm not properly taking.

[vanhauser-thc]

@kallsyms hey that is very cool!
and a lkm would make it much more portable than having it on a fixed kernel, we didnt think that would be possible though - so great job :)
I fear however as this is your code that it will be difficult for someone else to understand everything you did and take over your achievement.

[andreafioraldi]

@kallsyms good job! @vanhauser-thc I understood well what he did, just submitted a patch to have it working on lastest ubuntu lts. However, as you and Marc said, the code is rough (I guess cause is based on the original implementation) and I want to re-engineer it a bit when I'll have free time. For instance, maybe ftrace is much better than kprobes, avoid to trigger another trap each page fault is better.

[kallsyms]

@andreafioraldi yeah most of the actual snapshot logic is straight from the original repo with a few small changes to read out of the associated data structs we store instead of reading out of the modified kernel structs. Thanks for the PR! Just approved and merged. Re ftrace: I think kprobes can optimize to use the ftrace call stubs under the hood so in theory we aren't actually double faulting, but it's probably worth investigating if the optimization is actually happening.

[andreafioraldi]

I started to refactor it in https://github.com/AFLplusplus/AFL-Snapshot-LKM. For now, I just cleaned a bit obscure stuffs and ported to an ioctl instead of a new syscall. @kallsyms do want to be added as maintainer of the repo?

[kallsyms]

Sure!