WIP: QEMU exit handler (#1745)

* Added paging filtering.
Reworked address range filtering to fit with new generic code.

* Fix: renamed remaining QemuInstrumentationFilter instances.

* Renamed sync breakpoint to sync exit.

* Split emu in systemmode.rs / usermode.rs for specific code.
EmuExitHandler implementation.

* sync_backdoor.rs removal.
Formatting.

* Updated `bindgen` and `which`.
Adapting code to work with update.

* fix: reconfigure cleanly if prior configure was interrupted abruptly.

* Enable sanitizers in QEMU during debug.

* Added target-usable files.

* Added breakpoint structure.

* Adapted other files to work with ExitHandler.

* Adapted existing fuzzer to work with new exit handler.

* fix: use get to avoid crashes.

* Updated README to indicate cargo-make should be installed.

* Added QEMU internal exit handler.

* Adapted qemu_systemmode example with new exit handler.

* Fixed fuzzers to work with new exit handler.

* Trying to fix CI (#1739)

* test

* dummy

* dummy

* Added new examples.

* Forgot to add build scripts.

* format

* format

* clang-format

* python emulator adaptation.

* fixed python bindings.

* clippy fixes.

* python bindings.

* fix qemu_sugar.

* fix fuzzbench.

* fixed import issues.

* misc fixes.

* renamed crate.

* Updated x86_64 stub bindings.

* Fixed static naming.

* binding fmt

* clippy

* clippy

* Removed useless return statement.

* removed advice to install cargo-make in individual repositories.

* symcc_update (#1749)

* Remove unused create_anymap_for_trait macro (fixes #1719) (#1752)

* Fix `as_object` UB discussed in #1748 (#1751)

* Fix as_object UB discussed in #1748

* More cleanup, more less UB

* Fix fixes

* Added uninit_on_shmem api

* clippy

* fmt

* trying to fix fuzzers, libfuzzer wrapper

* Add OwnedRefMit::owned constructor, libfuzzer fix

* Some more fixes

* Add BacktaceObserver::owned fn

* fmt

* more fmt

* Ignore SigPipe by default (#1741)

* Ignore SigPipe by default

* Fix no_std

* fmt

* Fix incorrect imports (#1758)

* Fix incorrect imports

https://doc.rust-lang.org/core/simd/trait.SimdOrd.html

* Fix

* Try fix ci

* Documentation fixes (#1761)

* Documentation fixes

* Fix InProcessExecutor url

* Update all urls to latest

* Miri ignores for M1 regex (#1762)

* Enabling DrCov on Windows (#1765)

* Enabling DrCov for Windows

* Dedup common code in scheduler (#1702)

* dedup common code in scheduler

* del eco

* fixing

* fix

* replace `Emulator::new_empty` by `Emulator::get` calls outside of `emu.rs` for safety. (#1763)

* Add mute_inprocess_target fn, SimpleFdLogger::set_logger, and more (#1754)

* Add mute_inprocess_target fn, SimpleFdLogger::set_logger, set_error_print_panic_hook

* Trying to fix #1753

* typo

* More fix

* Fix test?

* more testcase fixes

* Fix: renamed remaining QemuInstrumentationFilter instances.

* Split emu in systemmode.rs / usermode.rs for specific code.
EmuExitHandler implementation.

* format

* format

* format

* Replace sync_exit with sync_backdoor.

* Rework command system.

* fix bad import.

* format.

* cargo fmt

* disable af-xdp as well to avoid linking errors.

* End of merging.

* format.

* Adaptation for usermode.

* format.

* injection support.

* usermode fixes.
format.

* clippy

* clippy + format

* Do not unwrap emu + format.

* fix: entry_point breakpoint

* inital commit.

* clippy

* tests

* clippy

* adapt example

* systemmode.

* renaming

* fmt

* fix lints.

* more lint fix.

* even more lint fixes.

* always more lint fixes.

* lint fix.

* allow unused qualifications for crate when it could be confusing.

* Still lint fixes.

* Lint fixes on generated code.

* Some lint fixes.

* merge continue.

* renamed modules as well.

* fixing merge.

* systemmode compiling.

* fmt

* fix early emulator drop.

* fmt

* fix cast to c_void of the wrong object.

* Added global enum for snapshot managers.
Some renaming.

* move things around.

* WIP: generic inclusion of QEMU Executor in exit handler.

* * Moved extern calls to `libafl_qemu_sys`
* Replaced old `Emulator` by `Qemu` and only kept C functions wrappers
* Now `Emulator` is for higher-level interactions with QEMU. Kept old functions for compatibility calling to `Qemu` functions
* A direct side effect of this slit is the removal of the `IsEmuExitHandler` trait dependency added in many parts of the code.
* Removed old dirty casting for `QemuExecutor` helpers and used the brand-new access to `QemuExecutorState` instead.
* Minor changes to `Qemu` and `Emulator` `get` methods for cleaner getters.

* Add missing `Qemu` function.

* Updated `qemu_systemmode` example.

* Adapted QEMU fuzzers + renaming.

* Fixed python.

* fix libafl_sugar with new implementation.

* fix dangling RefCell.
adapt new examples.
TODO: merge `libafl_systemmode.*` examples.

* clippy.

* fix more fuzzers.

* clippy.

* Implement `HasInstrumentationFilter` generically.
Updated `StdInstrumentationFilter` accordingly.

* Renamed breakpoint functions for QEMU.
`qemu.run()` handling.

* Removed OnceCell / RefCell in signature.
more explicit `MmapPerms` method names.

* minor code refactoring

* Emulator::run_handle refactoring

* deprecated Emulator functions calling directly to QEMU functions.

* IsSnapshotManager -> SnapshotManager

* IsEmuExitHandler -> EmuExitHandler + fmt

* Generic register when it makes sense.

* reverted IsSnapshotManager -> SnapshotManager because of a collision.

* fix syntax + clippy

* fmt

---------

Co-authored-by: Dongjia "toka" Zhang <tokazerkje@outlook.com>
Co-authored-by: Dominik Maier <domenukk@gmail.com>
Co-authored-by: lazymio <mio@lazym.io>
Co-authored-by: Bet4 <0xbet4@gmail.com>
Co-authored-by: mkravchik <mkravchik@hotmail.com>

fuzzers/fuzzbench_fork_qemu/src/fuzzer.rs

@@ -39,7 +39,7 @@ use libafl::{
 };
 use libafl_bolts::{
     current_nanos, current_time,
-    os::dup2,
+    os::{dup2, unix_signals::Signal},
     rands::StdRand,
     shmem::{ShMemProvider, StdShMemProvider},
     tuples::{tuple_list, Merge},
@@ -49,10 +49,10 @@ use libafl_qemu::{
     cmplog::{CmpLogMap, CmpLogObserver, QemuCmpLogChildHelper},
     edges::{QemuEdgeCoverageChildHelper, EDGES_MAP_PTR, EDGES_MAP_SIZE},
     elf::EasyElf,
-    emu::Emulator,
     filter_qemu_args,
     hooks::QemuHooks,
-    GuestReg, MmapPerms, QemuForkExecutor, Regs,
+    GuestReg, MmapPerms, Qemu, QemuExitReason, QemuExitReasonError, QemuForkExecutor,
+    QemuShutdownCause, Regs,
 };
 #[cfg(unix)]
 use nix::{self, unistd::dup};
@@ -148,33 +148,38 @@ fn fuzz(
 
     let args: Vec<String> = env::args().collect();
     let env: Vec<(String, String)> = env::vars().collect();
-    let emu = Emulator::new(&args, &env)?;
+    let qemu = Qemu::init(&args, &env)?;
 
     let mut elf_buffer = Vec::new();
-    let elf = EasyElf::from_file(emu.binary_path(), &mut elf_buffer)?;
+    let elf = EasyElf::from_file(qemu.binary_path(), &mut elf_buffer)?;
 
     let test_one_input_ptr = elf
-        .resolve_symbol("LLVMFuzzerTestOneInput", emu.load_addr())
+        .resolve_symbol("LLVMFuzzerTestOneInput", qemu.load_addr())
         .expect("Symbol LLVMFuzzerTestOneInput not found");
     println!("LLVMFuzzerTestOneInput @ {test_one_input_ptr:#x}");
 
-    emu.set_breakpoint(test_one_input_ptr); // LLVMFuzzerTestOneInput
-    unsafe { emu.run() };
+    qemu.set_breakpoint(test_one_input_ptr); // LLVMFuzzerTestOneInput
+    unsafe {
+        match qemu.run() {
+            Ok(QemuExitReason::Breakpoint(_)) => {}
+            _ => panic!("Unexpected QEMU exit."),
+        }
+    }
 
-    println!("Break at {:#x}", emu.read_reg::<_, u64>(Regs::Rip).unwrap());
+    println!("Break at {:#x}", qemu.read_reg::<_, u64>(Regs::Pc).unwrap());
 
-    let stack_ptr: u64 = emu.read_reg(Regs::Rsp).unwrap();
+    let stack_ptr: u64 = qemu.read_reg(Regs::Sp).unwrap();
     let mut ret_addr = [0; 8];
-    unsafe { emu.read_mem(stack_ptr, &mut ret_addr) };
+    unsafe { qemu.read_mem(stack_ptr, &mut ret_addr) };
     let ret_addr = u64::from_le_bytes(ret_addr);
 
     println!("Stack pointer = {stack_ptr:#x}");
     println!("Return address = {ret_addr:#x}");
 
-    emu.remove_breakpoint(test_one_input_ptr); // LLVMFuzzerTestOneInput
-    emu.set_breakpoint(ret_addr); // LLVMFuzzerTestOneInput ret addr
+    qemu.remove_breakpoint(test_one_input_ptr); // LLVMFuzzerTestOneInput
+    qemu.set_breakpoint(ret_addr); // LLVMFuzzerTestOneInput ret addr
 
-    let input_addr = emu.map_private(0, 4096, MmapPerms::ReadWrite).unwrap();
+    let input_addr = qemu.map_private(0, 4096, MmapPerms::ReadWrite).unwrap();
     println!("Placing input at {input_addr:#x}");
 
     let log = RefCell::new(
@@ -314,21 +319,28 @@ fn fuzz(
         }
 
         unsafe {
-            emu.write_mem(input_addr, buf);
-
-            emu.write_reg(Regs::Rdi, input_addr).unwrap();
-            emu.write_reg(Regs::Rsi, len as GuestReg).unwrap();
-            emu.write_reg(Regs::Rip, test_one_input_ptr).unwrap();
-            emu.write_reg(Regs::Rsp, stack_ptr).unwrap();
-
-            emu.run();
+            qemu.write_mem(input_addr, buf);
+
+            qemu.write_reg(Regs::Rdi, input_addr).unwrap();
+            qemu.write_reg(Regs::Rsi, len as GuestReg).unwrap();
+            qemu.write_reg(Regs::Rip, test_one_input_ptr).unwrap();
+            qemu.write_reg(Regs::Rsp, stack_ptr).unwrap();
+
+            match qemu.run() {
+                Ok(QemuExitReason::Breakpoint(_)) => {}
+                Ok(QemuExitReason::End(QemuShutdownCause::HostSignal(Signal::SigInterrupt))) => {
+                    process::exit(0)
+                }
+                Err(QemuExitReasonError::UnexpectedExit) => return ExitKind::Crash,
+                _ => panic!("Unexpected QEMU exit."),
+            }
         }
 
         ExitKind::Ok
     };
 
     let mut hooks = QemuHooks::new(
-        emu.clone(),
+        qemu.clone(),
         tuple_list!(
             QemuEdgeCoverageChildHelper::default(),
             QemuCmpLogChildHelper::default(),

fuzzers/fuzzbench_qemu/src/fuzzer.rs

@@ -38,7 +38,7 @@ use libafl::{
 };
 use libafl_bolts::{
     current_nanos, current_time,
-    os::dup2,
+    os::{dup2, unix_signals::Signal},
     rands::StdRand,
     shmem::{ShMemProvider, StdShMemProvider},
     tuples::{tuple_list, Merge},
@@ -53,11 +53,14 @@ use libafl_qemu::{
     elf::EasyElf,
     filter_qemu_args,
     hooks::QemuHooks,
-    Emulator,
     GuestReg,
     //snapshot::QemuSnapshotHelper,
     MmapPerms,
+    Qemu,
     QemuExecutor,
+    QemuExitReason,
+    QemuExitReasonError,
+    QemuShutdownCause,
     Regs,
 };
 #[cfg(unix)]
@@ -172,34 +175,39 @@ fn fuzz(
 
     let args: Vec<String> = env::args().collect();
     let env: Vec<(String, String)> = env::vars().collect();
-    let emu = Emulator::new(&args, &env).unwrap();
+    let qemu = Qemu::init(&args, &env).unwrap();
     // let (emu, asan) = init_with_asan(&mut args, &mut env).unwrap();
 
     let mut elf_buffer = Vec::new();
-    let elf = EasyElf::from_file(emu.binary_path(), &mut elf_buffer)?;
+    let elf = EasyElf::from_file(qemu.binary_path(), &mut elf_buffer)?;
 
     let test_one_input_ptr = elf
-        .resolve_symbol("LLVMFuzzerTestOneInput", emu.load_addr())
+        .resolve_symbol("LLVMFuzzerTestOneInput", qemu.load_addr())
         .expect("Symbol LLVMFuzzerTestOneInput not found");
     println!("LLVMFuzzerTestOneInput @ {test_one_input_ptr:#x}");
 
-    emu.set_breakpoint(test_one_input_ptr); // LLVMFuzzerTestOneInput
-    unsafe { emu.run() };
+    qemu.set_breakpoint(test_one_input_ptr); // LLVMFuzzerTestOneInput
+    unsafe {
+        match qemu.run() {
+            Ok(QemuExitReason::Breakpoint(_)) => {}
+            _ => panic!("Unexpected QEMU exit."),
+        }
+    }
 
-    println!("Break at {:#x}", emu.read_reg::<_, u64>(Regs::Rip).unwrap());
+    println!("Break at {:#x}", qemu.read_reg::<_, u64>(Regs::Pc).unwrap());
 
-    let stack_ptr: u64 = emu.read_reg(Regs::Rsp).unwrap();
+    let stack_ptr: u64 = qemu.read_reg(Regs::Sp).unwrap();
     let mut ret_addr = [0; 8];
-    unsafe { emu.read_mem(stack_ptr, &mut ret_addr) };
+    unsafe { qemu.read_mem(stack_ptr, &mut ret_addr) };
     let ret_addr = u64::from_le_bytes(ret_addr);
 
     println!("Stack pointer = {stack_ptr:#x}");
     println!("Return address = {ret_addr:#x}");
 
-    emu.remove_breakpoint(test_one_input_ptr); // LLVMFuzzerTestOneInput
-    emu.set_breakpoint(ret_addr); // LLVMFuzzerTestOneInput ret addr
+    qemu.remove_breakpoint(test_one_input_ptr); // LLVMFuzzerTestOneInput
+    qemu.set_breakpoint(ret_addr); // LLVMFuzzerTestOneInput ret addr
 
-    let input_addr = emu
+    let input_addr = qemu
         .map_private(0, MAX_INPUT_SIZE, MmapPerms::ReadWrite)
         .unwrap();
     println!("Placing input at {input_addr:#x}");
@@ -328,21 +336,28 @@ fn fuzz(
         }
 
         unsafe {
-            emu.write_mem(input_addr, buf);
-
-            emu.write_reg(Regs::Rdi, input_addr).unwrap();
-            emu.write_reg(Regs::Rsi, len as GuestReg).unwrap();
-            emu.write_reg(Regs::Rip, test_one_input_ptr).unwrap();
-            emu.write_reg(Regs::Rsp, stack_ptr).unwrap();
-
-            emu.run();
+            qemu.write_mem(input_addr, buf);
+
+            qemu.write_reg(Regs::Rdi, input_addr).unwrap();
+            qemu.write_reg(Regs::Rsi, len as GuestReg).unwrap();
+            qemu.write_reg(Regs::Rip, test_one_input_ptr).unwrap();
+            qemu.write_reg(Regs::Rsp, stack_ptr).unwrap();
+
+            match qemu.run() {
+                Ok(QemuExitReason::Breakpoint(_)) => {}
+                Ok(QemuExitReason::End(QemuShutdownCause::HostSignal(Signal::SigInterrupt))) => {
+                    process::exit(0)
+                }
+                Err(QemuExitReasonError::UnexpectedExit) => return ExitKind::Crash,
+                _ => panic!("Unexpected QEMU exit."),
+            }
         }
 
         ExitKind::Ok
     };
 
     let mut hooks = QemuHooks::new(
-        emu.clone(),
+        qemu.clone(),
         tuple_list!(
             QemuEdgeCoverageHelper::default(),
             QemuCmpLogHelper::default(),

fuzzers/qemu_cmin/src/fuzzer.rs

@@ -21,6 +21,7 @@ use libafl::{
 use libafl_bolts::{
     core_affinity::Cores,
     current_nanos,
+    os::unix_signals::Signal,
     rands::StdRand,
     shmem::{ShMemProvider, StdShMemProvider},
     tuples::tuple_list,
@@ -29,9 +30,8 @@ use libafl_bolts::{
 use libafl_qemu::{
     edges::{QemuEdgeCoverageChildHelper, EDGES_MAP_PTR, EDGES_MAP_SIZE},
     elf::EasyElf,
-    emu::Emulator,
-    ArchExtras, CallingConvention, GuestAddr, GuestReg, MmapPerms, QemuForkExecutor, QemuHooks,
-    Regs,
+    ArchExtras, CallingConvention, GuestAddr, GuestReg, MmapPerms, Qemu, QemuExitReason,
+    QemuExitReasonError, QemuForkExecutor, QemuHooks, QemuShutdownCause, Regs,
 };
 
 #[derive(Default)]
@@ -113,31 +113,31 @@ pub fn fuzz() -> Result<(), Error> {
 
     env::remove_var("LD_LIBRARY_PATH");
     let env: Vec<(String, String)> = env::vars().collect();
-    let emu = Emulator::new(&options.args, &env).unwrap();
+    let qemu = Qemu::init(&options.args, &env).unwrap();
 
     let mut elf_buffer = Vec::new();
-    let elf = EasyElf::from_file(emu.binary_path(), &mut elf_buffer).unwrap();
+    let elf = EasyElf::from_file(qemu.binary_path(), &mut elf_buffer).unwrap();
 
     let test_one_input_ptr = elf
-        .resolve_symbol("LLVMFuzzerTestOneInput", emu.load_addr())
+        .resolve_symbol("LLVMFuzzerTestOneInput", qemu.load_addr())
         .expect("Symbol LLVMFuzzerTestOneInput not found");
     log::debug!("LLVMFuzzerTestOneInput @ {test_one_input_ptr:#x}");
 
-    emu.entry_break(test_one_input_ptr);
+    qemu.entry_break(test_one_input_ptr);
 
-    let pc: GuestReg = emu.read_reg(Regs::Pc).unwrap();
+    let pc: GuestReg = qemu.read_reg(Regs::Pc).unwrap();
     log::debug!("Break at {pc:#x}");
 
-    let ret_addr: GuestAddr = emu.read_return_address().unwrap();
+    let ret_addr: GuestAddr = qemu.read_return_address().unwrap();
     log::debug!("Return address = {ret_addr:#x}");
-    emu.set_breakpoint(ret_addr);
+    qemu.set_breakpoint(ret_addr);
 
-    let input_addr = emu
+    let input_addr = qemu
         .map_private(0, MAX_INPUT_SIZE, MmapPerms::ReadWrite)
         .unwrap();
     log::debug!("Placing input at {input_addr:#x}");
 
-    let stack_ptr: GuestAddr = emu.read_reg(Regs::Sp).unwrap();
+    let stack_ptr: GuestAddr = qemu.read_reg(Regs::Sp).unwrap();
 
     let mut shmem_provider = StdShMemProvider::new().expect("Failed to init shared memory");
 
@@ -201,24 +201,29 @@ pub fn fuzz() -> Result<(), Error> {
         let len = len as GuestReg;
 
         unsafe {
-            emu.write_mem(input_addr, buf);
-            emu.write_reg(Regs::Pc, test_one_input_ptr).unwrap();
-            emu.write_reg(Regs::Sp, stack_ptr).unwrap();
-            emu.write_return_address(ret_addr).unwrap();
-            emu.write_function_argument(CallingConvention::Cdecl, 0, input_addr)
+            qemu.write_mem(input_addr, buf);
+            qemu.write_reg(Regs::Pc, test_one_input_ptr).unwrap();
+            qemu.write_reg(Regs::Sp, stack_ptr).unwrap();
+            qemu.write_return_address(ret_addr).unwrap();
+            qemu.write_function_argument(CallingConvention::Cdecl, 0, input_addr)
                 .unwrap();
-            emu.write_function_argument(CallingConvention::Cdecl, 1, len)
+            qemu.write_function_argument(CallingConvention::Cdecl, 1, len)
                 .unwrap();
-            emu.run();
+
+            match qemu.run() {
+                Ok(QemuExitReason::Breakpoint(_)) => {}
+                Ok(QemuExitReason::End(QemuShutdownCause::HostSignal(Signal::SigInterrupt))) => {
+                    process::exit(0)
+                }
+                Err(QemuExitReasonError::UnexpectedExit) => return ExitKind::Crash,
+                _ => panic!("Unexpected QEMU exit."),
+            }
         }
 
         ExitKind::Ok
     };
 
-    let mut hooks = QemuHooks::new(
-        emu.clone(),
-        tuple_list!(QemuEdgeCoverageChildHelper::default(),),
-    );
+    let mut hooks = QemuHooks::new(qemu, tuple_list!(QemuEdgeCoverageChildHelper::default(),));
 
     let mut executor = QemuForkExecutor::new(
         &mut hooks,