Extend elfuse

[doanbaotrung]

Dear owner,

I'm interesting with your work and planning to extend it to run android binary also.
Current I found some missing syscall. Let me share the running logs:

ELF entry   : 0x218000
Load range  : 0x200000 – 0x286728
Segments    : 4
calling guest_bootstrap_prepare...
15:23:19 DEBUG src/core/bootstrap.c:219: ELF entry=0x218000, 4 segments, load range [0x200000, 0x286728), machine=aarch64
15:23:19 DEBUG src/core/bootstrap.c:228: IPA size: 40 bits (1024 GiB primary)
15:23:19 DEBUG src/core/bootstrap.c:257: shim loaded at offset 0x100000 (4912 bytes)
15:23:19 DEBUG src/core/bootstrap.c:329: TTBR0=0x10000, IPA base=0x0
15:23:19 DEBUG src/core/bootstrap.c:71: L0[0]=0x11003
15:23:19 DEBUG src/core/bootstrap.c:78: L1[0]=0x12003
15:23:19 DEBUG src/core/bootstrap.c:85: L2[0]=0x14003
15:23:19 DEBUG src/core/bootstrap.c:85: L2[1]=0x15003
15:23:19 DEBUG src/core/bootstrap.c:85: L2[8]=0x60000001000765
15:23:19 DEBUG src/core/bootstrap.c:85: L2[9]=0x60000001200765
15:23:19 DEBUG src/core/bootstrap.c:85: L2[10]=0x60000001400765
15:23:19 DEBUG src/core/bootstrap.c:85: L2[11]=0x60000001600765
15:23:19 DEBUG src/core/bootstrap.c:85: L2[12]=0x60000001800765
15:23:19 DEBUG src/core/bootstrap.c:85: L2[13]=0x60000001a00765
15:23:19 DEBUG src/core/bootstrap.c:85: L2[14]=0x60000001c00765
15:23:19 DEBUG src/core/bootstrap.c:85: L2[15]=0x60000001e00765
15:23:19 DEBUG src/core/bootstrap.c:370: SP=0x7fff100, entry=0x218000
guest_bootstrap_prepare returned: 0
calling guest_bootstrap_create_vcpu...
15:23:19 DEBUG src/core/bootstrap.c:410: SCTLR_EL1 default=0x0
15:23:19 DEBUG src/core/bootstrap.c:428: vCPU configured: PC=0x100000 SCTLR=0x34d0d984 VBAR=0x100800 TTBR0=0x10000 TCR=0x25b5903510
15:23:19 DEBUG src/core/bootstrap.c:431: ELR_EL1=0x218000 SP_EL0=0x7fff100 SP_EL1=0x400000
15:23:19 DEBUG src/core/bootstrap.c:435: main thread registered with SP_EL1=0x400000
guest_bootstrap_create_vcpu returned: 0
calling vcpu_run_loop...
15:23:19 DEBUG src/syscall/proc.c:1048: elfuse: [0] vcpu_run PC=0x100000
15:23:19 DEBUG src/syscall/proc.c:1105: elfuse: HVC #4
15:23:19 DEBUG src/syscall/proc.c:1196: elfuse: HVC #4 set reg 4 = 0x34d0d985
15:23:19 DEBUG src/syscall/proc.c:1048: elfuse: [1] vcpu_run PC=0x10001c
15:23:19 DEBUG src/syscall/proc.c:1105: elfuse: HVC #5
15:23:19 DEBUG src/syscall/syscall.c:1783: syscall 172@0x25ec88(0x7fff068, 0x7fff050, 0x2180cc, 0x7fff0c0, 0x7fff050, 0x0)
15:23:19 DEBUG src/syscall/syscall.c:1815:   -> 1 (0x1)
15:23:19 DEBUG src/syscall/proc.c:1048: elfuse: [2] vcpu_run PC=0x1012a8
15:23:19 DEBUG src/syscall/proc.c:1105: elfuse: HVC #5
15:23:19 DEBUG src/syscall/syscall.c:1783: syscall 222@0x25f948(0x0, 0x3000, 0x3, 0x22, 0xffffffff, 0x0)
15:23:19 DEBUG src/syscall/syscall.c:709:   mmap(0x0, 0x3000) → 0x200000000
15:23:19 DEBUG src/syscall/syscall.c:1815:   -> 8589934592 (0x200000000)
15:23:19 DEBUG src/syscall/proc.c:1048: elfuse: [3] vcpu_run PC=0x1012a8
15:23:19 DEBUG src/syscall/proc.c:1105: elfuse: HVC #5
15:23:19 DEBUG src/syscall/syscall.c:1783: syscall 96@0x261388(0x285808, 0x200000000, 0x3, 0x22, 0xffffffff, 0x0)
15:23:19 DEBUG src/syscall/syscall.c:1815:   -> 1 (0x1)
15:23:19 DEBUG src/syscall/proc.c:1048: elfuse: [4] vcpu_run PC=0x1012a8
15:23:19 DEBUG src/syscall/proc.c:1105: elfuse: HVC #5
15:23:19 DEBUG src/syscall/syscall.c:1783: syscall 48@0x25f508(0xffffff9c, 0x203e92, 0x4, 0x0, 0xffffffff, 0x0)
15:23:19 DEBUG src/syscall/syscall.c:1815:   -> 0 (0x0)
15:23:19 DEBUG src/syscall/syscall.c:1821:   path="/dev/urandom"
15:23:19 DEBUG src/syscall/proc.c:1048: elfuse: [5] vcpu_run PC=0x1012a8
15:23:19 DEBUG src/syscall/proc.c:1105: elfuse: HVC #5
15:23:19 DEBUG src/syscall/syscall.c:1783: syscall 278@0x2615c8(0x7ffeee0, 0x28, 0x1, 0x0, 0xffffffff, 0x0)
15:23:19 DEBUG src/syscall/syscall.c:1815:   -> 40 (0x28)
15:23:19 DEBUG src/syscall/proc.c:1048: elfuse: [6] vcpu_run PC=0x1012a8
15:23:19 DEBUG src/syscall/proc.c:1105: elfuse: HVC #5
15:23:19 DEBUG src/syscall/syscall.c:1783: syscall 222@0x25f948(0x0, 0x450, 0x3, 0x22, 0xffffffff, 0x0)
15:23:19 DEBUG src/syscall/syscall.c:709:   mmap(0x0, 0x450) → 0x200004000
15:23:19 DEBUG src/syscall/syscall.c:1815:   -> 8589950976 (0x200004000)
15:23:19 DEBUG src/syscall/proc.c:1048: elfuse: [7] vcpu_run PC=0x1012a8
15:23:19 DEBUG src/syscall/proc.c:1105: elfuse: HVC #5
15:23:19 DEBUG src/syscall/syscall.c:1783: syscall 167@0x25e888(0x53564d41, 0x0, 0x200004000, 0x450, 0x20062a, 0x0)
15:23:19 DEBUG src/syscall/syscall.c:1815:   -> 0 (0x0)
15:23:19 DEBUG src/syscall/proc.c:1048: elfuse: [8] vcpu_run PC=0x1012a8
15:23:19 DEBUG src/syscall/proc.c:1105: elfuse: HVC #7
15:23:19 DEBUG src/syscall/proc.c:1346: elfuse: MRS trap: Op0=3 Op1=0 CRn=0 CRm=0 Op2=0 -> 0x610f0000
15:23:19 DEBUG src/syscall/proc.c:1048: elfuse: [9] vcpu_run PC=0x10103c
15:23:19 DEBUG src/syscall/proc.c:1105: elfuse: HVC #7
15:23:19 DEBUG src/syscall/proc.c:1346: elfuse: MRS trap: Op0=3 Op1=0 CRn=0 CRm=0 Op2=0 -> 0x610f0000
15:23:19 DEBUG src/syscall/proc.c:1048: elfuse: [10] vcpu_run PC=0x10103c
15:23:19 DEBUG src/syscall/proc.c:1105: elfuse: HVC #5
15:23:19 DEBUG src/syscall/syscall.c:1783: syscall 120@0x260a48(0x0, 0x285698, 0x8, 0x1, 0x200004080, 0x20000407c)
15:23:19 WARN  src/syscall/syscall.c:1805: unimplemented syscall 120 (x0=0x0, x1=0x285698, x2=0x8, x3=0x1, x4=0x200004080, x5=0x20000407c)
15:23:19 DEBUG src/syscall/syscall.c:1815:   -> -38 (0xffffffffffffffda)
15:23:19 DEBUG src/syscall/proc.c:1048: elfuse: [11] vcpu_run PC=0x1012a8
15:23:19 DEBUG src/syscall/proc.c:1105: elfuse: HVC #5
15:23:19 DEBUG src/syscall/syscall.c:1783: syscall 198@0x247560(0x1, 0x80802, 0x0, 0x7ffe958, 0x7ffeb3a, 0x10)
15:23:19 DEBUG src/syscall/syscall.c:1815:   -> 3 (0x3)
15:23:19 DEBUG src/syscall/proc.c:1048: elfuse: [12] vcpu_run PC=0x1012a8
15:23:19 DEBUG src/syscall/proc.c:1105: elfuse: HVC #5
15:23:19 DEBUG src/syscall/syscall.c:1783: syscall 203@0x260588(0x3, 0x7ffe930, 0x6e, 0x6c, 0x7ffeb3a, 0x10)
15:23:19 DEBUG src/syscall/syscall.c:1815:   -> -2 (0xfffffffffffffffe)
15:23:19 DEBUG src/syscall/proc.c:1048: elfuse: [13] vcpu_run PC=0x1012a8
15:23:19 DEBUG src/syscall/proc.c:1105: elfuse: HVC #5
15:23:19 DEBUG src/syscall/syscall.c:1783: syscall 57@0x247560(0x3, 0x6e, 0x6c, 0x7ffeb3a, 0x10, 0x10)
15:23:19 DEBUG src/syscall/syscall.c:1815:   -> 0 (0x0)
15:23:19 DEBUG src/syscall/proc.c:1048: elfuse: [14] vcpu_run PC=0x1012a8
15:23:19 DEBUG src/syscall/proc.c:1105: elfuse: HVC #5
15:23:19 DEBUG src/syscall/syscall.c:1783: syscall 66@0x25f088(0x2, 0x7ffe930, 0x4, 0xffffff, 0x0, 0x0)
libc: pthread_create sched_getscheduler failed: Function not implemented
15:23:19 DEBUG src/syscall/syscall.c:1815:   -> 73 (0x49)
15:23:19 DEBUG src/syscall/proc.c:1048: elfuse: [15] vcpu_run PC=0x1012a8
15:23:19 DEBUG src/syscall/proc.c:1105: elfuse: HVC #5
15:23:19 DEBUG src/syscall/syscall.c:1783: syscall 222@0x25f948(0x0, 0xc000, 0x3, 0x22, 0xffffffff, 0x0)
15:23:19 DEBUG src/syscall/syscall.c:709:   mmap(0x0, 0xc000) → 0x200008000
15:23:19 DEBUG src/syscall/syscall.c:1815:   -> 8589967360 (0x200008000)
15:23:19 DEBUG src/syscall/proc.c:1048: elfuse: [16] vcpu_run PC=0x1012a8
15:23:19 DEBUG src/syscall/proc.c:1105: elfuse: HVC #5
15:23:19 DEBUG src/syscall/syscall.c:1783: syscall 226@0x25ed88(0x200008000, 0x4000, 0x0, 0x22, 0xffffffff, 0x0)
15:23:19 DEBUG src/syscall/syscall.c:1815:   -> 0 (0x0)
15:23:19 DEBUG src/syscall/proc.c:1048: elfuse: [17] vcpu_run PC=0x1012a8
15:23:19 DEBUG src/syscall/proc.c:1105: elfuse: HVC #5
15:23:19 DEBUG src/syscall/syscall.c:1783: syscall 132@0x25e948(0x7ffef60, 0x0, 0x0, 0x22, 0xffffffff, 0x0)
15:23:19 DEBUG src/syscall/syscall.c:1815:   -> 0 (0x0)
15:23:19 DEBUG src/syscall/proc.c:1048: elfuse: [18] vcpu_run PC=0x1012a8
15:23:19 DEBUG src/syscall/proc.c:1105: elfuse: HVC #5
15:23:19 DEBUG src/syscall/syscall.c:1783: syscall 167@0x25e888(0x53564d41, 0x0, 0x20000c000, 0x8000, 0x202eec, 0x0)
15:23:19 DEBUG src/syscall/syscall.c:1815:   -> 0 (0x0)
15:23:19 DEBUG src/syscall/proc.c:1048: elfuse: [19] vcpu_run PC=0x1012a8
15:23:19 DEBUG src/syscall/proc.c:1105: elfuse: HVC #5
15:23:19 DEBUG src/syscall/syscall.c:1783: syscall 222@0x25f948(0x0, 0x1000000, 0x0, 0x22, 0xffffffff, 0x0)
15:23:19 DEBUG src/syscall/syscall.c:709:   mmap(0x0, 0x1000000) → 0x200014000
15:23:19 DEBUG src/syscall/syscall.c:1815:   -> 8590016512 (0x200014000)
15:23:19 DEBUG src/syscall/proc.c:1048: elfuse: [20] vcpu_run PC=0x1012a8
15:23:19 DEBUG src/syscall/proc.c:1105: elfuse: HVC #5
15:23:19 DEBUG src/syscall/syscall.c:1783: syscall 226@0x25ed88(0x200014000, 0x4000, 0x3, 0x22, 0xffffffff, 0x0)
15:23:19 DEBUG src/syscall/syscall.c:1815:   -> 0 (0x0)
15:23:19 DEBUG src/syscall/proc.c:1048: elfuse: [21] vcpu_run PC=0x1012a8
15:23:19 DEBUG src/syscall/proc.c:1105: elfuse: HVC #5
15:23:19 DEBUG src/syscall/syscall.c:1783: syscall 226@0x25ed88(0x270000, 0x4000, 0x1, 0x273f80, 0x274000, 0x4)
15:23:19 DEBUG src/syscall/syscall.c:1815:   -> -22 (0xffffffffffffffea)
15:23:19 DEBUG src/syscall/proc.c:1048: elfuse: [22] vcpu_run PC=0x1012a8
15:23:19 DEBUG src/syscall/proc.c:1105: elfuse: HVC #5
15:23:19 DEBUG src/syscall/syscall.c:1783: syscall 66@0x25f088(0x2, 0x7ffe9b8, 0x2, 0xffffff, 0x0, 0x0)
WriteProtected mprotect 1 failed: Invalid argument
15:23:19 DEBUG src/syscall/syscall.c:1815:   -> 51 (0x33)
15:23:19 DEBUG src/syscall/proc.c:1048: elfuse: [23] vcpu_run PC=0x1012a8
15:23:19 DEBUG src/syscall/proc.c:1105: elfuse: HVC #5
15:23:19 DEBUG src/syscall/syscall.c:1783: syscall 198@0x247560(0x1, 0x80802, 0x0, 0x0, 0x0, 0x63646b6860651f30)
15:23:19 DEBUG src/syscall/syscall.c:1815:   -> 3 (0x3)
15:23:19 DEBUG src/syscall/proc.c:1048: elfuse: [24] vcpu_run PC=0x1012a8
15:23:19 DEBUG src/syscall/proc.c:1105: elfuse: HVC #5
15:23:19 DEBUG src/syscall/syscall.c:1783: syscall 203@0x260588(0x3, 0x7ffe8c0, 0x6e, 0x6c, 0x0, 0x63646b6860651f30)
15:23:19 DEBUG src/syscall/syscall.c:1815:   -> -2 (0xfffffffffffffffe)
15:23:19 DEBUG src/syscall/proc.c:1048: elfuse: [25] vcpu_run PC=0x1012a8
15:23:19 DEBUG src/syscall/proc.c:1105: elfuse: HVC #5
15:23:19 DEBUG src/syscall/syscall.c:1783: syscall 57@0x247560(0x3, 0x6e, 0x6c, 0x0, 0x63646b6860651f30, 0x63646b6860651f30)
15:23:19 DEBUG src/syscall/syscall.c:1815:   -> 0 (0x0)
15:23:19 DEBUG src/syscall/proc.c:1048: elfuse: [26] vcpu_run PC=0x1012a8
15:23:19 DEBUG src/syscall/proc.c:1105: elfuse: HVC #5
15:23:19 DEBUG src/syscall/syscall.c:1783: syscall 66@0x25f088(0x2, 0x7ffe8c0, 0x4, 0xffffff, 0x0, 0x0)
libc: WriteProtected mprotect 1 failed: Invalid argument
15:23:19 DEBUG src/syscall/syscall.c:1815:   -> 57 (0x39)
15:23:19 DEBUG src/syscall/proc.c:1048: elfuse: [27] vcpu_run PC=0x1012a8
15:23:19 DEBUG src/syscall/proc.c:1105: elfuse: HVC #5
15:23:19 DEBUG src/syscall/syscall.c:1783: syscall 222@0x25f948(0x0, 0x4b, 0x3, 0x22, 0xffffffff, 0x0)
15:23:19 DEBUG src/syscall/syscall.c:709:   mmap(0x0, 0x4b) → 0x201014000
15:23:19 DEBUG src/syscall/syscall.c:1815:   -> 8606793728 (0x201014000)
15:23:19 DEBUG src/syscall/proc.c:1048: elfuse: [28] vcpu_run PC=0x1012a8
15:23:19 DEBUG src/syscall/proc.c:1105: elfuse: HVC #5
15:23:19 DEBUG src/syscall/syscall.c:1783: syscall 167@0x25e888(0x53564d41, 0x0, 0x201014000, 0x4b, 0x200825, 0x0)
15:23:19 DEBUG src/syscall/syscall.c:1815:   -> 0 (0x0)
15:23:19 DEBUG src/syscall/proc.c:1048: elfuse: [29] vcpu_run PC=0x1012a8
15:23:19 DEBUG src/syscall/proc.c:1105: elfuse: HVC #5
15:23:19 DEBUG src/syscall/syscall.c:1783: syscall 135@0x260388(0x1, 0x7ffee78, 0x0, 0x8, 0xa, 0xfffffffe06fea9c0)
15:23:19 DEBUG src/syscall/syscall.c:1815:   -> 0 (0x0)
15:23:19 DEBUG src/syscall/proc.c:1048: elfuse: [30] vcpu_run PC=0x1012a8
15:23:19 DEBUG src/syscall/proc.c:1105: elfuse: HVC #5
15:23:19 DEBUG src/syscall/syscall.c:1783: syscall 172@0x247560(0x7ffee78, 0x0, 0x8, 0xa, 0xfffffffe06fea9c0, 0x63646b6860651f30)
15:23:19 DEBUG src/syscall/syscall.c:1815:   -> 1 (0x1)
15:23:19 DEBUG src/syscall/proc.c:1048: elfuse: [31] vcpu_run PC=0x1012a8
15:23:19 DEBUG src/syscall/proc.c:1105: elfuse: HVC #5
15:23:19 DEBUG src/syscall/syscall.c:1783: syscall 178@0x247560(0x0, 0x8, 0xa, 0xfffffffe06fea9c0, 0x63646b6860651f30, 0x63646b6860651f30)
15:23:19 DEBUG src/syscall/syscall.c:1815:   -> 1 (0x1)
15:23:19 DEBUG src/syscall/proc.c:1048: elfuse: [32] vcpu_run PC=0x1012a8
15:23:19 DEBUG src/syscall/proc.c:1105: elfuse: HVC #5
15:23:19 DEBUG src/syscall/syscall.c:1783: syscall 174@0x25e008(0x1, 0x8, 0xa, 0xfffffffe06fea9c0, 0x63646b6860651f30, 0x63646b6860651f30)
15:23:19 DEBUG src/syscall/syscall.c:1815:   -> 1000 (0x3e8)
15:23:19 DEBUG src/syscall/proc.c:1048: elfuse: [33] vcpu_run PC=0x1012a8
15:23:19 DEBUG src/syscall/proc.c:1105: elfuse: HVC #5
15:23:19 DEBUG src/syscall/syscall.c:1783: syscall 240@0x21b4f4(0x1, 0x1, 0x6, 0x7ffeec0, 0x63646b6860651f30, 0x63646b6860651f30)
15:23:19 DEBUG src/syscall/syscall.c:851: rt_tgsigqueueinfo(tgid=1, tid=1, sig=6, si_code=-1)
15:23:19 DEBUG src/syscall/syscall.c:1815:   -> 0 (0x0)
vcpu_run_loop returned: 0

As you can see in the logs, the errors as follow:

15:23:19 WARN  src/syscall/syscall.c:1805: unimplemented syscall 120 (x0=0x0, x1=0x285698, x2=0x8, x3=0x1, x4=0x200004080, x5=0x20000407c)
libc: pthread_create sched_getscheduler failed: Function not implemented
WriteProtected mprotect 1 failed: Invalid argument

May I know if you have plan to support mprotect, munmap, and mmap(MAP_FIXED), and sched_getscheduler yet? If not, do you have plan to add Syscall Hook API support?

Thanks,
Trung

[jserv]

I'm interesting with your work and planning to extend it to run android binary also. Current I found some missing syscall. Let me share the running logs:

Before discussing support for Android executable files, could you provide a procedure to reproduce the issue? For example, please include the host environment (macOS version, SDK version, hardware information, and the status of make check), along with the binary files you attempted to execute as a minimal reproducible example (MRE). It would also be helpful to include the exact commands used, the observed behavior or error messages, and whether the issue is reproducible on a clean environment.

[doanbaotrung]

Mac OS 26.4.1 (25E253)
Model Name: MacBook Pro
Model Identifier: Mac14,9
Model Number: Z17G000ELSA/A
Chip: Apple M2 Pro
Total Number of Cores: 10 (6 Performance and 4 Efficiency)
Memory: 32 GB
System Firmware Version: 18000.101.7
OS Loader Version: 18000.101.7

test_return_42.c

int main() {
    return 42;
}

compile.sh

#!/bin/zsh
binary_output=build/bin/test_return_42
compiler_android_clang=$ANDROID_NDK_HOME/toolchains/llvm/prebuilt/darwin-x86_64/bin/aarch64-linux-android35-clang
readelf_command=$ANDROID_NDK_HOME/toolchains/llvm/prebuilt/darwin-x86_64/bin/llvm-readelf

mkdir -p build/bin
$compiler_android_clang -static -Wl,-z,max-page-size=4096 tests/assets/elf/test_return_42.c -o ${binary_output}

This is my function

int GuestRunner::run(const GuestRunnerConfig& cfg)
{
    // -----------------------------------------------------------------------
    // 0. Initialise elfuse logging
    // -----------------------------------------------------------------------
    log_init();
    if (cfg.verbose) {
        log_set_level(LOG_DEBUG);
    }

    // -----------------------------------------------------------------------
    // 1. Prepare argv / sysroot as C strings
    // -----------------------------------------------------------------------
    const char* elf_path = cfg.elf_path.c_str();
    const char* sysroot  = cfg.sysroot.empty() ? nullptr
                                                : cfg.sysroot.c_str();

    int guest_argc = static_cast<int>(cfg.argv.size());
    const char** guest_argv = to_cstrings(cfg.argv);

    if (!guest_argv) {
        throw std::runtime_error("GuestRunner: out of memory allocating argv");
    }

    // -----------------------------------------------------------------------
    // 2. Bootstrap the guest
    // -----------------------------------------------------------------------
    guest_t      g;
    bool         guest_initialized = false;
    guest_bootstrap_t boot;

    std::printf("[Muplar] calling guest_bootstrap_prepare...\n");
    int rc = guest_bootstrap_prepare( &g, elf_path, sysroot, guest_argc, guest_argv, environ, shim_bin, shim_bin_len, cfg.verbose, &guest_initialized, &boot );
    std::printf("[Muplar] guest_bootstrap_prepare returned: %d\n", rc);

    free_cstrings(guest_argv, guest_argc);

    if (rc < 0) {
        if (guest_initialized) guest_destroy(&g);
        throw std::runtime_error(
            "GuestRunner: guest_bootstrap_prepare failed for: " + cfg.elf_path
        );
    }

    // -----------------------------------------------------------------------
    // 3. Create the vCPU
    // -----------------------------------------------------------------------
    hv_vcpu_t       vcpu;
    hv_vcpu_exit_t* vexit;

    std::printf("[Muplar] calling guest_bootstrap_create_vcpu...\n");
    rc = guest_bootstrap_create_vcpu(&g, &boot, cfg.verbose, &vcpu, &vexit);
    std::printf("[Muplar] guest_bootstrap_create_vcpu returned: %d\n", rc);

    if (rc < 0) {
        guest_destroy(&g);
        throw std::runtime_error(
            "GuestRunner: guest_bootstrap_create_vcpu failed"
        );
    }

    // -----------------------------------------------------------------------
    // 4. Run the guest
    // -----------------------------------------------------------------------
    std::printf("[Muplar] calling vcpu_run_loop...\n");
    int exit_code = vcpu_run_loop(vcpu, vexit, &g, cfg.verbose, cfg.timeout_sec);
    std::printf("[Muplar] vcpu_run_loop returned: %d\n", rc);

    // -----------------------------------------------------------------------
    // 5. Tear down
    // -----------------------------------------------------------------------
    guest_destroy(&g);

    return exit_code;
}

[jserv]

15:23:19 WARN  src/syscall/syscall.c:1805: unimplemented syscall 120 (x0=0x0, x1=0x285698, x2=0x8, x3=0x1, x4=0x200004080, x5=0x20000407c)
libc: pthread_create sched_getscheduler failed: Function not implemented
WriteProtected mprotect 1 failed: Invalid argument

These syscalls are implemented via #22 .

[doanbaotrung]

Nice. I saw sched_getscheduler works now and don't see error about syscall 120 any more.
But still see the error of mprotect
libc: WriteProtected mprotect 1 failed: Invalid argument

calling vcpu_run_loop...
19:11:45 DEBUG src/syscall/proc.c:1048: elfuse: [0] vcpu_run PC=0x100000
19:11:45 DEBUG src/syscall/proc.c:1105: elfuse: HVC #4
19:11:45 DEBUG src/syscall/proc.c:1196: elfuse: HVC #4 set reg 4 = 0x34d0d985
19:11:45 DEBUG src/syscall/proc.c:1048: elfuse: [1] vcpu_run PC=0x10001c
19:11:45 DEBUG src/syscall/proc.c:1105: elfuse: HVC #5
19:11:45 DEBUG src/syscall/syscall.c:1790: syscall 172@0x25ec88(0x7fff048, 0x7fff030, 0x2180cc, 0x7fff0a0, 0x7fff030, 0x0)
19:11:45 DEBUG src/syscall/syscall.c:1822:   -> 1 (0x1)
19:11:45 DEBUG src/syscall/proc.c:1048: elfuse: [2] vcpu_run PC=0x1012a8
19:11:45 DEBUG src/syscall/proc.c:1105: elfuse: HVC #5
19:11:45 DEBUG src/syscall/syscall.c:1790: syscall 222@0x25f948(0x0, 0x3000, 0x3, 0x22, 0xffffffff, 0x0)
19:11:45 DEBUG src/syscall/syscall.c:716:   mmap(0x0, 0x3000) → 0x200000000
19:11:45 DEBUG src/syscall/syscall.c:1822:   -> 8589934592 (0x200000000)
19:11:45 DEBUG src/syscall/proc.c:1048: elfuse: [3] vcpu_run PC=0x1012a8
19:11:45 DEBUG src/syscall/proc.c:1105: elfuse: HVC #5
19:11:45 DEBUG src/syscall/syscall.c:1790: syscall 96@0x261388(0x285808, 0x200000000, 0x3, 0x22, 0xffffffff, 0x0)
19:11:45 DEBUG src/syscall/syscall.c:1822:   -> 1 (0x1)
19:11:45 DEBUG src/syscall/proc.c:1048: elfuse: [4] vcpu_run PC=0x1012a8
19:11:45 DEBUG src/syscall/proc.c:1105: elfuse: HVC #5
19:11:45 DEBUG src/syscall/syscall.c:1790: syscall 48@0x25f508(0xffffff9c, 0x203e92, 0x4, 0x0, 0xffffffff, 0x0)
19:11:45 DEBUG src/syscall/syscall.c:1822:   -> 0 (0x0)
19:11:45 DEBUG src/syscall/syscall.c:1828:   path="/dev/urandom"
19:11:45 DEBUG src/syscall/proc.c:1048: elfuse: [5] vcpu_run PC=0x1012a8
19:11:45 DEBUG src/syscall/proc.c:1105: elfuse: HVC #5
19:11:45 DEBUG src/syscall/syscall.c:1790: syscall 278@0x2615c8(0x7ffeec0, 0x28, 0x1, 0x0, 0xffffffff, 0x0)
19:11:45 DEBUG src/syscall/syscall.c:1822:   -> 40 (0x28)
19:11:45 DEBUG src/syscall/proc.c:1048: elfuse: [6] vcpu_run PC=0x1012a8
19:11:45 DEBUG src/syscall/proc.c:1105: elfuse: HVC #5
19:11:45 DEBUG src/syscall/syscall.c:1790: syscall 222@0x25f948(0x0, 0x450, 0x3, 0x22, 0xffffffff, 0x0)
19:11:45 DEBUG src/syscall/syscall.c:716:   mmap(0x0, 0x450) → 0x200004000
19:11:45 DEBUG src/syscall/syscall.c:1822:   -> 8589950976 (0x200004000)
19:11:45 DEBUG src/syscall/proc.c:1048: elfuse: [7] vcpu_run PC=0x1012a8
19:11:45 DEBUG src/syscall/proc.c:1105: elfuse: HVC #5
19:11:45 DEBUG src/syscall/syscall.c:1790: syscall 167@0x25e888(0x53564d41, 0x0, 0x200004000, 0x450, 0x20062a, 0x0)
19:11:45 DEBUG src/syscall/syscall.c:1822:   -> 0 (0x0)
19:11:45 DEBUG src/syscall/proc.c:1048: elfuse: [8] vcpu_run PC=0x1012a8
19:11:45 DEBUG src/syscall/proc.c:1105: elfuse: HVC #7
19:11:45 DEBUG src/syscall/proc.c:1346: elfuse: MRS trap: Op0=3 Op1=0 CRn=0 CRm=0 Op2=0 -> 0x610f0000
19:11:45 DEBUG src/syscall/proc.c:1048: elfuse: [9] vcpu_run PC=0x10103c
19:11:45 DEBUG src/syscall/proc.c:1105: elfuse: HVC #7
19:11:45 DEBUG src/syscall/proc.c:1346: elfuse: MRS trap: Op0=3 Op1=0 CRn=0 CRm=0 Op2=0 -> 0x610f0000
19:11:45 DEBUG src/syscall/proc.c:1048: elfuse: [10] vcpu_run PC=0x10103c
19:11:45 DEBUG src/syscall/proc.c:1105: elfuse: HVC #5
19:11:45 DEBUG src/syscall/syscall.c:1790: syscall 120@0x260a48(0x0, 0x285698, 0x8, 0x1, 0x200004080, 0x20000407c)
19:11:45 DEBUG src/syscall/syscall.c:1822:   -> 0 (0x0)
19:11:45 DEBUG src/syscall/proc.c:1048: elfuse: [11] vcpu_run PC=0x1012a8
19:11:45 DEBUG src/syscall/proc.c:1105: elfuse: HVC #5
19:11:45 DEBUG src/syscall/syscall.c:1790: syscall 222@0x25f948(0x0, 0xc000, 0x3, 0x22, 0xffffffff, 0x0)
19:11:45 DEBUG src/syscall/syscall.c:716:   mmap(0x0, 0xc000) → 0x200008000
19:11:45 DEBUG src/syscall/syscall.c:1822:   -> 8589967360 (0x200008000)
19:11:45 DEBUG src/syscall/proc.c:1048: elfuse: [12] vcpu_run PC=0x1012a8
19:11:45 DEBUG src/syscall/proc.c:1105: elfuse: HVC #5
19:11:45 DEBUG src/syscall/syscall.c:1790: syscall 226@0x25ed88(0x200008000, 0x4000, 0x0, 0x22, 0xffffffff, 0x0)
19:11:45 DEBUG src/syscall/syscall.c:1822:   -> 0 (0x0)
19:11:45 DEBUG src/syscall/proc.c:1048: elfuse: [13] vcpu_run PC=0x1012a8
19:11:45 DEBUG src/syscall/proc.c:1105: elfuse: HVC #5
19:11:45 DEBUG src/syscall/syscall.c:1790: syscall 132@0x25e948(0x7ffef40, 0x0, 0x0, 0x22, 0xffffffff, 0x0)
19:11:45 DEBUG src/syscall/syscall.c:1822:   -> 0 (0x0)
19:11:45 DEBUG src/syscall/proc.c:1048: elfuse: [14] vcpu_run PC=0x1012a8
19:11:45 DEBUG src/syscall/proc.c:1105: elfuse: HVC #5
19:11:45 DEBUG src/syscall/syscall.c:1790: syscall 167@0x25e888(0x53564d41, 0x0, 0x20000c000, 0x8000, 0x202eec, 0x0)
19:11:45 DEBUG src/syscall/syscall.c:1822:   -> 0 (0x0)
19:11:45 DEBUG src/syscall/proc.c:1048: elfuse: [15] vcpu_run PC=0x1012a8
19:11:45 DEBUG src/syscall/proc.c:1105: elfuse: HVC #5
19:11:45 DEBUG src/syscall/syscall.c:1790: syscall 222@0x25f948(0x0, 0x1000000, 0x0, 0x22, 0xffffffff, 0x0)
19:11:45 DEBUG src/syscall/syscall.c:716:   mmap(0x0, 0x1000000) → 0x200014000
19:11:45 DEBUG src/syscall/syscall.c:1822:   -> 8590016512 (0x200014000)
19:11:45 DEBUG src/syscall/proc.c:1048: elfuse: [16] vcpu_run PC=0x1012a8
19:11:45 DEBUG src/syscall/proc.c:1105: elfuse: HVC #5
19:11:45 DEBUG src/syscall/syscall.c:1790: syscall 226@0x25ed88(0x200014000, 0x4000, 0x3, 0x22, 0xffffffff, 0x0)
19:11:45 DEBUG src/syscall/syscall.c:1822:   -> 0 (0x0)
19:11:45 DEBUG src/syscall/proc.c:1048: elfuse: [17] vcpu_run PC=0x1012a8
19:11:45 DEBUG src/syscall/proc.c:1105: elfuse: HVC #5
19:11:45 DEBUG src/syscall/syscall.c:1790: syscall 226@0x25ed88(0x270000, 0x4000, 0x1, 0x273f80, 0x274000, 0x4)
19:11:45 DEBUG src/syscall/syscall.c:1822:   -> -22 (0xffffffffffffffea)
19:11:45 DEBUG src/syscall/proc.c:1048: elfuse: [18] vcpu_run PC=0x1012a8
19:11:45 DEBUG src/syscall/proc.c:1105: elfuse: HVC #5
19:11:45 DEBUG src/syscall/syscall.c:1790: syscall 66@0x25f088(0x2, 0x7ffe998, 0x2, 0xffffff, 0x0, 0x0)
WriteProtected mprotect 1 failed: Invalid argument
19:11:45 DEBUG src/syscall/syscall.c:1822:   -> 51 (0x33)
19:11:45 DEBUG src/syscall/proc.c:1048: elfuse: [19] vcpu_run PC=0x1012a8
19:11:45 DEBUG src/syscall/proc.c:1105: elfuse: HVC #5
19:11:45 DEBUG src/syscall/syscall.c:1790: syscall 198@0x247560(0x1, 0x80802, 0x0, 0x0, 0x0, 0x63646b6860651f30)
19:11:45 DEBUG src/syscall/syscall.c:1822:   -> 3 (0x3)
19:11:45 DEBUG src/syscall/proc.c:1048: elfuse: [20] vcpu_run PC=0x1012a8
19:11:45 DEBUG src/syscall/proc.c:1105: elfuse: HVC #5
19:11:45 DEBUG src/syscall/syscall.c:1790: syscall 203@0x260588(0x3, 0x7ffe8a0, 0x6e, 0x6c, 0x0, 0x63646b6860651f30)
19:11:45 DEBUG src/syscall/syscall.c:1822:   -> -2 (0xfffffffffffffffe)
19:11:45 DEBUG src/syscall/proc.c:1048: elfuse: [21] vcpu_run PC=0x1012a8
19:11:45 DEBUG src/syscall/proc.c:1105: elfuse: HVC #5
19:11:45 DEBUG src/syscall/syscall.c:1790: syscall 57@0x247560(0x3, 0x6e, 0x6c, 0x0, 0x63646b6860651f30, 0x63646b6860651f30)
19:11:45 DEBUG src/syscall/syscall.c:1822:   -> 0 (0x0)
19:11:45 DEBUG src/syscall/proc.c:1048: elfuse: [22] vcpu_run PC=0x1012a8
19:11:45 DEBUG src/syscall/proc.c:1105: elfuse: HVC #5
19:11:45 DEBUG src/syscall/syscall.c:1790: syscall 66@0x25f088(0x2, 0x7ffe8a0, 0x4, 0xffffff, 0x0, 0x0)
libc: WriteProtected mprotect 1 failed: Invalid argument
19:11:45 DEBUG src/syscall/syscall.c:1822:   -> 57 (0x39)
19:11:45 DEBUG src/syscall/proc.c:1048: elfuse: [23] vcpu_run PC=0x1012a8
19:11:45 DEBUG src/syscall/proc.c:1105: elfuse: HVC #5
19:11:45 DEBUG src/syscall/syscall.c:1790: syscall 222@0x25f948(0x0, 0x4b, 0x3, 0x22, 0xffffffff, 0x0)
19:11:45 DEBUG src/syscall/syscall.c:716:   mmap(0x0, 0x4b) → 0x201014000
19:11:45 DEBUG src/syscall/syscall.c:1822:   -> 8606793728 (0x201014000)
19:11:45 DEBUG src/syscall/proc.c:1048: elfuse: [24] vcpu_run PC=0x1012a8
19:11:45 DEBUG src/syscall/proc.c:1105: elfuse: HVC #5
19:11:45 DEBUG src/syscall/syscall.c:1790: syscall 167@0x25e888(0x53564d41, 0x0, 0x201014000, 0x4b, 0x200825, 0x0)
19:11:45 DEBUG src/syscall/syscall.c:1822:   -> 0 (0x0)
19:11:45 DEBUG src/syscall/proc.c:1048: elfuse: [25] vcpu_run PC=0x1012a8
19:11:45 DEBUG src/syscall/proc.c:1105: elfuse: HVC #5
19:11:45 DEBUG src/syscall/syscall.c:1790: syscall 135@0x260388(0x1, 0x7ffee58, 0x0, 0x8, 0xa, 0xfffffffe06fea9a0)
19:11:45 DEBUG src/syscall/syscall.c:1822:   -> 0 (0x0)
19:11:45 DEBUG src/syscall/proc.c:1048: elfuse: [26] vcpu_run PC=0x1012a8
19:11:45 DEBUG src/syscall/proc.c:1105: elfuse: HVC #5
19:11:45 DEBUG src/syscall/syscall.c:1790: syscall 172@0x247560(0x7ffee58, 0x0, 0x8, 0xa, 0xfffffffe06fea9a0, 0x63646b6860651f30)
19:11:45 DEBUG src/syscall/syscall.c:1822:   -> 1 (0x1)
19:11:45 DEBUG src/syscall/proc.c:1048: elfuse: [27] vcpu_run PC=0x1012a8
19:11:45 DEBUG src/syscall/proc.c:1105: elfuse: HVC #5
19:11:45 DEBUG src/syscall/syscall.c:1790: syscall 178@0x247560(0x0, 0x8, 0xa, 0xfffffffe06fea9a0, 0x63646b6860651f30, 0x63646b6860651f30)
19:11:45 DEBUG src/syscall/syscall.c:1822:   -> 1 (0x1)
19:11:45 DEBUG src/syscall/proc.c:1048: elfuse: [28] vcpu_run PC=0x1012a8
19:11:45 DEBUG src/syscall/proc.c:1105: elfuse: HVC #5
19:11:45 DEBUG src/syscall/syscall.c:1790: syscall 174@0x25e008(0x1, 0x8, 0xa, 0xfffffffe06fea9a0, 0x63646b6860651f30, 0x63646b6860651f30)
19:11:45 DEBUG src/syscall/syscall.c:1822:   -> 1000 (0x3e8)
19:11:45 DEBUG src/syscall/proc.c:1048: elfuse: [29] vcpu_run PC=0x1012a8
19:11:45 DEBUG src/syscall/proc.c:1105: elfuse: HVC #5
19:11:45 DEBUG src/syscall/syscall.c:1790: syscall 240@0x21b4f4(0x1, 0x1, 0x6, 0x7ffeea0, 0x63646b6860651f30, 0x63646b6860651f30)
19:11:45 DEBUG src/syscall/syscall.c:858: rt_tgsigqueueinfo(tgid=1, tid=1, sig=6, si_code=-1)
19:11:45 DEBUG src/syscall/syscall.c:1822:   -> 0 (0x0)

[jserv]

Nice. I saw sched_getscheduler works now and don't see error about syscall 120 any more. But still see the error of mprotect libc: WriteProtected mprotect 1 failed: Invalid argument

Please determine whether #23 helps resolve the mprotect issues.

[doanbaotrung]

Thank you so much. It fixed. I can run with no error now.