Multi-variant execution (MVX) source code. MvArmor aims to provide both high performance and security, and was published at the DSN'16 conference:
Secure and Efficient Multi-variant Execution Using Hardware-assisted Process Virtualization by Koen Koning, Herbert Bos and Cristiano Giuffrida.
The MVX engine consists of two parts: the back-end providing all MVX-related semantics such as copying and comparing system calls, and the front-end intercepting all system calls and other behavior of a program.
The back-end is implemented as a shared library and can be found in the
libmultivar directory. It contains a number of call-backs that the front-end
must invoke, for instance when a system call is executed. A front-end must also
provide some information and function-pointers to the library during
initialization, which may differ per front-end. This includes how to allocate
memory, how to allocate shared memory (visible to all variants) and how to
access the address space of the variants.
This repository contains several front-end implementations:
ptrace: slow implementation using a
ptracemonitor, for debugging. Does currently not support full variant-generation and intercepting
dune_sandbox: efficient implementation using Dune. Only front-end that supports full variant generation.
Setup and usage
MvArmor requires a 64-bit Linux, and has been tested on Debian Jessie (with 3.2 kernel) and Ubuntu 14.04. Especially for Dune, the Linux and glibc versions installed may not be supported. Linux 3.19 should work, higher versions most likely won't compile the Dune kernel module.
External dependencies include libcap, libunwind and Linux headers. On
sudo apt-get install build-essential libcap-dev linux-headers-amd64 libunwind-dev
Or on Ubuntu:
sudo apt-get install build-essential libcap-dev linux-headers-generic libunwind8-dev liblzma-dev
Furthermore, there are several dependency projects. All of these can be
automatically downloaded and compiled using
make deps. These are currently
only required when using the Dune-based front-end.
- libumem fork: sparse heap for variant generation (https://github.com/vusec/libumem-mvx)
- shalloc: simple allocator for shared memory regions (https://github.com/vusec/shalloc)
- dune fork: mvarmor-specific compatibility patches (https://github.com/vusec/dune)
(Optional: set up dependencies for Dune front-end using
make deps, or set
them up manualy and modify
make should build the MVX library and all fronte-ends.
For the Dune frontend, insert the kernel module with
make -C test test_ptrace or
make -C test test_dunesb. This
executes a number of small test cases, and should complete within a few seconds.
Note that the
rdtsc test case fails for ptrace.
Configure the number of variants (default 2):
When running testcases where the monitor should stay attached to all procs
(i.e., normally the process used to start MVX will exit once all variants are
running, with this option set it will wait until all variants have exited).
This is useful also when benchmarking SPEC for instance.
libumem (different allocator) for followers:
Optionally, configure slab behavior of
export MV_SLAB_MAX=1 # max objects per slab export MV_SLAB_PAD=1K # padding per slab
Then prepend the binary of the choosen front-end to normal execution, e.g.:
cd dune_sandbox ./multivar_dune_sandbox /bin/ls
When compiling libmultivar:
cd libmultivar make clean make OPT=val OPT2=val2 ...
Valid options are:
RINGBUFFER_SIZE: Size of ringbuffer (how much leader and followers can deviate when there are no
unsafesyscalls. (default: 10)
RINGBUFFER_SLEEP: Are processes waiting for the ringbuffer allowed to sleep (instead of spinlock) after a while? Going to sleep (using
futex) is a relatively expensive operation (so the time after which they do this should be finetuned too), but it does free up a core for other executions. (default: 1)
SYSCALL_ORDERING: Force ordering of system calls between different processes/threads across variants. (default: 1)
TIMING_ENABLE: Enable timing (benchmarking) of regions of the monitor. (default: 0)
ENABLE_ASSERT: When disabled, any assert statement in the code will be removed by the preprocessor. (default: 1)
VARN_OPEN_RO: Allow followers to open their own files when they are read-only. (default: 1)
SEC_POL: Security policy to use (i.e., which system calls are unsafe). (default: 2)
- 0 = all regular (performance, no lockstep at all)
- 1 = all unsafe (comprehensive)
- 2 = exec-related unsafe (code execution)
- 3 = write and similar unsafe (information leakage)
Additionally, the following command can be used to force recompilation of the security component:
cd libmultivar make secpol SEC_POL=n
When starting the application, the monitor will read the environment for some additional options. Some may not be available for all front-ends.
MV_NUM_PROC: Number of variant to run. E.g., a value of 2 will result in 1 leader and 1 follower. (default: 2)
MV_DONT_DETACH: When set, the process which starts all variants will stay attached. Useful when running or benchmarking non-server applications such as SPEC or the testcases. (default: 0)
MV_OUT_TO_FILE: Send all debug-output of libmultivar (enabled in
multivar.c) to variant-specific files. Normally, all output is sent to stderr, which may result in overlapping of messages. (default: 0)
MV_UMEM: Enable the alternative
libumemheap allocator for followers. (default: 0)
MV_UMEM) Maximum objects allowed in a slab. (default: 10000)
MV_UMEM) Size of the guard-zone after every slab. Can be a string such as "128K" or "4G". (default: 0)
MV_VAR_NUM: used internally When spawning the variants, this variable is temporarily set to indicate what variant each process is. It is unset before the actual application is started, to provide an identical environment to every variant.
error while loading shared libraries: libmultivar.so: cannot open shared object file: No such file or directory
$ export LD_LIBRARY_PATH=/path/to/libmultivar
Permission errors with ptrace frontend (e.g., failed to open
or ptrace errors in dmesg):
$ echo 0 | sudo tee /proc/sys/kernel/yama/ptrace_scope
Assertion `inherit_mem_id != -1' failed.
$ sudo sysctl -w kernel.shmmax=1073717248
segfault at 0 before dune-mode: Does the shalloc region overlap with Dune sandbox memory layout?
include/shalloc/shalloc.h: -# define SHALLOC_BASE_ADDR ((unsigned) 0x60000000) -# define SHALLOC_LAST_ADDR ((unsigned) 0x9FFFFFFF) +# define SHALLOC_BASE_ADDR ((unsigned) 0x80000000) +# define SHALLOC_LAST_ADDR ((unsigned) 0xBFFFFFFF)