aWsm - An Awesome Wasm Compiler and Runtime
What is aWsm?
aWsm is a compiler and runtime for compiling Web Assembly (Wasm) code into llvm bytecode, then into sandboxed binaries you can run on various platforms. It focuses on generating very fast code (best of breed for Web Assembly), having a simple and extensible code-base, and on portability.
What is Web Assembly? Wasm is a standard binary format that is platform agnostic, debuggable, fast, and safe. Please see the official webpage, and its specification. Many languages (including C/C++, Rust, C#, Go, Kotlin, Swift, and more, see a more complete list) compile to Wasm. Wasm can be run in all the major browsers, thus is broadly deployed and available.
We're interested in taking Wasm out of the browser, and into all different types of computers (from servers to microcontrollers). Wasm provides a number of benefits outside of the web including:
Sandboxing. Regardless which language is compiled to Wasm, the Wasm runtime can execute them safely (including those languages with footguns like C). This uses Software Fault Isolation (SFI) -- to ensure that loads and stores are only within the sandbox-- and Control-Flow Integrity (CFI) -- only allowing the execution of code intentionally generated by the compiler -- to provide safe execution. Restricting SFI protects the surrounding code from faulty or malicious code, and CFI prevents a compromise from hijacking the flow of execution (see buffer overflows and ROP chains).
Sandboxing can act as a process replacement, especially on systems that don't have hardware support for processes, or where high-density is required.
Ubiquity. Wasm is a high-level assembly, independent from any specific hardware. This has the potential to provide a universal specification of computation that can execute in the cloud, on the edge, or in an embedded device.
A note on naming.
aWsm started out as the
silverfish compiler, the brainchild of Gregor Peach when he was a researcher in the group.
There are still quite a few lingering
Please have patience as we update those to
Why would we implement a Wasm compiler and runtime? The Web Assembly eco-system is still developing, and we see the need for a system focusing on:
- Performance. aWsm is an ahead-of-time compiler that leverages the LLVM compiler to optimize code, and target different architectural backends. We have evaluated aWsm on x86-64, aarch64 (Raspberry Pi), and thumb (ARM Cortex-M4 and M7), and performance on the microprocessors is within 10% of native, and within 40% on the microcontrollers on Polybench benchmarks.
- Simplicity. The entire code base for the compiler and runtime is relatively small. The compiler is <3.5K lines of Rust, and the runtime (for all platforms) is <5K lines of C. It is nearly trivial to implement different means of sandboxing memory accesses. We've implemented seven different mechanisms for this!
- Portability. Both the compiler and runtime are mostly platform-independent code, and porting to a new platform only really requires additional work if you need to tweak stack sizes (microcontrollers), or use architectural features (e.g., MPX, segmentation, etc...). aWsm only links what is needed, so it's possible to avoid microcontroller-expensive operations such as f64, f32, and even dynamic memory.
The final output of aWsm is simple
*.oelf objects that can be linked into larger systems. This enables the trivial composition of sandboxes together, and sandboxes into larger programs.
We believe that aWsm is one of the best options for ahead-of-time compilation for Wasm execution outside of the browser.
PolyBench/C benchmarks for x86-64 (slowdown over native C):
|Wasmer||WAVM||Node.js + Emscripten||Lucet||aWsm|
|Stdev. in Slowdown||194.09||53.09||107.84||117.25||34.65|
PolyBench/C benchmarks for Arm aarch64 (slowdown over native C):
|Stdev. of Slowdown||19.38|
Polybench/C benchmarks for Arm Cortex-M microcontrollers (slowdown over native C):
|Cortex-M7 Avg. slowdown||40.2%|
|Cortex-M4 Avg. slowdown||24.9%|
Note: these numbers are from May 2020.
There are many compelling runtimes, but we believe that aWsm is useful in generating very fast code, while being simple and extensible.
Comparison to Existing Wasm Ecosystems
There are many existing compilers and runtimes. aWsm fills the niche of a compiler
- based on ahead-of-time compilation using the popular LLVM infrastructure,
- that generates fast, safe code (even on microcontrollers), and
- that is designed for to be lightweight and easily extended.
Adding runtime functions and changing safety checking mechanisms are trivial operations.
|aWsm||AoT||✓||✓||✓||✓||You are here|
This is not an exhaustive list! There are many others as this is a pretty active area.
Note: this table is from the best of our understanding of each system in July 2020.
aWsm 0.1.0 Gregor Peach <firstname.lastname@example.org> USAGE: silverfish [FLAGS] [OPTIONS] <input> FLAGS: -h, --help Prints help information -i, --inline-constant-globals Force inlining of constant globals -u, --fast-unsafe-implementations Allow unsafe instruction implementations that may be faster --runtime-globals Don't generate native globals, let the runtime handle it -V, --version Prints version information OPTIONS: -o, --output <output> Output bc file --target <target> Set compilation target ARGS: <input> Input wasm file
Installation from Source
git clone https://github.com/gwsystems/awsm.git cd awsm ./install_deb.sh
The compiler can now be run via
- Install Rust
- Install LLVM
- Link Your Clang Installation so
llvm-config-9are in your path and invokable as
llvm-config. For example, the following commands accomplish this using
update-alternativesafter replacing LLVM_VERSION with the version returned you installed above (In *NIX systems, this can be confirmed with
LLVM_VERSION=9 sudo update-alternatives --install /usr/bin/clang clang /usr/bin/clang-$LLVM_VERSION 100 sudo update-alternatives --install /usr/bin/llvm-config llvm-config /usr/bin/llvm-config-$LLVM_VERSION 100
- Install the C++ Standard Library for LLVM
- Clone and build awsm
git clone https://github.com/gwsystems/awsm.git cd awsm cargo build --release
- The awsm binary is built at
target/release/silverfish. Copy this to the appropriaate place for your platform and add to your PATH if neccessary.
Executing and Testing aWsm
The tests can run with
cd code_benches; python run.py
Please see the design to understand the pipeline, and see
run.py for an example of how to connect existing compilers (to generate Wasm, and generate machine code from LLVM IR) with aWsm.
Note that aWsm is still a research prototype, so might have some rough edges. It is not currently turn-key. You likely need to understand how to generate Wasm, and use an LLVM compile chain. We're happy to accept PRs with fixes, and "quality of life" improvements.
We're in the processes of standing up a CI infrastructure.
Tour of the Source
The source is organized as such:
src/- the Rust source of the compiler, and the
silverfishbinary. Look here for the logic for taking in Wasm, and generating LLVM bytecode.
cargoguides the compilation of this code.
code_benches/- This is a relatively large set of benchmarks, many of which are derived from Polybench (
pb_*), MiBench (
mb_*), or various applications (
custom_*including NN inference, sqlite, a PID controller, and an extended Kalman filter). The
run.pyfile guides the compilation and execution of these as effectively a set of unit tests, and is an example of the compilation structure of an application with the runtime in aWsm.
example_code/- More atomic tests for specific Wasm features (bounds checks, indirect function invocations, etc...). This ensures that changes to the compiler don't break Wasm sandboxing, and provides regression tests.
runtime/- The aWsm runtime. This includes most of the code that provides the sandboxing guarantees (for example, including bounds checks, indirect function call type checks and indirection). Microcontroller-specific runtime code (for Arm Cortex-M processors) referred to in
eWasm, is in
libc/cortex_m_backing.c. Various pluggable bounds checks can be found in
runtime/memory/. The runtime is compiled separately, and combined with the LLVM IR output by aWsm (using LTO) to generate the final sandboxed object.
doc/- Documentation directory. See the
Limitations and Assumptions
Additional Wasm instruction support needed. aWsm has not focused on a complete implementation of every Wasm instruction, instead focusing on supporting the code generated by the LLVM front-end. The code is extensible, and we've added instructions as needs-be. Please make a PR with support for additional instructions if you need them!
Limited support for future Wasm features. The features on the Wasm roadmap are not supported. We're quite interested in supporting as many of these feature sets as possible, and would happily accept PRs.
Advice for Wasm Standardization
We've done quite a bit of work targeting microcontrollers (Arm Cortex-Ms), and have discovered a number of limitations of the current Wasm Specification for that domain. We provide details in Section 7 of our EMSOFT publication. We believe that some changes to the specification, or the creation of an embedded profile might be warranted. The main limitations:
- Variant page sizes selected by the runtime. Wasm uses 64KiB pages. That is far too large for embedded systems. aWsm uses smaller pages, while simulating the larger default pages. Enabling smaller pages would make Wasm's use on microcontrollers -- often with only 64KiB of memory total -- easier. In aWsm, the page size is configurable.
- Separation of read-only and read-write memory. In Wasm, all data is accessed in linear memory, which provides strong sandboxing, but does not discriminate between RW and RO data. Microcontrollers often use Execute in Place (XIP) read-only memory, which allows RO data to be accessed and executed directly from flash. This is beneficial in microcontrollers with on the order of 64KiB of memory (SRAM), but around 1MIB of flash. Wasm does not separate RO and RW memory, preventing this optimization that is essential for density on such systems.
- Allow undefined behavior on Out of Bounds (OoB) accesses. The specification requires any access outside of the allocated bounds of linear memory to be caught, and the sandbox terminated. We show in the publication that relaxing this requirement, and allowing undefined behavior on OoB accesses can significantly speed up execution, and shrink code sizes, while maintaining strong sandboxed isolation.
About Us & Acknowledgments
The GWU Systems group focuses on low-level system design and implementation. One of our main research vehicles is the Composite component-based operating system, which we aim to integrate with Wasm through aWsm. If you're interested in low-level hacking and system building, in secure and reliable systems, in embedded systems, or in models for parallelism, don't hesitate to contact Gabe about doing a PhD or becoming involved with the group.
Our collaborations with Arm Research during a lot of the maturation of the aWsm infrastructure have been instrumental in its development. Support from SRC, ARM, and NSF have all contributed greatly to Wasm's research.