Skip to content
Lucet, the Sandboxing WebAssembly Compiler.
Branch: master
Clone or download
awortman-fastly Merge pull request #132 from fastly/aw/module-data-trap-tables-ii
Move trap information to lucet-module-data
Latest commit 1b5e538 Apr 26, 2019
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
.cargo Replace -W,-export-dynamic with -rdynamic Apr 11, 2019
assemblyscript The WASI bindings are now in the AssemblyScript standard library Apr 1, 2019
assets Don't consider assets/*.png files as text Apr 11, 2019
benchmarks For some reason, nix forgot to define unistd::sync() on macOS Apr 18, 2019
cranelift @ 894cecc bump cranelift, faerie, and expand readme Mar 15, 2019
faerie @ ebe9edf bump cranelift, faerie, and expand readme Mar 15, 2019
helpers Bail out if helpers/install.sh is run on a non-Linux system Apr 20, 2019
lucet-analyze Merge pch/return_to_clifwasm into master, closing #44 Apr 24, 2019
lucet-builtins Merge pch/return_to_clifwasm into master, closing #44 Apr 24, 2019
lucet-idl Just a massive update to lucet-idl Apr 22, 2019
lucet-module-data correct off by one in interpretation of function size Apr 25, 2019
lucet-runtime Merge pull request #132 from fastly/aw/module-data-trap-tables-ii Apr 25, 2019
lucet-spectest write a function manifest into lucetc artifacts Apr 25, 2019
lucet-wasi-fuzz Merge branch 'master' into acf/more-fuzz Apr 22, 2019
lucet-wasi-sdk Add sysroot to the linker flags Apr 20, 2019
lucet-wasi Short reads from readv(2) don't mean EOF Apr 23, 2019
lucetc write a function manifest into lucetc artifacts Apr 25, 2019
sightglass @ 4999f90 Update sightglass again Apr 16, 2019
.clang-format add clang-format and gitignore Jan 25, 2019
.dockerignore add dockerignore: dont send target/ as part of build context Mar 27, 2019
.editorconfig add Makefile exception to .editorconfig Apr 10, 2019
.gitattributes gitattributes: wasm is also binary Apr 17, 2019
.gitignore Merge branch 'master' into acf/more-fuzz Apr 22, 2019
.gitmodules Merge pch/return_to_clifwasm into master, closing #44 Apr 24, 2019
.travis.yml Simplify Travis builds Apr 11, 2019
CODE_OF_CONDUCT.md add contributor covenant 1.4 Jan 31, 2019
Cargo.lock write a function manifest into lucetc artifacts Apr 25, 2019
Cargo.toml
Dockerfile dockerfile: upgrade to wasi-sdk 4.0 Apr 23, 2019
LICENSE add Apache-2.0 WITH LLVM-exception license throughout Jan 28, 2019
Makefile Merge pch/return_to_clifwasm into master, closing #44 Apr 24, 2019
README.md Update README.md now that we have basic support for macOS Apr 18, 2019
SECURITY.md
config.inc Support bash in devenv scripts Mar 28, 2019
devenv_build_container.sh Build a "lucet" layer on top of "lucet-dev" Apr 4, 2019
devenv_run.sh Old versions of Docker didn't support regexes in filters Apr 11, 2019
devenv_setenv.sh Support bash in devenv scripts Mar 28, 2019
devenv_start.sh Old versions of Docker didn't support regexes in filters Apr 11, 2019
devenv_stop.sh Support bash in devenv scripts Mar 28, 2019
platform.info [lucet-wasi-fuzz] creduce driver and test predicate improvements Apr 4, 2019
rust-toolchain Pin rustup toolchain to stable Apr 2, 2019

README.md

Lucet   Build Status

Lucet is a native WebAssembly compiler and runtime. It is designed to safely execute untrusted WebAssembly programs inside your application.

Check out our announcement post on the Fastly blog.

Lucet uses, and is developed in collaboration with, Mozilla's Cranelift code generator.

Lucet powers Fastly's Terrarium platform.


Status

Lucet supports running WebAssembly programs written in C (via clang), Rust, and AssemblyScript. It does not yet support the entire WebAssembly spec, but full support is coming in the near future.

Lucet's runtime currently only supports x86-64 based Linux systems, with experimental support for macOS.

Contents

lucetc

lucetc is the Lucet Compiler.

The Rust crate lucetc provides an executable lucetc. It compiles WebAssembly modules (.wasm or .wat files) into native code (.o or .so files).

lucet-runtime

lucet-runtime is the runtime for WebAssembly modules compiled through lucetc. It is a Rust crate that provides the functionality to load modules from shared object files, instantiate them, and call exported WebAssembly functions. lucet-runtime manages the resources used by each WebAssembly instance (linear memory & globals), and the exception mechanisms that detect and recover from illegal operations.

The bulk of the library is defined in the child crate lucet-runtime-internals. The public API is exposed in lucet-runtime. Test suites are defined in the child crate lucet-runtime-tests. Many of these tests invoke lucetc and the wasi-sdk tools.

lucet-runtime is usable as a Rust crate or as a C library. The C language interface is found at lucet-runtime/include/lucet.h.

lucet-wasi

lucet-wasi is a crate providing runtime support for the WebAssembly System Interface (WASI). It can be used as a library to support WASI in another application, or as an executable, lucet-wasi, to execute WASI programs compiled through lucetc.

See "Your first Lucet application" for an example that builds a C program and executes it with lucet-wasi.

For details on WASI's implementation, see lucet-wasi/README.md.

lucet-wasi-sdk

wasi-sdk is a Cranelift project that packages a build of the Clang toolchain, the WASI reference sysroot, and a libc based on WASI syscalls. lucet-wasi-sdk is a Rust crate that provides wrappers around these tools for building C programs into Lucet modules. We use this crate to build test cases in lucet-runtime-tests and lucet-wasi.

lucet-module-data

lucet-module-data is a crate with data structure definitions and serialization functions that we emit into shared objects with lucetc, and read with lucet-runtime.

lucet-analyze

lucet-analyze is a Rust executable for inspecting the contents of a shared object generated by lucetc.

lucet-idl

lucet-idl is a Rust executable that implements code generation via an Interface Description Language (IDL). The generated code provides zero-copy accessor and constructor functions for datatypes that have the same representation in both the WebAssembly guest program and the host program.

Functionality is incomplete at the time of writing, and not yet integrated with other parts of the project. Rust code generator, definition of import and export function interfaces, and opaque type definitions are planned for the near future.

lucet-spectest

lucet-spectest is a Rust crate that uses lucetc and lucet-runtime, as well as the (external) wabt crate, to run the official WebAssembly spec test suite, which is provided as a submodule in this directory. Lucet is not yet fully spec compliant, and the implementation of lucet-spectest has not been maintained very well during recent codebase evolutions. We expect to fix this up and reach spec compliance in the near future.

lucet-builtins

lucet-builtins is a C library that provides optimized native versions of libc primitives. lucetc can substitute the implementations defined in this library for the WebAssembly implementations.

lucet-builtins/wasmonkey is the Rust crate that lucetc uses to transform function definitions in a WebAssembly module into uses of an import function.

Vendor libraries

Lucet is tightly coupled to several upstream dependencies, and Lucet development often requires making changes to these dependencies which are submitted upstream once fully baked. To reduce friction in this development cycle, we use git submodules to vendor these modules into the Lucet source tree.

Cranelift

We keep the primary Cranelift project repository as a submodule at /cranelift.

Cranelift provides the native code generator used by lucetc, and a ton of supporting infrastructure.

Cranelift was previously known as Cretonne. Project developers hang out in the #cranelift channel on irc.mozilla.org:6697.

Faerie

faerie is a Rust crate for producing ELF files. Faerie is used by Cranelift (through the module system's cranelift-faerie backend) and also directly by lucetc, for places where the cranelift-module API can't do everything we need.

Tests

Most of the crates in this repository have some form of unit tests. In addition, lucet-runtime/lucet-runtime-tests defines a number of integration tests for the runtime, and lucet-wasi has a number of integration tests using WASI C programs.

Benchmarks

We created the sightglass benchmarking tool to measure the runtime of C code compiled through a standard native toolchain against the Lucet toolchain. It is provided as a submodule at /sightglass.

Sightglass ships with a set of microbenchmarks called shootout. The scripts to build the shootout tests with native and various versions of the Lucet toolchain are in /benchmarks/shootout.

Furthermore, there is a suite of benchmarks of various Lucet runtime functions, such as instance creation and teardown, in /benchmarks/lucet-benchmarks.

Development Environment

Operating System

Lucet is developed and tested on Linux and macOS. We expect it to work on any POSIX system which supports shared libraries.

Dependencies

Lucet requires:

  • Stable Rust, and rustfmt. We typically track the latest stable release.
  • wasi-sdk, providing a Clang toolchain with wasm-ld, the WASI reference sysroot, and a libc based on WASI syscalls.
  • GNU Make, CMake, & various standard Unix utilities for the build system.

Getting started

The easiest way to get started with the Lucet toolchain is by using the provided Docker-based development environment.

This repository includes a Dockerfile to build a complete environment for compiling and running WebAssembly code with Lucet, but you shouldn't have to use Docker commands directly. A set of shell scripts with the devenv_ prefix are used to manage the container.

Setting up the environment

  1. The Lucet repository uses git submodules. Make sure they are checked out by running git submodule init && git submodule update.

  2. Install and run the docker service. We do not support podman at this time. On MacOS, Docker for Mac is an option.

  3. Once Docker is running, in a terminal, and at the root of the cloned repository, run: source devenv_setenv.sh. (This command requires the current shell to be zsh, ksh or bash). After a couple minutes, the Docker image is built and a new container is run.

  4. Check that new commands are now available:

lucetc --help

You're now all set!

Your first Lucet application

The devenv_setenv.sh shell script ensures the Lucet executables are available in your shell. Under the hood, these commands are executed in the Docker container. The container has limited visibility into the host's filesystem - it can only see files under the lucet repository.

Create a new work directory in the lucet directory:

mkdir -p src/hello

cd src/hello

Save the following C source code as hello.c:

#include <stdio.h>

int main(void)
{
    puts("Hello world");
    return 0;
}

Time to compile to WebAssembly! The development environment includes a version of the Clang toolchain that is built to generate WebAssembly by default. The related commands are accessible from your current shell, and are prefixed by wasm32-unknown-wasi-.

For example, to create a WebAssembly module hello.wasm from hello.c:

wasm32-unknown-wasi-clang -Ofast -o hello.wasm hello.c

The next step is to use Lucet to build native x86_64 code from that WebAssembly file:

lucetc-wasi -o hello.so hello.wasm

lucetc is the WebAssembly to native code compiler. The lucetc-wasi command runs the same compiler, but automatically configures it to target WASI.

hello.so is created and ready to be run:

lucet-wasi hello.so

Additional shell commands

  • ./devenv_build_container.sh rebuilds the container image. This is never required unless you edit the Dockerfile.
  • ./devenv_run.sh [<command>] [<arg>...] runs a command in the container. If a command is not provided, an interactive shell is spawned. In this container, Lucet tools are installed in /opt/lucet by default. The command source /opt/lucet/bin/devenv_setenv.sh can be used to initialize the environment.
  • ./devenv_start.sh and ./devenv_stop.sh start and stop the container.

Security

The lucet project aims to provide support for secure execution of untrusted code. Security is achieved through a combination of lucet-supplied security controls and user-supplied security controls. See SECURITY.md for more information on the lucet security model.

Reporting Security Issues

The Lucet project team welcomes security reports and is committed to providing prompt attention to security issues. Security issues should be reported privately via Fastly’s security issue reporting process. Remediation of security vulnerabilities is prioritized. The project teams endeavors to coordinate remediation with third-party stakeholders, and is committed to transparency in the disclosure process.

You can’t perform that action at this time.