Enigma VM

An implementation of the Erlang VM in Rust. We aim to be complete, correct and fast (in that order of importance). However my TotallySerious™ fibonacci microbenchmarks are currently on-par with OTP (but I'm missing 99% of the runtime :)

We aim to be OTP 22+ compatible (sans the distributed bits for now) — all your code should eventually run on Enigma unchanged. Deprecated opcodes won't be supported.

Why?

Because it's fun and I've been learning a lot. BEAM and HiPE are awesome, but they're massive (~300k SLOC). A small implementation makes it easier for new people to learn Erlang internals. We also get a platform to quickly iterate on ideas for inclusion into BEAM.

Why Rust?

I read the BEAM book followed by the Rust book. Two birds with one stone?

Installation

Only prerequisite to building Enigma is Rust. Use rustup (or your preferred package manager) to install latest rust (minimum version is the 2018 edition / ‎1.31, and stable is supported).

Run cargo install to install the dependencies, cargo run to build and run the VM. Expect heavy crashes, but a basic spawn + send multi-process model already works.

We will distribute binaries for various platforms, once we reach a certain level of usability.

Goals / ideas / experiments

• Be able to run the Erlang bootstrap (and all OTP)
• Be able to run Elixir
• Write more documentation about more sparsely documented BEAM aspects (binary matching, time wheel, process monitors, etc).
• Ideally one day, feature parity with OTP
• Explore using immix as a GC for Erlang
• Explore using RocksDB / Sled / embedded DB equivalent for the ETS implementation.
• Process as a generator function (yield to suspend/on reduce)
• Use Commentz-Walter for binary matching. "Commentz-Walter is an algorithm that combines Aho-Corasick with Boyer-Moore. (Only implementation I know of is in GNU grep.)")
• Cross-compile to WebAssembly (threading is coming)

Initial non-goals

Until we can run a large subset of OTP code, it doesn't make sense to consider these.

• Distributed Erlang nodes
• Tracing / debugging support
• BEAM compatible NIFs / FFI

Note: NIF/FFI compatibility with OTP is going to be quite some work. At the moment I plan to trick the inet & file implementations by fake-loading the internal NIFs, then re-implementing those via a compatible Rust / BIF interface.

Feature status

This section is a quick overview of what's supported, and what's the next general features that will be worked on.

You can view a detailed breakdown on opcode or BIF progress.

Plan:

• implement all the instructions
• implement enough BIFs to get preloaded bootstrap to load
• implement enough to get the full system to boot (init:start)
• get OTP tests to run

Features:

• Floating point math (float registers)
• Spawn & message sending
• Lambdas / anonymous functions
• Stack traces
• Exceptions
• Process Dictionary
• Monitors (rbtree)
• Signal queue
• error_handler system hooks (export stubs)
• Deep term comparison (lists, tuples, maps)
• Timers
• Maps
  • Basic type implementation
  • BIF functions
  • Map specific opcodes
• Binaries
  • Basic type implementation
  • Binary building
  • Binary matching
  • Binary searching
    • multi pattern via aho-corasick
    • single pattern via boyer-moore | needle booyer-moore | regex - booyer-moore
  • Bitstring (bit-level) matching
• File IO
• NIF
• Ports
  • inet_drv
  • ram_file_drv
• External Term representation
  • Most of decoding
  • Encoding
• ETS
• GC!
• Code reloading
• Tracing/debugging support
• beam_makeops compatible load-time opcode transformer
• Optimize select_val with a jump table
• Directly embed imports as some form of a pointer reference

Contributing

Contributors are very welcome!

The easiest way to get started is to look at the notes folder and pick a BIF or an opcode to implement. Take a look at src/bif.rs and the bif folder on how other BIFs are implemented. There's also a few issues open with the good first issue tag, which would be a good introduction to the codebase.

Test coverage is currently lacking, and there's varying levels of documentation; I will be addressing these as soon as I solidify the core data structures.

I'm also relying on a few external rust crates for various parts of implementation, I'd like to bring the dependency count down once we are able to run the OTP emulator tests.

Acknowledgements

• Yorick Peterse's Inko, from which I've stolen the process scheduling code and a lot of the VM design.
• @kvakvs for ErlangRT which I've used for an extensive reference, along with his BEAM Wisdoms website.
• The BEAM Book, which spurred this interest in the first place.
• bumpalo for the basis of bump allocation.