Skip to content

Latest commit

 

History

143 Commits

Folders and files

NameName
Last commit message
Last commit date
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

██████╗ ██╗      █████╗ ███╗   ██╗ ██████╗
██╔══██╗██║     ██╔══██╗████╗  ██║██╔════╝
██████╔╝██║     ███████║██╔██╗ ██║██║  ███╗
██╔══██╗██║     ██╔══██║██║╚██╗██║██║   ██║
██║  ██║███████╗██║  ██║██║ ╚████║╚██████╔╝
╚═╝  ╚═╝╚══════╝╚═╝  ╚═╝╚═╝  ╚═══╝ ╚═════╝

CI Rust Docs license status

[R, COMPILED TO BYTECODE — AOT-NATIVE, NOT TREE-WALKED]

"GNU R walks the tree. rlang compiles it."

R in Rust — a compiled R runtime, hosted on the fusevm bytecode VM with a Cranelift ahead-of-time compiler (--aot) — the same engine behind zshrs, stryke, awkrs, elisp, and rubylang.


Table of Contents


[0x00] OVERVIEW

GNU R evaluates R by walking a parse tree in C. rlang lexes and parses R to an AST, lowers it to fusevm bytecode, and runs it on a compiled VM. rlang carries no VM of its own. Highlights:

  • Compiled, not tree-walkedfor, while, repeat, if, && and || lower to native fusevm jumps and integer loop counters, scalar + - * to native arithmetic ops, numeric literals ride unboxed, and a whole-program top level's locals bind to native frame slots (GetVar/SetVar by index, not a name hash) when the unit is slot-safe. On the interpreter a scalar loop runs within ~1.2× of GNU R — faster on some (a for (v in x) s <- s + v reduction beats it ~1.5×).
  • AOT native loops crush GNU R — because the hot path of a scalar loop is now entirely builtin-free (unboxed literals, native arithmetic and slots, and range bounds computed with native ops — folded at compile time for 1:LIT, inferred native for 1:n when n is a proven scalar), --aot lets fusevm's Cranelift backend lower the whole loop to register arithmetic — fadd/iadd on typed slots, no operand stack, no dispatch. A 10-million-iteration accumulator, standalone .fvm vs GNU R 4.6.1: for(i in 1:1e7) 12.5 ms vs 188.7 ms — 15× faster; for(i in 1:n) (runtime bound) 26.4 ms — 7× faster (100× the interpreter), bit-identical results.
  • No tracing JIT — fusevm's tracer cannot compile R (it rejects any trace containing a builtin call, and R lowers element fetch, comparisons, %%, and indexing to builtins), so rlang leaves it off: its per-loop trace-cache probe was pure overhead (~12% on a scalar loop). The AOT path above is where native execution comes from instead.
  • No per-operation setup — the costs that do not belong in a hot path have been taken out of it. A closure call reuses a VM parked under its closure id (VM::reset keeps the builtin table and the numeric hook), so entering a function no longer constructs a VM, re-registers every builtin and copies the body chunk. x[i] reads the source vector through a borrow instead of a copy, which takes one element read from O(length(x)) to O(1) and a read loop from quadratic to linear. Type tests ask the value's kind rather than cloning its data to look at the tag. Compiled regexes are memoised by pattern text, so grepl/sub/gsub/strsplit in a loop build the engine once instead of per call — its literal prefilter costs far more than matching a short subject.
  • fusevm-hosted — no local vm.rs / jit.rs; the shared engine behind zshrs, stryke, awkrs, elisp, and rubylang. jit-disk-cache persists native code across runs.
  • Everything is a vector — there are no scalars: 1 is a double vector of length one, every value carries attributes (names, dim, class), and every operator recycles and propagates NA.
  • Copy-on-modifyy <- x; y[1] <- 9 leaves x alone, and complex targets (l$v[2] <- 9, names(x) <- v) compile the way R defines them: rebuild the container, then re-bind it.
  • Three-valued logicNA & FALSE is FALSE and NA | TRUE is TRUE, because the answer is decided regardless of the missing value.
  • S3 dispatchUseMethod walks the class vector then .default, with implicit classes for the builtin types.
  • AOP intercepts — a glob-matched before/after/around call-intercept registry, the same design as zshrs's function intercepts.
  • Native executablesRscript --aot FILE lowers the script to a fusevm object and links it against the rlang runtime into a standalone .fvm binary (user closures embedded, no interpreter startup). The executable has the same CRAN reach as the interpreter: base R runs natively, per-call package routines delegate to embedded R through the bridge, and a whole-script fallback re-runs non-standard-evaluation programs (dplyr, data.table) in embedded R — the original source is embedded for that path. Runtime errors surface on stderr with a non-zero exit rather than being swallowed.
  • Inline-Rust FFI.rust("…Rust source…") compiles a self-contained Rust block to a cached cdylib on first use; .Call(name, …) — R's own native-call verb — invokes its exports, marshalling length-1 vectors to i64/f64/string and back.
  • CRAN bridge — rlang runs its own compiled path for base R and, for library(pkg) and any CRAN routine (including compiled C/C++/Fortran), it dlopens the system libR at run time and delegates to an embedded GNU R — no re-implementation of R's package system or C API. Loaded lazily and only if R is installed, so the base runtime is unaffected when it isn't.
  • Runs on wasm — the same crate builds for wasm32-unknown-unknown (pure interpreter, no Cranelift) and exports rlang_eval for a web-worker host.
  • Editor-ready — an LSP server and a DAP adapter over stdio, introspection dumps (--dump-tokens, --dump-ast, --disasm), and a REPL on a persistent host where a function defined at one prompt completes at the next.
  • Differential parity — a hand-authored snippet corpus plus a grammar-driven fuzzer, both diffed live against the reference Rscript; the corpus is frozen and replayed in CI with no R installed.

[0x01] INSTALL

# Via Homebrew tap (bumped by each release; formula is `rlang`)
brew tap MenkeTechnologies/menketech
brew install rlang

# Or from source
git clone https://github.com/MenkeTechnologies/rlang
cd rlang
cargo build

# run a file, a one-liner, or the REPL
./target/debug/Rscript script.R
./target/debug/Rscript -e 'print(sum(1:100))'
./target/debug/Rscript --repl

rlang is a standalone Rust crate (an explicit empty [workspace] keeps it independent of the meta repo). On native targets fusevm is pulled from crates.io with the jit, jit-disk-cache, aot, and ffi features; the wasm build uses the bare interpreter. Run the tests with cargo test.

# AOT-compile to a standalone native executable
./target/debug/Rscript --aot script.R && ./script.fvm

# build the wasm engine (web-worker host; exports rlang_eval / rlang_alloc / rlang_free)
cargo rustc --lib --crate-type cdylib --target wasm32-unknown-unknown

Zsh tab completion

cp completions/_Rscript /usr/local/share/zsh/site-functions/_Rscript
# or: fpath=(/path/to/rlang/completions $fpath) in .zshrc
autoload -Uz compinit && compinit

[0x02] USAGE

fib <- function(n) if (n < 2) n else fib(n - 1) + fib(n - 2)
print(sapply(0:10, fib))
# [1]  0  1  1  2  3  5  8 13 21 34 55

x <- c(a = 1, b = 2, c = 3)
print(x[x > 1])
#  b  c
#  2  3

counter <- function() {
  n <- 0
  function() {
    n <<- n + 1
    n
  }
}
tick <- counter()
tick(); tick()
print(tick())        # [1] 3

m <- matrix(1:6, nrow = 2)
print(m[, 2])        # [1] 3 4

c(3, 1, 2) |> sort() |> rev()   # [1] 3 2 1

Base R runs on rlang's own compiled path; CRAN packages are delegated to an embedded GNU R (needs R installed):

library(jsonlite)
cat(toJSON(1:3), "\n")          # [1,2,3]

library(stringi)
print(stri_reverse("hello"))    # [1] "olleh"

[0x03] LANGUAGE FEATURES

Implemented and checked against the reference Rscript:

  • Vectors & types — logical, integer, double, character, and list vectors with NA in every atomic type, recycling, type promotion in c(), and the L integer-literal suffix.
  • Attributesnames, dim, class, and arbitrary attr(), preserved through arithmetic and subsetting.
  • All four index forms — positive, negative (exclusion), logical (recycled), and character (by name), plus [[, $, and N-D array indexing a[i, j, k].
  • Assignment<-, =, ->, <<-, growing assignment past the end, index/$/[[ targets, nested targets, and replacement functions (names(x) <-, dim(x) <-, class(x) <-, user-defined `f<-`).
  • Functions — defaults that may refer to other arguments, ... forwarding, R's exact/partial/positional argument matching, lexical closures, return(), and function-position lookup that skips non-function bindings.
  • Control flowif/else as an expression, for, while, repeat, break, next, short-circuiting && / ||, and lazy switch (only the selected branch is evaluated, with fall-through and numeric selection).
  • Operators — the full precedence ladder from ?Syntax, %%/%/% with the sign of the divisor, %in%, user-defined %op%, and the native pipe |>.
  • S3class(), inherits(), structure(), UseMethod dispatch with implicit classes and .default fallback.
  • Primitive library — the apply family (lapply/sapply/Map/Filter/ Reduce/do.call), string and regex functions (paste, sprintf, substr, strsplit, grepl, sub, gsub), numeric summaries (sum, mean, median, var, sd, cumsum, diff), sequence and set functions, and matrix helpers.
  • R's printing[n] index prefixes with 80-column wrapping, shared decimal widths, quoted and left-justified character vectors, named-vector column pairs, [i,]/[,j] matrix layout, and $name / [[n]] list sections.

[0x04] COMMAND-LINE FLAGS

Flag Effect
FILE Run a .R script.
-e SRC Run a one-liner.
--repl Interactive REPL on a persistent host.
--lsp Language Server Protocol over stdio.
--dap Debug Adapter Protocol over stdio (handshake + run to completion).
--build FILE AOT-compile the script's bytecode into the on-disk cache.
--aot FILE AOT-compile the script to a standalone native .fvm executable (override the path with -o OUT).
-o OUT Output path for --aot (default: the script's name with a .fvm extension).
--dump-tokens FILE Print the lexer token stream.
--dump-ast FILE Print the parsed AST.
--disasm FILE Disassemble the lowered fusevm chunk.
--tiers FILE Run it, then report which fusevm execution tier took each of its chunks.

[0x05] ARCHITECTURE

rlang contains no virtual machine or JIT of its own. The execution path mirrors how zshrs hosts zsh and rubylang hosts Ruby:

R source → lexer → parser (AST) → lower to fusevm bytecode → fusevm VM + Cranelift JIT
                                          │
                              RHost heap (vectors, attributes, environments, closures)
Piece How
fusevm-hosted No local vm.rs / jit.rs. R lowers to fusevm bytecode and runs on the shared three-tier Cranelift JIT; jit-disk-cache persists native code across runs.
Native control flow Loops and branches lower to native fusevm jumps over native integer counters, so hot loops trace-compile.
Vectors on the host heap Every R value is a Value::Obj handle into the RHost heap, because R has no scalars and any value can carry attributes.
Environments by reference Frames are Rc<RefCell<..>> environments chained to their enclosure — R's lexical scoping, and what lets <<- reach the defining frame.
R truthiness A condition must be a single non-NA logical, so conditions normalize through a TRUTHY op before a native branch.
Complex assignment f(x) <- v compiles to x <- `f<-`(x, v) and x[i] <- v rebuilds and re-binds x, so nested targets unwind through the same two rules.

[0x06] PARITY HARNESS

Behaviour is checked against the reference Rscript by a differential parity harnesscargo run --bin parity diffs the snippet corpus (tests/data/parity_corpus.R) live against the system R, and tests/parity.rs replays the frozen outputs in CI with no R installed. Nothing is faked as working: an unimplemented primitive raises could not find function.

The examples/ directory holds runnable programs that double as tests: the scripts embed stopifnot assertions that abort on any divergence from R, and tests/examples.rs runs every example through the binary in CI, asserting a clean exit and stdout matching the frozen reference output (cargo run --bin parity -- --freeze-examples regenerates it).

Where the fixed corpus is hand-authored, the differential fuzzercargo run --bin parity-fuzz — generates thousands of grammar-driven R snippets across 63 surfaces (vectors, seq/rep, apply family, sprintf/formatC, matrices and linear algebra, factor/table, factor subsetting and factor operators, set/bit ops, trig, gamma/choose, pmax/pmin, string translation, closure deparse, rbind/cbind seam labels, dimnames replacement, sort/order with missing values, NA-versus-NaN propagation, S3 methods on the generic primitives, cat argument handling, non-ASCII string measurement and layout, character collation, …) and runs each through the reference Rscript --vanilla -e and rlang's own Rscript -e, reporting every case where stdout or exit code diverges. Both binaries share the name Rscript, so each is resolved by absolute path — the reference from a system path, rlang's from this harness's own directory — and can never be confused. Generators emit only deterministic-output programs (no Sys.time, RNG, or environment prints), so any divergence is a genuine gap. A finding is delta-debugged to its minimal reproducer and replays exactly with --seed <N> --once.

cargo build --bin parity-fuzz
./target/debug/parity-fuzz --count 5000                       # sweep all modes
./target/debug/parity-fuzz --sprintf --count 2000             # one surface
./target/debug/parity-fuzz --seed 52 --once                   # replay one case
./target/debug/parity-fuzz --count 5000 \
    --baseline tests/data/parity_fuzz_baseline.txt            # gate on NEW gaps only

The fuzzer currently reports zero divergences across its 63 surfaces over repeated multi-seed sweeps, so tests/data/parity_fuzz_baseline.txt is empty; with --baseline the run exits non-zero the moment any new divergence class appears — a regression, or a surface that just started diverging. Like parity, it needs R on PATH (or RLANG_FUZZ_RSCRIPT), so it is a development tool, not a CI gate.

A zero is only worth reading if the run looked at something, so every case is accounted as compared, drained (the oracle timed out or would not spawn, so the case was never judged) or no-signal (neither side printed anything, which agrees trivially). All three are printed, and a run whose compared count is zero exits 2 rather than reporting a clean sheet — --count 0, an oracle that never answers, and a timeout short enough to kill every reference run all fail now instead of passing.


[0x07] STATUS & ROADMAP

The standalone Rscript binary, the REPL, the rkyv bytecode cache, the --aot native-executable emitter, the inline-Rust FFI bridge (.rust / .Call), the wasm32 build, the AOP call-intercept registry, the LSP server, and the DAP adapter (handshake plus run-to-completion; stepping is a later wave) are all in the tree. The parity corpus and every example match the reference R byte-for-byte.

Arguments are evaluated eagerly rather than as promises, so substitute() / quote() / non-standard evaluation are not available; data frames, complex numbers, and part of the linear-algebra surface (solve, det, eigen) are not implemented yet. Factors (ordered included, surviving subsetting and reordering, comparing by label), table, %*%, outer, crossprod, cbind/rbind with R's deparsed seam labels, and apply over matrix margins (carrying the margin's dimnames) all work; the condition system is complete through restarts (withCallingHandlers resumes, invokeRestart("muffleWarning"), withRestarts/computeRestarts, suppressWarnings/suppressMessages); tryCatch / try / on.exit / local / NextMethod work, with conditions carrying no call; and a closure prints and deparses its own source. See BUGS.md for the full known-gaps list.


[0x08] DOCUMENTATION

  • Read the Docs — the HUD documentation site.
  • Engineering Report — architecture, value model, roadmap, dependency posture.
  • Primitive Reference — every primitive and operator with its signature and what it does, generated from the same corpus the language server completes from.
  • BUGS.md — the honest known-gaps list.

[0xFF] LICENSE

MIT — free and open source. See LICENSE.

About

No description, website, or topics provided.

Resources

Stars

0 stars

Watchers

0 watching

Forks

Releases

Packages

Used by

Contributors

Languages