Skip to content

OrchestrateLang v0.15.0

Choose a tag to compare

@Immanuel-Fowler Immanuel-Fowler released this 21 Sep 18:12
· 71 commits to main since this release

The sandbox catches up with the C ABI, and every claim is measured.

Added

  • Arrays and structs across the sandbox boundary. A sandboxed serverlet carried int,
    float, bool, and string; it now carries int[], float[], bool[], and structs
    whose fields are numbers, booleans, or such structs — the set the C ABI carries since
    0.14.0, under the same rules. An array crosses as a pointer and a count into the guest's
    memory, a struct as its #[repr(C)] bytes, each field at its own offset and padding
    zero, so neither side reads padding or depends on the other's endianness. What the host
    writes for a call it frees after the call; what the guest returns the host copies and
    frees. The guest's allocator is now 8-byte aligned. A handler's state may be of these
    types, and a handler may return it.

    An array of strings or structs, or a struct holding one, does not cross the sandbox,
    though the in-process, secret, and landline boundaries carry them. That gap is written
    down in docs/limitations-notes.md.

  • grant call on a sandboxed serverlet. In a library build, a sandboxed serverlet
    takes the same grant lines a landline does, and there they are the only way the guest
    reaches the host at all: each grant is exactly one import defined in the guest's
    wasmtime linker, behind which the Host trait method runs; nothing is defined for
    anything ungranted, so it does not exist inside the guest. A handler that calls an
    ungranted host function is refused by orchestrate check, which names the call and the
    grant that would allow it. Arguments and results cross in the wire encoding landline
    grants use — numbers, booleans, strings, arrays, and structs. A host error or panic
    fails that one call inside the guest, which logs it and continues with the default. A
    host call counts against the serverlet's timeout, since the guest is still inside its
    call while the host runs. Step 7 of docs/features/sandboxed-serverlets.md, and the
    point where containment and consent become one mechanism.

  • on_crash on a sandboxed serverlet runs on the host after a call ran past its
    timeout, exhausted its memory, or stopped itself, with the trap's message bound, before
    the guest is replaced and the caller gets the default. It cannot reach the guest's
    state, which is inside the instance being thrown away; a handler that names a state
    binding is a compile error saying so.

  • The boundary ladder benchmark covers every rung. benchmarks/landline_latency
    measures the same echo handlers on the in-process, sandboxed, secret, Python, and
    TypeScript boundaries, and the two rungs below a serverlet call, shared let and a
    plain let, timed per batch of a thousand statements. Beside the payload sweep there
    is a call-rate sweep — the int payload paced at 100 Hz and 1 kHz with the caller
    asleep between calls, and at 10 kHz spinning — and a concurrent-callers sweep with 2, 4,
    and 8 callers sharing one serverlet. run.py writes CSV, JSON, and Markdown under
    benchmarks/results/ with an environment header (CPU, OS, rustc, Python, the
    TypeScript toolchain, wasmtime, sample counts, commit SHA), and the README's tables are
    pasted from that output rather than typed. The TypeScript rung is skipped, and said to
    be, when Bun or TypeScript 7 is missing.

  • The tick cost is a committed benchmark. benchmarks/tick_cost/run.py builds six
    small library programs — an empty tick, one and five host calls, an instance let
    increment, a shared let increment, and one host-fired event per tick — drives each
    through tick_sync on a current-thread runtime in release mode, and reports the median
    of 21 rounds of 200,000 ticks beside the Host trait method called through its vtable
    alone. It writes CSV, JSON, and Markdown with the same environment header as the
    boundary ladder. The 8 ns tick and 1.5 ns host-call figures from the 0.8.1 notes come
    from this measurement now, not from a release note.

  • Deterministic mode is tested at length. One scripted sequence of 10,000 ticks and
    events — host-fired events, events fired by handlers, handlers that sleep on host time
    across ticks, and instance state — replayed on fresh instances, on a current-thread and
    a multithreaded runtime, through tick_blocking and tick_sync, produces a
    byte-identical host-call trace. A second test asserts that each documented exclusion
    (a spawned worker, a serverlet, a landline, a sleep outside an event handler) fails
    the way docs/library-mode.md now says, with the deterministic-mode message.

  • The diagnostics corpus covers the language. tests/error_cases/diagnostics/ grew
    from 30 to 88 deliberately invalid programs, across types, events, serverlets of every
    kind, processes, C, Rust, and WebAssembly sidecars, and module boundaries, with the
    module fixtures they import beside them. diagnostics_never_leak_rustc now runs each
    through orchestrate check first and reports the count — 81 of 88 are rejected by
    check, 6 more by the build before Cargo, 1 reaches rustc — and holds the leaks to
    KNOWN_LEAKS.txt, so a leak that appears or disappears fails the test until the list
    says so. benchmarks/diagnostics_coverage.py writes the same classification as CSV,
    JSON, and Markdown.

  • One command regenerates every table. python3 benchmarks/run_all.py builds the
    release compiler and runs the boundary ladder, the tick cost, and the diagnostics
    coverage in order, writing every CSV, JSON, and Markdown table under
    benchmarks/results/ and joining the reports into results.md. benchmarks/README.md
    documents it and the three scripts.

  • What orchestrate check guarantees is written down, in one paragraph of
    docs/language-reference.md before "Debugging Generated Code": what a check does and
    finds, and what it does not — no codegen, no Cargo, library-only rules unenforced,
    module serverlet bodies unwalked, foreign sources unchecked.

Changed

  • The limitations notes live under docs/. LIMITATIONS-notes.md sat at the repo
    root, where CONTRIBUTING.md allows only the README, changelog, contributing guide, and
    license; it is now docs/limitations-notes.md, and every reference to it, including
    the comment in KNOWN_LEAKS.txt, follows.

  • The documentation says what the language is before it says what it does. The four
    foundational principles each have a document under docs/design/: concurrency is the
    structure, asynchronous first and synchronous second, polyglot as coordination and
    attachment, and as many choices as possible made in syntax. Each states the claim, what
    it buys, and what it costs, with examples that typecheck and numbers taken from
    benchmarks/. docs/design-philosophy.md is now the index of those four plus the
    eight principles that govern how the language is built.

    The per-feature design documents moved from docs/design/ to docs/features/, where
    README.md lists them with their status. Every cross-reference in the repository was
    updated, so no link moved without its target.

  • The README is a front door rather than a second reference. Installing comes first,
    then the feature set, then the benchmarks, then the four ideas in thirty seconds and
    the principles they rest on. The features, benchmarks, and philosophy sections are
    self-contained: the benchmark tables are the generated output spliced in, not a pointer
    at benchmarks/, and nothing in those three sections sends a reader to another page to
    learn what the section is about. The 430-line syntax walkthrough it used to contain is
    in docs/language-reference.md, where it was already documented.

Fixed

  • A function that can fall off its end without returning is refused by check.
    fn f(n: int) -> int { if n > 0 { return 1 } } reached rustc as E0317 ("if may be
    missing an else clause"). orchestrate check now requires a body that declares a
    return type to end in a value or to return on every path — through both branches of an
    if/else, every arm of a match, both sides of a try/catch — and says which
    function, task, or handler can reach its end without one. fn_missing_return leaves
    KNOWN_LEAKS.txt; of the 88 corpus programs, 81 are rejected by check and one still
    reaches rustc. Patch-level: an invalid program is refused earlier, and no valid program
    changes — every example, benchmark program, and test program still checks.
  • A name that is a Rust keyword no longer breaks the generated Rust. A handler
    named move, a fn type(), a let mut, a struct field ref, a module called impl,
    a C sidecar function move — any of the fifty-odd Rust keywords that are not
    OrchestrateLang keywords — reached rustc as a syntax error in generated code; only
    gen was escaped, and only in library builds. Every such name is now emitted as a raw
    identifier (r#move) from one keyword list covering editions 2021 and 2024, across
    functions, tasks, handlers, parameters, bindings, struct fields, enum variants, match
    bindings, program and shared state, module names, and foreign wrappers. A name that
    crosses a boundary keeps its spelling there: a sandbox export carries
    #[export_name = "move"], a landline handler is move on the wire and in Python or
    TypeScript, a C symbol is move for the linker. self, Self, super, and crate
    cannot be raw identifiers at all, so orchestrate check refuses them as names with an
    error that says so; four corpus cases hold that, taking the corpus to 88 programs. Regression tests use keyword names on every boundary, on a C
    sidecar and module, and in a library build. Patch-level: valid programs that were
    rejected now compile, and nothing that compiled before changes.
  • The runtime tests clean up after themselves. Each runtime_tests case built its
    program in its own directory under the system temp dir and left it there, about 140 MB
    a plain case and 400–600 MB a sandbox case, so a full run left roughly 7 GB behind
    and a nearly full disk saw the linker fail mid-run in what looked like a test failure.
    A case now removes its build directory when it passes and keeps it when it fails;
    ORCH_KEEP_TEST_BUILDS=1 keeps every one for a fast rerun. Documented in
    CONTRIBUTING.md. Patch-level: the tests changed, the compiler did not.
  • A serverlet call no longer consumes the caller's argument. let n = s.shout(payload)
    followed by any later use of payload failed in rustc with E0382, because the call
    moved its arguments into the message; the same for an array or a struct, on every
    boundary. A binding or a field passed to a handler is now copied at the call site, so
    the caller keeps it; a temporary is still moved, and neither the message nor any wire
    protocol changed. Regression tests reuse a string, an array, and a struct after a call
    in-process, on a secret serverlet, in a sandbox, and over the Python and TypeScript
    landlines. Patch-level: a valid program that was rejected now compiles, and nothing
    that compiled before behaves differently.
  • Sixteen wrong programs are told so by orchestrate check instead of rustc. The
    corpus found them: a call into a module or a foreign function with the wrong argument
    types, or to a function the module does not have (which now lists what it has); a
    serverlet handler called with the wrong arity or types, where only the handler's name
    was checked before; a handler declared twice in any serverlet, not only a landline; a
    secret serverlet handler type the wire cannot carry, reported by check rather than by
    the child's build; start X(args) on a serverlet, which takes none; start of
    something that is not a process; for over a number or a string; ? in a function
    that returns neither result nor option; a piped value of the wrong type; a second
    orchestrator main; and a sandboxed handler that waits, which a guest cannot do,
    reported at build. orchestrate check also registers Rust sidecar signatures now, as
    the build always did, so a call into a Rust foreign module or the standard library is
    typed during a check instead of unknown.
  • A block whose last statement touches shared state takes the lock. while c { hits = hits + 1 }, if c { hits = hits + 1 }, and on_tick(dt: float) { hits = hits + 1 }
    all end in a block's tail expression, which was compiled without the guard and reached
    rustc as an unknown __shared. Found by the tick benchmark's shared let case. Every
    block tail that touches shared state now takes the lock exactly as a statement does,
    and a tail that also waits is refused with the same message.
  • A let that reads shared state keeps its binding in scope. let snapshot = hits
    wrapped the whole declaration in the lock's block, so snapshot was gone on the next
    line and rustc reported it missing. The lock now wraps only the initialiser.
  • A sandboxed serverlet no longer leaks guest memory on every string argument. The
    host allocated each string in guest memory and never freed it, so a serverlet that took
    strings walked into its memory cap and then failed every call: under an 8mb cap, 20,000
    one-kilobyte calls failed four times and reset the guest each time. Every allocation the
    host makes for a call is released after it, and a test sends those 20,000 calls.
  • A handler may return its own state. return seen, where seen is an array or a
    string the serverlet declares, moved the value out of the actor, the child, or the guest,
    and rustc refused with a message about moved values in generated code — on every
    boundary. A handler that returns a state binding now returns a copy. For a number the
    copy is what happened before.
  • A sandbox handler type that cannot cross is rejected by orchestrate check, by
    name, as the compiler's own error. It used to surface from the guest crate's build as a
    compile_error! after codegen.