Repository navigation
Releases: Immanuel-Fowler/OrchestrateLang
Release list
OrchestrateLang v0.15.1
Everything the build used to catch, check catches first.
Added
orchestrate check --libchecks a program the waybuild --libbuilds it:
hostblocks,on_tick, and grants are allowed, and a library's own rules hold —
host names, grants that name a host function, one typedon_tick, lifecycle hooks in
the entry file, the shape ofmain. Those rules now live in one function thatbuild
andcheckboth call, so the two cannot drift apart. Plaincheckapplies the
standalone rulesrunandbuildapply, so it refuses ahostblock or anon_tick,
which the build without--libalways refused; check a library with--lib.
Patch-level, under CONTRIBUTING.md's rule for a command that only reports on existing
code: nothing compiles or runs differently, and the only programs plainchecknow
refuses are ones its matching build already refused.
Fixed
- A struct or enum named with a Rust keyword compiles. 0.15.0 escaped every other
kind of name but left type names out, sostruct ref { ... }orenum loop { ... }
still reached rustc. The type is now a raw identifier wherever generated Rust names
it — its declaration, literals, variant paths, the wire and sandbox codecs, library
host signatures, and C and TypeScript FFI wrappers — while the sandbox codec functions
built from its name keep the plain spelling, which is always valid. Patch-level:
programs that failed now compile; nothing else changes. checkwalks the modules a program imports, and refuses a foreign language the
build does not support. Each imported module's own declarations are typechecked in
the module's own scope, with its foreign functions callable by their bare names, so a
type error in a serverlet declared in a module is acheckerror that names the
module's file and line; it used to passcheckand fail in Cargo, as the new corpus
programmodule_serverlet_body_type_errordid. Aload_foreignin a language the
compiler does not support is the build's own error, now fromcheckas well. With the
checks above, all 89 programs in the diagnostics corpus are rejected bycheck: none
reaches the build, and none reaches rustc. Patch-level: invalid programs and modules
are refused earlier, and every valid one still checks.orchestrate checkrefuses a wait where the compiler cannot put one. Four
mistakes that only the build caught arecheckerrors now: afnthat calls a task
(a module's too),sleep, a serverlet, or aparallelblock anywhere in its body; a
sandboxed handler that waits; a statement that both waits and touches shared state,
which would hold the lock across the wait; and ashared letof a type that cannot be
shared, such as a serverlet client. What counts as a wait is codegen's own rule, except
that a call through a receiver not known to hold a serverlet client never counts, so
the language server, which registers no modules, never mistakes a module call for one.
Patch-level: wrong programs are refused earlier, and every valid program in the
repository still checks.- The whole test suite cleans up after itself. 0.15.0 did it for
runtime_tests;
the rest still left their builds: about 4 GB intests/programs/.orch_cacheand
more intests/error_cases/, both inside the source tree, a folder per landline test
in the temp directory, and one per run from the dependency unit tests. Each is
removed when its test passes, and kept when it fails or under
ORCH_KEEP_TEST_BUILDS=1. Patch-level: only the tests changed. - Calling another orchestrator type-checks again. 0.15.0's check for a second
orchestrator mainsat in front of the arm that registers each orchestrator's
signature, so that arm never ran andhelper()on anorchestrator helper()was
"unknown function 'helper'" incheck, where 0.14.0 accepted it. Both now run in one
arm. Patch-level: it restores what 0.14.0 did. - A deterministic-mode refusal tells the host why. A worker, serverlet, or landline
started in deterministic mode, or asleepin a tick, stops the library with a panic
naming deterministic mode, but the host saw only "library startup task failed" or
"tick task failed"; the reason reached the process's panic hook and nothing else.
ready,tick, andfixed_ticknow append the panic's message to that error, and
any other stop is reported the same way. Patch-level: the errors keep their prefixes
and only gain the reason. - A library instance with nothing to unpack leaves no temp folder. Every
scripts::startcreatedorch_library_<pid>_<n>in the temp directory, even when
the program embeds no landline or secret executables, and removed it only on a clean
shutdown, so a host that exited without one left an empty folder each time. The folder
is now created only when there is something to put in it. Patch-level: nothing a host
can observe changes. - The embedded standard library no longer leaves a temp folder per run. An installed
orchestratewith nostdlib/beside it unpackedlistsandstringsinto a fresh
orchestrate_stdlib_<pid>_<n>folder in every process and never removed it; 38 had
piled up on one machine. It now unpacks one copy per compiler build, named by the
version and a hash of the embedded sources, shared by every process and renamed into
place whole, so a racing process never reads a half-written copy. Patch-level: how a
module resolves and compiles is unchanged. - Conflicting generic arguments are refused by
orchestrate check.same(1, "x")
againstfn same<T>(a: T, b: T)passedcheckand reached rustc as E0308 against
generated code, the last program in the diagnostics corpus that did. The typechecker
now binds each type parameter to one type and names the argument that disagrees; an
argument whose type is not known yet, such as[]ornone, binds nothing.
KNOWN_LEAKS.txtis empty. Patch-level: a wrong program is refused earlier, and no
valid program changes.
OrchestrateLang v0.15.0
The sandbox catches up with the C ABI, and every claim is measured.
Added
-
Arrays and structs across the sandbox boundary. A sandboxed serverlet carried
int,
float,bool, andstring; it now carriesint[],float[],bool[], and structs
whose fields are numbers, booleans, or such structs — the set the C ABI carries since
0.14.0, under the same rules. An array crosses as a pointer and a count into the guest's
memory, a struct as its#[repr(C)]bytes, each field at its own offset and padding
zero, so neither side reads padding or depends on the other's endianness. What the host
writes for a call it frees after the call; what the guest returns the host copies and
frees. The guest's allocator is now 8-byte aligned. A handler's state may be of these
types, and a handler may return it.An array of strings or structs, or a struct holding one, does not cross the sandbox,
though the in-process, secret, and landline boundaries carry them. That gap is written
down indocs/limitations-notes.md. -
grant callon a sandboxed serverlet. In a library build, a sandboxed serverlet
takes the same grant lines a landline does, and there they are the only way the guest
reaches the host at all: each grant is exactly one import defined in the guest's
wasmtime linker, behind which theHosttrait method runs; nothing is defined for
anything ungranted, so it does not exist inside the guest. A handler that calls an
ungranted host function is refused byorchestrate check, which names the call and the
grant that would allow it. Arguments and results cross in the wire encoding landline
grants use — numbers, booleans, strings, arrays, and structs. A host error or panic
fails that one call inside the guest, which logs it and continues with the default. A
host call counts against the serverlet'stimeout, since the guest is still inside its
call while the host runs. Step 7 ofdocs/features/sandboxed-serverlets.md, and the
point where containment and consent become one mechanism. -
on_crashon a sandboxed serverlet runs on the host after a call ran past its
timeout, exhausted its memory, or stopped itself, with the trap's message bound, before
the guest is replaced and the caller gets the default. It cannot reach the guest's
state, which is inside the instance being thrown away; a handler that names a state
binding is a compile error saying so. -
The boundary ladder benchmark covers every rung.
benchmarks/landline_latency
measures the same echo handlers on the in-process, sandboxed, secret, Python, and
TypeScript boundaries, and the two rungs below a serverlet call,shared letand a
plainlet, timed per batch of a thousand statements. Beside the payload sweep there
is a call-rate sweep — theintpayload paced at 100 Hz and 1 kHz with the caller
asleep between calls, and at 10 kHz spinning — and a concurrent-callers sweep with 2, 4,
and 8 callers sharing one serverlet.run.pywrites CSV, JSON, and Markdown under
benchmarks/results/with an environment header (CPU, OS,rustc, Python, the
TypeScript toolchain, wasmtime, sample counts, commit SHA), and the README's tables are
pasted from that output rather than typed. The TypeScript rung is skipped, and said to
be, when Bun or TypeScript 7 is missing. -
The tick cost is a committed benchmark.
benchmarks/tick_cost/run.pybuilds six
small library programs — an empty tick, one and five host calls, an instancelet
increment, ashared letincrement, and one host-fired event per tick — drives each
throughtick_syncon a current-thread runtime in release mode, and reports the median
of 21 rounds of 200,000 ticks beside theHosttrait method called through its vtable
alone. It writes CSV, JSON, and Markdown with the same environment header as the
boundary ladder. The 8 ns tick and 1.5 ns host-call figures from the 0.8.1 notes come
from this measurement now, not from a release note. -
Deterministic mode is tested at length. One scripted sequence of 10,000 ticks and
events — host-fired events, events fired by handlers, handlers that sleep on host time
across ticks, and instance state — replayed on fresh instances, on a current-thread and
a multithreaded runtime, throughtick_blockingandtick_sync, produces a
byte-identical host-call trace. A second test asserts that each documented exclusion
(a spawned worker, a serverlet, a landline, asleepoutside an event handler) fails
the waydocs/library-mode.mdnow says, with the deterministic-mode message. -
The diagnostics corpus covers the language.
tests/error_cases/diagnostics/grew
from 30 to 88 deliberately invalid programs, across types, events, serverlets of every
kind, processes, C, Rust, and WebAssembly sidecars, and module boundaries, with the
module fixtures they import beside them.diagnostics_never_leak_rustcnow runs each
throughorchestrate checkfirst and reports the count — 81 of 88 are rejected by
check, 6 more by the build before Cargo, 1 reaches rustc — and holds the leaks to
KNOWN_LEAKS.txt, so a leak that appears or disappears fails the test until the list
says so.benchmarks/diagnostics_coverage.pywrites the same classification as CSV,
JSON, and Markdown. -
One command regenerates every table.
python3 benchmarks/run_all.pybuilds the
release compiler and runs the boundary ladder, the tick cost, and the diagnostics
coverage in order, writing every CSV, JSON, and Markdown table under
benchmarks/results/and joining the reports intoresults.md.benchmarks/README.md
documents it and the three scripts. -
What
orchestrate checkguarantees is written down, in one paragraph of
docs/language-reference.mdbefore "Debugging Generated Code": what a check does and
finds, and what it does not — no codegen, no Cargo, library-only rules unenforced,
module serverlet bodies unwalked, foreign sources unchecked.
Changed
-
The limitations notes live under
docs/.LIMITATIONS-notes.mdsat at the repo
root, where CONTRIBUTING.md allows only the README, changelog, contributing guide, and
license; it is nowdocs/limitations-notes.md, and every reference to it, including
the comment inKNOWN_LEAKS.txt, follows. -
The documentation says what the language is before it says what it does. The four
foundational principles each have a document underdocs/design/: concurrency is the
structure, asynchronous first and synchronous second, polyglot as coordination and
attachment, and as many choices as possible made in syntax. Each states the claim, what
it buys, and what it costs, with examples that typecheck and numbers taken from
benchmarks/.docs/design-philosophy.mdis now the index of those four plus the
eight principles that govern how the language is built.The per-feature design documents moved from
docs/design/todocs/features/, where
README.mdlists them with their status. Every cross-reference in the repository was
updated, so no link moved without its target. -
The README is a front door rather than a second reference. Installing comes first,
then the feature set, then the benchmarks, then the four ideas in thirty seconds and
the principles they rest on. The features, benchmarks, and philosophy sections are
self-contained: the benchmark tables are the generated output spliced in, not a pointer
atbenchmarks/, and nothing in those three sections sends a reader to another page to
learn what the section is about. The 430-line syntax walkthrough it used to contain is
indocs/language-reference.md, where it was already documented.
Fixed
- A function that can fall off its end without returning is refused by
check.
fn f(n: int) -> int { if n > 0 { return 1 } }reached rustc as E0317 ("ifmay be
missing anelseclause").orchestrate checknow requires a body that declares a
return type to end in a value or to return on every path — through both branches of an
if/else, every arm of amatch, both sides of atry/catch— and says which
function, task, or handler can reach its end without one.fn_missing_returnleaves
KNOWN_LEAKS.txt; of the 88 corpus programs, 81 are rejected bycheckand one still
reaches rustc. Patch-level: an invalid program is refused earlier, and no valid program
changes — every example, benchmark program, and test program still checks. - A name that is a Rust keyword no longer breaks the generated Rust. A handler
namedmove, afn type(), alet mut, a struct fieldref, a module calledimpl,
a C sidecar functionmove— any of the fifty-odd Rust keywords that are not
OrchestrateLang keywords — reached rustc as a syntax error in generated code; only
genwas escaped, and only in library builds. Every such name is now emitted as a raw
identifier (r#move) from one keyword list covering editions 2021 and 2024, across
functions, tasks, handlers, parameters, bindings, struct fields, enum variants, match
bindings, program and shared state, module names, and foreign wrappers. A name that
crosses a boundary keeps its spelling there: a sandbox export carries
#[export_name = "move"], a landline handler ismoveon the wire and in Python or
TypeScript, a C symbol ismovefor the linker.self,Self,super, andcrate
cannot be raw identifiers at all, soorchestrate checkrefuses them as names with an
error that says so; four corpus cases hold that, taking the corpus to 88 programs. Regression tests use keyword names on every boundary, on a C
sidecar and module, and in a library build. Patch-level: valid programs that were
rejected now compile, and nothing that compiled before changes. - The runtime tests clean up after themselves. Each
runtime_testscase built its
program in its own directory under the system temp dir...
OrchestrateLang v0.14.0
Arrays and structs cross the fastest boundary.
Added
-
Arrays and structs across the C ABI. C, C++, Zig, and Swift sidecars carried
int,
float,bool,string, andhandle; they now carryint[],float[],bool[], and
structs declared in the program.total(items: int[]) -> int scaled(items: float[], by: float) -> float[] shift(p: Point) -> PointAn array parameter is two C parameters — a pointer and a count, in that order. A
returned array adds one more, along long *the function writes the count through, and
returns the pointer. The ownership rule is the one strings already follow: what the
foreign side returns ismalloc'd, and the generated wrapper copies it and frees it; an
array parameter is borrowed for the call and must not be kept.A struct crosses by value, because generated structs are now
#[repr(C)]. Its fields
must line up with the foreign declaration, field for field and in order.Arrays carry
int,float, andbool. An array of strings, of handles, or of arrays
does not cross yet, and neither does a struct with those fields.
Changed
- Structs generated from a
structdeclaration are#[repr(C)], so their layout is the
one a foreign declaration sees. Nothing else about them changed.
OrchestrateLang v0.13.0
State a worker can touch, declared rather than assumed.
Added
-
shared let. A top-levelletbelongs to the instance and only hooks reach it,
because a spawned worker can run while a tick holds it. The existing answer for state
several concurrent things touch is a serverlet, at about 7.5 µs a call.shared letis
the rung between: one mutex instead of a channel round trip.let counter = 0 // unchanged: instance-owned, hooks only, free shared let hits = 0 // any task, any fn, synchronisedThe guarantee is one sentence: a statement that touches shared state is atomic with
respect to all shared state. All shared bindings live behind a single mutex, taken once
per statement, sohits = hits + 1is correct without anyone thinking about it and there
is no lock order to get wrong. Reads clone, because a value cannot be moved out of a
guard; for a number that is a copy.A statement may wait, or touch shared state — not both. Holding the lock across a
wait would block every other reader, so it is a compile error saying to split the
statement. Because the lock is synchronous, a plainfncan touch shared state, which
afncould never do with instance state.A shared binding is never captured by a worker or an event handler: capturing would copy
the value and quietly undo the sharing, so it is reached through the lock wherever it is
used. In library mode it lives on the instance, so two libraries started in one process
do not share it. Data can be shared —int,float,bool,string, arrays, options
and structs; a serverlet client, a process, or a closure is refused by name.A program without
shared letgains no mutex, no field, and no code. The design, and the
five questions it opened and how each was answered, are indocs/design/shared-state.md.
OrchestrateLang v0.12.0
Wrong programs are told so in OrchestrateLang.
Fixed
An audit wrote twenty-eight deliberately wrong programs across the language and checked how
each one failed. Eleven reached rustc, reporting on generated code the user never wrote,
with advice about traits and moves that means nothing here. All eleven are now the
compiler's own errors, and the corpus is a test so they stay that way.
- Calls check argument types, not just how many there are.
- An unknown function is an error, not a warning followed by a rustc failure.
print
is generic over anything displayable, as the generated code always was. - A serverlet client only answers handlers its serverlet declares, and the error lists
the ones it has. Handlers reached through a module alias resolve correctly. start X()must name a serverlet that exists.triggermust name a declared event, with the right number and types of arguments.- A struct literal must fill every field.
- Comparing two unrelated types is an error, rather than a missing
PartialOrd. - A function declared to return nothing cannot return a value.
fn,task, andprocessnames must be unique, and an enum cannot repeat a variant.process[...]must name processes, and each name must be declared.
Added
tests/error_cases/diagnostics/holds the audit corpus, anddiagnostics_never_leak_rustc
asserts that every file there is rejected by the compiler rather than by Cargo, and that
none is accepted.
OrchestrateLang v0.11.0
The language owns its own errors.
Fixed
- String concatenation no longer consumes its operands.
a + bmoved both sides, so a
string could be used exactly once ands + sdid not compile at all
(error[E0382]: use of moved value). Addition now borrows:let c = a + bleaves both
aandbusable, anda + aworks.intandfloataddition is unchanged in
behaviour and in cost. - A match on a unit enum variant that binds a value is a type error, naming the variant
and how to write it, instead of rustc's "expected tuple struct or tuple variant, found
unit variant". The mirror case is caught too: a variant that carries a value and is
matched without binding it. So is a variant that does not exist, which now lists the ones
that do. - A
tryblock whose body waits compiles.try { fetch(1) } catch e { 0 }, where
fetchis a task, reached rustc as "awaitis only allowed insideasyncfunctions"
because the block generated a synchronous closure. A block that waits now generates an
async block instead, so?still works and the handler may wait too. A block that does
not wait generates exactly what it did before. - A
fn,task, orprocessthat names top-level state says so. Top-levellets
belong to the instance and only hooks reach them — a spawned task can run while a tick
holds that state, so the restriction is deliberate. It used to surface as rustc's
"cannot find value in this scope"; it is now a diagnostic naming the function, the
binding, and the way out.
Changed
docs/roadmap.mdno longer describes sandboxed serverlets as unshipped (0.9.0) or C# as
upcoming (0.10.0).
OrchestrateLang v0.10.1
A fn that cannot wait says so in OrchestrateLang.
Fixed
- A
fnthat calls a serverlet, a landline, atask,sleep, or aparallelblock now
fails with an OrchestrateLang error naming the function and the call, and saying to
declare it as atask. It used to reach Cargo and surface as rustc's
"awaitis only allowed insideasyncfunctions", pointing at generated code the user
never wrote.fncompiles to a synchronous Rust function andtaskto an async one,
which is the rule the message now states;docs/language-reference.md§2.5 states it too.
OrchestrateLang v0.10.0
C# runs in-process, about five nanoseconds a call.
Added
-
load_foreign "csharp". A C# file becomes a native shared library through .NET's
Native AOT compiler and links into the program like any other foreign module. A call
costs about 5 ns on an Apple M2, against 1–2 ns for the same function in C; the
difference is the transition a reverse P/Invoke makes entering and leaving managed code.
That is 20× cheaper than reaching the same C# through a WebAssembly module.load_foreign "csharp" "./Math.cs"A method is exported by carrying
[UnmanagedCallersOnly(EntryPoint = "...")], and the
name it gives must match the sidecar; everything else in the file stays ordinary C#.
intislong,floatisdouble,voidisvoid, andboolis the exception —
it is not blittable in an export signature, so the sidecar'sboolis a C#bytethat
is 0 or 1. Strings, arrays, structs, andhandledo not cross yet.Each module publishes its own shared library rather than a static archive. .NET can
publish static, and it would link a little faster, but two Native AOT static archives
cannot go into one program — each embeds its own runtime — which would cap a program at
one C# module and fail as duplicate symbols from the linker rather than as a diagnostic.
A test covers two C# modules in one program.The .NET SDK 8 or newer must be on
PATH, orORCH_DOTNETmust point at it. The first
build downloads the Native AOT compiler.check-foreigndoes not cover C#, because
dotnethas no syntax-only check for one file outside a project.A C# module brings .NET's garbage collector into the process. For a host with a frame
budget, keep the hot path allocation-free, or use a boundary that keeps the collector
elsewhere. Behaviour under an allocating handler is not measured yet; the plan in
docs/plans/csharp-native-backend.mdsays so.
OrchestrateLang v0.9.0
Untrusted code runs contained, and any .wasm module is callable.
Added
-
Sandboxed serverlets contain their code.
serverlet X sandbox(memory_limit: "64mb", timeout: "5s")has parsed since 0.5, and since 0.6 its handlers compiled to a
wasm32-wasip1guest, but the serverlet still ran in-process and the compiler warned
that nothing was isolated. It is isolated now: the guest runs under
wasmtime,memory_limitcaps its linear memory,timeout
bounds a single call through epoch interruption, and its state lives inside the guest
between calls. Imports are denied by default, so a guest that reaches for the host traps
instead of arriving. From the caller's side nothing changed — samestart, same client,
same methods.int,float,bool,string, andvoidcross the boundary.A call that exceeds a limit or stops itself is logged, answered with the return type's
default, and the program carries on. The guest is then replaced, because a trap abandons
it where it stands rather than unwinding it — so a failed call resets that serverlet's
state. The guest is given two host functions and no others: one carries its own stderr
out, so a panic or an allocation failure is reported as[orchestrate] sandbox guest: ..., and one lets it stop itself. Neither is a capability; they exist so a contained
failure can say what it was.Not yet:
grantandon_crashon a sandboxed serverlet are compile errors rather than
holes that open quietly, and arrays and structs do not cross the boundary. -
load_foreign "wasm". A module's exports become ordinary functions:load_foreign "wasm" "./math.wasm"There is no toolchain to install and no language to name, because a
.wasmis already
compiled — whatever produced it is the author's business. The compiler reads the
module's own export table and checks every sidecar signature against it, so a name that
is not exported, or a type that does not line up, is a build error naming both sides;
orchestrate checkruns the same check without building. The module is embedded in the
program, and its imports are denied by default, so it reaches nothing it was not given.
intisi64,floatisf64,boolis ani32, and astringis a pointer and a
length, which means a module carrying strings must exportorch_alloc(i32) -> i32and
orch_free(i32, i32)so both sides use one allocator.
Changed
sandbox(...)validatesmemory_limitandtimeoutwhen it parses them, so a typo is
an error rather than a limit that quietly misses.orchestrate checknow registers C, C++, Zig, and Swift sidecar signatures as well as
TypeScript's, so a call into one of those modules is typed during a check instead of
drawing an "unknown method call" warning.- Programs that use neither a wasm module nor a sandboxed serverlet gain no new
dependency. Those that do gainwasmtime, which is large; building a sandboxed
serverlet also needs thewasm32-wasip1target.
OrchestrateLang v0.8.1
The glue around a synchronous tick is a few nanoseconds, down from about forty.
Changed
- Library mode: the glue around a synchronous tick is a few nanoseconds.
tick_syncno
longer enters the runtime, scopes a task-local, or takes an async mutex per frame: the
program's state sits in a slot entered with two flag stores and two loads, every hook
body binds its instance once so a host call is the trait call and one branch, the clock
advances with an integer add, and an idle tick never builds the event drain. On an Apple
M2 an empty tick went from 45 ns to 8 ns pertick_sync, and each host call in a hook
from about 3 ns to about 1.5 ns, no more than the trait call itself. Nothing observable changed:tick,
fixed_tick, theblock_onfallback for a body that waits, deterministic mode,
stop_orch(), shutdown, event ordering, and the "a tick is already running" error on a
re-entered tick are as before. The ignored testglue_costintests/library_tests.rs
measures the three cases. - Library mode:
sleep, landline budgets, and workers started inside a hook no longer need
the calling thread to be inside the runtime's context; the library carries the handle it
was started with, sotick_syncworks from any thread that holds the runtime. - Library mode: a failed host call is reported as
[orchestrate] host call <group>.<name> failed: <error>— the call's name is now part of
the message — still at error level throughHost::log, still yielding the return type's
default, through one shared cold function rather than a closure at every call site.