Repository navigation
Execution Model
This page covers what happens once source is parsed: how a call becomes a process, what a command call is as a value, the streaming/reactive model, task lifecycle, and session-scoped state. See Language-Reference for the grammar that produces the AST this page executes, and Agent-Surface for how the same execution surfaces over the wire.
One rule, keyed to syntax position (TDD §1.2):
-
Statement position, in an interactive session — the call is the whole statement — the child
gets a real PTY, passed through raw: colors, progress bars, cursor tricks, everything,
byte-identical to bash — while the byte stream is simultaneously teed into the CAS.
itbinds to the resulting outcome afterward, with structured parsing applied lazily to the teed bytes on first structured access.cargo buildat the prompt looks exactly like it does today, and is still a queryable value afterward. - Value position — the call feeds a chain, an assignment, an argument, or any expression — the child gets pipes and no controlling tty. It self-detects non-interactive, emits clean machine output, and shoal captures it structurally.
Scripts and non-tty sessions are all value-position. interact <cmd> forces a PTY anywhere;
adapters can declare class = "tui" to get PTY behavior automatically. Redirects (>, >>) force
capture regardless of position, because the tee makes it work either way.
R0 ergonomics fix. An outcome now carries streamed: bool — true only when its bytes
actually hit the real terminal via the PtyTee path. The REPL's render_result re-renders any
outcome whose streamed is false, which fixes a real defect: statement-position builtins
(echo hello, ls, cat) never went through PtyTee (they have no external process to hand a
PTY to) and so used to render nothing at the prompt until you explicitly pulled .out — only
genuine external commands in statement position (already visible live on the real tty) correctly
skipped the double-render. exit [code: int = 0] / quit are now REPL builtins too — they set a
host-level exit flag (Flow/pending_exit) rather than calling std::process::exit directly,
so the same code works identically whether the host is the interactive REPL, a script, or an
embedded kernel session.
Capture (value position) |
PtyTee (statement position) |
|
|---|---|---|
| Streams | stdout/stderr piped, drained concurrently | raw bytes to the real terminal, simultaneously teed |
| Process group | own group (setpgid(0,0)), no controlling tty |
session leader on a real PTY |
| stdin | per StdinSpec (Null/Inherit/Bytes/File) |
real stdin forwarded, terminal in raw mode for the duration |
| stderr | separate buffer | merged into stdout (PTY semantics — this is why a live run can't split .stderr back out; only a captured run can) |
Implemented at shoal-exec (crates/shoal-exec), which is deliberately blocking/thread-based —
no tokio. Cancellation escalates SIGINT → the process group, SIGTERM after 3s, SIGKILL after
3 more; signal deaths surface honestly as signal: Some("SIGSEGV") rather than bash's 128+n
encoding, and children are always reaped (no zombies). When the host process itself has no tty
(CI), PTY mode degrades gracefully: a PTY is still allocated for the child, but raw mode and stdin
forwarding are skipped.
Under outcome unification, every command call — builtin or external — evaluates to a single
outcome value: {status: int, ok: bool, out, err, dur, pid, cmd}. .out is the outcome's
structured payload, parsed lazily. This means echo, ls, and an arbitrary external binary are
all the same shape of result, just with different fidelity depending on whether an adapter is
present (Ecosystem-and-Tooling).
# spec/cases/outcome.toml
(echo hi).out # -> "hi"
(echo hi).ok # -> true
(echo hi).status # -> 0
echo hi # -> outcome(status: 0, ok: true) (bare call at value position)
echo a && echo b && echo c # -> outcome(status: 0, ok: true) (chain returns the deciding operand verbatim)
echo hi && false # -> false (lhs truthy, && proceeds; rhs is a plain bool)
true && (echo hi) # -> outcome(status: 0, ok: true)
A builtin's .out is exactly its own structured return value — echo's .out is the joined
argument string verbatim, never re-sniffed as JSON even if it looks like [1,2,3]. A real
subprocess with no parsed preset does auto-upgrade bracket/brace-shaped stdout to a
structured list/record on first access — the T1 sniffing tier described in
Ecosystem-and-Tooling. An adapter-backed call (T2) skips guessing entirely: .out is
whatever output.parse promised, typed.
ok_codes (adapter field, default {0}) decides what "success" means per tool — non-zero raises
stays the default without pretending every tool agrees that 0 is the only success code.
The complete type list (TDD §4.1): null bool int(i64) float(f64) str path glob regex size(u64 bytes) duration(i64 ns) datetime time bytes list<T> record table stream<T> error outcome task plan cmd secret.
-
pathis bytes-backed (OsString) —path → stris fallible (.str()errors on invalid UTF-8;.display()is lossy-with-replacement). Non-UTF-8 filenames are handled end-to-end as bytes, never mangled. -
secretis opaque: renders assecret(NAME), cannot be interpolated into astr(a type error), is injected by the kernel at spawn time, and only its name — never its material — reaches the journal. -
tableis semanticallylist<record>— there's no separate table type at the language level, just a rendering/storage optimization over the same shape. - Equality is structural for data types, identity for
task/stream— comparing two streams is a runtime error, not a silentfalse.
The real Value enum (crates/shoal-value/src/lib.rs) additionally carries Range, Closure,
and CmdRef (an alias/partial-application node, TDD §1.8 — alias gs = git status binds gs to a
partial call AST node; invoking gs -sb appends to that AST, never splices text).
stream<T> is lazy (constructing one does no work until it's driven), single-consumption
(TDD §1.9 — a second .each/.collect/render on an already-driven stream is a runtime error,
code stream_consumed), pull-based with bounded backpressure, and cancellable sink-to-source.
Comparing streams is an error; there is no stream == stream.
Implemented (roadmap wave R1 — crates/shoal-eval/src/streams.rs + channels.rs; corpus in
spec/cases/streams.toml, 33 cases). This is the honest replacement for tail -f | grep,
fswatch | xargs, and sentinel-file polling loops: every source and sink shares the same
combinators as an ordinary collection — a live filesystem watch and a static list<record> are
driven with the identical .where/.map vocabulary.
Sources: .stream() promotes any collection or string (line-split) to a finite stream;
watch(path | glob) (via notify, cross-platform) → stream<{path, kind}>; tail(file, from_start: bool=false) → stream<str>, following external appends; every(duration) →
stream<datetime> ticks; channel(name).events() → stream<event>.
Combinators: .where .map .scan(init, f) .window(n) .dedupe .distinct .flat_map .enumerate .buffer(n) .zip(other) .merge(other) .take(n) .take_until(pred) .throttle(dur) .debounce(dur).
Sinks: .collect() (finite only — an unbounded stream errors stream_unbounded, use .take(n)
first), .each(f), .sum() .first() .len() .sort() (collection sinks fall through on a finite
stream), .into(channel(name)), .save(path), .feed(cmd).
# spec/cases/streams.toml — real, passing cases
[1, 2, 3, 4, 5, 6].stream().where(x => x % 2 == 0).map(x => x * 10).collect()
# -> [20, 40, 60]
[1, 2, 3, 4].stream().scan(0, (a, x) => a + x).collect() # -> [1, 3, 6, 10]
[1, 1, 2, 2, 2, 3, 1].stream().dedupe().collect() # -> [1, 2, 3, 1]
(1..5).stream().map(x => x * x).collect() # -> [1, 4, 9, 16]
"error: a\ninfo: b\nerror: c".stream()
.where(l => l.starts_with("error")).collect() # -> ["error: a", "error: c"]
let s = [1, 2, 3].stream()
s.collect()
s.collect() # -> error, stream_consumed
channel("x").events().collect() # -> error, stream_unbounded
# the tail/watch/every sources are non-deterministic (real time, real fs events) so they're
# covered by shoal-eval's own unit tests rather than the corpus, but the shape is real and runs:
tail("app.log").where(.contains("ERROR")).each(render)
watch("src/**/*.rs").debounce(200ms).each(_ => cargo test)
every(1s).map(_ => ps.where(.name == "myapp")).each(render)
In-language channel(name) (emit/latest/events/take, ring-buffered with a seq cursor) is the
same substrate the wire's user.{name} events use — see Agent-Surface §7 for the pair-shelling
use case and the exact method list.
spawn { … } produces a task (structured concurrency — children are cancelled with it).
task.await() / .cancel() / .suspend() / .resume(); parallel(a, b, …) and
.parallel(f, max: n) gather with fail-fast by default (settle: true collects errors instead).
jobs is the live task table. Ctrl-C cancels the foreground task tree (SIGINT to the process
group, escalating); Ctrl-Z suspends and returns to the prompt; fg <task> re-fronts it with a
PTY. The kernel is the parent of every job — detaching a client never kills what it started;
background output is buffered per-task and rendered as discrete blocks on resume.
At the kernel level (see Kernel-and-Agents), exec {asynchronous: true} returns a task ref
immediately and the kernel runs the same evaluation on a worker thread; task.list / task.await
/ task.cancel operate on it. task.suspend is implemented as an explicit, honest error today
("task suspension is unavailable for evaluator-owned processes") rather than a silent no-op.
The session owns cwd and env. cd and env.NAME = v are legal at session top level
(journaled) and illegal inside fn bodies — the error names with cwd: / with env: as the
fix, because a function silently mutating session-global state is exactly the ambient-mutation
disease export causes in bash. with cwd: p, env: {…} { … } scopes both dynamically and restores
on any exit path — including an error thrown inside the block.
# spec/cases/reef.toml — cwd/env purity is a load-bearing regression fence,
# because reef's project-scope resolution (see [[Reef]]) depends on it.
let before = pwd
with cwd: "sub" { }
let after = pwd
before == after # -> true
let before = pwd
try { with cwd: "sub" { 1 / 0 } } catch { null }
let after = pwd
before == after # -> true (restores even when the body raises)
fn f() { cd "sub" }
f() # -> custom error, "with cwd:" in the message
env.SHOAL_REEF_ENV_ASSIGN = "via-assign"
sh { printf '%s' "$SHOAL_REEF_ENV_ASSIGN" }.out # -> "via-assign" (session env write, child observes it)
with env: {SHOAL_REEF_MARKER2: "set"} { (env SHOAL_REEF_MARKER2).out } # -> "set"
with reef: {tool: "version"} { … } is the same dynamic-scoping primitive applied to tool
resolution — see Reef.
Session-scoped cwd navigation beyond plain cd, all top-level-only for the same reason cd is
(and all routed through one change_cwd choke point so directory history keeps recording):
-
cd -returns to the previous directory via a session-scopedOLDPWD; erroringcustom("OLDPWD"in the message) if nothing has been recorded yet. -
pushd dir/popd/dirs— a directory stack.pushd dircds and pushes; no-argpushdswaps the top two instead;popdpops and cds there (customerror,"empty", on an empty stack);dirsreturns the stack as alist<path>, current directory first, so it dot-chains (dirs.len(),dirs.first()). -
jump/j— frecency-rankedcd: ranks previously-visited directories by a recency+frequency score and cds to the best match for a fuzzy query.
# spec/cases/dir-stack.toml
let start = pwd
pushd "sub"
dirs.len() # -> 2
popd
start == pwd # -> true
- Language-Reference — the grammar and coercion rules feeding every call here.
- Reef — how the command head resolves to a binary before spawn ever happens.
- Leash-and-Security — what gates a spawn before it's allowed to happen at all.
-
Agent-Surface —
shoal_exec {position: "stmt"|"value"}is this exact rule, over the wire.