Repository navigation
Developer Guide
How to build shoal from source, how the workspace is organized, how to run the tests that matter,
and how to contribute. The two files to read before touching anything non-trivial are
docs/TDD.md (the language/semantics
contract) and docs/CONTRACTS.md
(the pinned inter-crate Rust API contracts) — this page is a map onto both, not a replacement for
either.
Rust, edition 2024. No other toolchain requirements.
cargo build --workspace
cargo run -p shoal # interactive REPL
cargo run -p shoal -- -c $'let answer = 6 * 7\nanswer' # one-shot
cargo run -p shoal -- path/to/script.shl # run a .shl fileInside the REPL: statement-position commands get a real PTY (colors, progress bars, password
prompts, TUIs all behave normally); expressions and assignments capture clean output instead. The
line editor (reedline) gives persistent history, multiline input, completion, and a cwd/git-aware
prompt out of the box.
Developer-facing subcommands, all on the shoal binary itself:
shoal fmt [--check] [FILES] # format .shl source (stdin when no files given)
shoal doctor [--json] # diagnose the installation — see Ecosystem-and-Tooling
shoal lsp # run the language-server companion
shoal mcp # run the MCP companion, talking to a running shoal-kernel
shoal completions <bash|zsh|fish>The dependency graph is acyclic and pinned in docs/CONTRACTS.md — re-derive it any time with
grep -oE '^shoal-[a-z]+' crates/*/Cargo.toml. By tier:
Tier 0 — leaf (no shoal-* deps):
shoal-ast shoal-auth shoal-config shoal-journal shoal-leash
shoal-proto shoal-reef shoal-secret shoal-wasm shoal-prompt
Tier 1 — depend only on Tier 0:
shoal-value -> ast
shoal-syntax -> ast (owns the canonical builtin-command-head registry, `commands::builtin_names()`)
shoal-exec -> leash (NOT a leaf — sandboxing hooks around spawn)
shoal-history-> journal
shoal-lsp -> syntax (consumes shoal-syntax's builtin registry; no shoal-eval dep)
Tier 2 — depend on Tier 0/1:
shoal-adapters -> ast, value
shoal-picker -> value
Tier 3 — the domain core:
shoal-eval -> adapters, ast, exec, journal, leash, picker, reef, secret, syntax, value
Tier 4 — composition roots (daemon + tools):
shoal-doctor -> adapters, journal, leash
shoal-kernel -> ast, auth, eval, exec, journal, leash, proto, syntax, value
Tier 5 — entrypoints (binaries):
shoal -> adapters, ast, config, doctor, eval, prompt, syntax, value
shoal-mcp, shoal-lsp -- spawned by `shoal` as companion subprocesses (shoal mcp / shoal lsp),
NOT Cargo dependencies of `shoal`. shoal-mcp has zero shoal-* deps and
talks to shoal-kernel purely over the wire protocol; shoal-lsp, unlike
shoal-mcp, DOES link a shoal-* crate (shoal-syntax — see Tier 1 above).
shoal-kernel -- the long-lived per-user daemon; `shoal` never depends on or spawns it
shoal-prompt is a pure, dependency-free domain crate (only serde+toml, shoal-value only
as a dev-dependency for a render-parity test) that renders a shell prompt from a structured
PromptContext snapshot — no IO, no subprocess calls, so it's independently benchmarkable
(benches/render) and testable without a live session.
Hexagonal ports (done — roadmap wave R4): shoal-eval's domain logic depends on trait objects
for every OS-facing effect family rather than calling std::fs/std::process/std::time
directly. Fs/Clock/Opener/SecretPort live in shoal-value/src/ports.rs (reachable from a
leaf crate); Exec lives in shoal-eval/src/ports.rs because its signature is stated in terms of
shoal-exec types. Every trait has a Std* default adapter that performs the exact call the
evaluator made inline before the ports existed, so this is behavior-preserving — its value is
testability: a test can swap a fake to interpose on reads/writes/spawns/time without touching the
real OS.
Ownership, per CONTRACTS.md: shoal-ast, shoal-value, shoal-syntax, shoal-eval, and the
shoal binary are integrator-owned core; shoal-exec, shoal-journal, shoal-adapters,
shoal-reef, shoal-value/src/{methods,render}.rs, and spec/cases/*.toml are delegated modules
that build to the pinned contracts in that file. If you're touching one of the delegated pieces,
build against the contract, don't renegotiate it in the same PR.
cargo test --workspace --all-targets
cargo clippy --workspace --all-targets -- -D warningsThe single most important test in the whole workspace:
cargo test -p shoal --test conformance -- --nocaptureThis walks every [[case]] in spec/cases/*.toml (1,218 cases across 74 files as of this writing —
ls spec/cases/ for the full, ever-growing list; roughly organized around core, literals,
strings*, operators*, coercion*, collections, match*, outcome*, closures*, reef*,
io*, streams*, namespaces*, and many more focused files), builds a fresh Evaluator per
case rooted at a fresh temp-dir cwd
(materializing any fixture files first), parses and evaluates src, and checks the rendered
final value (or the expected error/parse_error code) against the case's expectation. It's one
umbrella #[test] so a single cargo test invocation reports every case name that failed, not
just the first.
The corpus is the normative spec (TDD §12: "the corpus decides disputes"). A case encodes the
correct behavior per docs/TDD.md/docs/CONTRACTS.md — not necessarily what today's
implementation does. If a test passes but violates the TDD, the test (or the implementation) is
wrong; don't edit the TDD to match an emergent bug. A case may be marked skip = "reason"
(host-dependent behavior — see spec/cases/reef.toml for real examples: a real tool's resolved
hash/provider varies by machine, so those specific cases are skipped with a reason rather than
faked into determinism).
Case schema (CONTRACTS §5):
[[case]]
name = "unique-kebab-name"
src = """
let x = 2 + 3
x * 2
"""
value = "10" # render_inline of the final statement's value
# OR error = "type_error" # eval error code (+ optional error_contains = "substr")
# OR parse_error = true # (+ optional parse_error_contains = "substr")
fixture = ["a.txt", "sub/b.log"] # optional: empty files created under a fresh temp cwd first
skip = "reason" # optional: harness skips with a reason instead of assertingAdding a new language behavior or fixing a bug? Add or adjust a case here first — it's the sharpest, smallest possible regression test, and it's what the whole team (and CI) trusts as ground truth. Case names must be globally unique across every file; the harness fails loudly on a collision.
Most crates carry their own focused unit tests (shoal-leash's policy/enforcement tests,
shoal-exec's PTY/signal tests — gracefully skipped when the test runner itself has no tty,
shoal-reef's lockfile/provider tests, shoal-adapters' canned-fixture parser tests that need no
real binaries or network). cargo fuzz targets exist for the lexer/parser (never panic on
arbitrary bytes) and are built — not yet required to pass — on CI's nightly fuzz-build job.
See Ecosystem-and-Tooling §8 for the full breakdown: a real Linux + macOS matrix on every push
and PR to main (build, test, conformance corpus, fmt, clippy), a separate release-build gate on
both OSes, and a continue-on-error nightly fuzz-build job.
- Read
docs/TDD.mdanddocs/CONTRACTS.mdbefore changing evaluator or wire-protocol behavior — they are decisions, not options. - Prefer adding a
spec/cases/*.tomlcase over an ad-hoc unit test for anything about language semantics (parsing, coercion, evaluation, error codes) — it doubles as documentation. - Adding a new adapter is the lowest-friction contribution: one TOML file under
adapters/, no code changes, verified with a canned-fixture-bytes unit test (seecrates/shoal-adapters' existing tests for the pattern) — no live binary or network access required. - If you're touching a delegated crate (
shoal-exec,shoal-journal,shoal-adapters,shoal-reef,shoal-value/src/methods.rs+render.rs), build to the pinned signatures indocs/CONTRACTS.mdand update that file in the same PR if a signature has to change — it's a contract other in-flight work depends on. - Never touch the workspace root
Cargo.tomlcasually; add crate-local dependencies withcargo add -p <your-crate> <dep>. - License is MIT OR Apache-2.0.
-
Language-Reference, Execution-Model — what
shoal-syntax/shoal-evalactually implement. - Reef, Leash-and-Security — the delegated crates with the richest design docs.
- Ecosystem-and-Tooling — the adapter pack, CI, and release pipeline in more detail.