Skip to content

Developer Guide

Allie Coleman edited this page Jul 16, 2026 · 6 revisions

Developer Guide

How to build shoal from source, how the workspace is organized, how to run the tests that matter, and how to contribute. The two files to read before touching anything non-trivial are docs/TDD.md (the language/semantics contract) and docs/CONTRACTS.md (the pinned inter-crate Rust API contracts) — this page is a map onto both, not a replacement for either.

Building

Rust, edition 2024. No other toolchain requirements.

cargo build --workspace
cargo run -p shoal                                          # interactive REPL
cargo run -p shoal -- -c $'let answer = 6 * 7\nanswer'        # one-shot
cargo run -p shoal -- path/to/script.shl                       # run a .shl file

Inside the REPL: statement-position commands get a real PTY (colors, progress bars, password prompts, TUIs all behave normally); expressions and assignments capture clean output instead. The line editor (reedline) gives persistent history, multiline input, completion, and a cwd/git-aware prompt out of the box.

Developer-facing subcommands, all on the shoal binary itself:

shoal fmt [--check] [FILES]     # format .shl source (stdin when no files given)
shoal doctor [--json]           # diagnose the installation — see Ecosystem-and-Tooling
shoal lsp                        # run the language-server companion
shoal mcp                        # run the MCP companion, talking to a running shoal-kernel
shoal completions <bash|zsh|fish>

Crate map

The dependency graph is acyclic and pinned in docs/CONTRACTS.md — re-derive it any time with grep -oE '^shoal-[a-z]+' crates/*/Cargo.toml. By tier:

Tier 0 — leaf (no shoal-* deps):
  shoal-ast  shoal-auth  shoal-config  shoal-journal  shoal-leash
  shoal-proto  shoal-reef  shoal-secret  shoal-wasm  shoal-prompt

Tier 1 — depend only on Tier 0:
  shoal-value  -> ast
  shoal-syntax -> ast              (owns the canonical builtin-command-head registry, `commands::builtin_names()`)
  shoal-exec   -> leash            (NOT a leaf — sandboxing hooks around spawn)
  shoal-history-> journal
  shoal-lsp    -> syntax           (consumes shoal-syntax's builtin registry; no shoal-eval dep)

Tier 2 — depend on Tier 0/1:
  shoal-adapters -> ast, value
  shoal-picker   -> value

Tier 3 — the domain core:
  shoal-eval -> adapters, ast, exec, journal, leash, picker, reef, secret, syntax, value

Tier 4 — composition roots (daemon + tools):
  shoal-doctor -> adapters, journal, leash
  shoal-kernel -> ast, auth, eval, exec, journal, leash, proto, syntax, value

Tier 5 — entrypoints (binaries):
  shoal                  -> adapters, ast, config, doctor, eval, prompt, syntax, value
  shoal-mcp, shoal-lsp    -- spawned by `shoal` as companion subprocesses (shoal mcp / shoal lsp),
                             NOT Cargo dependencies of `shoal`. shoal-mcp has zero shoal-* deps and
                             talks to shoal-kernel purely over the wire protocol; shoal-lsp, unlike
                             shoal-mcp, DOES link a shoal-* crate (shoal-syntax — see Tier 1 above).
  shoal-kernel            -- the long-lived per-user daemon; `shoal` never depends on or spawns it

shoal-prompt is a pure, dependency-free domain crate (only serde+toml, shoal-value only as a dev-dependency for a render-parity test) that renders a shell prompt from a structured PromptContext snapshot — no IO, no subprocess calls, so it's independently benchmarkable (benches/render) and testable without a live session.

Hexagonal ports (done — roadmap wave R4): shoal-eval's domain logic depends on trait objects for every OS-facing effect family rather than calling std::fs/std::process/std::time directly. Fs/Clock/Opener/SecretPort live in shoal-value/src/ports.rs (reachable from a leaf crate); Exec lives in shoal-eval/src/ports.rs because its signature is stated in terms of shoal-exec types. Every trait has a Std* default adapter that performs the exact call the evaluator made inline before the ports existed, so this is behavior-preserving — its value is testability: a test can swap a fake to interpose on reads/writes/spawns/time without touching the real OS.

Ownership, per CONTRACTS.md: shoal-ast, shoal-value, shoal-syntax, shoal-eval, and the shoal binary are integrator-owned core; shoal-exec, shoal-journal, shoal-adapters, shoal-reef, shoal-value/src/{methods,render}.rs, and spec/cases/*.toml are delegated modules that build to the pinned contracts in that file. If you're touching one of the delegated pieces, build against the contract, don't renegotiate it in the same PR.

Testing

cargo test --workspace --all-targets
cargo clippy --workspace --all-targets -- -D warnings

The conformance corpus

The single most important test in the whole workspace:

cargo test -p shoal --test conformance -- --nocapture

This walks every [[case]] in spec/cases/*.toml (1,218 cases across 74 files as of this writing — ls spec/cases/ for the full, ever-growing list; roughly organized around core, literals, strings*, operators*, coercion*, collections, match*, outcome*, closures*, reef*, io*, streams*, namespaces*, and many more focused files), builds a fresh Evaluator per case rooted at a fresh temp-dir cwd (materializing any fixture files first), parses and evaluates src, and checks the rendered final value (or the expected error/parse_error code) against the case's expectation. It's one umbrella #[test] so a single cargo test invocation reports every case name that failed, not just the first.

The corpus is the normative spec (TDD §12: "the corpus decides disputes"). A case encodes the correct behavior per docs/TDD.md/docs/CONTRACTS.md — not necessarily what today's implementation does. If a test passes but violates the TDD, the test (or the implementation) is wrong; don't edit the TDD to match an emergent bug. A case may be marked skip = "reason" (host-dependent behavior — see spec/cases/reef.toml for real examples: a real tool's resolved hash/provider varies by machine, so those specific cases are skipped with a reason rather than faked into determinism).

Case schema (CONTRACTS §5):

[[case]]
name = "unique-kebab-name"
src  = """
let x = 2 + 3
x * 2
"""
value = "10"                      # render_inline of the final statement's value
# OR error = "type_error"          # eval error code (+ optional error_contains = "substr")
# OR parse_error = true             # (+ optional parse_error_contains = "substr")
fixture = ["a.txt", "sub/b.log"]    # optional: empty files created under a fresh temp cwd first
skip    = "reason"                   # optional: harness skips with a reason instead of asserting

Adding a new language behavior or fixing a bug? Add or adjust a case here first — it's the sharpest, smallest possible regression test, and it's what the whole team (and CI) trusts as ground truth. Case names must be globally unique across every file; the harness fails loudly on a collision.

Everything else

Most crates carry their own focused unit tests (shoal-leash's policy/enforcement tests, shoal-exec's PTY/signal tests — gracefully skipped when the test runner itself has no tty, shoal-reef's lockfile/provider tests, shoal-adapters' canned-fixture parser tests that need no real binaries or network). cargo fuzz targets exist for the lexer/parser (never panic on arbitrary bytes) and are built — not yet required to pass — on CI's nightly fuzz-build job.

CI

See Ecosystem-and-Tooling §8 for the full breakdown: a real Linux + macOS matrix on every push and PR to main (build, test, conformance corpus, fmt, clippy), a separate release-build gate on both OSes, and a continue-on-error nightly fuzz-build job.

Contributing

  • Read docs/TDD.md and docs/CONTRACTS.md before changing evaluator or wire-protocol behavior — they are decisions, not options.
  • Prefer adding a spec/cases/*.toml case over an ad-hoc unit test for anything about language semantics (parsing, coercion, evaluation, error codes) — it doubles as documentation.
  • Adding a new adapter is the lowest-friction contribution: one TOML file under adapters/, no code changes, verified with a canned-fixture-bytes unit test (see crates/shoal-adapters' existing tests for the pattern) — no live binary or network access required.
  • If you're touching a delegated crate (shoal-exec, shoal-journal, shoal-adapters, shoal-reef, shoal-value/src/methods.rs+render.rs), build to the pinned signatures in docs/CONTRACTS.md and update that file in the same PR if a signature has to change — it's a contract other in-flight work depends on.
  • Never touch the workspace root Cargo.toml casually; add crate-local dependencies with cargo add -p <your-crate> <dep>.
  • License is MIT OR Apache-2.0.

Cross-links

Clone this wiki locally