Repository navigation
Ecosystem and Tooling
This page covers what turns arbitrary external binaries into first-class, typed shoal citizens (adapters), how tool acquisition works (reef providers — see Reef for the full resolution model), the picker/LSP/plugin ecosystem, and how the project builds and ships itself.
Every command head resolves to one of three fidelity tiers, in order of how much shoal actually knows about the tool:
-
T0 — no adapter, unknown signature. Every word passes through as
str, verbatim, always (TDD §4.2.2). Effects are declaredOpaque(leash can't reason about what it might do — Leash-and-Security). The outcome's.outis raw bytes with no structured attempt. -
T1 — PATH-found, adapter-less, sniffed. The binary is found (via reef, Reef), but there's
no declarative schema for it. A real subprocess's outcome, with no
parsedvalue preset, does auto-upgrade bracket/brace-shaped stdout to a structured list/record on first.outaccess — a lazy, best-effort structural guess, never a claim of a promised shape. -
T2 — a declarative adapter. A TOML file teaches shoal exactly how to invoke a tool (typed
params, flags, positionals, an
invokeargv template) and how to parse its output into a promised, typed value (output.parse+output.type) — no guessing, no sniffing, a type mismatch degrades to bytes plus a warning rather than a silent lie.
# adapters/git.toml — real, shipped adapter (crates/shoal-adapters + adapters/*.toml)
[cmd.git]
bin = "git"
class = "cli"
ok_codes = [0]
effects = ["fs.read(cwd)"]
[cmd.git.sub.status]
params = { short = "bool", branch = "bool", untracked_files = "str?" }
flags = { short = { s = "short", b = "branch" } }
invoke = ["status", "--porcelain=v2"]
# `--short`/`-s` remain valid, recognized flags a user can still type, but they
# are "consumed": never forwarded to git's own argv, because git's format
# flags are last-wins and would silently switch git to a shape the
# porcelain-v2 parser can't read. The porcelain-v2 table already carries
# everything --short/--branch would otherwise reveal.
consumed = ["short", "branch"]
output = { parse = "porcelain-v2", type = "table<{status: str, path: path, orig: path?}>" }
effects = ["fs.read(cwd)"]
[cmd.git.sub.log]
params = { n = "int?", path = "path?", follow = "bool" }
positional = ["path"]
invoke = ["log", "--pretty=format:%H%x00%an%x00%aI%x00%s", "-z"]
output = { parse = "z-records", type = "table<{hash: str, author: str, date: datetime, subject: str}>" }
effects = ["fs.read(cwd)"]
[cmd.git.sub.push]
params = { remote = "str?", refspec = "str?", force = "bool", dry_run = "bool", set_upstream = "bool" }
positional = ["remote", "refspec"]
flags = { short = { f = "force", u = "set_upstream", n = "dry_run" } }
effects = ["fs.read(cwd)", "net.connect($remote)"]Note the net.connect($remote) and fs.read($paths) forms above — adapter effects are
parametric: the concrete host/path in the actual call substitutes into the effect instance
that leash evaluates, so git push origin main derives NetConnect{host:"origin",...}, not a
blanket "this tool can touch the network."
Output parser strategies (output.parse): json, ndjson, csv, tsv, lines, kv,
z-records (NUL-delimited records, used by git log -z), porcelain-v2 (git's own status
format), or none. output.type is the promised shoal type (table<{...}>, list<path>, …) —
informational in v1, checked against what the parser actually produced.
42 adapters ship under adapters/ as of this writing (ls adapters/ for the current, growing
list) — everything below is a representative sample, not the complete set:
| Tool | Highlights |
|---|---|
git |
status (porcelain-v2 → table, with a semantic .state field), log (z-records → table), branch, push, pull, diff (ok_codes = [0, 1] — a diff found is not a failure), add, commit
|
cargo |
build/test/run/check, each declaring fs.write(cwd/target) and either net.connect(crates.io:443) (build) or proc.spawn(...) (test/run) |
rg |
--json invoke template → ndjson parse, ok_codes = [0, 1] (no matches isn't a failure) |
jq |
filter/files params, raw/compact/slurp/null-input flags, output.parse = "json"
|
curl |
net.connect($url) + fs.write($output) effects, duration-typed --max-time
|
docker |
class = "daemon"; ps/images pin a --format template and consumed = ["quiet"] (same last-wins-flag rationale as git status above) → tsv parse |
kubectl |
get --output=json → record; effects are all net.connect(kubernetes-api)
|
tar, fd
|
classic flag-heavy CLIs, each with typed path/glob params and structured (lines/tsv) output |
du, df, stat
|
tsv-headerless parse (no header line at all, column shape from output.type); du/df's size/used/avail columns are size_kb-typed — the pinned -k/-kP block counts scaled ×1024 into a genuine, comparable Value::Size ((du).where(.size > 1mb) works exactly like ls's builtin .size) |
node, python, ruby, deno
|
interpreter-class (docs/IO.md §2) — a raw trailing { … } block is the program, not a -c-quoted string |
npm, pnpm, bun, yarn, pip, uv, go, rustup, terraform, helm, gcloud, aws, gh, jj, sqlite3, systemctl, systemd-analyze, ip, ss, ps, bash, brew, podman, unzip, zip, yq
|
the rest of the shipped pack — package managers, cloud CLIs, systemd tooling, and general Unix utilities |
Adding a new tool is one TOML file — no shell code, no parser changes, no shoal-eval recompile.
Covered fully on Reef. In short: adapters answer "how do I invoke and parse this tool";
reef answers "which binary is git right now, and can I prove it hasn't changed since I
locked it." The provider pack (crates/shoal-reef/src/provider/) is real: system (canonical
roots + ambient PATH), mise (reads mise's own install tree directly, no shims), cargo
(~/.cargo/bin), npm (node_modules/.bin, scoped), venv (.venv/bin).
shoal-picker is a crossterm-powered fuzzy finder backing .pick(). Grapheme-aware scoring
rewards sequential matches and word boundaries over scattered ones; matches are highlighted in the
TUI; both single-item and multi-item (BTreeSet-backed) selection modes exist, and results map
directly back into shoal Value lists — no text scraping between the picker and the language.
shoal-lsp (tower-lsp) gives editors live diagnostics (parse errors, incomplete-syntax
notifications), auto-formatting via shoal_syntax::format_program, keyword/lexical completion
(scraping in-scope let/var/alias declarations from the open document), and contextual hover
markdown for core operators and execution primitives. It's a companion subprocess
(shoal lsp), spawned the same way shoal-mcp is — see Kernel-and-Agents §1.
shoal-wasm hosts WASM components (via wasmtime) declared by a manifest (name, version,
a component path, and a list of commands). This is a real, working component loader with
resource limits (StoreLimits) — but per CONTRACTS.md it is not yet wired into
shoal-eval's command dispatch: no in-workspace crate currently depends on it as a library.
Treat it as an early foundation for third-party command plugins, not a shipped extension point yet.
shoal-config merges system (/etc/shoal/shoal.toml), user (~/.config/shoal/shoal.toml), and
project (.shoal.toml) configuration. Real, serde-backed sections:
| Section | Covers |
|---|---|
[prompt] |
prompt template (rendered by the new shoal-prompt crate — a pure, dependency-free domain crate that renders a prompt from a structured context snapshot, no IO) |
[history] |
enabled, max entries, custom path |
[render] |
table width, color |
[editor] |
edit mode, bracketed paste |
[kernel] |
enabled, default session name |
[adapters] |
extra directories to load .toml adapter specs from |
[journal] |
enabled, custom state dir |
[leash] |
path to the policy file (Leash-and-Security) |
[init] |
files to source at startup |
shoal doctor [--json] runs a real diagnostic sweep: leash enforcement status (honest, per
Leash-and-Security §4), tty/PTY availability, runtime/state/config directory writability, the
kernel socket, loaded adapters and each adapter's underlying tool, and the journal — each reported
as ok/warn/FAIL with a process exit code (0 clean, 1 warnings, 2 failures) suitable for
scripting.
CI (.github/workflows/ci.yml) runs on every push/PR to main, on a real
Linux + macOS matrix (ubuntu-latest, macos-latest):
-
test:cargo build --workspace --all-targets --locked,cargo test --workspace --locked, then the conformance corpus specifically (cargo test -p shoal --test conformance --locked -- --nocapture) — on both OSes. -
lint(Linux):cargo fmt --all --check,cargo clippy --workspace --all-targets --locked -- -D warnings. -
release-build: a release-profile build on both OSes, as a gate independent of the actual release pipeline. -
fuzz-build(continue-on-error): builds thecargo-fuzztargets on nightly, so lexer/parser fuzzing stays buildable without blocking merges on it.
Releases (.github/workflows/release.yml) trigger on v* tags and build four targets:
x86_64-unknown-linux-gnu, aarch64-unknown-linux-gnu, x86_64-apple-darwin,
aarch64-apple-darwin — every workspace binary, packaged as a tarball with the README/license, and
attached to a GitHub release with auto-generated notes.
- Reef — the resolution/acquisition half of "run this tool."
- Leash-and-Security — the effect strings declared throughout the adapter pack above.
- Developer-Guide — building, testing, and adding your own adapter.