Skip to content

Ecosystem and Tooling

Allie Coleman edited this page Jul 16, 2026 · 4 revisions

Ecosystem and Tooling

This page covers what turns arbitrary external binaries into first-class, typed shoal citizens (adapters), how tool acquisition works (reef providers — see Reef for the full resolution model), the picker/LSP/plugin ecosystem, and how the project builds and ships itself.

1. Adapters — the T0/T1/T2 tiers

Every command head resolves to one of three fidelity tiers, in order of how much shoal actually knows about the tool:

  • T0 — no adapter, unknown signature. Every word passes through as str, verbatim, always (TDD §4.2.2). Effects are declared Opaque (leash can't reason about what it might do — Leash-and-Security). The outcome's .out is raw bytes with no structured attempt.
  • T1 — PATH-found, adapter-less, sniffed. The binary is found (via reef, Reef), but there's no declarative schema for it. A real subprocess's outcome, with no parsed value preset, does auto-upgrade bracket/brace-shaped stdout to a structured list/record on first .out access — a lazy, best-effort structural guess, never a claim of a promised shape.
  • T2 — a declarative adapter. A TOML file teaches shoal exactly how to invoke a tool (typed params, flags, positionals, an invoke argv template) and how to parse its output into a promised, typed value (output.parse + output.type) — no guessing, no sniffing, a type mismatch degrades to bytes plus a warning rather than a silent lie.
# adapters/git.toml — real, shipped adapter (crates/shoal-adapters + adapters/*.toml)
[cmd.git]
bin = "git"
class = "cli"
ok_codes = [0]
effects = ["fs.read(cwd)"]

[cmd.git.sub.status]
params = { short = "bool", branch = "bool", untracked_files = "str?" }
flags  = { short = { s = "short", b = "branch" } }
invoke = ["status", "--porcelain=v2"]
# `--short`/`-s` remain valid, recognized flags a user can still type, but they
# are "consumed": never forwarded to git's own argv, because git's format
# flags are last-wins and would silently switch git to a shape the
# porcelain-v2 parser can't read. The porcelain-v2 table already carries
# everything --short/--branch would otherwise reveal.
consumed = ["short", "branch"]
output  = { parse = "porcelain-v2", type = "table<{status: str, path: path, orig: path?}>" }
effects = ["fs.read(cwd)"]

[cmd.git.sub.log]
params = { n = "int?", path = "path?", follow = "bool" }
positional = ["path"]
invoke  = ["log", "--pretty=format:%H%x00%an%x00%aI%x00%s", "-z"]
output  = { parse = "z-records", type = "table<{hash: str, author: str, date: datetime, subject: str}>" }
effects = ["fs.read(cwd)"]

[cmd.git.sub.push]
params  = { remote = "str?", refspec = "str?", force = "bool", dry_run = "bool", set_upstream = "bool" }
positional = ["remote", "refspec"]
flags   = { short = { f = "force", u = "set_upstream", n = "dry_run" } }
effects = ["fs.read(cwd)", "net.connect($remote)"]

Note the net.connect($remote) and fs.read($paths) forms above — adapter effects are parametric: the concrete host/path in the actual call substitutes into the effect instance that leash evaluates, so git push origin main derives NetConnect{host:"origin",...}, not a blanket "this tool can touch the network."

Output parser strategies (output.parse): json, ndjson, csv, tsv, lines, kv, z-records (NUL-delimited records, used by git log -z), porcelain-v2 (git's own status format), or none. output.type is the promised shoal type (table<{...}>, list<path>, …) — informational in v1, checked against what the parser actually produced.

The shipped adapter pack (adapters/*.toml)

42 adapters ship under adapters/ as of this writing (ls adapters/ for the current, growing list) — everything below is a representative sample, not the complete set:

Tool Highlights
git status (porcelain-v2 → table, with a semantic .state field), log (z-records → table), branch, push, pull, diff (ok_codes = [0, 1] — a diff found is not a failure), add, commit
cargo build/test/run/check, each declaring fs.write(cwd/target) and either net.connect(crates.io:443) (build) or proc.spawn(...) (test/run)
rg --json invoke template → ndjson parse, ok_codes = [0, 1] (no matches isn't a failure)
jq filter/files params, raw/compact/slurp/null-input flags, output.parse = "json"
curl net.connect($url) + fs.write($output) effects, duration-typed --max-time
docker class = "daemon"; ps/images pin a --format template and consumed = ["quiet"] (same last-wins-flag rationale as git status above) → tsv parse
kubectl get --output=json → record; effects are all net.connect(kubernetes-api)
tar, fd classic flag-heavy CLIs, each with typed path/glob params and structured (lines/tsv) output
du, df, stat tsv-headerless parse (no header line at all, column shape from output.type); du/df's size/used/avail columns are size_kb-typed — the pinned -k/-kP block counts scaled ×1024 into a genuine, comparable Value::Size ((du).where(.size > 1mb) works exactly like ls's builtin .size)
node, python, ruby, deno interpreter-class (docs/IO.md §2) — a raw trailing { … } block is the program, not a -c-quoted string
npm, pnpm, bun, yarn, pip, uv, go, rustup, terraform, helm, gcloud, aws, gh, jj, sqlite3, systemctl, systemd-analyze, ip, ss, ps, bash, brew, podman, unzip, zip, yq the rest of the shipped pack — package managers, cloud CLIs, systemd tooling, and general Unix utilities

Adding a new tool is one TOML file — no shell code, no parser changes, no shoal-eval recompile.

2. Reef — tool acquisition and resolution

Covered fully on Reef. In short: adapters answer "how do I invoke and parse this tool"; reef answers "which binary is git right now, and can I prove it hasn't changed since I locked it." The provider pack (crates/shoal-reef/src/provider/) is real: system (canonical roots + ambient PATH), mise (reads mise's own install tree directly, no shims), cargo (~/.cargo/bin), npm (node_modules/.bin, scoped), venv (.venv/bin).

3. Fuzzy picker

shoal-picker is a crossterm-powered fuzzy finder backing .pick(). Grapheme-aware scoring rewards sequential matches and word boundaries over scattered ones; matches are highlighted in the TUI; both single-item and multi-item (BTreeSet-backed) selection modes exist, and results map directly back into shoal Value lists — no text scraping between the picker and the language.

4. Language Server

shoal-lsp (tower-lsp) gives editors live diagnostics (parse errors, incomplete-syntax notifications), auto-formatting via shoal_syntax::format_program, keyword/lexical completion (scraping in-scope let/var/alias declarations from the open document), and contextual hover markdown for core operators and execution primitives. It's a companion subprocess (shoal lsp), spawned the same way shoal-mcp is — see Kernel-and-Agents §1.

5. WASM plugin host (experimental)

shoal-wasm hosts WASM components (via wasmtime) declared by a manifest (name, version, a component path, and a list of commands). This is a real, working component loader with resource limits (StoreLimits) — but per CONTRACTS.md it is not yet wired into shoal-eval's command dispatch: no in-workspace crate currently depends on it as a library. Treat it as an early foundation for third-party command plugins, not a shipped extension point yet.

6. Configuration

shoal-config merges system (/etc/shoal/shoal.toml), user (~/.config/shoal/shoal.toml), and project (.shoal.toml) configuration. Real, serde-backed sections:

Section Covers
[prompt] prompt template (rendered by the new shoal-prompt crate — a pure, dependency-free domain crate that renders a prompt from a structured context snapshot, no IO)
[history] enabled, max entries, custom path
[render] table width, color
[editor] edit mode, bracketed paste
[kernel] enabled, default session name
[adapters] extra directories to load .toml adapter specs from
[journal] enabled, custom state dir
[leash] path to the policy file (Leash-and-Security)
[init] files to source at startup

7. shoal doctor

shoal doctor [--json] runs a real diagnostic sweep: leash enforcement status (honest, per Leash-and-Security §4), tty/PTY availability, runtime/state/config directory writability, the kernel socket, loaded adapters and each adapter's underlying tool, and the journal — each reported as ok/warn/FAIL with a process exit code (0 clean, 1 warnings, 2 failures) suitable for scripting.

8. CI and releases

CI (.github/workflows/ci.yml) runs on every push/PR to main, on a real Linux + macOS matrix (ubuntu-latest, macos-latest):

  • test: cargo build --workspace --all-targets --locked, cargo test --workspace --locked, then the conformance corpus specifically (cargo test -p shoal --test conformance --locked -- --nocapture) — on both OSes.
  • lint (Linux): cargo fmt --all --check, cargo clippy --workspace --all-targets --locked -- -D warnings.
  • release-build: a release-profile build on both OSes, as a gate independent of the actual release pipeline.
  • fuzz-build (continue-on-error): builds the cargo-fuzz targets on nightly, so lexer/parser fuzzing stays buildable without blocking merges on it.

Releases (.github/workflows/release.yml) trigger on v* tags and build four targets: x86_64-unknown-linux-gnu, aarch64-unknown-linux-gnu, x86_64-apple-darwin, aarch64-apple-darwin — every workspace binary, packaged as a tarball with the README/license, and attached to a GitHub release with auto-generated notes.

Cross-links

  • Reef — the resolution/acquisition half of "run this tool."
  • Leash-and-Security — the effect strings declared throughout the adapter pack above.
  • Developer-Guide — building, testing, and adding your own adapter.