Skip to content
Allie Coleman edited this page Jul 15, 2026 · 2 revisions

Reef

reef is shoal's tool resolver — a content-addressed, scoped, journaled replacement for PATH. It's implemented as its own crate (shoal-reef) with real providers, a real lockfile format, and a real synthesized-PATH view builder.

Status. shoal-reef is built, unit-tested, and wired as the live resolution path — project-scope .reef.toml walking genuinely changes what a bare tool name resolves to (verified directly: a scratch directory with a .reef.toml [tools] entry changes which's reported scope/chain to name that manifest). The conformance corpus (spec/cases/reef.toml) pins the deterministic, host-independent parts (which's arity and structural shape, with reef: scoping, cwd/env purity) and skips only what genuinely needs a real host tool inventory to assert against (drift detection, the live binding table) — those are testing-determinism gaps, not functionality gaps. One real functionality gap remains: bare-path-head ergonomics (./script.py with no run) only routes through reef's [runners] table for .shl; other extensions need the explicit run script.py spelling today (§5).

0. Why PATH had to go

Every tool-version manager that exists — mise, asdf, nvm, rbenv, pyenv, direnv, nix-shell — is a workaround for the same design fossil: PATH is a flat, ordered, mutable string of directories where the first name match wins. That one fact produces the daily misery of the terminal: version selection is ambient global mutation inherited invisibly at fork; "which binary am I actually running" needs which -a/type/shim forensics; project-locality gets reinvented per tool (.nvmrc, .tool-versions, mise.toml, .envrc), each with its own directory walk and its own prompt-hook activation race; shims add a fork to every invocation and lie to which; and any writable directory on PATH is an injection vector.

A shell that already owns spawn (no execvp, no PATH search — shoal's exec resolves explicitly), already journals everything, and already pins binaries by content hash for its sandbox (Leash-and-Security) can delete the fossil instead of shimming it. Resolution becomes a pure, declarative, journaled function; PATH becomes an output synthesized for legacy children, never an input the shell lives in.

1. The resolution model

A command head X resolves through scopes, not directories, after the language-level resolution chain (session fn/alias first, TDD §3.1.3):

session fn/alias  →  adapter bin pin  →  project reef  →  user reef  →  system reef  →  ambient

Each scope is a manifest mapping tool names to constraints or explicit providers:

# .reef.toml at a project root (nearest wins) — or [reef] in ~/.config/shoal/shoal.toml for user scope
[tools]
node   = "22"                      # semver-ish constraint
python = "3.12"
rg     = "*"                       # any; locked on first resolve
go     = { provider = "mise" }     # force a provider

[runners]                          # content-type resolution, keyed on extension — §5
py  = "python"
js  = "node"
ts  = { tool = "deno", args = ["run"] }
sh  = "sh"
shl = "self"

[options]
hermetic = false                   # child PATH: synthesized-only (true), or +system tail (false)
  • Project scope: nearest .reef.toml walking up from cwd. A pure function of (cwd, manifest files) — no activation, no hooks, no env mutation, ever. cd re-scopes the next resolution; nothing else changes.
  • User scope: [reef] in ~/.config/shoal/shoal.toml.
  • System reef: a discovered inventory of /usr/bin etc. Ambient PATH survives only as the final fallback and is reported as scope ambient in diagnostics — never silently preferred.
  • Foreign manifests: the mise provider reads mise.toml/.tool-versions as read-only manifests, so existing repos work day one without adopting .reef.toml. mise is one provider among several, not baked in — delete it and nothing else changes.
# spec/cases/reef.toml — dynamic scoping mirrors with cwd:/with env:
with reef: {node: "20"} { }                              # -> null (pure no-op body)
with reef: {"fake-tool-9f3a": "1"} { ^fake-tool-9f3a }     # -> reef_unlocked (constrained but not locked)

2. The lock — resolution you can trust

reef.lock (TOML, committed) records every resolved tool: name, exact version, provider, absolute path, blake3 of the binary, and a resolved-at timestamp:

// crates/shoal-reef/src/lock.rs — real, shipped shape
pub struct LockEntry {
    pub name: String,
    pub version: String,
    pub provider: String,
    pub path: PathBuf,
    pub blake3: String,       // hex of the binary at lock time
    pub resolved_at: String,  // RFC3339 UTC
}
  • Interactive sessions auto-lock on first resolve, with a one-line notice and a journal entry. Scripts/CI error on unlocked constraints (reef_unlocked) — a deterministic artifact doesn't get to guess, mirroring the same interactive/script split TDD already applies to it and watchdogs.
  • At spawn, the on-disk binary is re-hashed (cached by dev/inode/mtime): a mismatch against the lock is reef_drift — a hard error naming old and new hashes, fix-it reef lock --refresh. This drift check is real and independent of anything below.
  • Corrected from a previous revision of this page. The natural next link — leash's proc.spawn{bin_hash} grant pinning against this same lock, so policy and resolution verify one identical name → version → hash → grant chain — is not wired today, verified directly against source: crates/shoal-eval/src/plan_derive.rs always constructs Effect::ProcSpawn with an empty bin_hash, and the real spawn path never calls shoal-leash's effect evaluator with a real one. See Leash-and-Security §4 ("Spawn hash pinning") for the exact, verified state — the function that would do this check exists and is unit-tested in isolation, it's just not called from a live spawn yet. Treat "leash pins spawns against reef's locked hash" as a docs/ROADMAP.md item, not a shipped guarantee.
  • Version conflicts across scopes are surfaced as values naming both sources — never silent first-wins.

3. Providers — acquisition as adapters

pub trait Provider: Send + Sync {
    fn name(&self) -> &'static str;
    fn discover(&self, tool: &str) -> Vec<Candidate>;                       // fast, cached
    fn fetch(&self, tool: &str, req: &Constraint) -> Option<Result<Candidate, ProviderError>>;
}

Shipped providers (crates/shoal-reef/src/provider/): system (scans canonical roots + ambient PATH, versions via a cached --version probe), mise (reads ~/.local/share/mise/installs/** directly — no shims, no mise exec, no extra forks), cargo (~/.cargo/bin), npm (node_modules/.bin as a declared, scoped provider — the exact thing everyone hand-hacks PATH for today), and venv (.venv/bin when present). Providers only enumerate; only an explicit reef fetch <tool> may install, and it delegates to the provider's own machinery — reef never installs implicitly.

4. Legacy children — PATH as an output

Children that themselves search PATH (a build script spawning cc, npm spawning node) still need a coherent world. At spawn, exec builds a synthesized PATH: a per-resolution-context view directory ($XDG_RUNTIME_DIR/shoal/views/<hash-of-binding-set>/bin, symlinks to the locked binaries) plus the system tail, unless hermetic = true drops the tail entirely. Views are content-addressed and reused — building one costs O(bindings). The shell never consults PATH for its own resolution; it only ever emits one, for guests. leash's strongest tier can force hermetic per grant.

5. Runners — the poly-runner question, dissolved

"How do I run ./script.py" is the same question as "what does node denote" — resolution keyed on content-type instead of name, so it lives in reef, not the language core. The [runners] table maps an extension (falling back to shebang sniffing) to a tool + argv template. The explicit spelling, run ./x.py args / run x.py args, resolves the runner, resolves its tool through reef, and spawns — journaled identically to any other spawn, verified directly against the binary. Defaults ship for py js ts sh shl rb lua; rs intentionally has no default (compile-vs-script ambiguity is a real design decision, not an oversight) — one TOML line configures it. .shl is special: its runner is the literal "self" — run in a child evaluator of the current shoal process, not a subprocess; no binary is spawned and no reef resolution happens for the interpreter itself.

Not yet wired: the bare-path-head ergonomics case — ./script.py args with no leading run — for anything other than .shl. Verified against the binary: a bare ./script.py at command head position currently attempts to exec the file directly (permission error if it isn't itself marked executable) rather than consulting [runners]; only .shl bare-path heads work today, via the separate, pre-existing "self" child-evaluator path above. Use the explicit run spelling for other extensions until this generalizes.

6. Surface

  • which node → a record {name, scope, constraint, version, path, hash, provider, chain} — the full resolution chain, as a value, not forensics. which -a-style: which node --all.
  • reef → the current binding table (name, constraint, version, hash8, provider, scope).
  • reef add node@22 (writes the nearest manifest and locks it), reef lock [--refresh], reef fetch <tool>, reef doctor (drift, orphans, shadowed ambient tools).
  • with reef: {node: "20"} { … } — dynamic scoping, restore-on-exit, exactly like with cwd:/ with env: (Execution-Model §6).
  • Command-not-found did-you-mean gains reef-aware hints: "node is constrained (22) but not installed — reef fetch node" and "found in ambient PATH but shadowed by project reef".
  • Every journal entry records the resolution-context hash; every spawn records the (tool, version, bin hash) actually used — "which node built this artifact three weeks ago" is a journal.query, not forensics.
# spec/cases/reef.toml — host-independent field is guaranteed even before full integration lands
(which "sh").name                                            # -> "sh"  (always echoes the query)
(which "definitely-not-a-real-binary-xyz-9f3a").out           # -> null
which "a" "b"                                                  # -> arg_error  (which takes one name)

7. Error codes

reef_unlocked reef_drift reef_conflict reef_not_found reef_provider (CONTRACTS §4).

8. Non-goals (v1)

reef is not a package manager (fetch delegates entirely to providers' own machinery), not an env manager (env stays session state — reef only synthesizes the child PATH), and ships no shims, no hooks, no prompt integration, and no implicit installs. Windows resolution semantics are deferred with the rest of the Windows port.

Cross-links