Repository navigation
Reef
reef is shoal's tool resolver — a content-addressed, scoped, journaled replacement for PATH.
It's implemented as its own crate (shoal-reef) with real providers, a real lockfile format, and a
real synthesized-PATH view builder.
Status.
shoal-reefis built, unit-tested, and wired as the live resolution path — project-scope.reef.tomlwalking genuinely changes what a bare tool name resolves to (verified directly: a scratch directory with a.reef.toml[tools]entry changeswhich's reportedscope/chainto name that manifest). The conformance corpus (spec/cases/reef.toml) pins the deterministic, host-independent parts (which's arity and structural shape,with reef:scoping,cwd/envpurity) andskips only what genuinely needs a real host tool inventory to assert against (drift detection, the live binding table) — those are testing-determinism gaps, not functionality gaps. One real functionality gap remains: bare-path-head ergonomics (./script.pywith norun) only routes through reef's[runners]table for.shl; other extensions need the explicitrun script.pyspelling today (§5).
Every tool-version manager that exists — mise, asdf, nvm, rbenv, pyenv, direnv, nix-shell — is a
workaround for the same design fossil: PATH is a flat, ordered, mutable string of directories
where the first name match wins. That one fact produces the daily misery of the terminal: version
selection is ambient global mutation inherited invisibly at fork; "which binary am I actually
running" needs which -a/type/shim forensics; project-locality gets reinvented per tool
(.nvmrc, .tool-versions, mise.toml, .envrc), each with its own directory walk and its own
prompt-hook activation race; shims add a fork to every invocation and lie to which; and any
writable directory on PATH is an injection vector.
A shell that already owns spawn (no execvp, no PATH search — shoal's exec resolves explicitly),
already journals everything, and already pins binaries by content hash for its sandbox
(Leash-and-Security) can delete the fossil instead of shimming it. Resolution becomes a pure,
declarative, journaled function; PATH becomes an output synthesized for legacy children, never
an input the shell lives in.
A command head X resolves through scopes, not directories, after the language-level
resolution chain (session fn/alias first, TDD §3.1.3):
session fn/alias → adapter bin pin → project reef → user reef → system reef → ambient
Each scope is a manifest mapping tool names to constraints or explicit providers:
# .reef.toml at a project root (nearest wins) — or [reef] in ~/.config/shoal/shoal.toml for user scope
[tools]
node = "22" # semver-ish constraint
python = "3.12"
rg = "*" # any; locked on first resolve
go = { provider = "mise" } # force a provider
[runners] # content-type resolution, keyed on extension — §5
py = "python"
js = "node"
ts = { tool = "deno", args = ["run"] }
sh = "sh"
shl = "self"
[options]
hermetic = false # child PATH: synthesized-only (true), or +system tail (false)-
Project scope: nearest
.reef.tomlwalking up fromcwd. A pure function of(cwd, manifest files)— no activation, no hooks, no env mutation, ever.cdre-scopes the next resolution; nothing else changes. -
User scope:
[reef]in~/.config/shoal/shoal.toml. -
System reef: a discovered inventory of
/usr/binetc. AmbientPATHsurvives only as the final fallback and is reported as scopeambientin diagnostics — never silently preferred. -
Foreign manifests: the
miseprovider readsmise.toml/.tool-versionsas read-only manifests, so existing repos work day one without adopting.reef.toml.miseis one provider among several, not baked in — delete it and nothing else changes.
# spec/cases/reef.toml — dynamic scoping mirrors with cwd:/with env:
with reef: {node: "20"} { } # -> null (pure no-op body)
with reef: {"fake-tool-9f3a": "1"} { ^fake-tool-9f3a } # -> reef_unlocked (constrained but not locked)
reef.lock (TOML, committed) records every resolved tool: name, exact version, provider, absolute
path, blake3 of the binary, and a resolved-at timestamp:
// crates/shoal-reef/src/lock.rs — real, shipped shape
pub struct LockEntry {
pub name: String,
pub version: String,
pub provider: String,
pub path: PathBuf,
pub blake3: String, // hex of the binary at lock time
pub resolved_at: String, // RFC3339 UTC
}-
Interactive sessions auto-lock on first resolve, with a one-line notice and a journal entry.
Scripts/CI error on unlocked constraints (
reef_unlocked) — a deterministic artifact doesn't get to guess, mirroring the same interactive/script split TDD already applies toitand watchdogs. - At spawn, the on-disk binary is re-hashed (cached by dev/inode/mtime): a mismatch against the
lock is
reef_drift— a hard error naming old and new hashes, fix-itreef lock --refresh. This drift check is real and independent of anything below. -
Corrected from a previous revision of this page. The natural next link — leash's
proc.spawn{bin_hash}grant pinning against this same lock, so policy and resolution verify one identical name → version → hash → grant chain — is not wired today, verified directly against source:crates/shoal-eval/src/plan_derive.rsalways constructsEffect::ProcSpawnwith an emptybin_hash, and the real spawn path never callsshoal-leash's effect evaluator with a real one. See Leash-and-Security §4 ("Spawn hash pinning") for the exact, verified state — the function that would do this check exists and is unit-tested in isolation, it's just not called from a live spawn yet. Treat "leash pins spawns against reef's locked hash" as adocs/ROADMAP.mditem, not a shipped guarantee. - Version conflicts across scopes are surfaced as values naming both sources — never silent first-wins.
pub trait Provider: Send + Sync {
fn name(&self) -> &'static str;
fn discover(&self, tool: &str) -> Vec<Candidate>; // fast, cached
fn fetch(&self, tool: &str, req: &Constraint) -> Option<Result<Candidate, ProviderError>>;
}Shipped providers (crates/shoal-reef/src/provider/): system (scans canonical roots +
ambient PATH, versions via a cached --version probe), mise (reads
~/.local/share/mise/installs/** directly — no shims, no mise exec, no extra forks),
cargo (~/.cargo/bin), npm (node_modules/.bin as a declared, scoped provider — the
exact thing everyone hand-hacks PATH for today), and venv (.venv/bin when present).
Providers only enumerate; only an explicit reef fetch <tool> may install, and it delegates to
the provider's own machinery — reef never installs implicitly.
Children that themselves search PATH (a build script spawning cc, npm spawning node) still
need a coherent world. At spawn, exec builds a synthesized PATH: a per-resolution-context view
directory ($XDG_RUNTIME_DIR/shoal/views/<hash-of-binding-set>/bin, symlinks to the locked
binaries) plus the system tail, unless hermetic = true drops the tail entirely. Views are
content-addressed and reused — building one costs O(bindings). The shell never consults PATH
for its own resolution; it only ever emits one, for guests. leash's strongest tier can force
hermetic per grant.
"How do I run ./script.py" is the same question as "what does node denote" — resolution keyed
on content-type instead of name, so it lives in reef, not the language core. The [runners] table
maps an extension (falling back to shebang sniffing) to a tool + argv template. The explicit
spelling, run ./x.py args / run x.py args, resolves the runner, resolves its tool through
reef, and spawns — journaled identically to any other spawn, verified directly against the binary.
Defaults ship for py js ts sh shl rb lua; rs intentionally has no default (compile-vs-script
ambiguity is a real design decision, not an oversight) — one TOML line configures it. .shl is
special: its runner is the literal "self" — run in a child evaluator of the current shoal
process, not a subprocess; no binary is spawned and no reef resolution happens for the interpreter
itself.
Not yet wired: the bare-path-head ergonomics case — ./script.py args with no leading run —
for anything other than .shl. Verified against the binary: a bare ./script.py at command
head position currently attempts to exec the file directly (permission error if it isn't itself
marked executable) rather than consulting [runners]; only .shl bare-path heads work today,
via the separate, pre-existing "self" child-evaluator path above. Use the explicit run spelling
for other extensions until this generalizes.
-
which node→ a record{name, scope, constraint, version, path, hash, provider, chain}— the full resolution chain, as a value, not forensics.which -a-style:which node --all. -
reef→ the current binding table (name, constraint, version, hash8, provider, scope). -
reef add node@22(writes the nearest manifest and locks it),reef lock [--refresh],reef fetch <tool>,reef doctor(drift, orphans, shadowed ambient tools). -
with reef: {node: "20"} { … }— dynamic scoping, restore-on-exit, exactly likewith cwd:/with env:(Execution-Model §6). - Command-not-found did-you-mean gains reef-aware hints: "
nodeis constrained (22) but not installed —reef fetch node" and "found in ambient PATH but shadowed by project reef". - Every journal entry records the resolution-context hash; every spawn records the (tool, version,
bin hash) actually used — "which node built this artifact three weeks ago" is a
journal.query, not forensics.
# spec/cases/reef.toml — host-independent field is guaranteed even before full integration lands
(which "sh").name # -> "sh" (always echoes the query)
(which "definitely-not-a-real-binary-xyz-9f3a").out # -> null
which "a" "b" # -> arg_error (which takes one name)
reef_unlocked reef_drift reef_conflict reef_not_found reef_provider (CONTRACTS §4).
reef is not a package manager (fetch delegates entirely to providers' own machinery), not
an env manager (env stays session state — reef only synthesizes the child PATH), and ships no
shims, no hooks, no prompt integration, and no implicit installs. Windows resolution semantics are
deferred with the rest of the Windows port.
- Language-Reference §2/§3.1 — how a bare word becomes a command head in the first place.
-
Execution-Model §6 —
withscoping shared acrosscwd/env/reef. - Leash-and-Security — spawn hash pinning consumes reef's lockfile hashes directly.
-
docs/REEF.md— the full design contract this page summarizes.