diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index 3cdffcba..d79df1b2 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -255,6 +255,103 @@ jobs: --component-dir smoke-test/agent/components \ --raw + # The host document-filesystem contract, on every target a release ships. + # + # `test-deno`, `test-node`, and `test-bun` run the whole corpus on Linux x64 + # and prove the contract holds there. What they cannot prove is that it holds + # on the other four triples: the host adapter's containment is path arithmetic + # plus `realpath`, and both are the platform's — Windows has drive letters, + # UNC paths, junctions, and reparse points that POSIX does not, and the two + # macOS rows resolve `/var` through a symlink that a naive comparison reads as + # an escape. + # + # So this row is focused rather than exhaustive: one suite, plus a compiled + # probe, on each of the five. The compiled probe is the second half of the + # claim — the shipped artifact is a binary, and the adapter reaches + # `node:path`, `node:fs`, and `node:os` through whatever `deno compile` put in + # its graph. + filesystem-contract: + strategy: + fail-fast: false + matrix: + include: + - runner: macos-15 + target: aarch64-apple-darwin + - runner: macos-15-intel + target: x86_64-apple-darwin + - runner: ubuntu-24.04 + target: x86_64-unknown-linux-gnu + - runner: ubuntu-24.04-arm + target: aarch64-unknown-linux-gnu + - runner: windows-2025 + target: x86_64-pc-windows-msvc + runs-on: ${{ matrix.runner }} + defaults: + run: + shell: bash + steps: + - uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6 + + - uses: denoland/setup-deno@e95548e56dfa95d4e1a28d6f422fafe75c4c26fb # v2.0.3 + with: + deno-version: v2.9.5 + + - uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020 # v4 + with: + node-version: "22" + + - uses: pnpm/action-setup@b906affcce14559ad1aafd4ab0e942779e9f58b1 # v4 + + - uses: oven-sh/setup-bun@0c5077e51419868618aeaa5fe8019c62421857d6 # v2 + with: + bun-version: 1.3.14 + + # `deno install` rather than `deno task deps`: the task also caches the + # graphs a browser build and a release compile walk, and it reaches them + # by spawning a child — which does not survive the Windows runner's path + # handling. Nothing here builds the bundle or compiles the CLI, so the + # plain frozen install is the whole preparation this job needs. + - name: Install dependencies + run: deno install --frozen + + # The compile below runs under `--node-modules-dir=none`, which resolves + # npm packages from the Deno cache rather than from `node_modules`. This + # caches the probe's graph in that mode without touching the layout the + # step above just created. + - name: Cache the probe's graph for a compile + run: > + deno install --entrypoint --node-modules-dir=none --frozen + scripts/files-contract-probe.ts + + # Deno first, and the compile with it: `pnpm install` and `bun install` + # each rewrite `node_modules` into their own layout, so a Deno step after + # one of them resolves through links the other pruned (#279). + - name: Host contract under Deno + run: deno test --allow-all --frozen packages/runtime/tests/host-files.test.ts + + - name: Host contract as a compiled binary + run: | + set -eu + deno compile --node-modules-dir=none --cached-only --frozen --allow-all \ + --output dist/files-contract-probe scripts/files-contract-probe.ts + if [ -f dist/files-contract-probe.exe ]; then + ./dist/files-contract-probe.exe + else + ./dist/files-contract-probe + fi + + - name: Install the Node layout + run: pnpm install + + - name: Host contract under Node + run: pnpm exec tsx --tsconfig tsconfig.node.json --test packages/runtime/tests/host-files.test.ts + + - name: Install the Bun layout + run: bun install + + - name: Host contract under Bun + run: bun test --timeout=300000 packages/runtime/tests/host-files.test.ts + # The same chain `deno task verify:clean` runs locally. It is the regression # for #279's ownership claim: a build that installs anything moves the # prepared-state fingerprint, prunes pnpm's links, and fails the resolution @@ -384,7 +481,18 @@ jobs: run: bun run packages/cli/src/bun.ts test smoke-test/test-agent/README.md --raw green: - needs: [lint, test-deno, jsr, smoke, composability, site, test-node, test-bun] + needs: + [ + lint, + test-deno, + jsr, + smoke, + filesystem-contract, + composability, + site, + test-node, + test-bun, + ] if: always() runs-on: ubuntu-latest steps: diff --git a/architecture.md b/architecture.md index f8d18266..8d921091 100644 --- a/architecture.md +++ b/architecture.md @@ -42,6 +42,10 @@ Existing documents and code get aligned to this section retroactively. | blocker | execution needs input (auth, a human answer); not a failure | | suspension | a durable wait: a crash restarts into the same wait | | Workspace | the provider-neutral, run-owned environment that supplies retained filesystem, repository, process and working-directory capabilities to a workflow | +| document filesystem | the files a document names in its own text, reached only through `API.Files`; distinct from the host paths the engine's own control plane reads | +| Files provider | the installed implementation of `API.Files` for a document execution: the host provider under `xmd run`, a transaction-bound provider under a workflow run. There is no default, and no provider falls back to another | +| stable host namespace | the condition the host Files provider's containment is stated under: no other process replaces a directory, symlink, junction or reparse point between the moment a path is observed and the moment it is used | +| Files infrastructure failure | a Files provider that is absent, that refuses an operation, or that broke its own contract; fatal like a durability failure, and never a printed error | | ephemeral | a replay classification for an operation, context or attachment that runs again to reconstruct live execution; its result is not substituted from the journal and it owns no durable workflow state | | live binding | an execution-owned value reconstructed ephemerally for the current document execution; it is visible only to constructs that explicitly consume the live binding overlay and never enters interpolation or the journal | | attached service | a scoped host process that publishes its authenticated loopback endpoint through the XMD service handshake protocol and remains supervised for the lifetime of its service attachment | @@ -688,7 +692,7 @@ the process may terminate, and a later document execution arrives back at the same wait. An error that reveals a blocker (an expired login) reaches suspension through middleware; waiting itself is never raised. -### 8. Durability failures are outside the model +### 8. Durability and Files infrastructure failures are outside the model A durability failure (§6.11) says the journal no longer describes the document execution. No middleware sees it; it is never the document's own outcome. A @@ -705,6 +709,69 @@ throws. A pre-persistence policy rejection remains the policy's ordinary document failure; the guarded stream marks that boundary before the backing append and does not activate the fail-stop state. +A Files infrastructure failure is outside the model on the same terms. A +missing provider, a refused operation, and a provider that broke its own +contract are none of them things the document did or can act on, and printing +one would let every step after the file work run as though the file work had +happened. No middleware converts one, and no printing boundary prints one. + +Both are discovered through one cycle-safe traversal of the whole cause graph, +and precedence is decided by kind rather than by position: a durability failure +first, then a Files infrastructure failure, then a documentation failure. The +selected failure comes back by identity, because a fail-stop that records "the +first error" has to record the one that happened. + +The two kinds are *recognized* by different mechanisms, and the difference is +deliberate. A durability failure is recognized exactly as it always has been — +by class, against the failure types durable-streams exports — and that is +unchanged. A **Files** infrastructure failure is recognized structurally: a +stable tag on frozen data, the fixed diagnostic for its kind, and no cause, +never `instanceof`. It has to be, because the Files boundary is the one a +separately loaded copy of a package reaches across, where `instanceof` answers +false. Recognizing one is also a decision to let that exact object travel +onward, so a candidate carrying anything else — a raw message, a cause chain, +an extra field — fails the contract and is replaced by a fresh invariant rather +than preserved. + +The traversal reads values it did not create, so every read in it is total: a +thrown Proxy or a failing accessor narrows what discovery finds instead of +replacing the failure being classified. + +## The document filesystem boundary + +Contextual routing is not authority. `API.Files` decides *which* provider +answers a document's file operations; what a provider is allowed to do is +decided by the provider, from identities the contextual layer cannot supply. + +The Api's operations are whole semantic acts — read this path, replace this +path, list what these patterns select — rather than steps a caller sequences. +The one preliminary operation, `checkFilePath`, is deliberately weak: pure path +arithmetic, no filesystem access, and it returns nothing usable. `` calls +it to decide whether a write's children may expand, and the later write repeats +the same admission from the same authored path. Nothing is handed between them, +so a check that was skipped or answered elsewhere authorizes nothing. + +The two providers make different containment claims, and both are stated rather +than implied: + +- **`xmd run`** resolves document paths in the caller's own filesystem. It + refuses empty, absolute, and lexically escaping paths without touching the + filesystem, and refuses an observed outward symlink once resolution can see + one. That holds while the host namespace is stable; it is not a sandbox, and + closing the replacement window would require a native dependency this project + does not take. +- **A workflow run** resolves document paths in the run-owned Workspace's + logical filesystem, inside the caller-owned transaction. A document path + never becomes a host path, so there is no host namespace to replace. + +Neither claim covers a native command a document runs. + +Failure data crosses the boundary as a plain frozen object under a stable tag, +carrying a reason from a fixed vocabulary and the phase it came from. No +message, errno code, resolved path, temporary name, or symlink target crosses. +Consumers parse that data before reading a field; a write whose data does not +validate is a provider-contract failure rather than a commit state to invent. + ## Attempts - The journal records every attempt in full; replay restores the outcome @@ -831,6 +898,9 @@ Status is measured against main. | Workspace coordination API | fails closed by default and lets a Workspace operation explicitly select provider coordination | built on the #365 stack; the atomic Deno Workspace handler is unbuilt | | explicit WorkflowRun journal route | binds one already-filtered publication to one exact active transaction and otherwise uses ordinary serialized journal storage | built on the #365 stack | | `API.Service` / `startService()` | creates an authenticated, supervised loopback service attachment through a provider-neutral operation | built on main | +| `API.Files` | routes every document filesystem operation to the installed provider, with no host default and structural failure data | built on the #227 stack | +| host Files provider / `useHostFiles()` | resolves document paths in the caller's filesystem, containing them while the host namespace is stable; installed by all four CLI entrypoints | built on the #227 stack | +| transaction-bound Files provider | resolves document paths in the run-owned logical Workspace inside the caller-owned transaction | unbuilt; the adapter is #227's second layer, and workflow effect coordination and CLI reachability remain unbuilt | | `service=` | publishes the attachment's endpoint into the live binding overlay for its invocation | built on main | | `ephemeral eval` | reconstructs live middleware and bindings without a journal entry | built on main | | `useWorkflowServiceDenial()` | provides and tests a non-delegating workflow service denial provider; #366 will install it in future start and resume scopes | built on main; no workflow CLI execution branch exists yet | diff --git a/packages/cli/src/bun.ts b/packages/cli/src/bun.ts index 5a72f684..6f8ba15f 100644 --- a/packages/cli/src/bun.ts +++ b/packages/cli/src/bun.ts @@ -8,7 +8,7 @@ import { main } from "effection"; import { fileURLToPath } from "node:url"; import process from "node:process"; -import { API } from "@executablemd/runtime"; +import { API, useHostFiles } from "@executablemd/runtime"; import { compileDataUri } from "@executablemd/core"; import { runXmd } from "./cli.ts"; import { useBunService } from "./bun-service.ts"; @@ -30,5 +30,10 @@ await main(function* (args) { }, { at: "min" }, ); + // Document filesystem access resolves in the caller's own filesystem here. + // It is installed explicitly, and at the same depth, because `API.Files` has + // no host default: a run with no provider must fail rather than reach the + // host by accident. + yield* useHostFiles(); yield* runXmd(args, useBunService); }); diff --git a/packages/cli/src/compiled.ts b/packages/cli/src/compiled.ts index 3ac11182..e1abf66a 100644 --- a/packages/cli/src/compiled.ts +++ b/packages/cli/src/compiled.ts @@ -7,7 +7,7 @@ import { main } from "effection"; import process from "node:process"; -import { API } from "@executablemd/runtime"; +import { API, useHostFiles } from "@executablemd/runtime"; import { compileDataUri } from "@executablemd/core"; import { runXmd } from "./cli.ts"; import { useCompiledService } from "./compiled-service.ts"; @@ -27,5 +27,10 @@ await main(function* (args) { }, { at: "min" }, ); + // Document filesystem access resolves in the caller's own filesystem here. + // It is installed explicitly, and at the same depth, because `API.Files` has + // no host default: a run with no provider must fail rather than reach the + // host by accident. + yield* useHostFiles(); yield* runXmd(args, useCompiledService); }); diff --git a/packages/cli/src/deno.ts b/packages/cli/src/deno.ts index 8d0ff3ea..4415aebc 100644 --- a/packages/cli/src/deno.ts +++ b/packages/cli/src/deno.ts @@ -10,7 +10,7 @@ import { main } from "effection"; import { fileURLToPath } from "node:url"; import process from "node:process"; -import { API } from "@executablemd/runtime"; +import { API, useHostFiles } from "@executablemd/runtime"; import { compileDataUri } from "@executablemd/core"; import { runXmd } from "./cli.ts"; import { useDenoService } from "./deno-service.ts"; @@ -32,5 +32,10 @@ await main(function* (args) { }, { at: "min" }, ); + // Document filesystem access resolves in the caller's own filesystem here. + // It is installed explicitly, and at the same depth, because `API.Files` has + // no host default: a run with no provider must fail rather than reach the + // host by accident. + yield* useHostFiles(); yield* runXmd(args, useDenoService); }); diff --git a/packages/cli/src/node.ts b/packages/cli/src/node.ts index 01d20f05..878ab6fc 100755 --- a/packages/cli/src/node.ts +++ b/packages/cli/src/node.ts @@ -14,7 +14,7 @@ import { main } from "effection"; import { fileURLToPath } from "node:url"; import process from "node:process"; -import { API } from "@executablemd/runtime"; +import { API, useHostFiles } from "@executablemd/runtime"; import { compileTempFile } from "@executablemd/core"; import { runXmd } from "./cli.ts"; import { useNodeService } from "./node-service.ts"; @@ -37,5 +37,10 @@ await main(function* (args) { }, { at: "min" }, ); + // Document filesystem access resolves in the caller's own filesystem here. + // It is installed explicitly, and at the same depth, because `API.Files` has + // no host default: a run with no provider must fail rather than reach the + // host by accident. + yield* useHostFiles(); yield* runXmd(args, useNodeService); }); diff --git a/packages/core/src/components/File.ts b/packages/core/src/components/File.ts index 8b3607b1..20568ff7 100644 --- a/packages/core/src/components/File.ts +++ b/packages/core/src/components/File.ts @@ -4,77 +4,69 @@ * * Both forms take one relative `path`, resolved against `Env.cwd`, so a * document composes with `` without choosing where anything lives. - * Everything the component touches is confined to that directory, checked in - * two stages that answer different questions at different times. + * What "resolved" means belongs to the installed `API.Files` provider: under + * `xmd run` it is a path in the caller's own filesystem, and under a workflow + * run it names an entry in a logical filesystem the run owns. This component + * names neither, and makes no filesystem call of its own. * - * The first stage is pure path arithmetic against `Env.cwd`: an empty path, an - * absolute path, and a lexical `..` escape are refused with no filesystem call - * at all. It runs **before** the children expand, so an unusable path costs - * nothing and its printed error is written before there is any child failure to - * report alongside it. + * What it does own is **order**, and the order is the whole design. A write + * happens in two stages that answer different questions at different times. * - * The second stage resolves what is actually on disk and re-checks the result, - * which is what catches a symlink leaving the workspace. It runs **after** the - * children have finished, because a child can change what a path means — - * replacing a directory with a symlink out of the workspace, for instance — - * and a destination resolved earlier would not be the one the write lands on. + * The first is `checkFilePath`: pure path arithmetic, no filesystem access, and + * nothing usable comes back. An empty path, an absolute path, and a lexical + * `..` escape are refused there. It runs **before** the children expand, so an + * unusable path costs nothing and its printed error is written before there is + * any child failure to report alongside it. * - * Writes go through a sibling temporary file and a rename. The rename is the - * **commit point**: everything before it can fail or be cancelled with the - * previous file untouched, and once it begins the result is the complete old - * file or the complete new one, never a partial write. It is not a - * transaction — a commit that has happened is not rolled back by a later - * cancellation. The temporary also closes the one containment hole resolution - * cannot: a dangling symlink has nothing to resolve, and `rename` replaces the - * link rather than following it wherever it points. + * The second is `writeTextFile`, which runs **after** the children have + * finished, because a child can change what a path means — replacing a + * directory with a symlink out of the workspace, for instance — and a + * destination resolved earlier would not be the one the write lands on. That + * call repeats admission from the same authored path and then owns every later + * step: resolution, target classification, parent creation, and the commit. The + * earlier check therefore authorizes nothing; it only decides whether children + * run. * - * Printed errors name only the path the document wrote. A resolved workspace - * path, the destination a symlink pointed at, a temporary file, and a rejected - * absolute path are all withheld — §1.2 keeps absolute paths out of - * printed errors, and a containment failure is the last place to start reporting - * them. A platform error carries the path it failed on, so every filesystem - * call is wrapped and nothing from the error it caught is reproduced: the errno - * code *selects* a phrase from the allowlist in `fs-error-phrases.ts`. + * Printed errors name only the path the document wrote. A resolved path, the + * destination a symlink pointed at, a temporary file, and a rejected absolute + * path never cross the provider boundary at all — what comes back is a + * `FilesReason` from a fixed vocabulary, which *selects* a phrase from + * `fs-error-phrases.ts`. §1.2 keeps absolute paths out of printed errors, and a + * containment failure is the last place to start reporting them. * - * A failed cleanup of the temporary is the one thing reported that the document - * did not ask for, and it is reported alongside the write's own outcome rather - * than instead of it — a file the document did not create may be sitting in its - * directory, which it cannot learn any other way. + * A failed cleanup of a provider's temporary is the one thing reported that the + * document did not ask for, and it is reported alongside the write's own + * outcome rather than instead of it — a file the document did not create may be + * sitting in its directory, which it cannot learn any other way. * - * Every filesystem call goes through the contextual `API.Fs`, so a host can - * observe or sandbox a document's own file access on the same terms as the - * engine's component resolution. + * ## Failure, and the two kinds of it + * + * An ordinary filesystem condition arrives as a structured `Err` and becomes a + * printed error. A provider that is not installed, that refuses an operation, + * or that breaks its own contract throws instead, and `invokeFiles` keeps it + * fatal: there is nothing for the document to read in "no filesystem provider + * is installed", and printing it would let the siblings after this one run as + * though the file work had happened. * * ## Threat model * - * Containment is judged against the filesystem as this component observes it. - * That is sound while the filesystem is stable, and every guarantee here is - * stated on that basis. It is not a sandbox: nothing prevents another process - * from replacing a directory with a symlink between the moment a path is - * validated and the moment it is used. Deferring resolution until immediately - * before the write narrows that window and covers the document's own children, - * which is the case a document controls; closing it entirely needs a - * capability or a platform-enforced sandbox, and is issue #227. + * Containment is the provider's claim, not this component's. The host provider + * judges it against the filesystem as it observes it, which is sound while the + * host pathname namespace is stable; a transaction-bound provider resolves + * logical paths that never reach a host filesystem call at all. Deferring the + * second stage until immediately before the write is what covers the + * document's own children under either. */ -import { basename, dirname, isAbsolute, join, relative, resolve, sep } from "node:path"; -import { randomUUID } from "node:crypto"; -import { ensure, scoped } from "effection"; import type { Operation } from "effection"; import { printErrors } from "../component-failures.ts"; -import { - cwd, - ensureDir, - readTextFile, - realpath, - remove, - rename, - stat, - writeTextFile, -} from "@executablemd/runtime"; +import { cwd } from "@executablemd/runtime"; +import { parseFilesFailure } from "@executablemd/runtime"; +import type { FilesFailureData, FileWriteFailureData, FilesReason } from "@executablemd/runtime"; import { content } from "../component-api.ts"; import { hasContent } from "../content-context.ts"; import { ContentError } from "../errors.ts"; +import { checkFilePath, readFileText, writeFileText } from "../files.ts"; import type { Json } from "../types.ts"; import { reason } from "./fs-error-phrases.ts"; @@ -95,45 +87,31 @@ export class FileAccessError extends Error { } } -/** One sanitized sentence: the document's own path, and an allowlisted phrase. */ -function sentence(requested: string, verb: string, error: unknown): string { - return `cannot ${verb} "${requested}": ${reason(error)}.`; -} - -/** - * Run a filesystem operation, replacing whatever it throws with a printed error - * built only from the path the document wrote and an allowlisted phrase. - * - * Nothing is passed through, including a `FileAccessError`. This component's - * own checks throw outside guarded calls, so an error surfacing from inside one - * came from the Api — and an error's class says nothing about whether its - * message is safe to show. Trusting one would let middleware choose the text of - * a printed error by choosing what to throw. - * - * Cancellation is not a thrown error in Effection — halting resumes the - * generator through `return()` — so this never converts a halt into a failure. - */ -function* guard(requested: string, verb: string, operation: Operation): Operation { - try { - return yield* operation; - } catch (error) { - throw new FileAccessError(sentence(requested, verb, error)); - } -} - export default printErrors(function* (props: Record): Operation { const requested = String(props.path); - const admitted = yield* admissible(requested); + const directory = yield* cwd(); if (yield* hasContent()) { + const admitted = yield* checkFilePath({ cwd: directory, path: requested }); + if (!admitted.ok) { + throw new FileAccessError(refusal(requested, "write", parseFilesFailure(admitted.error))); + } + // The children run only once the path is known to be usable, and the // destination is resolved only once they are done. const text = yield* rendered(requested); - yield* write(requested, yield* destination(admitted), text); + const failure = yield* writeFileText({ cwd: directory, path: requested, content: text }); + if (failure !== undefined) { + throw new FileAccessError(report(requested, failure)); + } return ""; } - return yield* read(requested, yield* destination(admitted)); + const read = yield* readFileText({ cwd: directory, path: requested }); + if (!read.ok) { + throw new FileAccessError(refusal(requested, "read", parseFilesFailure(read.error))); + } + return read.value; }); /** @@ -145,7 +123,7 @@ export default printErrors(function* (props: Record): Operation` renders nothing: this @@ -173,259 +151,130 @@ function* rendered(requested: string): Operation { } } -/** A path that has passed the lexical stage, carried to the resolving one. */ -interface Admissible { - /** The path the document wrote, for printed errors. */ - requested: string; - /** `requested` joined onto the contextual directory and normalized. */ - lexical: string; -} +const EMPTY = "path is empty; give a path relative to the working directory."; +const ABSOLUTE = "an absolute path is not accepted; give a path relative to the working directory."; /** - * Stage one: what can be decided without touching the filesystem. + * The sentence for a failure that stopped before anything was written. * - * An empty path, an absolute path, and a `..` escape are all answerable from - * `Env.cwd` and path arithmetic alone. Deciding them here means an unusable - * path is refused before the children of a write run at all, and that the - * printed error never has to mention a path that was rejected for being absolute. + * The lexical refusals and the two containment escapes are named rather than + * phrased, because each says something about the path the document wrote rather + * than about a filesystem condition. A rejected absolute path is deliberately + * not echoed: §1.2 keeps absolute paths out of printed errors, and the whole + * reason this one was refused is that it named somewhere else. * - * `resolve` normalizes `..` lexically, so this holds against the contextual - * directory as given — canonicalizing it is stage two's job and would only - * move the same comparison onto a different pair of strings. + * Everything else is the verb, the document's own path, and an allowlisted + * phrase. */ -function* admissible(requested: string): Operation { - if (requested.length === 0) { - throw new FileAccessError("path is empty; give a path relative to the working directory."); +function refusal( + requested: string, + verb: "read" | "write", + data: FilesFailureData | FileWriteFailureData | undefined, +): string { + if (data === undefined) { + return `cannot ${verb} "${requested}": ${reason(undefined)}.`; } - if (isAbsolute(requested)) { - throw new FileAccessError( - "an absolute path is not accepted; give a path relative to the working directory.", - ); + if (data.reason === "empty-path") { + return EMPTY; } - - const directory = yield* cwd(); - const lexical = resolve(directory, requested); - if (!within(directory, lexical)) { - throw new FileAccessError(`"${requested}" resolves outside the working directory.`); + if (data.reason === "absolute-path") { + return ABSOLUTE; } - - return { requested, lexical }; -} - -/** - * Stage two: the path as the filesystem currently has it. - * - * Resolves the part of the path that is already on disk — the file itself when - * it is there, the deepest existing ancestor when it is not — and re-checks - * the result, which is what catches a symlink pointing out of the workspace. - * What comes back is that resolved path, so an internal symlink is followed to - * the file it names rather than being replaced by the write. - * - * Both sides of the comparison are canonical here, so a working directory - * reached through a symlink — macOS's `/var` against `/private/var` — does not - * read as an escape. - */ -function* destination({ requested, lexical }: Admissible): Operation { - const directory = yield* cwd(); - const base = (yield* guard(requested, "resolve", realpath(directory))) ?? directory; - - const effective = yield* guard(requested, "resolve", resolveExisting(lexical)); - if (!within(base, effective)) { - throw new FileAccessError( - `"${requested}" leads through a symlink outside the working directory.`, - ); + if (data.reason === "lexical-escape") { + return `"${requested}" resolves outside the working directory.`; } - - return effective; -} - -/** - * Whether `path` names the working directory or something inside it. - * - * The directory itself is contained — `.` is not an escape. What it is instead - * is a directory, which is a question about the target rather than about - * containment, so it belongs to the read and write checks that come after. - * - * Only a complete `..` segment leaves. A name that merely starts with two dots - * — `..notes.md`, `..config/settings.json` — is an ordinary file inside, and a - * prefix test would refuse it. - */ -function within(base: string, path: string): boolean { - const rel = relative(base, path); - if (isAbsolute(rel)) { - return false; + if (data.reason === "resolved-escape") { + return `"${requested}" leads through a symlink outside the working directory.`; + } + if (data.phase === "resolution") { + return `cannot resolve "${requested}": ${reason(data.reason)}.`; } - if (rel.length === 0) { - return true; + if (data.phase === "target") { + return classified(requested, verb, data.reason); } - return rel !== ".." && !rel.startsWith(`..${sep}`); + return `cannot ${verb} "${requested}": ${reason(data.reason)}.`; } /** - * `path` with every symlink in its existing prefix resolved. + * What the target turned out to be, when that is the answer rather than an + * errno. * - * `realpath` needs the whole path to exist, and a write commonly names one - * that does not yet, so the walk gives up one trailing segment at a time until - * something answers and then puts the segments back. The working directory - * always exists, so the loop terminates there at the latest. + * A directory is not text, and saying so is better than whatever reading one + * produces on a given platform. "No such file" is the read's own phrasing of + * the same idea: the document asked for a file, and there is not one. */ -function* resolveExisting(path: string): Operation { - const trailing: string[] = []; - let current = path; - - while (true) { - const resolved = yield* realpath(current); - if (resolved !== undefined) { - return trailing.length === 0 ? resolved : join(resolved, ...trailing); +function classified(requested: string, verb: "read" | "write", why: FilesReason | undefined) { + if (verb === "read") { + if (why === "missing") { + return `cannot read "${requested}": no such file.`; } - const parent = dirname(current); - if (parent === current) { - return join(current, ...trailing); + if (why === "directory") { + return `cannot read "${requested}": it is a directory, not a text file.`; } - trailing.unshift(basename(current)); - current = parent; - } -} - -function* read(requested: string, target: string): Operation { - const info = yield* guard(requested, "read", stat(target)); - if (!info.exists) { - throw new FileAccessError(`cannot read "${requested}": no such file.`); - } - if (info.isDirectory) { - throw new FileAccessError(`cannot read "${requested}": it is a directory, not a text file.`); - } - if (!info.isFile) { - throw new FileAccessError(`cannot read "${requested}": it is not a regular file.`); - } - - return yield* guard(requested, "read", readTextFile(target)); -} - -/** - * Which step a write failed at, which is what decides what can be said about - * the target afterwards. - */ -type Step = "preparation" | "commit"; - -/** - * Replace `target` with exactly `text`. - * - * The content is whole by the time this runs — the children expanded first — - * so a child that failed never reaches the filesystem at all. What remains is - * the write, and it has one commit point. - * - * Everything up to the rename is preparation: a failure or a cancellation - * there leaves the previous file untouched, because nothing has replaced it - * yet. The rename is the commit, and it is a commit rather than a transaction — - * `rename` is a single filesystem call that cannot be interrupted once started, - * and a cancellation arriving after it has completed does not undo it. What is - * promised is that no write is ever half visible, not that a finished write can - * be taken back. - * - * A rename that *throws* is the one case where the outcome cannot be inferred. - * `rename` is an operation on the contextual Fs Api, and an `around` handler - * legitimately does work on both sides of `next()` — so a throw may arrive - * before the underlying rename ran or after it succeeded, and there is no way - * from here to tell which. What still holds is atomicity: the target is the - * complete previous content or the complete replacement, never a partial write. - * Claiming the previous file survived would be a guess, and it would be wrong - * exactly when a handler failed after committing. - * - * Removal of the temporary is registered before it is written rather than - * after. `writeTextFile` is where an interruption is most likely to land, and - * a cleanup installed on the far side of it would not run for the one failure - * it exists to handle. `remove` is forced, so registering it for a file that - * was never created — or one the rename has already consumed — is a no-op. - * - * A removal that does fail is reported, because a file the document did not - * create may be left in its directory. It is reported *alongside* whatever the - * write did rather than instead of it: both outcomes are collected here and - * raised together, so neither hides the other. - */ -function* write(requested: string, target: string, text: string): Operation { - const info = yield* guard(requested, "write", stat(target)); - if (info.exists && !info.isFile) { - throw new FileAccessError( - `cannot write "${requested}": it is a ${info.isDirectory ? "directory" : "special file"}, ` + - "not a text file.", - ); - } - - yield* guard(requested, "write", ensureDir(dirname(target))); - - // Both halves are collected rather than thrown, then reported together. A - // destructor that threw would replace the failure it is unwinding, or be - // aggregated into it in a shape a document cannot read; and the write's own - // failure must not hide the fact that a temporary was left behind. - const failed: string[] = []; - const uncleaned: string[] = []; - let step: Step = "preparation"; - - yield* scoped(function* () { - const temporary = `${target}.xmd-${randomUUID().slice(0, 8)}.tmp`; - yield* ensure(() => discard(requested, temporary, uncleaned)); - try { - yield* writeTextFile(temporary, text); - step = "commit"; - yield* rename(temporary, target); - } catch (error) { - failed.push(sentence(requested, "write", error)); + if (why === "special-file") { + return `cannot read "${requested}": it is not a regular file.`; } - }); - - if (failed.length === 0 && uncleaned.length === 0) { - return; } - throw new FileAccessError( - [ - ...failed, - ...uncleaned, - outcome(failed.length > 0 ? step : undefined), - ...(uncleaned.length > 0 ? [LEFTOVER] : []), - ].join(" "), - ); + if (verb === "write" && (why === "directory" || why === "special-file")) { + const kind = why === "directory" ? "directory" : "special file"; + return `cannot write "${requested}": it is a ${kind}, not a text file.`; + } + return `cannot ${verb} "${requested}": ${reason(why)}.`; } /** * What can be said about the target, given where the write stopped. * - * Only two of these are conclusions. A preparation failure changed nothing, - * and a rename that returned committed. A rename that threw is the honest - * "unknown": atomicity still holds, so the answer is one of two whole files, - * but which one is not observable from here. + * Only three of these are conclusions. Everything up to the commit changed + * nothing, a commit that returned committed, and a transaction that rolled its + * change back is back where it started. A commit that *threw* is the honest + * "unknown": the provider cannot tell whether the underlying replacement ran, so + * the answer is one of two whole files, but not which. + * + * The phases before the commit have no outcome sentence at all. Nothing was + * attempted on the target, so there is nothing to report about it beyond why the + * write did not start. */ -function outcome(failedAt: Step | undefined): string { - if (failedAt === undefined) { - return "The file was written."; - } - if (failedAt === "preparation") { - return "The previous file is unchanged."; - } - return ( +const OUTCOMES: ReadonlyMap = new Map([ + ["temporary", "The previous file is unchanged."], + [ + "commit", "Whether the replacement committed is unknown: the target holds either the " + - "complete previous content or the complete replacement, never a partial write." - ); -} + "complete previous content or the complete replacement, never a partial write.", + ], + ["cleanup", "The file was written."], + ["transaction", "The Workspace change was rolled back."], +]); /** - * Orthogonal to `outcome`: a temporary that could not be removed is a separate - * fact about the directory, and composes with any of the three. + * Orthogonal to the outcome: a temporary that could not be removed is a separate + * fact about the directory, and composes with any of them. */ const LEFTOVER = "A temporary file beside it may remain."; /** - * Remove the temporary, recording a sanitized sentence if it cannot be. + * The complete report for a failed write. * - * Runs during teardown, sometimes the teardown of a write that is already - * failing, so it records rather than throws — see `write`. What it records - * names the document's own path: the temporary is generated, and a document - * that never chose that name cannot be shown it. + * The write's own failure and a failed cleanup are both reported, in that order, + * because neither may hide the other: one says what is known about the target, + * the other that something was left behind, and a reader needs both to know what + * the directory now holds. The outcome sentence follows them, and the leftover + * warning follows that. */ -function* discard(requested: string, temporary: string, uncleaned: string[]): Operation { - try { - yield* remove(temporary, { force: true }); - } catch (error) { - uncleaned.push(sentence(requested, "clean up", error)); +function report(requested: string, data: FileWriteFailureData): string { + const parts: string[] = []; + if (data.reason !== undefined) { + parts.push(refusal(requested, "write", data)); + } + if (data.cleanup !== undefined) { + parts.push(`cannot clean up "${requested}": ${reason(data.cleanup)}.`); + } + const outcome = OUTCOMES.get(data.phase); + if (outcome !== undefined) { + parts.push(outcome); + } + if (data.cleanup !== undefined) { + parts.push(LEFTOVER); } + return parts.join(" "); } diff --git a/packages/core/src/components/Glob.ts b/packages/core/src/components/Glob.ts index 94f8845e..7c0d8436 100644 --- a/packages/core/src/components/Glob.ts +++ b/packages/core/src/components/Glob.ts @@ -16,17 +16,17 @@ * be the answer to "that pattern was a mistake". The same stage refuses an * empty pattern, which matches nothing by construction. * - * Everything else about matching belongs to the Fs Api's `glob`, which is the - * dialect. This component adds no syntax of its own, which is why a leading dot - * needs no special prop: `*` matches one, so a pattern that names a hidden file - * finds it and a pattern that does not, does not. + * Everything else about matching belongs to the `API.Files` provider, which is + * the dialect and owns the whole search. This component adds no syntax of its + * own, which is why a leading dot needs no special prop: `*` matches one, so a + * pattern that names a hidden file finds it and a pattern that does not, does + * not. * * Only regular files come back. A symbolic link is a link rather than a file, * so it is never a result and a link to a directory is never descended into — - * which is also what keeps traversal inside `Env.cwd` and free of cycles. The - * Fs Api exposes symlink following, but nothing there confines a resolved - * destination to the root or detects a traversal cycle, so following one cannot - * be offered safely yet. + * which is also what keeps traversal inside `Env.cwd` and free of cycles. + * Following one cannot be offered safely yet: nothing here confines a resolved + * destination to the root or detects a traversal cycle. * * Printed errors name the pattern the document wrote, or nothing at all. A * traversal failure names no path: what failed is a directory somewhere under @@ -40,10 +40,11 @@ * the search runs again against whatever is on disk now. */ -import { isAbsolute } from "node:path"; import type { Operation } from "effection"; import { printErrors } from "../component-failures.ts"; -import { cwd, glob, stat } from "@executablemd/runtime"; +import { cwd, parseFilesFailure } from "@executablemd/runtime"; +import type { FilesFailureData } from "@executablemd/runtime"; +import { globFiles } from "../files.ts"; import type { Json } from "../types.ts"; import { reason } from "./fs-error-phrases.ts"; @@ -78,14 +79,11 @@ export default printErrors(function* (props: Record): Operation entry.isFile).map((entry) => entry.path); - - return [...new Set(files)].sort(byCodePoint); + const found = yield* globFiles({ cwd: yield* cwd(), include, exclude }); + if (!found.ok) { + throw failed(parseFilesFailure(found.error), [...include, ...exclude]); + } + return found.value; }); /** @@ -117,7 +115,7 @@ function patterns(prop: string, value: Json | undefined): string[] { "give a pattern relative to the working directory.", ); } - if (isAbsolute(pattern)) { + if (absolute(pattern)) { throw new GlobError( `${prop} pattern "${pattern}" is absolute; ` + "give a pattern relative to the working directory.", @@ -132,95 +130,44 @@ function patterns(prop: string, value: Json | undefined): string[] { } /** - * The contextual working directory, once it is known to be searchable. + * Whether a pattern names an absolute location. * - * Checked here rather than left to the traversal, because the two failures read - * very differently to an author: a directory that is missing or is a file is - * something about the document's own environment, while a traversal failure is - * something about one entry inside a directory that was fine. + * Decided from the pattern's own grammar rather than the running platform's. + * Patterns match POSIX-relative paths on every host — that is what makes one + * document mean one thing everywhere — so a leading `/` is absolute wherever + * this runs, and so is a drive-letter prefix, which is absolute on the host that + * has drives and matches nothing on the hosts that do not. A leading backslash + * is left alone: in this dialect it escapes the character after it. */ -function* directory(): Operation { - const root = yield* cwd(); - - const info = yield* guard(stat(root)); - if (!info.exists) { - throw new GlobError("the working directory does not exist."); - } - if (!info.isDirectory) { - throw new GlobError("the working directory is not a directory."); - } - - return root; -} - -interface Search { - root: string; - patterns: string[]; - exclude: string[]; +function absolute(pattern: string): boolean { + return pattern.startsWith("/") || /^[A-Za-z]:[\\/]/.test(pattern); } /** - * Run the search, reporting a pattern that cannot be compiled as the authoring - * error it is. + * One sanitized sentence for a failed search. * - * The Api compiles patterns as it starts, so an unusable one — an unterminated - * character class — arrives as a `SyntaxError` from `RegExp` rather than as an - * errno. It is separated from a traversal failure because it is the one failure - * here the document can fix by editing a pattern, and because a `RegExp` - * message describes a translated regular expression the author never wrote. + * The two questions an author can act on are separated from the rest. A working + * directory that is missing or is a file is something about the document's own + * environment; a pattern the dialect cannot compile — an unterminated character + * class — is something about the document's own text. Which pattern it was does + * not survive the provider boundary, so the sentence lists the candidates + * instead of naming one. They are the document's own text. * - * Which pattern it was is not recoverable from the error, so the sentence lists - * the candidates instead of naming one. They are the document's own text. + * Everything else names no path. What failed is the working directory or + * something under it, and both are absolute paths the document did not write + * (§1.2). */ -function* search(options: Search): Operation> { - try { - return yield* glob(options); - } catch (error) { - if (error instanceof SyntaxError) { - const all = [...options.patterns, ...options.exclude]; - throw new GlobError( - `one of these patterns cannot be used: ${all.map((p) => `"${p}"`).join(", ")}.`, - ); - } - throw failed(error); +function failed(data: FilesFailureData | undefined, candidates: string[]): GlobError { + if (data?.phase === "target" && data.reason === "missing") { + return new GlobError("the working directory does not exist."); } -} - -/** - * Run a filesystem operation, replacing whatever it throws with a sanitized - * sentence. - * - * Nothing is passed through, including a `GlobError`: this component's own - * checks throw outside guarded calls, so an error surfacing from inside one came - * from the Api, and an error's class says nothing about whether its message is - * safe to show. - */ -function* guard(operation: Operation): Operation { - try { - return yield* operation; - } catch (error) { - throw failed(error); - } -} - -/** - * One sanitized sentence for a failed filesystem call. - * - * It names no path. What failed is the working directory or something under it, - * and both are absolute paths the document did not write (§1.2). - */ -function failed(error: unknown): GlobError { - return new GlobError(`cannot search the working directory: ${reason(error)}.`); -} - -// Code point order, not `localeCompare`: what a document branches on must not -// depend on the locale the host happens to be configured with. -function byCodePoint(left: string, right: string): number { - if (left < right) { - return -1; + if (data?.phase === "target" && data.reason === "not-directory") { + return new GlobError("the working directory is not a directory."); } - if (left > right) { - return 1; + if (data?.phase === "pattern") { + return new GlobError( + `one of these patterns cannot be used: ${candidates.map((p) => `"${p}"`).join(", ")}.`, + ); } - return 0; + return new GlobError(`cannot search the working directory: ${reason(data?.reason)}.`); } diff --git a/packages/core/src/components/TempDir.ts b/packages/core/src/components/TempDir.ts index ea08fb68..3c61f7c1 100644 --- a/packages/core/src/components/TempDir.ts +++ b/packages/core/src/components/TempDir.ts @@ -6,19 +6,24 @@ * finishes, fails, or is cancelled. Written self-closing it is an allocation: * the directory is retained at the invocation site so a later sibling can use * the path it renders, and removed with that scope. + * + * The directory comes from the installed `API.Files` provider, so this + * component neither creates nor removes anything itself. A provider that has no + * temporary directories to give — one whose whole filesystem is a database + * transaction — refuses the operation outright, and that refusal is fatal + * rather than a printed error: there is no directory to run inside, so the + * content must not run and the siblings after it must not proceed as though it + * had. */ -import { ensure, resource } from "effection"; import type { Operation } from "effection"; import { printErrors } from "../component-failures.ts"; -import { rm } from "@effectionx/fs"; -import { API } from "@executablemd/runtime"; +import { API, parseFilesFailure } from "@executablemd/runtime"; import { ReplayGuard, StaleInputError } from "@executablemd/durable-streams"; -import { mkdtempSync, realpathSync } from "node:fs"; -import { tmpdir } from "node:os"; -import { join } from "node:path"; import { content, retain } from "../component-api.ts"; import { hasContent } from "../content-context.ts"; +import { temporaryDirectory } from "../files.ts"; +import { reason } from "./fs-error-phrases.ts"; export const props = { type: "object", @@ -26,33 +31,34 @@ export const props = { additionalProperties: false, }; +/** A temporary directory that could not be created. */ +export class TempDirError extends Error { + constructor(message: string) { + super(message); + this.name = "TempDirError"; + } +} + /** - * A directory this call created, named by its canonical path. - * - * Creation is synchronous so that nothing can suspend between it and the - * `ensure` that removes it. `until()` cannot cancel the promise it is waiting - * on, so an asynchronous `mkdtemp` halted mid-flight would go on to create a - * directory after the generator had already stopped — one nothing owns and - * nothing removes. Reading the directory does not suspend either, so the whole - * acquisition is a single uninterruptible step. + * A directory that lives as long as the acquiring scope. * - * `mkdtemp` names and creates at once, so the directory is never one an - * earlier run left behind. The path is then canonicalized: on macOS `tmpdir()` - * is a symlink (`/var/folders/…`) while a child process resolves it - * (`/private/var/…`), and canonicalizing is what makes the rendered path, - * `Env.cwd`, and a subprocess's own `cwd` the same string. - * - * `@effectionx/fs` has neither operation, so both come from Node. + * The provider owns creation and removal as one acquisition, so nothing can + * land between them and leave a directory nobody holds. What this adds is the + * unwrapping: an ordinary acquisition failure is a printed error naming no path + * — the directory is generated and the document never chose its name — while a + * provider that refuses the operation has already thrown past here. * * Exported for the lifetime tests, which drive acquisition directly. Not part * of the package's public surface — `mod.ts` does not re-export it. */ -export function useTemporaryDirectory(): Operation { - return resource(function* (provide) { - const created = mkdtempSync(join(tmpdir(), "xmd-tempdir-")); - yield* ensure(() => rm(created, { recursive: true, force: true })); - yield* provide(realpathSync(created)); - }); +export function* useTemporaryDirectory(): Operation { + const acquired = yield* temporaryDirectory(); + if (!acquired.ok) { + throw new TempDirError( + `cannot create a temporary directory: ${reason(parseFilesFailure(acquired.error)?.reason)}.`, + ); + } + return acquired.value; } /** diff --git a/packages/core/src/components/fs-error-phrases.ts b/packages/core/src/components/fs-error-phrases.ts index 2d3b59be..4a1506f7 100644 --- a/packages/core/src/components/fs-error-phrases.ts +++ b/packages/core/src/components/fs-error-phrases.ts @@ -1,64 +1,54 @@ /** - * The vocabulary a failed filesystem call may be reported in. + * The vocabulary a failed filesystem operation may be reported in. * - * Shared by the components that touch the filesystem — `` (§6.13) and + * Shared by the components that reach the filesystem — `` (§6.13) and * `` (§6.14) — because the constraint is the same for both. A platform * error message names the path it failed on: `ENOTDIR: not a directory, stat * '/private/var/…'`. That is the resolved path §1.2 keeps out of printed errors, * and it can be a path the document never wrote — a generated temporary, or a * file somewhere under a directory that was only ever named by a pattern. * - * So nothing from the caught error is reproduced. The errno code **selects** a - * phrase written here, and an unrecognized code selects the generic one. The - * code itself is never emitted, because whatever implements the Fs Api chooses - * it and can put a path, markup, or a newline there as easily as `ENOENT`. + * So nothing from the failure is reproduced. The provider has already reduced + * whatever its platform produced to a `FilesReason`, and that reason **selects** + * a phrase written here. An unrecognized reason selects the generic one, which + * is also what a provider reports when it could not classify the condition. */ +import type { FilesReason } from "@executablemd/runtime"; + /** * Every phrase, and the whole allowlist. * - * A `Map` rather than an object, because a lookup on an object literal answers - * for inherited keys — `REASONS["toString"]` would hand back a function whose - * source would then be interpolated into a printed error. + * A `Map` rather than an object literal, because a lookup on one answers for + * inherited keys — `PHRASES["toString"]` would hand back a function whose source + * would then be interpolated into a printed error. + * + * The reasons that are not here are the ones a component has a sentence of its + * own for: the lexical refusals, the containment escapes, a target that is a + * directory or a special file, and a pattern that cannot be compiled. */ -const REASONS: ReadonlyMap = new Map([ - ["ENOENT", "no such file or directory"], - ["ENOTDIR", "a component of the path is not a directory"], - ["EISDIR", "it is a directory"], - ["ENOTEMPTY", "the directory is not empty"], - ["EACCES", "permission denied"], - ["EPERM", "permission denied"], - ["EROFS", "the filesystem is read-only"], - ["ELOOP", "too many levels of symbolic links"], - ["ENAMETOOLONG", "the path is too long"], - ["ENOSPC", "no space left on the device"], - ["EDQUOT", "the disk quota is exhausted"], - ["EXDEV", "the destination is on a different filesystem"], - ["EBUSY", "the file is in use"], - ["EMFILE", "too many open files"], +const PHRASES: ReadonlyMap = new Map([ + ["missing", "no such file or directory"], + ["not-directory", "a component of the path is not a directory"], + ["directory", "it is a directory"], + ["directory-not-empty", "the directory is not empty"], + ["permission-denied", "permission denied"], + ["read-only", "the filesystem is read-only"], + ["too-many-symlinks", "too many levels of symbolic links"], + ["path-too-long", "the path is too long"], + ["no-space", "no space left on the device"], + ["quota-exhausted", "the disk quota is exhausted"], + ["cross-device", "the destination is on a different filesystem"], + ["busy", "the file is in use"], + ["too-many-open-files", "too many open files"], ]); const UNRECOGNIZED = "the filesystem operation failed"; -function errorCode(error: unknown): string | undefined { - if (typeof error !== "object" || error === null || !("code" in error)) { - return undefined; - } - const { code } = error; - return typeof code === "string" ? code : undefined; -} - -/** - * A phrase for a failure, chosen from `REASONS` or the generic one. - * - * `code` is attacker-supplied as far as a component is concerned — anything - * installed as Fs middleware can put a path, markup, or a newline in it — so it - * is used to select a phrase and never to build one. - */ -export function reason(error: unknown): string { - const code = errorCode(error); - if (code === undefined) { +/** A phrase for a failure, chosen from `PHRASES` or the generic one. */ +export function reason(value: FilesReason | undefined): string { + if (value === undefined) { return UNRECOGNIZED; } - return REASONS.get(code) ?? UNRECOGNIZED; + return PHRASES.get(value) ?? UNRECOGNIZED; } diff --git a/packages/core/src/errors.ts b/packages/core/src/errors.ts index aa9f508b..f6463b89 100644 --- a/packages/core/src/errors.ts +++ b/packages/core/src/errors.ts @@ -7,6 +7,13 @@ import { StaleInputError, TerminalDivergenceError, } from "@executablemd/durable-streams"; +// The leaf module rather than the package root. This module is in the graph a +// separately loaded copy of `printErrors` bundles, and the runtime's root +// re-exports the host Apis — process, fetch, filesystem — one of which carries a +// native addon no bundler can inline. Recognizing a Files failure needs none of +// that: it is a structural tag and a parser. +import { asFilesFatal } from "@executablemd/runtime/files"; +import type { FilesFatalFailure } from "@executablemd/runtime/files"; import { InvocationTeardownError } from "./invocation.ts"; import type { ErrorSegment } from "./types.ts"; @@ -192,13 +199,13 @@ export type DurabilityFailure = | ContinuePastCloseDivergenceError; /** A failure that ends the execution rather than becoming a printed error. */ -export type FatalFailure = DocumentationError | DurabilityFailure; +export type FatalFailure = DocumentationError | DurabilityFailure | FilesFatalFailure; /** * The error that ends the execution, if this failure carries one. * * Expansion turns a failure into a printed error the document can render, which - * is right for anything the document itself got wrong. Two kinds are not that, + * is right for anything the document itself got wrong. Three kinds are not that, * and every generic catch in the engine rethrows them: * * - `DocumentationError` — the error mode has already decided this execution @@ -212,31 +219,54 @@ export type FatalFailure = DocumentationError | DurabilityFailure; * to a note. It would also bury *where* the journal stopped describing the * run: expansion that carried on would reach another durable operation, whose * own mismatch is then the one reported. + * - a **Files infrastructure failure** — no document filesystem provider is + * installed, one refused an operation outright, or one broke its own + * contract. None of those is a filesystem condition a document could have + * caused or could act on, and a provider that is not there must not become a + * printed comment that lets later siblings run as if the file work had + * happened. * * This looks through the three ways the engine and the platform aggregate * failures and returns the fatal error itself rather than the wrapper, which is - * the one worth reporting. The two kinds travel differently: a durability - * failure stays fatal through the complete cause graph, while a + * the one worth reporting. The kinds travel differently: durability and Files + * failures stay fatal through the complete cause graph, while a * `DocumentationError` remains fatal unless it crossed an explicit * `ContentError` recovery boundary — a component that recovered decided what * the document reports. An uncaught private content transport restores its * original `DocumentationError` explicitly at the function-component boundary, * not through this traversal. * - * **A durability failure outranks a documentation failure**, wherever each sits - * in the graph. A wrapper carries whatever failed together, in whatever order - * the platform happened to collect it, and one of those orders would otherwise - * report the document's failure and let the loop record an `error` outcome onto - * a journal already known not to describe this run. Precedence is therefore - * decided by kind, not by position: the graph is searched for a durability - * failure first, and only a graph without one reports a documentation failure. + * **Precedence is durability, then Files, then documentation**, wherever each + * sits in the graph. A wrapper carries whatever failed together, in whatever + * order the platform happened to collect it, and one of those orders would + * otherwise report the document's failure and let the loop record an `error` + * outcome onto a journal already known not to describe this run. Precedence is + * therefore decided by kind, not by position: each kind is searched for across + * the whole graph in turn, so nesting and aggregate member order cannot change + * the answer. * - * The two searches reach different parts of the same graph. A content failure - * ends the documentation search and not the durability one — see - * `isRecoveredContent` for why the asymmetry is the point. + * The searches reach different parts of the same graph. A content failure ends + * the documentation search and neither of the others — see `isRecoveredContent` + * for why the asymmetry is the point. */ export function fatalCause(error: unknown): FatalFailure | undefined { - return durabilityFailure(error) ?? documentationFailure(error); + return durabilityFailure(error) ?? filesFatalFailure(error) ?? documentationFailure(error); +} + +/** + * The Files infrastructure failure this one carries, if any. + * + * Recognized structurally rather than with `instanceof`: two copies of the + * runtime package can be loaded at once, and a provider failure constructed by + * the other one has to be found on the same terms as one constructed here. + * + * Like durability discovery, this walks the whole graph and treats no node as a + * leaf. A `ContentError` in the chain says a component recovered from failed + * content and reported a failure of its own; that decides which *document* + * failure is reported, and a missing provider is not a document failure. + */ +export function filesFatalFailure(error: unknown): FilesFatalFailure | undefined { + return firstCause(error, asFilesFatal); } /** @@ -260,8 +290,9 @@ export function documentationFailure(error: unknown): DocumentationError | undef * whether the document gets to read what happened. * * An `output` decision says yes — that is the whole difference between the mode - * a region installs and the mode documentation installs. A `throw` decision and - * a durability failure say no. + * a region installs and the mode documentation installs. A `throw` decision, a + * durability failure, and a Files infrastructure failure all say no: none of + * them is a decision about what the document reports. */ export function decidedByOutput(failure: FatalFailure): boolean { return failure instanceof DocumentationError && failure.mode === "output"; @@ -318,12 +349,13 @@ type OpaqueFailure = (error: object) => boolean; * replaced — the component's contextual printed error would be built and then * discarded in favour of the child's. * - * Durability discovery looks straight through it. Only the engine's own + * Durability and Files discovery look straight through it. Only the engine's own * projection failures are known to carry a documentation failure and nothing * else; this class is public, so an author constructs and subclasses it and may * put anything underneath — and a durability failure stays fatal however it is - * wrapped (§6.11). Recovery decides which failure the document *reports*, and a - * durability failure is not one of the things a document reports. + * wrapped (§6.11). Recovery decides which failure the document *reports*, and + * neither a durability failure nor a missing filesystem provider is one of the + * things a document reports. */ function isRecoveredContent(error: object): boolean { return error instanceof ContentError; @@ -355,14 +387,14 @@ function walkCauses( opaque: OpaqueFailure | undefined, seen: Set, ): T | undefined { - const selected = select(error); + const selected = attempt(() => select(error)); if (selected !== undefined) { return selected; } if (typeof error !== "object" || error === null || seen.has(error)) { return undefined; } - if (opaque !== undefined && opaque(error)) { + if (opaque !== undefined && attempt(() => opaque(error)) === true) { return undefined; } seen.add(error); @@ -375,16 +407,49 @@ function walkCauses( return undefined; } -/** The wrapper contracts a failure can aggregate other failures through. */ +/** + * The wrapper contracts a failure can aggregate other failures through. + * + * Every read here is of a value the engine did not create. A thrown object may + * be a Proxy, or carry an accessor that fails, and one of those refusing to + * answer must not become the failure this traversal was called to classify — + * every generic catch in expansion asks `fatalCause` first, so a throw here + * would replace the real failure with a failure about inspecting it. An + * unreadable wrapper simply aggregates nothing. + */ function causesOf(error: object): unknown[] { - if (error instanceof InvocationTeardownError) { - return error.causes; - } - if (error instanceof AggregateError) { - return error.errors; - } - if (error instanceof Error && error.cause !== undefined) { - return [error.cause]; + return ( + attempt(() => { + if (error instanceof InvocationTeardownError) { + return members(error.causes); + } + if (error instanceof AggregateError) { + return members(error.errors); + } + if (error instanceof Error && error.cause !== undefined) { + return [error.cause]; + } + return []; + }) ?? [] + ); +} + +/** A wrapper's members, when it really holds a list of them. */ +function members(value: unknown): unknown[] { + return Array.isArray(value) ? [...value] : []; +} + +/** + * Read a value that may refuse to be read. + * + * `undefined` means "this said nothing", which every caller here already treats + * as "not recognized" — so a hostile shape narrows what discovery finds rather + * than replacing what it was discovering. + */ +function attempt(read: () => T): T | undefined { + try { + return read(); + } catch { + return undefined; } - return []; } diff --git a/packages/core/src/files.ts b/packages/core/src/files.ts new file mode 100644 index 00000000..118b1f04 --- /dev/null +++ b/packages/core/src/files.ts @@ -0,0 +1,358 @@ +/** + * The engine's one door to `API.Files`. + * + * Every document filesystem operation a component performs goes through here, + * because the contract has a part no component can enforce on its own: a + * provider is allowed to *fail*, and it is not allowed to *throw*. An ordinary + * filesystem condition comes back as `Err` with structural data; anything that + * throws is an installation or provider-contract failure and must end the + * execution rather than become something a document renders. + * + * Deciding that in one place is what makes the rule hold. A component that + * caught a throw would have to guess whether the thing it caught was safe to + * report, and an error's class is no evidence about its message — which is + * exactly the guess `` and `` already refuse to make about the + * platform. + * + * This lives in core rather than in the runtime package because the ordering + * below reaches durability failures, which belong to `@executablemd/durable- + * streams`. Core already depends on both; the runtime must not acquire a + * dependency on durable-streams to answer a question only the engine asks. + */ + +import { + Files, + FilesError, + filesFailure, + FilesInvariantError, + fileWriteFailure, + parseFileWriteFailure, + parseFileWriteSuccess, + parseFilesFailure, +} from "@executablemd/runtime"; +import type { + FilePathInput, + FilesOperation, + FilesPhase, + FileWriteFailureData, + FileWriteInput, + GlobInput, +} from "@executablemd/runtime"; +import { Err, Ok } from "effection"; +import type { Operation, Result } from "effection"; +import { durabilityFailure, filesFatalFailure } from "./errors.ts"; + +/** + * Perform one provider call, converting an illegal throw into a failure the + * engine already knows how to fence. + * + * The search order is the one thing here that is not obvious. A provider call + * can happen underneath work that has *already* failed — a durability failure + * unwinding through a component's teardown, or a Files failure from a nested + * operation — and the first failure is the one that describes what went wrong. + * Replacing it with a fresh invariant would report the symptom and lose the + * cause, and for a durability failure it would also lose the identity that + * #394's fail-stop records as "the first error". + * + * So: an existing durability failure is rethrown as it stands, then an existing + * Files infrastructure failure, and only something that is neither becomes a new + * `protocol` invariant. That last one carries no cause, message, errno text, or + * host value — a handler that threw an arbitrary object is precisely the case + * where nothing it produced can be trusted. + * + * Cancellation does not arrive here. Halting resumes a generator through + * `return()` rather than by throwing, so no `catch` in Effection converts a + * cancellation into a failure. + */ +export function* invokeFiles(call: Operation): Operation { + try { + return yield* call; + } catch (error) { + throw ( + durabilityFailure(error) ?? filesFatalFailure(error) ?? new FilesInvariantError("protocol") + ); + } +} + +/** + * Whether a member is there at all, or `undefined` when asking threw. + * + * Asked separately from reading it, because absence and unreadability are + * different answers and only one of them is ever legitimate. A `has` trap can + * refuse the question, which is why even this is total. + */ +function present(target: unknown, name: string): boolean | undefined { + if (typeof target !== "object" || target === null) { + return undefined; + } + try { + return Reflect.has(target, name); + } catch { + return undefined; + } +} + +/** + * A member of a value the engine did not create, or `undefined` when it is + * absent or reading it threw. + * + * A `Result` is only conventionally a `Result`: what a provider actually + * returned is an arbitrary runtime value, and reading `ok` on a Proxy that + * refuses can throw. The type says otherwise, which is exactly why the check + * belongs here — the signature is a claim about the provider, not a guarantee. + * + * The value comes back boxed so that a member which really is `undefined` is + * distinguishable from one that could not be read. Collapsing those is how a + * container that never described its outcome gets mistaken for one that + * described an empty one. + */ +function read(target: unknown, name: string): { readonly value: unknown } | undefined { + if (present(target, name) !== true) { + return undefined; + } + try { + return { value: Reflect.get(Object(target), name) }; + } catch { + return undefined; + } +} + +/** + * Whether this really is an array, or `undefined` when asking threw. + * + * Recognizing the brand is itself an operation on provider-controlled data: + * `Array.isArray` throws on a revoked Proxy. A value whose array identity + * cannot be inspected has not described itself, which is the same answer as + * not being one. + */ +function isArray(value: unknown): boolean | undefined { + try { + return Array.isArray(value); + } catch { + return undefined; + } +} + +/** + * How a `Result` settled, or `undefined` when it will not say. + * + * A value that does not describe its own outcome cannot be reported as either + * one, so `undefined` here is a provider-contract failure rather than a result. + */ +function settlement(result: unknown): boolean | undefined { + const ok = read(result, "ok"); + return typeof ok?.value === "boolean" ? ok.value : undefined; +} + +/** + * The generic failure a non-write operation falls back to. + * + * Built fresh from this operation's own identity rather than from anything the + * provider returned, so the sentence a document reads is the one the vocabulary + * already has for "the filesystem operation failed" — and nothing the provider + * put in its place travels with it. + */ +function generic(operation: FilesOperation, phase: FilesPhase): FilesError { + return filesFailure({ operation, phase, reason: "operation-failed" }); +} + +/** + * The failure half of a non-write outcome, rebuilt from validated parts. + * + * The two unreadable cases are not the same as the unrecognized one. A + * container whose `error` is absent or refuses to be read never described what + * went wrong, and there is nothing to report — that is a provider-contract + * failure. An `error` that reads fine but carries data the vocabulary does not + * recognize *did* describe a failure, just not one this version knows: the + * generic sentence covers it and the document carries on. + */ +function failure(result: unknown, operation: FilesOperation, phase: FilesPhase): Result { + const reported = read(result, "error"); + if (reported === undefined) { + throw new FilesInvariantError("protocol"); + } + const data = parseFilesFailure(reported.value); + if (data === undefined || data.operation !== operation) { + return Err(generic(operation, phase)); + } + return Err(filesFailure({ operation: data.operation, phase: data.phase, reason: data.reason })); +} + +/** + * One non-write operation whose success carries a payload, with its whole + * outcome rebuilt from validated parts. + * + * Nothing a provider returned reaches a component: not the container, not the + * error object, not the payload. By the time `` or `` reads + * `result.error` it is reading an object this module made. + */ +function* outcome( + call: Operation, + contract: { + operation: FilesOperation; + phase: FilesPhase; + /** + * Recognize this operation's payload, or answer `undefined`. + * + * Every contract here is **total**: it inspects provider-controlled data, + * so it answers rather than throws. Guarding the call site instead would + * make the guards inside each contract untestable, and an untestable guard + * is one nobody notices going missing. + */ + payload: (value: unknown) => { readonly value: T } | undefined; + }, +): Operation> { + const result = yield* invokeFiles(call); + const settled = settlement(result); + if (settled === undefined) { + throw new FilesInvariantError("protocol"); + } + if (!settled) { + return failure(result, contract.operation, contract.phase); + } + const carried = read(result, "value"); + if (carried === undefined) { + throw new FilesInvariantError("protocol"); + } + const payload = contract.payload(carried.value); + if (payload === undefined) { + throw new FilesInvariantError("protocol"); + } + return Ok(payload.value); +} + +function text(value: unknown): { readonly value: string } | undefined { + return typeof value === "string" ? { value } : undefined; +} + +/** + * A search result, copied out of whatever the provider handed back. + * + * Every step is provider-controlled — recognizing the array brand, reading + * `length`, and reading each element — so each goes through a total reader and + * the walk is by index rather than by `for…of`. Even `Array.isArray` can throw: + * a revoked Proxy refuses it. `undefined` from any step is a refusal to describe + * the result, which the caller turns into one fixed invariant. + * + * The array itself is rebuilt rather than passed along. A provider could return + * something array-shaped whose elements are accessors, or one it goes on + * mutating afterwards — and a document binds this value. + */ +function paths(value: unknown): { readonly value: string[] } | undefined { + if (isArray(value) !== true) { + return undefined; + } + const length = read(value, "length"); + if (typeof length?.value !== "number" || !Number.isInteger(length.value) || length.value < 0) { + return undefined; + } + const copied: string[] = []; + for (let index = 0; index < length.value; index++) { + const entry = read(value, String(index)); + if (typeof entry?.value !== "string") { + return undefined; + } + copied.push(entry.value); + } + return { value: copied }; +} + +/** + * Whether this authored path is admissible, with nothing usable coming back. + * + * The success carries no payload, and Effection spells that as its shared + * `Unit` — `{ ok: true }`, with no `value` member at all. So an **absent** + * `value` is the ordinary successful answer here, and this is the one operation + * where that is true. What is still refused is a `value` that cannot be read, + * and one that is present but is something other than `undefined`: the first + * means the container never described its outcome, and the second means it + * described one this operation does not have. + */ +export function* checkFilePath(input: FilePathInput): Operation> { + const result = yield* invokeFiles(Files.operations.checkFilePath(input)); + const settled = settlement(result); + if (settled === undefined) { + throw new FilesInvariantError("protocol"); + } + if (!settled) { + return failure(result, "check-file-path", "lexical"); + } + const carried = present(result, "value"); + if (carried === undefined) { + throw new FilesInvariantError("protocol"); + } + if (carried) { + // Present, so it has to be readable *and* be the absent payload. Asking + // only whether the value is `undefined` would let a member that refused to + // be read pass as one that read as nothing. + const value = read(result, "value"); + if (value === undefined || value.value !== undefined) { + throw new FilesInvariantError("protocol"); + } + } + return Ok(undefined); +} + +export function readFileText(input: FilePathInput): Operation> { + return outcome(Files.operations.readTextFile(input), { + operation: "read", + phase: "access", + payload: text, + }); +} + +export function globFiles(input: GlobInput): Operation> { + return outcome(Files.operations.globFiles(input), { + operation: "glob", + phase: "traversal", + payload: paths, + }); +} + +export function temporaryDirectory(): Operation> { + return outcome(Files.operations.temporaryDirectory(), { + operation: "temporary-directory", + phase: "acquire", + payload: text, + }); +} + +/** + * What a write reported, or nothing when it succeeded. + * + * A write is the one operation whose outcome makes a claim about the world: the + * file was replaced, or it was not, or nobody can tell. Every sentence a + * consumer could print asserts one of those, so an outcome that does not + * validate leaves none of them available — and a provider that cannot describe + * what it did is treated as one that may not have done it. Both a malformed + * success and a malformed failure are therefore fatal, and so is a container + * that will not say which it is. + * + * The data that comes back is rebuilt from validated parts, like every other + * outcome here. + */ +export function* writeFileText(input: FileWriteInput): Operation { + const result = yield* invokeFiles(Files.operations.writeTextFile(input)); + const settled = settlement(result); + if (settled === undefined) { + throw new FilesInvariantError("protocol"); + } + if (settled) { + const carried = read(result, "value"); + if (carried === undefined || parseFileWriteSuccess(carried.value) === undefined) { + throw new FilesInvariantError("protocol"); + } + return undefined; + } + const reported = read(result, "error"); + if (reported === undefined) { + throw new FilesInvariantError("protocol"); + } + const data = parseFileWriteFailure(reported.value); + if (data === undefined) { + throw new FilesInvariantError("protocol"); + } + return parseFileWriteFailure( + fileWriteFailure({ phase: data.phase, reason: data.reason, cleanup: data.cleanup }), + ); +} diff --git a/packages/core/tests/component-registration.test.ts b/packages/core/tests/component-registration.test.ts index def79cff..a54b845f 100644 --- a/packages/core/tests/component-registration.test.ts +++ b/packages/core/tests/component-registration.test.ts @@ -12,7 +12,7 @@ import { expect } from "@executablemd/test-support/expect"; import { ensure, resource, scoped, spawn, until } from "effection"; import type { Operation } from "effection"; import { ensureDir, rm, writeTextFile } from "@effectionx/fs"; -import { API } from "@executablemd/runtime"; +import { API, useHostFiles } from "@executablemd/runtime"; import { InMemoryStream } from "@executablemd/durable-streams"; import type { Json } from "@executablemd/durable-streams"; import { mkdtemp, realpath } from "node:fs/promises"; @@ -107,6 +107,9 @@ function* thrown(body: () => Operation): Operation { function run(dir: string, componentDirs: string[] = [dir]): Operation { return scoped(function* () { + // `API.Files` has no host default, and `` is one of the components + // these cases resolve. + yield* useHostFiles(); return yield* collect( yield* execute({ path: join(dir, "doc.md"), stream: new InMemoryStream(), componentDirs }), ); @@ -575,6 +578,7 @@ describe("Tier CR — selection is journaled", () => { function runOn(dir: string, stream: InMemoryStream): Operation { return scoped(function* () { + yield* useHostFiles(); return yield* collect( yield* execute({ path: join(dir, "doc.md"), stream, componentDirs: [dir] }), ); diff --git a/packages/core/tests/fatal-cause.test.ts b/packages/core/tests/fatal-cause.test.ts index ba375fa8..54cce018 100644 --- a/packages/core/tests/fatal-cause.test.ts +++ b/packages/core/tests/fatal-cause.test.ts @@ -18,8 +18,21 @@ import { StaleInputError, TerminalDivergenceError, } from "@executablemd/durable-streams"; +import { + FILES_FATAL, + FilesInvariantError, + FilesOperationDeniedError, + FilesProviderUnavailableError, +} from "@executablemd/runtime"; import { InvocationTeardownError } from "../src/invocation.ts"; -import { ContentError, DocumentationError, durabilityFailure, fatalCause } from "../src/errors.ts"; +import { + ContentError, + decidedByOutput, + DocumentationError, + durabilityFailure, + fatalCause, + filesFatalFailure, +} from "../src/errors.ts"; import { Component } from "../src/component-api.ts"; import { expandSegments } from "../src/expand.ts"; import { renderSegments } from "../src/render.ts"; @@ -79,6 +92,39 @@ const DURABILITY_FAILURES: Array<() => Error> = [ () => new DurablePersistenceError("yield", new Error("journal unavailable")), ]; +/** + * One of each Files infrastructure failure. + * + * All three are structurally tagged and carry no cause, so what a failing + * assertion has to distinguish is the kind, not the class. + */ +const FILES_FAILURES: Array<() => Error> = [ + () => new FilesProviderUnavailableError(), + () => new FilesOperationDeniedError("temporary-directory"), + () => new FilesInvariantError("authority"), + () => new FilesInvariantError("savepoint"), + () => new FilesInvariantError("protocol"), + () => new FilesInvariantError("teardown"), +]; + +/** + * A Files failure as a **separately loaded copy** of the runtime package would + * build it. + * + * Two copies can be resolved at once — a repository component reaching its own + * runtime beside the engine's — and `instanceof` answers false across them. The + * structural tag is the whole mechanism, so this is built by hand rather than + * through the constructor: nothing about this object shares a class identity + * with the one core imported. + */ +function foreignFilesFatal(): Error { + const error = new Error("Files provider is not installed"); + error.name = "FilesProviderUnavailableError"; + return Object.assign(error, { + data: Object.freeze({ type: FILES_FATAL, kind: "provider-unavailable" }), + }); +} + describe("Tier FA — Fatal error discovery", () => { it("FA1: an error that is its own cause terminates the search", function* () { const error = new Error("ordinary"); @@ -316,4 +362,155 @@ describe("Tier FA — Fatal error discovery", () => { expect(durabilityFailure(cyclic)).toBe(planted); expect(fatalCause(wrapper)).toBe(planted); }); + + // FA20–FA27: a Files infrastructure failure is the third fatal kind. It is + // not something the document did and not something it can act on, so it + // travels like a durability failure — through every wrapper, and through a + // recovery boundary — and ranks between the two existing kinds. + it("FA20: every Files infrastructure failure is discovered as fatal", function* () { + for (const make of FILES_FAILURES) { + const planted = make(); + expect(fatalCause(planted)).toBe(planted); + expect(filesFatalFailure(planted)).toBe(planted); + + const wrapped = new AggregateError([planted], "wrapped"); + expect(fatalCause(wrapped)).toBe(planted); + expect(filesFatalFailure(wrapped)).toBe(planted); + + // Not a durability failure: the two questions stay separate. + expect(durabilityFailure(planted)).toBeUndefined(); + } + }); + + it("FA21: a Files failure is found through every wrapper the engine builds", function* () { + const planted = new FilesProviderUnavailableError(); + + expect(fatalCause(new InvocationTeardownError([planted]))).toBe(planted); + expect(fatalCause(new AggregateError([new Error("other"), planted]))).toBe(planted); + expect(fatalCause(new Error("wrapper", { cause: planted }))).toBe(planted); + + const deep = new InvocationTeardownError([ + new AggregateError( + [new Error("noise"), new Error("component exploded", { cause: planted })], + "mixed", + ), + ]); + expect(fatalCause(deep)).toBe(planted); + }); + + it("FA22: a Files failure outranks a documentation failure in either order", function* () { + for (const make of FILES_FAILURES) { + const planted = make(); + const doc = documentation(); + + expect(fatalCause(new AggregateError([doc, planted], "mixed"))).toBe(planted); + expect(fatalCause(new AggregateError([planted, doc], "mixed"))).toBe(planted); + expect(fatalCause(new InvocationTeardownError([doc, planted]))).toBe(planted); + expect(fatalCause(new InvocationTeardownError([planted, doc]))).toBe(planted); + } + }); + + it("FA23: a durability failure outranks a Files failure in either order", function* () { + for (const make of DURABILITY_FAILURES) { + const durability = make(); + const files = new FilesInvariantError("protocol"); + + expect(fatalCause(new AggregateError([files, durability], "mixed"))).toBe(durability); + expect(fatalCause(new AggregateError([durability, files], "mixed"))).toBe(durability); + } + }); + + it("FA24: all three kinds at once resolve durability, then Files, then documentation", function* () { + const durability = stale(); + const files = new FilesInvariantError("savepoint"); + const doc = documentation(); + + // All six orders of the three. Position is exactly what must not decide + // this, so a sample of the orderings would leave the claim partly untested — + // and the one ordering left out is the one that would ship the bug. + for (const members of [ + [durability, files, doc], + [durability, doc, files], + [files, durability, doc], + [files, doc, durability], + [doc, durability, files], + [doc, files, durability], + ]) { + expect(fatalCause(new AggregateError(members, "mixed"))).toBe(durability); + // The other wrapper the engine builds, with the same members. + expect(fatalCause(new InvocationTeardownError(members))).toBe(durability); + } + + expect(fatalCause(new AggregateError([doc, files], "mixed"))).toBe(files); + expect(fatalCause(new AggregateError([files, doc], "mixed"))).toBe(files); + + // Nesting cannot change it either: the shallowest member loses to the kind. + const nested = new AggregateError( + [doc, new InvocationTeardownError([new AggregateError([files], "inner")])], + "outer", + ); + expect(fatalCause(nested)).toBe(files); + }); + + it("FA25: a content failure does not hide a Files failure it carries", function* () { + for (const make of FILES_FAILURES) { + const planted = make(); + expect(filesFatalFailure(recovered(planted))).toBe(planted); + expect(fatalCause(recovered(planted))).toBe(planted); + expect(fatalCause(new AuthorContentError(planted))).toBe(planted); + } + + // And against the documentation failure a recovery boundary would otherwise + // let the search stop at. + const files = new FilesInvariantError("teardown"); + const contextual = new Error("component exploded", { + cause: recovered(new AggregateError([documentation(), files], "content")), + }); + expect(fatalCause(contextual)).toBe(files); + }); + + it("FA26: a cyclic graph carrying a Files failure terminates and still finds it", function* () { + const planted = new FilesProviderUnavailableError(); + const noise = new Error("noise"); + const teardown = new InvocationTeardownError([noise, planted]); + noise.cause = teardown; + + expect(fatalCause(teardown)).toBe(planted); + + const cyclic = recovered(); + const wrapper = new AggregateError([cyclic, planted], "mixed"); + cyclic.cause = wrapper; + expect(filesFatalFailure(cyclic)).toBe(planted); + }); + + it("FA27: a Files failure from a separately loaded runtime copy is recognized", function* () { + const foreign = foreignFilesFatal(); + + // The mechanism, stated as a fact about this object: no class identity. + expect(foreign instanceof FilesProviderUnavailableError).toBe(false); + + expect(filesFatalFailure(foreign)).toBe(foreign); + expect(fatalCause(new InvocationTeardownError([documentation(), foreign]))).toBe(foreign); + + // A tag that is not this one is not recognized, so recognition is the tag + // rather than the presence of a `data` member. + const impostor = Object.assign(new Error("looks similar"), { + data: { type: "some.other/v1", kind: "provider-unavailable" }, + }); + expect(filesFatalFailure(impostor)).toBeUndefined(); + expect(fatalCause(impostor)).toBeUndefined(); + }); + + it("FA28: only an output-mode documentation failure is decided by output", function* () { + const output = new DocumentationError({ type: "error", message: "wrong" }, "output"); + expect(decidedByOutput(output)).toBe(true); + expect(decidedByOutput(documentation())).toBe(false); + + for (const make of FILES_FAILURES) { + expect(decidedByOutput(make())).toBe(false); + } + for (const make of DURABILITY_FAILURES) { + expect(decidedByOutput(make())).toBe(false); + } + }); }); diff --git a/packages/core/tests/file-component.test.ts b/packages/core/tests/file-component.test.ts index f1c85a0e..4902f582 100644 --- a/packages/core/tests/file-component.test.ts +++ b/packages/core/tests/file-component.test.ts @@ -17,7 +17,7 @@ import { expect } from "@executablemd/test-support/expect"; import { ensure, race, resource, scoped, sleep, suspend, until } from "effection"; import type { Operation } from "effection"; import { exists, readTextFile, rm, writeTextFile } from "@effectionx/fs"; -import { API } from "@executablemd/runtime"; +import { API, useHostFiles } from "@executablemd/runtime"; import { InMemoryStream } from "@executablemd/durable-streams"; import type { Json } from "@executablemd/durable-streams"; import { execute } from "../src/execute.ts"; @@ -62,11 +62,16 @@ function useFixture(): Operation { } /** - * Install the workspace as the contextual working directory. + * Install the workspace as the contextual working directory, and the host + * document filesystem provider beneath it. * * The document lives in the workspace too, but nothing depends on that: * `` resolves against `Env.cwd`, which this installs explicitly rather * than inheriting from the process. + * + * `API.Files` has no host default, so a suite driving `execute()` directly + * installs the provider the way an entrypoint does. Everything below then + * exercises the real host adapter rather than a stand-in. */ function* useWorkspaceCwd(fixture: Fixture): Operation { yield* API.Env.around( @@ -78,6 +83,7 @@ function* useWorkspaceCwd(fixture: Fixture): Operation { }, { at: "min" }, ); + yield* useHostFiles(); } function run(fixture: Fixture, source: string): Operation { diff --git a/packages/core/tests/files-fatal.test.ts b/packages/core/tests/files-fatal.test.ts new file mode 100644 index 00000000..905d01e5 --- /dev/null +++ b/packages/core/tests/files-fatal.test.ts @@ -0,0 +1,1253 @@ +/** + * Tier FF — Files infrastructure failure (spec §§6.9, 6.11, 6.13, 6.14). + * + * Every other filesystem tier asks what a document reads when something goes + * wrong. This one asks the opposite question: when a document filesystem + * *provider* is missing, refuses an operation, or breaks its own contract, + * nothing is written into the document at all. The execution ends. + * + * The distinction matters because the two are easy to confuse from inside a + * component. "No such file" is something the document did and can fix; "no + * filesystem provider is installed" is not, and printing it as a comment would + * let every sibling after it run as though the file work had happened. + * + * These drive the real components through `execute()` and install providers + * that misbehave in exactly one way each. + */ + +import { describe, it, beforeAll } from "@executablemd/test-support/bdd"; +import { expect } from "@executablemd/test-support/expect"; +import { ensure, Err, Ok, resource, scoped, spawn, suspend, until, withResolvers } from "effection"; +import type { Operation, Result } from "effection"; +import { exists, readTextFile, rm, writeTextFile } from "@effectionx/fs"; +import { + API, + FILES_ERROR, + FILES_FATAL, + FILES_WRITE_SUCCESS, + Files, + FilesError, + FilesInvariantError, + FilesOperationDeniedError, + FilesProviderUnavailableError, + fileWriteSuccess, + hostFilesHandler, + parseFileWriteFailure, + parseFileWriteSuccess, + parseFilesFailure, + parseFilesFatal, + useHostFiles, +} from "@executablemd/runtime"; +import type { FilePathInput, FileWriteInput, FileWriteSuccess } from "@executablemd/runtime"; +import { InMemoryStream, StaleInputError } from "@executablemd/durable-streams"; +import { InvocationTeardownError } from "../src/invocation.ts"; +import { execute } from "../src/execute.ts"; +import { collect } from "../src/collect.ts"; +import { useTempFileCompiler } from "../src/temp-file-compiler.ts"; +import { fatalCause, filesFatalFailure } from "../src/errors.ts"; +import { invokeFiles } from "../src/files.ts"; +import { mkdtemp, realpath } from "node:fs/promises"; +import { tmpdir } from "node:os"; +import { join } from "node:path"; + +function useFixture(): Operation { + return resource(function* (provide) { + const dir = yield* until(mkdtemp(join(tmpdir(), "ff-test-"))); + yield* ensure(() => rm(dir, { recursive: true, force: true })); + yield* provide(yield* until(realpath(dir))); + }); +} + +interface Outcome { + ok: boolean; + error: unknown; + output: string; +} + +/** + * Run `source` as a document, reporting both what it rendered and how the + * execution ended. + * + * Both halves are needed here: a fatal failure is distinguished from a printed + * error precisely by leaving nothing in the output, and by stopping what comes + * after it. + */ +function run(dir: string, source: string, install?: () => Operation): Operation { + return scoped(function* () { + yield* writeTextFile(join(dir, "doc.md"), source); + yield* API.Env.around( + { + // deno-lint-ignore require-yield + *cwd() { + return dir; + }, + }, + { at: "min" }, + ); + if (install) { + yield* install(); + } + const chunks: string[] = []; + const result = yield* scoped(function* () { + const execution = yield* execute({ + path: join(dir, "doc.md"), + stream: new InMemoryStream(), + componentDirs: [dir], + }); + try { + chunks.push(String(yield* collect(execution))); + } catch { + // Collection stops where the failure did; the execution's own outcome + // below is what says why. + } + return yield* execution; + }); + return { + ok: result.ok, + error: result.ok ? undefined : result.error, + output: chunks.join(""), + }; + }); +} + +function invariantCategory(error: unknown): string | undefined { + const data = parseFilesFatal(fatalCause(error)); + return data?.kind === "invariant" ? data.category : undefined; +} + +/** + * Whether anything a hostile provider planted reached the outside. + * + * The rendered document, the failure's own text, its cause chain, and its + * enumerable data are the places a value could surface. Only the *outcome* is + * inspected — never the hostile object — so nothing here runs a trap the + * boundary was supposed to run first. + */ +function leaked(outcome: Outcome): boolean { + const failure = outcome.error; + const shown = [ + outcome.output, + String(failure), + failure instanceof Error ? String(failure.cause) : "", + JSON.stringify(failure instanceof Error ? { ...failure } : {}), + ].join(" "); + return shown.includes("planted") || shown.includes("EACCES"); +} + +/** + * Data as it arrives from outside the type system. + * + * A provider is a contextual handler: at run time it can return whatever it + * likes, and the contract this suite exercises is precisely the one the types + * cannot enforce. Round-tripping through JSON is how a malformed value is + * built here without asserting a type it does not have. + */ +function fromOutside(value: unknown): T { + return JSON.parse(JSON.stringify(value)); +} + +/** Install a provider that answers exactly one way, and nothing else. */ +function useFiles(handler: { + checkFilePath?: (input: FilePathInput) => Operation>; + readTextFile?: (input: FilePathInput) => Operation>; + writeTextFile?: (input: FileWriteInput) => Operation>; + globFiles?: (input: { + cwd: string; + include: string[]; + exclude: string[]; + }) => Operation>; + temporaryDirectory?: () => Operation>; +}): Operation { + return Files.around({ + ...(handler.checkFilePath === undefined + ? {} + : { + *checkFilePath([input], next) { + return yield* (handler.checkFilePath ?? next)(input); + }, + }), + ...(handler.readTextFile === undefined + ? {} + : { + *readTextFile([input], next) { + return yield* (handler.readTextFile ?? next)(input); + }, + }), + ...(handler.writeTextFile === undefined + ? {} + : { + *writeTextFile([input], next) { + return yield* (handler.writeTextFile ?? next)(input); + }, + }), + ...(handler.globFiles === undefined + ? {} + : { + *globFiles([input], next) { + return yield* (handler.globFiles ?? next)(input); + }, + }), + ...(handler.temporaryDirectory === undefined + ? {} + : { + *temporaryDirectory(_args, next) { + return yield* (handler.temporaryDirectory ?? next)(); + }, + }), + }); +} + +/** A `` write whose child leaves a marker on disk if it ever runs. */ +function writeDocument(dir: string): string { + return [ + '', + "```sh exec", + `touch ${join(dir, "child-ran.txt")}`, + "```", + "", + "", + "```sh exec", + `touch ${join(dir, "sibling-ran.txt")}`, + "```", + ].join("\n"); +} + +describe("Tier FF — Files infrastructure failure", () => { + beforeAll(() => useTempFileCompiler()); + + // FF1: no provider at all. The check a write performs before its children is + // the first Files call the document makes, so the failure lands before the + // children — and nothing after the component runs either. + it("FF1: an absent provider fails the execution before children or siblings", function* () { + const dir = yield* useFixture(); + const touched: string[] = []; + + const outcome = yield* run(dir, writeDocument(dir), function* () { + yield* API.Fs.around({ + *writeTextFile([path, content], next) { + touched.push(path); + return yield* next(path, content); + }, + *rename([from, to], next) { + touched.push(from); + return yield* next(from, to); + }, + }); + }); + + expect(outcome.ok).toBe(false); + expect(parseFilesFatal(fatalCause(outcome.error))).toEqual({ + type: "executablemd.runtime.files-fatal/v1", + kind: "provider-unavailable", + }); + // Nothing was written into the document, and nothing ran on either side. + expect(outcome.output).not.toContain("ERROR"); + expect(outcome.output).not.toContain("Files provider"); + expect(yield* exists(join(dir, "child-ran.txt"))).toBe(false); + expect(yield* exists(join(dir, "sibling-ran.txt"))).toBe(false); + // And the low-level host Api was never reached: absence does not fall back. + expect(touched).toEqual([]); + }); + + // FF2: the same for every other form. Each stops at its first Files call. + it("FF2: read, Glob and TempDir all stop at their first provider call", function* () { + const dir = yield* useFixture(); + + for (const source of [ + '\n\nAFTER', + '\n\nAFTER', + "INSIDE\n\nAFTER", + // The self-closing form acquires through `retain()`, which owns the + // resource at the invocation site — a different path to the same call, + // and one whose wrapper could otherwise launder the failure into an + // ordinary one. + '\n\nAFTER', + ]) { + const outcome = yield* run(dir, source); + expect(outcome.ok).toBe(false); + expect(parseFilesFatal(fatalCause(outcome.error))?.kind).toBe("provider-unavailable"); + expect(outcome.output).not.toContain("AFTER"); + expect(outcome.output).not.toContain("INSIDE"); + } + }); + + // FF3: a provider that exists but refuses one operation. A logical filesystem + // owned by a transaction has no temporary directories to give, and inventing + // one would be worse than refusing: the document would run inside somewhere + // the run does not own. + it("FF3: a refused operation is fatal, with its own fixed diagnostic", function* () { + const dir = yield* useFixture(); + + const outcome = yield* run(dir, "INSIDE\n\nAFTER", function* () { + yield* useHostFiles(); + yield* useFiles({ + // deno-lint-ignore require-yield + *temporaryDirectory(): Operation> { + throw new FilesOperationDeniedError("temporary-directory"); + }, + }); + }); + + expect(outcome.ok).toBe(false); + expect(parseFilesFatal(fatalCause(outcome.error))).toEqual({ + type: "executablemd.runtime.files-fatal/v1", + kind: "operation-denied", + operation: "temporary-directory", + }); + expect(String(outcome.error)).toContain("Files provider does not support temporary-directory"); + expect(outcome.output).not.toContain("INSIDE"); + expect(outcome.output).not.toContain("AFTER"); + }); + + // FF4: the ordering the whole two-stage write exists for. A check that fails + // means the children never expand and no second provider call is made — the + // provider-level statement of FL18b and FL18c. + it("FF4: a refused check expands no children and makes no later provider call", function* () { + const dir = yield* useFixture(); + const calls: string[] = []; + + const outcome = yield* run(dir, writeDocument(dir), function* () { + yield* useFiles({ + // deno-lint-ignore require-yield + *checkFilePath(): Operation> { + calls.push("checkFilePath"); + return Err( + new FilesError( + Object.freeze({ + type: FILES_ERROR, + operation: "check-file-path", + phase: "lexical", + reason: "lexical-escape", + }), + ), + ); + }, + // deno-lint-ignore require-yield + *writeTextFile(): Operation> { + calls.push("writeTextFile"); + throw new Error("the write must never be reached"); + }, + }); + }); + + expect(calls).toEqual(["checkFilePath"]); + // An ordinary refusal, so the document reads it and carries on. + expect(outcome.ok).toBe(true); + expect(outcome.output).toContain("resolves outside the working directory"); + expect(yield* exists(join(dir, "child-ran.txt"))).toBe(false); + expect(yield* exists(join(dir, "sibling-ran.txt"))).toBe(true); + }); + + // FF5: a write outcome nobody can read. Every sentence a component could + // print makes a claim about whether the file was replaced, so a provider that + // cannot describe what it did is treated as one that may not have done it. + it("FF5: malformed write data and a malformed success are both fatal", function* () { + const dir = yield* useFixture(); + + const malformed: Array<() => Operation>> = [ + // A phase and a target that contradict each other. + // deno-lint-ignore require-yield + function* () { + return Err( + new FilesError( + Object.freeze({ + type: FILES_ERROR, + operation: "write", + phase: "temporary", + reason: "no-space", + target: "committed", + }), + ), + ); + }, + // A reason outside the vocabulary. + // deno-lint-ignore require-yield + function* () { + return Err( + new FilesError( + fromOutside({ + type: FILES_ERROR, + operation: "write", + phase: "commit", + reason: "the disk caught fire", + target: "commit-unknown", + }), + ), + ); + }, + // A success that does not describe a publication. + // deno-lint-ignore require-yield + function* () { + return Ok(fromOutside({ type: FILES_WRITE_SUCCESS, publication: "made-up" })); + }, + ]; + + for (const writeTextFile of malformed) { + const outcome = yield* run(dir, 'content\n\nAFTER', function* () { + yield* useHostFiles(); + yield* useFiles({ writeTextFile }); + }); + + expect(outcome.ok).toBe(false); + expect(parseFilesFatal(fatalCause(outcome.error))).toEqual({ + type: "executablemd.runtime.files-fatal/v1", + kind: "invariant", + category: "protocol", + }); + expect(String(outcome.error)).toContain("Files provider invariant failed"); + // The category is control data, not text. + expect(String(outcome.error)).not.toContain("protocol"); + expect(outcome.output).not.toContain("AFTER"); + } + }); + + // FF6: malformed data on a *non-write* failure is not fatal. Nothing about a + // target is at stake, and the component already has a sentence for "the + // operation failed", so the document reads that and carries on. + it("FF6: a malformed non-write failure becomes the generic printed error", function* () { + const dir = yield* useFixture(); + + const outcome = yield* run(dir, '\n\nAFTER', function* () { + yield* useFiles({ + // deno-lint-ignore require-yield + *readTextFile(): Operation> { + return Err( + Object.assign(new Error("read failed at /planted/absolute/path"), { + data: { type: FILES_ERROR, operation: "read", phase: "nowhere" }, + }), + ); + }, + }); + }); + + expect(outcome.ok).toBe(true); + expect(outcome.output).toContain('cannot read "notes.md": the filesystem operation failed.'); + expect(outcome.output).not.toContain("/planted/absolute/path"); + expect(outcome.output).toContain("AFTER"); + }); + + // FF7: a handler that throws something arbitrary. Nothing it produced can be + // trusted, so it is replaced rather than wrapped — no cause, no message, no + // errno text, and no host value survives. + it("FF7: an arbitrary throw becomes a sanitized protocol invariant", function* () { + const dir = yield* useFixture(); + const planted = Object.assign(new Error("EACCES: denied, at '/planted/secret.txt'"), { + code: "EACCES", + }); + + let thrown: unknown; + yield* scoped(function* () { + yield* useFiles({ + // deno-lint-ignore require-yield + *readTextFile(): Operation> { + throw planted; + }, + }); + try { + yield* invokeFiles(Files.operations.readTextFile({ cwd: dir, path: "notes.md" })); + } catch (error) { + thrown = error; + } + }); + + expect(thrown).toBeInstanceOf(FilesInvariantError); + expect(thrown).not.toBe(planted); + expect(parseFilesFatal(thrown)?.kind).toBe("invariant"); + expect(thrown instanceof Error ? thrown.message : "").toBe("Files provider invariant failed"); + expect(thrown instanceof Error ? thrown.cause : "unset").toBeUndefined(); + expect(JSON.stringify(thrown instanceof Error ? { ...thrown } : {})).not.toContain("planted"); + }); + + // FF8: an already-meaningful failure is preserved by identity instead. The + // first failure is the one that describes what went wrong, and for a + // durability failure the identity is what the shared fail-stop records. + it("FF8: invokeFiles rethrows an existing durability or Files failure unchanged", function* () { + const dir = yield* useFixture(); + + const durability = new StaleInputError("the journal no longer describes this run"); + const files = new FilesInvariantError("authority"); + + for (const planted of [durability, files]) { + let thrown: unknown; + yield* scoped(function* () { + yield* useFiles({ + // deno-lint-ignore require-yield + *readTextFile(): Operation> { + // Nested, and wrapped, which is how each of them actually arrives. + throw new AggregateError([new Error("noise"), planted], "teardown"); + }, + }); + try { + yield* invokeFiles(Files.operations.readTextFile({ cwd: dir, path: "notes.md" })); + } catch (error) { + thrown = error; + } + }); + + expect(thrown).toBe(planted); + } + }); + + // FF9: precedence between the two, at the wrapper itself. A durability + // failure wins wherever it sits, because a Files invariant raised while one + // is already unwinding is the symptom rather than the cause. + it("FF9: a durability failure beneath a Files failure is the one preserved", function* () { + const dir = yield* useFixture(); + const durability = new StaleInputError("the journal no longer describes this run"); + + let thrown: unknown; + yield* scoped(function* () { + yield* useFiles({ + // deno-lint-ignore require-yield + *readTextFile(): Operation> { + throw new AggregateError([new FilesInvariantError("teardown"), durability], "unwinding"); + }, + }); + try { + yield* invokeFiles(Files.operations.readTextFile({ cwd: dir, path: "notes.md" })); + } catch (error) { + thrown = error; + } + }); + + expect(thrown).toBe(durability); + expect(filesFatalFailure(thrown)).toBeUndefined(); + }); + + // FF11: none of this parsing may throw. Every value it reads is a value a + // provider handed back, and a provider is as free to return a Proxy that + // refuses to be inspected as a plain record. These parsers run from + // `fatalCause`, which every generic catch in expansion consults — so one of + // them throwing would replace the failure being classified with a failure + // about classifying it, exactly when the engine is deciding whether the + // execution may continue. + it("FF11: hostile shapes are not recognized, and never throw", function* () { + const explode = () => { + throw new Error("EACCES: refused, at '/planted/secret.txt'"); + }; + + const hostile: Array<{ name: string; build: () => unknown }> = [ + { + name: "a throwing data accessor", + build: () => + Object.defineProperty(new Error("wrapper"), "data", { get: explode, configurable: true }), + }, + { + name: "data whose fields throw", + build: () => + Object.assign(new Error("wrapper"), { + data: new Proxy({}, { get: explode }), + }), + }, + { + name: "data whose key enumeration throws", + build: () => + Object.assign(new Error("wrapper"), { + data: new Proxy( + { type: FILES_FATAL, kind: "provider-unavailable" }, + { ownKeys: explode }, + ), + }), + }, + { + name: "an unreadable prototype", + build: () => new Proxy(new Error("wrapper"), { getPrototypeOf: explode }), + }, + { + name: "a throwing cause", + build: () => + Object.defineProperty(new Error("wrapper"), "cause", { + get: explode, + configurable: true, + }), + }, + { + name: "unreadable aggregate members", + build: () => + Object.defineProperty(new AggregateError([], "wrapper"), "errors", { + get: explode, + configurable: true, + }), + }, + { + name: "aggregate members that are not a list", + build: () => Object.assign(new AggregateError([], "wrapper"), { errors: 7 }), + }, + { + name: "unreadable teardown causes", + build: () => + Object.defineProperty(new InvocationTeardownError([]), "causes", { + get: explode, + configurable: true, + }), + }, + ]; + + for (const shape of hostile) { + const candidate = shape.build(); + // Every parser answers, and none of them recognizes the shape. + expect(parseFilesFatal(candidate)).toBeUndefined(); + expect(parseFilesFailure(candidate)).toBeUndefined(); + expect(parseFileWriteFailure(candidate)).toBeUndefined(); + expect(parseFileWriteSuccess(candidate)).toBeUndefined(); + expect(fatalCause(candidate)).toBeUndefined(); + expect(filesFatalFailure(candidate)).toBeUndefined(); + + // And a real failure underneath one is still found, so totality narrows + // what a hostile wrapper hides rather than what discovery reaches. + const planted = new FilesInvariantError("authority"); + expect(fatalCause(new AggregateError([candidate, planted], "mixed"))).toBe(planted); + } + }); + + // FF12: a candidate that carries the right tag but breaks the rest of the + // contract is not preserved. Recognition is a decision to let that exact + // object travel onward by identity, so an Error carrying a raw platform + // message and a cause chain would carry both past the boundary the reason + // vocabulary exists to hold. + it("FF12: a correctly tagged but unsafe failure is replaced, not preserved", function* () { + const dir = yield* useFixture(); + + const unsafe: Array<{ name: string; build: () => Error }> = [ + { + name: "a raw message", + build: () => + Object.assign(new Error("EACCES: denied, at '/planted/secret.txt'"), { + data: Object.freeze({ type: FILES_FATAL, kind: "provider-unavailable" }), + }), + }, + { + name: "a cause chain", + build: () => + Object.assign( + new Error("Files provider is not installed", { + cause: new Error("ENOENT at /planted/secret.txt"), + }), + { data: Object.freeze({ type: FILES_FATAL, kind: "provider-unavailable" }) }, + ), + }, + { + name: "mutable data", + build: () => + Object.assign(new Error("Files provider is not installed"), { + data: { type: FILES_FATAL, kind: "provider-unavailable" }, + }), + }, + { + name: "an extra data field", + build: () => + Object.assign(new Error("Files provider is not installed"), { + data: Object.freeze({ + type: FILES_FATAL, + kind: "provider-unavailable", + path: "/planted/secret.txt", + }), + }), + }, + { + name: "a path-bearing name", + build: () => + Object.assign(new Error("Files provider is not installed"), { + name: `FilesProviderUnavailableError: /planted/secret.txt`, + data: Object.freeze({ type: FILES_FATAL, kind: "provider-unavailable" }), + }), + }, + { + name: "an extra Error-level path property", + build: () => + Object.assign(new Error("Files provider is not installed"), { + name: "FilesProviderUnavailableError", + data: Object.freeze({ type: FILES_FATAL, kind: "provider-unavailable" }), + path: "/planted/secret.txt", + }), + }, + { + name: "an enumerable symbol payload", + build: () => { + const error = Object.assign(new Error("Files provider is not installed"), { + name: "FilesProviderUnavailableError", + data: Object.freeze({ type: FILES_FATAL, kind: "provider-unavailable" }), + }); + // Enumerable, so it survives a spread and an `Object.assign` copy — + // the two ways a consumer would carry a failure onward. + Object.defineProperty(error, Symbol.for("planted"), { + value: "/planted/secret.txt", + enumerable: true, + }); + return error; + }, + }, + { + name: "hostile key enumeration", + build: () => + new Proxy( + Object.assign(new Error("Files provider is not installed"), { + name: "FilesProviderUnavailableError", + data: Object.freeze({ type: FILES_FATAL, kind: "provider-unavailable" }), + }), + { + ownKeys() { + throw new Error("refused, at '/planted/secret.txt'"); + }, + }, + ), + }, + { + name: "hostile descriptor access", + build: () => + new Proxy( + Object.assign(new Error("Files provider is not installed"), { + name: "FilesProviderUnavailableError", + data: Object.freeze({ type: FILES_FATAL, kind: "provider-unavailable" }), + }), + { + getOwnPropertyDescriptor() { + throw new Error("refused, at '/planted/secret.txt'"); + }, + }, + ), + }, + ]; + + for (const shape of unsafe) { + const candidate = shape.build(); + expect(filesFatalFailure(candidate)).toBeUndefined(); + + let thrown: unknown; + yield* scoped(function* () { + yield* useFiles({ + // deno-lint-ignore require-yield + *readTextFile(): Operation> { + throw candidate; + }, + }); + try { + yield* invokeFiles(Files.operations.readTextFile({ cwd: dir, path: "notes.md" })); + } catch (error) { + thrown = error; + } + }); + + // Replaced rather than preserved, and nothing it carried came along. + expect(thrown).not.toBe(candidate); + expect(thrown).toBeInstanceOf(FilesInvariantError); + expect(parseFilesFatal(thrown)?.kind).toBe("invariant"); + expect(thrown instanceof Error ? thrown.message : "").toBe("Files provider invariant failed"); + expect(thrown instanceof Error ? thrown.cause : "unset").toBeUndefined(); + // Neither stringification nor enumerable output retains the planted value. + expect(String(thrown)).not.toContain("planted"); + expect(JSON.stringify(thrown instanceof Error ? { ...thrown } : {})).not.toContain("planted"); + expect( + JSON.stringify( + Object.getOwnPropertySymbols(thrown instanceof Error ? thrown : {}).map(String), + ), + ).not.toContain("planted"); + } + + // And the real constructors, which have to stay recognizable: the contract + // above is what they produce, not a stricter shape nothing satisfies. + for (const genuine of [ + new FilesProviderUnavailableError(), + new FilesOperationDeniedError("temporary-directory"), + new FilesInvariantError("teardown"), + ]) { + expect(filesFatalFailure(genuine)).toBe(genuine); + } + }); + + // FF13: a host cleanup that fails while cancellation is unwinding leaves the + // scope as an infrastructure failure, and the engine's own discovery is what + // has to find it — that is what makes it consumable by a coordinator deciding + // what to fence. + it("FF13: a cleanup failure during cancellation is discovered as fatal", function* () { + const dir = yield* useFixture(); + yield* writeTextFile(join(dir, "notes.md"), "first"); + const suspended = withResolvers(); + const files = hostFilesHandler(); + + const write = yield* spawn(function* () { + yield* scoped(function* () { + yield* API.Fs.around({ + *writeTextFile([path, content], next) { + yield* next(path, content); + suspended.resolve(); + yield* suspend(); + }, + // deno-lint-ignore require-yield + *remove() { + throw Object.assign(new Error("EPERM: denied, at '/planted/secret.txt'"), { + code: "EPERM", + }); + }, + }); + yield* files.writeTextFile({ cwd: dir, path: "notes.md", content: "second" }); + }); + }); + + yield* suspended.operation; + let thrown: unknown; + try { + yield* write.halt(); + } catch (error) { + thrown = error; + } + + const selected = fatalCause(thrown); + expect(parseFilesFatal(selected)).toEqual({ + type: "executablemd.runtime.files-fatal/v1", + kind: "invariant", + category: "teardown", + }); + // Preserved by identity, which is what a shared fail-stop records. + expect(filesFatalFailure(thrown)).toBe(selected); + expect(String(selected)).not.toContain("planted"); + expect(yield* readTextFile(join(dir, "notes.md"))).toBe("first"); + }); + + // FF14: the `Result` a provider returns is only conventionally a Result. The + // TypeScript signature is a claim about the provider, not a guarantee, so a + // component that read `ok`, `value`, or `error` first would be the thing that + // ran the hostile accessor — outside anything that sanitizes. + // + // Every shape below is a live Proxy **around a real `Ok`/`Err`**, so it keeps + // the declared type without a cast and, more importantly, its traps have never + // run when the provider hands it back. The first thing to touch them is the + // normalization boundary. Serializing one here instead would invoke the getter + // inside the test and prove only the handler-throw path FF7 already covers. + it("FF14: a hostile Result never reaches a component", function* () { + const dir = yield* useFixture(); + yield* writeTextFile(join(dir, "notes.md"), "existing"); + + /** + * A Result whose named member is there and refuses to be read. + * + * The `has` trap matters as much as the `get` one: a payload-free success is + * Effection's `Unit`, which has no `value` member at all, so a `get` trap + * alone would never be consulted and the case would test nothing. + */ + function throwing(result: Result, member: string): Result { + return new Proxy(result, { + has(target, property) { + return property === member ? true : Reflect.has(target, property); + }, + get(target, property, receiver) { + if (property === member) { + throw new Error("EACCES: refused, at '/planted/secret.txt'"); + } + return Reflect.get(target, property, receiver); + }, + }); + } + + /** A Result whose named member is not there at all. */ + function absent(result: Result, member: string): Result { + return new Proxy(result, { + has(target, property) { + return property === member ? false : Reflect.has(target, property); + }, + get(target, property, receiver) { + return property === member ? undefined : Reflect.get(target, property, receiver); + }, + }); + } + + /** A Result that will not say how it settled. */ + function undecided(result: Result): Result { + return new Proxy(result, { + get(target, property, receiver) { + return property === "ok" ? "yes" : Reflect.get(target, property, receiver); + }, + }); + } + + const ordinary = new Error("EACCES: denied, at '/planted/secret.txt'"); + + /** + * Which settlement each shape actually bites. + * + * A trap on `value` is never consulted by a failure, and one on `error` is + * never consulted by a success — so pairing every shape with every + * settlement would demand a fatal outcome from combinations that + * legitimately take the ordinary path, and the test would measure the wrong + * thing. + */ + const shapes: Array<{ + name: string; + settlements: ReadonlyArray<"success" | "failure">; + skip?: string; + hostile: (result: Result) => Result; + }> = [ + { + name: "a throwing ok", + settlements: ["success", "failure"], + hostile: (result) => throwing(result, "ok"), + }, + { + name: "an absent ok", + settlements: ["success", "failure"], + hostile: (result) => absent(result, "ok"), + }, + { name: "a non-boolean ok", settlements: ["success", "failure"], hostile: undecided }, + { + name: "a throwing value", + settlements: ["success"], + hostile: (result) => throwing(result, "value"), + }, + { + name: "an absent value", + settlements: ["success"], + // `checkFilePath` succeeds with no payload, and Effection spells that as + // its shared `Unit` — `{ ok: true }`, with no `value` member at all. + // Absence is the ordinary success there, which FF14c asserts positively. + skip: "checkFilePath", + hostile: (result) => absent(result, "value"), + }, + { + name: "a throwing error", + settlements: ["failure"], + hostile: (result) => throwing(result, "error"), + }, + { + name: "an absent error", + settlements: ["failure"], + hostile: (result) => absent(result, "error"), + }, + ]; + + const operations: Array<{ + name: string; + source: string; + method: string; + ok: () => Result; + }> = [ + { + name: "read", + source: '\n\nAFTER', + method: "readTextFile", + ok: () => Ok("existing"), + }, + { + name: "write", + source: 'content\n\nAFTER', + method: "writeTextFile", + ok: () => Ok(fileWriteSuccess("host-committed")), + }, + { + name: "checkFilePath", + source: 'content\n\nAFTER', + method: "checkFilePath", + ok: () => Ok(undefined), + }, + { + name: "glob", + source: '\n\nAFTER', + method: "globFiles", + ok: () => Ok(["notes.md"]), + }, + { + name: "tempdir", + source: "INSIDE\n\nAFTER", + method: "temporaryDirectory", + ok: () => Ok(dir), + }, + ]; + + for (const operation of operations) { + for (const shape of shapes) { + if (shape.skip === operation.name) { + continue; + } + for (const settlement of shape.settlements) { + const settled = settlement === "success" ? operation.ok : () => Err(ordinary); + const outcome = yield* run(dir, operation.source, function* () { + yield* useHostFiles(); + yield* Files.around({ + // deno-lint-ignore require-yield + *[operation.method]() { + return shape.hostile(settled()); + }, + }); + }); + + // A container that will not describe its own outcome is a provider + // contract failure: the execution ends and later work stops. + expect(outcome.ok).toBe(false); + expect(invariantCategory(outcome.error)).toBe("protocol"); + expect(outcome.output).not.toContain("AFTER"); + expect(outcome.output).not.toContain("INSIDE"); + // No child of the write ever expanded, either. + expect(yield* exists(join(dir, "out.txt"))).toBe(false); + expect(leaked(outcome)).toBe(false); + } + } + } + }); + + // FF14b: the array a search returns is provider-controlled all the way down — + // its length and every element. The copy walks it by index through the same + // total reader, so a trap that refuses becomes the same fixed invariant rather + // than an exception escaping into expansion. + it("FF14b: a hostile search array becomes a sanitized invariant", function* () { + const dir = yield* useFixture(); + const explode = () => { + throw new Error("EACCES: refused, at '/planted/secret.txt'"); + }; + + const arrays: Array<{ name: string; build: () => string[] }> = [ + { + name: "a throwing element", + build: () => + new Proxy(["notes.md"], { + get(target, property, receiver) { + return property === "0" ? explode() : Reflect.get(target, property, receiver); + }, + }), + }, + { + name: "a throwing length", + build: () => + new Proxy(["notes.md"], { + get(target, property, receiver) { + return property === "length" ? explode() : Reflect.get(target, property, receiver); + }, + }), + }, + { + name: "a refused element", + build: () => + new Proxy(["notes.md"], { + has(target, property) { + return property === "0" ? false : Reflect.has(target, property); + }, + }), + }, + { + name: "an element that is not a string", + build: () => + new Proxy(["notes.md"], { + get(target, property, receiver) { + return property === "0" ? 7 : Reflect.get(target, property, receiver); + }, + }), + }, + ]; + + for (const array of arrays) { + const outcome = yield* run( + dir, + '\n\nAFTER', + function* () { + yield* useHostFiles(); + yield* Files.around({ + // deno-lint-ignore require-yield + *globFiles() { + return Ok(array.build()); + }, + }); + }, + ); + + expect(outcome.ok).toBe(false); + expect(invariantCategory(outcome.error)).toBe("protocol"); + expect(outcome.output).not.toContain("AFTER"); + expect(leaked(outcome)).toBe(false); + } + + // A revoked Proxy is the shape that escapes everything else: recognizing it + // as an array is itself an operation on provider-controlled data, and + // `Array.isArray` throws on one. It is built valid, wrapped in `Ok` without + // being inspected, and revoked before the provider returns — so the first + // thing to touch it is the boundary, and the brand check is what it touches + // it with. + const revoked = yield* run( + dir, + '\n\nAFTER', + function* () { + yield* useHostFiles(); + yield* Files.around({ + // deno-lint-ignore require-yield + *globFiles() { + const { proxy, revoke } = Proxy.revocable(["notes.md"], {}); + const result = Ok(proxy); + revoke(); + return result; + }, + }); + }, + ); + + expect(revoked.ok).toBe(false); + expect(invariantCategory(revoked.error)).toBe("protocol"); + expect(revoked.output).not.toContain("AFTER"); + // The platform's own failure is replaced, not carried. + const selected = fatalCause(revoked.error); + expect(selected).not.toBeInstanceOf(TypeError); + expect(selected instanceof Error ? selected.cause : "unset").toBeUndefined(); + const shown = [revoked.output, String(selected), String(revoked.error)].join(" "); + expect(shown).not.toContain("IsArray"); + expect(shown).not.toContain("revoked"); + + // The iterator is never consulted, because the walk is by index. A search + // whose only hostile trap is `Symbol.iterator` therefore copies cleanly — + // which is the derivation this kills: a `for…of` walk would have run it. + const untouched = yield* run( + dir, + '\n\nfound: {found}', + function* () { + yield* useHostFiles(); + yield* Files.around({ + // deno-lint-ignore require-yield + *globFiles() { + return Ok( + new Proxy(["b.md", "a.md"], { + get(target, property, receiver) { + return property === Symbol.iterator + ? explode() + : Reflect.get(target, property, receiver); + }, + }), + ); + }, + }); + }, + ); + expect(untouched.ok).toBe(true); + expect(untouched.output).toContain("found: b.md,a.md"); + + // And what comes back is the document's own array: mutating the provider's + // afterwards does not reach the bound value. + const source = ["b.md", "a.md"]; + const copied = yield* run( + dir, + '\n\nfound: {found}', + function* () { + yield* useHostFiles(); + yield* Files.around({ + // deno-lint-ignore require-yield + *globFiles() { + return Ok(source); + }, + }); + }, + ); + source.push("planted.md"); + expect(copied.ok).toBe(true); + expect(copied.output).toContain("found: b.md,a.md"); + expect(copied.output).not.toContain("planted.md"); + }); + + // FF14c: a legitimate payload-free success is still a success. Effection + // spells one as its shared `Unit` — `{ ok: true }`, with no `value` member — + // so requiring the member to be present would reject the very thing the host + // adapter returns from `checkFilePath`. + it("FF14c: a payload-free success is accepted, however it is spelled", function* () { + const dir = yield* useFixture(); + + for (const admitted of [() => Ok(undefined), () => Ok(void 0)]) { + const outcome = yield* run(dir, 'content\n\nAFTER', function* () { + yield* useHostFiles(); + yield* Files.around({ + // deno-lint-ignore require-yield + *checkFilePath() { + return admitted(); + }, + }); + }); + + expect(outcome.ok).toBe(true); + expect(outcome.output).toContain("AFTER"); + expect(yield* readTextFile(join(dir, "out.txt"))).toBe("content"); + } + + // A payload-free success that carries a payload anyway is not one. The + // payload arrives through a Proxy, so the declared type survives without a + // cast and the boundary is what first observes it. + const carrying = yield* run(dir, 'content\n\nAFTER', function* () { + yield* useHostFiles(); + yield* Files.around({ + // deno-lint-ignore require-yield + *checkFilePath() { + return new Proxy(Ok(undefined), { + has(target, property) { + return property === "value" ? true : Reflect.has(target, property); + }, + get(target, property, receiver) { + return property === "value" + ? "/planted/secret.txt" + : Reflect.get(target, property, receiver); + }, + }); + }, + }); + }); + expect(carrying.ok).toBe(false); + expect(invariantCategory(carrying.error)).toBe("protocol"); + expect(leaked(carrying)).toBe(false); + }); + + // FF15: the one hostile shape that is *not* fatal. A non-write failure whose + // data does not validate has no claim about a target in it, and the vocabulary + // already has a sentence for an operation that failed for an unrecognized + // reason — so the document reads that and carries on. What it must not read is + // anything the provider put there. + it("FF15: malformed non-write failure data alone stays printable", function* () { + const dir = yield* useFixture(); + + const cases: Array<{ source: string; method: string; expected: string }> = [ + { + source: '\n\nAFTER', + method: "readTextFile", + expected: 'cannot read "notes.md": the filesystem operation failed.', + }, + { + source: '\n\nAFTER', + method: "globFiles", + expected: "cannot search the working directory: the filesystem operation failed.", + }, + { + source: "INSIDE\n\nAFTER", + method: "temporaryDirectory", + expected: "cannot create a temporary directory: the filesystem operation failed.", + }, + ]; + + for (const shape of cases) { + const outcome = yield* run(dir, shape.source, function* () { + yield* useHostFiles(); + yield* Files.around({ + // deno-lint-ignore require-yield + *[shape.method]() { + return Err( + Object.assign(new Error("EACCES: denied, at '/planted/secret.txt'"), { + data: { type: FILES_ERROR, operation: "read", phase: "nowhere" }, + path: "/planted/secret.txt", + }), + ); + }, + }); + }); + + expect(outcome.ok).toBe(true); + expect(outcome.output).toContain(shape.expected); + expect(outcome.output).not.toContain("/planted/secret.txt"); + expect(outcome.output).not.toContain("EACCES"); + // The document carried on, which is the whole difference from FF14. + expect(outcome.output).toContain("AFTER"); + } + }); + + // FF10: an ordinary failure is still an ordinary failure. The fatal rule is + // for a provider that is missing or wrong, not for everything that goes + // wrong beneath one. + it("FF10: an ordinary provider failure stays a printed error", function* () { + const dir = yield* useFixture(); + + const outcome = yield* run(dir, '\n\nAFTER', function* () { + yield* useHostFiles(); + }); + + expect(outcome.ok).toBe(true); + expect(outcome.output).toContain('cannot read "absent.md": no such file.'); + expect(outcome.output).toContain("AFTER"); + }); +}); diff --git a/packages/core/tests/glob-component.test.ts b/packages/core/tests/glob-component.test.ts index a3a34512..cca795fe 100644 --- a/packages/core/tests/glob-component.test.ts +++ b/packages/core/tests/glob-component.test.ts @@ -18,7 +18,7 @@ import { expect } from "@executablemd/test-support/expect"; import { ensure, race, resource, scoped, sleep, suspend, until } from "effection"; import type { Operation } from "effection"; import { ensureDir, FsApi, rm, writeTextFile } from "@effectionx/fs"; -import { API } from "@executablemd/runtime"; +import { API, useHostFiles } from "@executablemd/runtime"; import { InMemoryStream } from "@executablemd/durable-streams"; import type { Json } from "@executablemd/durable-streams"; import { execute } from "../src/execute.ts"; @@ -92,6 +92,9 @@ function runWith( yield* writeTextFile(path, source); } yield* useCwd(fixture.workspace); + // `API.Files` has no host default, so a suite driving `execute()` directly + // installs the provider the way an entrypoint does. + yield* useHostFiles(); if (install) { yield* install(); } diff --git a/packages/core/tests/inline-root.test.ts b/packages/core/tests/inline-root.test.ts index 5f07b30b..25ed1329 100644 --- a/packages/core/tests/inline-root.test.ts +++ b/packages/core/tests/inline-root.test.ts @@ -23,7 +23,7 @@ import { expect } from "@executablemd/test-support/expect"; import { ensure, scoped, until } from "effection"; import type { Operation } from "effection"; import { InMemoryStream } from "@executablemd/durable-streams"; -import { API } from "@executablemd/runtime"; +import { API, useHostFiles } from "@executablemd/runtime"; import { useStubFs } from "@executablemd/runtime/test"; import { rm, writeTextFile } from "@effectionx/fs"; import { mkdtemp } from "node:fs/promises"; @@ -131,6 +131,7 @@ function* useWorkspace(files: Record): Operation { }, { at: "min" }, ); + yield* useHostFiles(); return root; } diff --git a/packages/core/tests/loaded-copy-files.test.ts b/packages/core/tests/loaded-copy-files.test.ts new file mode 100644 index 00000000..90b5c960 --- /dev/null +++ b/packages/core/tests/loaded-copy-files.test.ts @@ -0,0 +1,146 @@ +/** + * Tier LC — a Files failure built by a genuinely separate copy of the runtime. + * + * `FA27` builds a correctly tagged object by hand, which proves the parser + * accepts the shape. It does not prove the case the shape exists for: two + * copies of `@executablemd/runtime` evaluated in one process, where a + * repository component reached its own dependency beside the engine's. In that + * world `instanceof` answers false, every class identity differs, and the tag + * is the only thing the two copies share. + * + * So this bundles `packages/runtime/files.ts` the way an installed dependency + * would arrive, imports the bundle as its own module, constructs a failure with + * *that* copy's constructor, and asks the engine's own discovery about it. + * + * Bundling is also the second half of a claim the smoke job makes: this module + * has to stay reachable without the runtime's host Apis, one of which carries a + * native addon no bundler can inline. A regression there fails here as a + * bundling error rather than as a mysterious CI break. + * + * Deno-only: `deno bundle` is Deno's. + */ + +import { describe, it } from "@executablemd/test-support/bdd"; +import { expect } from "@executablemd/test-support/expect"; +import { ensure, resource, until } from "effection"; +import type { Operation } from "effection"; +import { exec } from "@effectionx/process"; +import { rm } from "@effectionx/fs"; +import { + FilesInvariantError, + FilesProviderUnavailableError, + parseFilesFatal, +} from "@executablemd/runtime"; +import { InvocationTeardownError } from "../src/invocation.ts"; +import { DocumentationError, fatalCause, filesFatalFailure } from "../src/errors.ts"; +import { mkdtemp } from "node:fs/promises"; +import { tmpdir } from "node:os"; +import { join } from "node:path"; +import { fileURLToPath } from "node:url"; +import process from "node:process"; + +const FILES_MODULE = fileURLToPath(new URL("../../runtime/files.ts", import.meta.url)); +const REPOSITORY = fileURLToPath(new URL("../../../", import.meta.url)); + +/** What a separately loaded copy of the module exposes to this test. */ +interface LoadedCopy { + FilesProviderUnavailableError: new () => Error; + FilesInvariantError: new (category: string) => Error; + parseFilesFatal: (error: unknown) => unknown; +} + +function isLoadedCopy(value: unknown): value is LoadedCopy { + if (typeof value !== "object" || value === null) { + return false; + } + const module: Record = Object.fromEntries(Object.entries(value)); + return ( + typeof module.FilesProviderUnavailableError === "function" && + typeof module.FilesInvariantError === "function" && + typeof module.parseFilesFatal === "function" + ); +} + +/** + * `packages/runtime/files.ts`, bundled and evaluated as its own module. + * + * The bundle is what makes the copy separate: importing the source path again + * would resolve to the module this test already holds, and share every class + * with it. + */ +function useSeparateCopy(): Operation { + return resource(function* (provide) { + const directory = yield* until(mkdtemp(join(tmpdir(), "lc-files-"))); + yield* ensure(() => rm(directory, { recursive: true, force: true })); + const bundle = join(directory, "files.js"); + + // `process.execPath` under Deno is the deno binary, so the driver stays + // typed against node:process rather than a runtime global. + const built = yield* exec(process.execPath, { + arguments: [ + "bundle", + "--frozen", + "--node-modules-dir=none", + FILES_MODULE, + "--output", + bundle, + ], + cwd: REPOSITORY, + }).join(); + if (built.code !== 0) { + throw new Error( + `could not bundle the runtime's Files module:\n${built.stdout}${built.stderr}`, + ); + } + + const loaded: unknown = yield* until(import(`file://${bundle}`)); + if (!isLoadedCopy(loaded)) { + throw new Error("the bundled copy does not expose the Files failure surface"); + } + yield* provide(loaded); + }); +} + +describe("Tier LC — a separately loaded runtime copy", () => { + it("LC1: a failure built by another copy is discovered as fatal, by identity", function* () { + const copy = yield* useSeparateCopy(); + const foreign = new copy.FilesProviderUnavailableError(); + + // The premise, stated as a fact about this object rather than assumed: + // nothing about it shares a class with the copy core imported. + expect(foreign instanceof FilesProviderUnavailableError).toBe(false); + expect(foreign.constructor).not.toBe(FilesProviderUnavailableError); + + // Recognized anyway, and returned as the object that was thrown. + expect(filesFatalFailure(foreign)).toBe(foreign); + expect(parseFilesFatal(foreign)).toEqual({ + type: "executablemd.runtime.files-fatal/v1", + kind: "provider-unavailable", + }); + + // Through every wrapper the engine builds, and ahead of a documentation + // failure sitting beside it. + const documentation = new DocumentationError({ type: "error", message: "wrong" }, "throw"); + expect(fatalCause(new InvocationTeardownError([documentation, foreign]))).toBe(foreign); + expect(fatalCause(new AggregateError([foreign, documentation], "mixed"))).toBe(foreign); + expect(fatalCause(new Error("wrapper", { cause: foreign }))).toBe(foreign); + }); + + it("LC2: recognition crosses in both directions", function* () { + const copy = yield* useSeparateCopy(); + + // This copy's failure, recognized by the other one's parser. + expect(copy.parseFilesFatal(new FilesInvariantError("savepoint"))).toEqual({ + type: "executablemd.runtime.files-fatal/v1", + kind: "invariant", + category: "savepoint", + }); + + // And the other's invariant, recognized here, category and all. + expect(parseFilesFatal(new copy.FilesInvariantError("authority"))).toEqual({ + type: "executablemd.runtime.files-fatal/v1", + kind: "invariant", + category: "authority", + }); + }); +}); diff --git a/packages/core/tests/loop.test.ts b/packages/core/tests/loop.test.ts index 845e06f7..fdd2c82c 100644 --- a/packages/core/tests/loop.test.ts +++ b/packages/core/tests/loop.test.ts @@ -12,6 +12,7 @@ import type { SourceOrigin } from "../src/scanner.ts"; import { renderSegments } from "../src/render.ts"; import { DivergenceError, InMemoryStream, StaleInputError } from "@executablemd/durable-streams"; import type { DurableEvent, Json, Result } from "@executablemd/durable-streams"; +import { useHostFiles } from "@executablemd/runtime"; import { useEchoExec, useStubFs } from "@executablemd/runtime/test"; import { execute } from "../src/execute.ts"; import { collect } from "../src/collect.ts"; @@ -694,6 +695,7 @@ describe("Tier LOOP — document execution", () => { it("LOOP35: every iteration journals its own eval entry", function* () { const stream = new InMemoryStream(); + yield* useHostFiles(); yield* useStubFs({ "test.md": ["", "", "```js eval", "output('RAN');", "```", "", ""].join( "\n", @@ -713,6 +715,7 @@ describe("Tier LOOP — document execution", () => { it("LOOP36: content skipped by writes no journal entry", function* () { const stream = new InMemoryStream(); + yield* useHostFiles(); yield* useStubFs({ "test.md": [ "", @@ -746,6 +749,7 @@ describe("Tier LOOP — document execution", () => { it("LOOP37: a binding accumulates across iterations", function* () { const stream = new InMemoryStream(); + yield* useHostFiles(); yield* useStubFs({ "test.md": [ "```js eval", @@ -774,6 +778,7 @@ describe("Tier LOOP — document execution", () => { it("LOOP38: a component in the loop body is imported once per iteration", function* () { const stream = new InMemoryStream(); + yield* useHostFiles(); yield* useStubFs({ "components/Step.md": "step\n", "test.md": "", @@ -800,6 +805,7 @@ describe("Tier LOOP — document execution", () => { ].join("\n"); const stream = new InMemoryStream(); + yield* useHostFiles(); yield* useStubFs({ "test.md": DOC }); yield* useEchoExec(); @@ -841,6 +847,7 @@ describe("Tier LOOP — document execution", () => { it("LOOP40: a chosen from a binding stops the document's loop", function* () { const stream = new InMemoryStream(); + yield* useHostFiles(); yield* useStubFs({ "test.md": [ "```js eval", @@ -882,6 +889,7 @@ describe("Tier LOOP — execution records", () => { function runDoc(doc: string, files: Record = {}) { return scoped(function* () { const stream = new InMemoryStream(); + yield* useHostFiles(); yield* useStubFs({ "test.md": doc, ...files }); yield* useEchoExec(); let output = ""; @@ -994,6 +1002,7 @@ describe("Tier LOOP — execution records", () => { reachedSecond.resolve(); } }; + yield* useHostFiles(); yield* useStubFs({ "test.md": DOC }); yield* useEchoExec(); @@ -1082,6 +1091,7 @@ describe("Tier LOOP — execution records", () => { enteredThird.resolve(); } }; + yield* useHostFiles(); yield* useStubFs({ "test.md": DOC }); yield* useEchoExec(); @@ -1105,6 +1115,7 @@ describe("Tier LOOP — execution records", () => { // The incomplete journal, read back and handed to a new execution. const resumed = yield* scoped(function* () { const stream = new InMemoryStream(interrupted); + yield* useHostFiles(); yield* useStubFs({ "test.md": DOC }); yield* useEchoExec(); const execution = yield* execute({ @@ -1184,6 +1195,7 @@ describe("Tier LOOP — replay validates the terminal record", () => { ) { return scoped(function* () { const stream = new InMemoryStream(); + yield* useHostFiles(); yield* useStubFs({ "test.md": doc, ...files }); yield* useEchoExec(); yield* collect(yield* execute({ path: "test.md", stream, props })); @@ -1201,6 +1213,7 @@ describe("Tier LOOP — replay validates the terminal record", () => { ) { return scoped(function* () { const stream = new InMemoryStream(journal); + yield* useHostFiles(); yield* useStubFs({ "test.md": doc, ...files }); yield* useEchoExec(); let failure: unknown; @@ -1303,6 +1316,7 @@ describe("Tier LOOP — replay validates the terminal record", () => { const cut = yield* scoped(function* () { const stream = new InMemoryStream(); + yield* useHostFiles(); yield* useStubFs({ "test.md": HELD }); yield* useEchoExec(); yield* collect(yield* execute({ path: "test.md", stream })); @@ -1328,6 +1342,7 @@ describe("Tier LOOP — replay validates the terminal record", () => { const run = yield* scoped(function* () { const stream = new InMemoryStream(); + yield* useHostFiles(); yield* useStubFs({ "test.md": "ab" }); yield* useEchoExec(); // No generic catch sits above the component, so the wrapper reaches the @@ -1367,6 +1382,7 @@ describe("Tier LOOP — replay validates the terminal record", () => { const run = yield* scoped(function* () { const stream = new InMemoryStream(); + yield* useHostFiles(); yield* useStubFs({ "test.md": "ab" }); yield* useEchoExec(); // Thrown from a component, so it travels through the generic catch that @@ -1429,6 +1445,7 @@ describe("Tier LOOP — replay validates the terminal record", () => { const complete = yield* scoped(function* () { const stream = new InMemoryStream(); + yield* useHostFiles(); yield* useStubFs({ "test.md": DIVERGING }); yield* useEchoExec(); yield* collect(yield* execute({ path: "test.md", stream })); @@ -1528,6 +1545,7 @@ describe("Tier BREAK — the projection boundary", () => { function runDoc(doc: string, files: Record) { return scoped(function* () { + yield* useHostFiles(); yield* useStubFs({ "test.md": doc, ...files }); yield* useEchoExec(); return asText( diff --git a/packages/core/tests/output-error-mode.test.ts b/packages/core/tests/output-error-mode.test.ts index fdcf1c0d..2160b407 100644 --- a/packages/core/tests/output-error-mode.test.ts +++ b/packages/core/tests/output-error-mode.test.ts @@ -18,7 +18,7 @@ import { ensure, scoped } from "effection"; import type { Operation } from "effection"; import { InMemoryStream } from "@executablemd/durable-streams"; import type { DurableEvent } from "@executablemd/durable-streams"; -import { API } from "@executablemd/runtime"; +import { API, useHostFiles } from "@executablemd/runtime"; import { useStubFs } from "@executablemd/runtime/test"; import { forEach } from "@effectionx/stream-helpers"; import { execute } from "../src/execute.ts"; @@ -68,6 +68,8 @@ function run(files: Record, stream = new InMemoryStream()): Oper return scoped(function* () { yield* useStubFs(files); yield* useStagedExec(); + // `` reaches `API.Files`, which has no host default. + yield* useHostFiles(); const execution = yield* execute({ path: "doc.md", stream }); const chunks: string[] = []; diff --git a/packages/core/tests/temp-dir.test.ts b/packages/core/tests/temp-dir.test.ts index 7ffbc84e..2abb06ba 100644 --- a/packages/core/tests/temp-dir.test.ts +++ b/packages/core/tests/temp-dir.test.ts @@ -22,6 +22,7 @@ import { import type { Operation } from "effection"; import { when } from "@effectionx/converge"; import { cwd, exists, readTextFile, rm, writeTextFile } from "@effectionx/fs"; +import { useHostFiles } from "@executablemd/runtime"; import { InMemoryStream, StaleInputError } from "@executablemd/durable-streams"; import type { Json } from "@executablemd/durable-streams"; import { execute } from "../src/execute.ts"; @@ -56,6 +57,7 @@ function writeDocument(dir: string, source: string): Operation { /** One bounded run, so the document scope closes before effects are read. */ function run(dir: string): Operation { return scoped(function* () { + yield* useHostFiles(); return yield* collect( yield* execute({ path: join(dir, "doc.md"), @@ -114,6 +116,7 @@ describe("Tier TD — TempDir", () => { const stream = new InMemoryStream(); const output = yield* scoped(function* () { + yield* useHostFiles(); return yield* collect( yield* execute({ path: join(dir, "doc.md"), stream, componentDirs: [dir] }), ); @@ -381,6 +384,7 @@ describe("Tier TD — TempDir", () => { const stream = new InMemoryStream(); const first = String( yield* scoped(function* () { + yield* useHostFiles(); return yield* collect( yield* execute({ path: join(dir, "doc.md"), stream, componentDirs: [dir] }), ); @@ -401,6 +405,7 @@ describe("Tier TD — TempDir", () => { (event) => !(event.type === "close" && event.coroutineId === "root"), ); const outcome = yield* scoped(function* () { + yield* useHostFiles(); const execution = yield* execute({ path: join(dir, "doc.md"), stream: new InMemoryStream(partial), @@ -423,6 +428,7 @@ describe("Tier TD — TempDir", () => { // one behind — whether it arrives while the directory is in use or before // the acquiring task has run at all. it("TD15: a cancelled acquisition leaves no directory behind", function* () { + yield* useHostFiles(); const before = yield* temporaries(); // Halted while the directory is live: the path is observed first, so the @@ -467,6 +473,7 @@ describe("Tier TD — TempDir", () => { const stream = new InMemoryStream(); const first = String( yield* scoped(function* () { + yield* useHostFiles(); return yield* collect( yield* execute({ path: join(dir, "doc.md"), stream, componentDirs: [dir] }), ); @@ -483,6 +490,7 @@ describe("Tier TD — TempDir", () => { (event) => !(event.type === "close" && event.coroutineId === "root"), ); const outcome = yield* scoped(function* () { + yield* useHostFiles(); const execution = yield* execute({ path: join(dir, "doc.md"), stream: new InMemoryStream(partial), @@ -521,6 +529,7 @@ describe("Tier TD — TempDir", () => { ); const outcome = yield* scoped(function* () { + yield* useHostFiles(); const execution = yield* execute({ path: join(dir, "doc.md"), stream: new InMemoryStream(), diff --git a/packages/runtime/apis.ts b/packages/runtime/apis.ts index 2e02f311..e3c07474 100644 --- a/packages/runtime/apis.ts +++ b/packages/runtime/apis.ts @@ -30,10 +30,15 @@ * * - **Process** — subprocess lifecycle has its own cancellation semantics * (killing processes on scope teardown). Middleware targets exec only. - * - **Fs** — reading, writing, and inspecting files form a cohesive file-IO - * surface used together for component resolution, replay guards, and the - * `` component. Middleware installed here sees a document's own file - * access on the same terms as the engine's. + * - **Fs** — the low-level host file surface: reading, writing, and inspecting + * paths the engine itself resolves, for component lookup, replay guards, and + * the root document. It is the host adapter's own dependency, not the + * boundary a document's paths cross. + * - **Files** — document filesystem access, in whole semantic operations + * (`files.ts`). ``, ``, and `` speak only this Api, so + * the same document means the same thing whether its paths resolve in the + * caller's filesystem or in a run-owned logical one. Its terminal handler + * throws: an uninstalled provider must not silently reach the host. * - **Fetch** — HTTP has distinct timeout/body/abort semantics. Merging * with Fs or Process would blur cancellation boundaries. * - **Env** — the host itself: metadata (env vars, platform) plus the two @@ -84,6 +89,7 @@ import { exec as processExec } from "@effectionx/process"; import { race, sleep, until } from "effection"; import type { Operation } from "effection"; import { timeout as contextualTimeout } from "./config.ts"; +import { Files } from "./files.ts"; import { Service } from "./service.ts"; /** @@ -342,6 +348,7 @@ interface EnvHandler { export const API: { Process: Api; Fs: Api; + Files: typeof Files; Fetch: Api; Env: Api; Service: typeof Service; @@ -545,6 +552,7 @@ export const API: { ); }, }), + Files, Service, }; diff --git a/packages/runtime/deno.json b/packages/runtime/deno.json index d1adb361..555d3941 100644 --- a/packages/runtime/deno.json +++ b/packages/runtime/deno.json @@ -3,6 +3,7 @@ "version": "0.8.0", "exports": { ".": "./mod.ts", + "./files": "./files.ts", "./test": "./test/mod.ts" } } diff --git a/packages/runtime/files.ts b/packages/runtime/files.ts new file mode 100644 index 00000000..c3fbe1a0 --- /dev/null +++ b/packages/runtime/files.ts @@ -0,0 +1,799 @@ +/** + * `API.Files` — the document filesystem boundary. + * + * A document names files with a path relative to the contextual working + * directory, and every one of those operations arrives here. What is on the + * other side is a provider's choice: `xmd run` installs a host adapter that + * resolves those paths in the caller's own filesystem, and `xmd workflow` + * installs one that resolves them in a logical filesystem owned by a database + * transaction. Neither is named in the components that call this Api, which is + * what lets one document mean the same thing under both. + * + * The operations are **semantic**, not primitive. `writeTextFile` is a whole + * replacement — admission, resolution, target classification, parent creation, + * and commit — rather than a sequence a caller assembles, because assembling it + * from outside is what would let a path admitted by one provider be used by + * another. `API.Fs` remains the low-level host surface a host adapter is built + * on; it is not this boundary. + * + * `checkFilePath` is the one exception, and it is deliberately weak: pure path + * arithmetic, no filesystem access, and nothing usable comes back — no path, no + * handle, no authority token. ``'s write form calls it to decide whether + * its children may expand at all, and the later `writeTextFile` repeats the + * same admission from the same authored path. A check that was skipped, + * replaced, or answered by another provider therefore authorizes nothing. + * + * ## Two kinds of failure + * + * An ordinary filesystem condition — missing, a directory, permission denied, + * no space — comes back as `Err(FilesError)` carrying frozen structural data. + * The consumer reads that data and selects a sentence from a fixed vocabulary; + * no message, errno code, resolved path, temporary path, or symlink target + * crosses this boundary. Cancellation is neither of these: it is not caught and + * never becomes a Result. + * + * A provider that is absent, that refuses an operation, or that breaks its own + * contract is not a filesystem condition. Those **throw**, with fixed + * diagnostics and no cause, and they end the execution rather than becoming + * something a document renders. A run whose Files provider is missing must not + * quietly reach the host instead. + * + * ## Why the data is structural + * + * Both the failures and the write outcome carry a plain frozen object under a + * stable `type` tag, and every consumer recognizes them by parsing that tag + * rather than with `instanceof`. Two copies of this package can be loaded at + * once — a repository component resolving its own runtime beside the engine's — + * and `instanceof` answers false across them, which would turn a provider + * failure into an unrecognized throw exactly when it matters most. + */ + +import { type Api, createApi } from "@effectionx/context-api"; +import type { Operation, Result } from "effection"; + +/** The stable discriminant on ordinary filesystem failure data. */ +export const FILES_ERROR = "executablemd.runtime.files-error/v1"; + +/** The stable discriminant on infrastructure failure data. */ +export const FILES_FATAL = "executablemd.runtime.files-fatal/v1"; + +/** The stable discriminant on a successful write's outcome data. */ +export const FILES_WRITE_SUCCESS = "executablemd.runtime.files-write-success/v1"; + +/** + * The vocabulary an ordinary failure is reported in. + * + * A provider maps whatever its platform produced onto one of these before + * returning. An unmapped condition becomes `operation-failed`, which is a real + * answer rather than a placeholder: the consumer has a sentence for it, and the + * unmapped value itself never crosses the boundary. + */ +export type FilesReason = + | "empty-path" + | "absolute-path" + | "lexical-escape" + | "resolved-escape" + | "missing" + | "directory" + | "special-file" + | "not-directory" + | "permission-denied" + | "read-only" + | "too-many-symlinks" + | "path-too-long" + | "no-space" + | "quota-exhausted" + | "cross-device" + | "busy" + | "too-many-open-files" + | "directory-not-empty" + | "invalid-pattern" + | "operation-failed"; + +const REASONS: readonly FilesReason[] = [ + "empty-path", + "absolute-path", + "lexical-escape", + "resolved-escape", + "missing", + "directory", + "special-file", + "not-directory", + "permission-denied", + "read-only", + "too-many-symlinks", + "path-too-long", + "no-space", + "quota-exhausted", + "cross-device", + "busy", + "too-many-open-files", + "directory-not-empty", + "invalid-pattern", + "operation-failed", +]; + +/** The operations whose failure carries no commit outcome. */ +export type FilesOperation = "check-file-path" | "read" | "glob" | "temporary-directory"; + +const OPERATIONS: readonly FilesOperation[] = [ + "check-file-path", + "read", + "glob", + "temporary-directory", +]; + +/** Where a non-write operation stopped. */ +export type FilesPhase = + | "lexical" + | "resolution" + | "target" + | "access" + | "pattern" + | "traversal" + | "acquire"; + +const PHASES: readonly FilesPhase[] = [ + "lexical", + "resolution", + "target", + "access", + "pattern", + "traversal", + "acquire", +]; + +/** Where a write stopped, which is what decides what may be said about the target. */ +export type FileWritePhase = + | "lexical" + | "resolution" + | "target" + | "parents" + | "temporary" + | "commit" + | "cleanup" + | "transaction"; + +/** + * What is known about the target afterwards. + * + * `commit-unknown` is an answer rather than a missing one: a commit that threw + * may have run or not, and no provider can tell which from where it stands. + * What still holds in that case is that the target is one complete version. + */ +export type FileWriteTarget = "unchanged" | "commit-unknown" | "committed" | "rolled-back"; + +export interface FilesFailureData { + readonly type: typeof FILES_ERROR; + readonly operation: FilesOperation; + readonly phase: FilesPhase; + readonly reason: FilesReason; +} + +export interface FileWriteFailureData { + readonly type: typeof FILES_ERROR; + readonly operation: "write"; + readonly phase: FileWritePhase; + readonly reason?: FilesReason; + readonly cleanup?: FilesReason; + readonly target: FileWriteTarget; +} + +export type FilesErrorData = FilesFailureData | FileWriteFailureData; + +/** + * The message every ordinary failure carries. + * + * Constant on purpose. A message is the part of an Error that gets printed by + * accident, and there is nothing safe to put in this one: the authored path + * belongs to the consumer that wrote it, and everything else belongs to the + * platform. + */ +export const FILES_ERROR_MESSAGE = "Files operation failed"; + +/** An ordinary filesystem failure. What it means is in `data`, never in the message. */ +export class FilesError extends Error { + readonly data: FilesErrorData; + + constructor(data: FilesErrorData) { + super(FILES_ERROR_MESSAGE); + this.name = "FilesError"; + this.data = data; + } +} + +export interface FileWriteSuccess { + readonly type: typeof FILES_WRITE_SUCCESS; + readonly publication: "host-committed" | "transaction-staged"; +} + +export interface FilePathInput { + readonly cwd: string; + readonly path: string; +} + +export interface FileWriteInput extends FilePathInput { + readonly content: string; +} + +export interface GlobInput { + readonly cwd: string; + readonly include: string[]; + readonly exclude: string[]; +} + +export interface FilesHandler { + /** + * Whether this authored path is admissible at all, decided from the path and + * `cwd` alone. No filesystem access, and nothing usable comes back. + */ + checkFilePath(input: FilePathInput): Operation>; + readTextFile(input: FilePathInput): Operation>; + writeTextFile(input: FileWriteInput): Operation>; + /** Sorted, deduplicated, POSIX-separated paths of the regular files that match. */ + globFiles(input: GlobInput): Operation>; + /** + * A directory that lives as long as the acquiring scope. A resource, so the + * caller holds it by acquisition rather than by remembering to remove it. + */ + temporaryDirectory(): Operation>; +} + +/** The operations a provider may refuse outright rather than fail at. */ +export type FilesDeniableOperation = "temporary-directory"; + +/** + * Which contract a provider broke. + * + * `authority` — the identity authorizing access is stale, foreign, or gone. + * `savepoint` — a nested transaction could not be rolled back or released. + * `protocol` — a handler threw, or returned data no consumer can trust. + * `teardown` — cleanup failed while the scope was already unwinding. + */ +export type FilesInvariantCategory = "authority" | "savepoint" | "protocol" | "teardown"; + +const INVARIANT_CATEGORIES: readonly FilesInvariantCategory[] = [ + "authority", + "savepoint", + "protocol", + "teardown", +]; + +/** + * Infrastructure failure data. + * + * Three kinds, each with fixed fields and nothing derived from the condition + * that produced it. A category is control data for a consumer deciding what to + * fence, not text: no diagnostic interpolates it. + */ +export type FilesFatalData = + | { readonly type: typeof FILES_FATAL; readonly kind: "provider-unavailable" } + | { + readonly type: typeof FILES_FATAL; + readonly kind: "operation-denied"; + readonly operation: FilesDeniableOperation; + } + | { + readonly type: typeof FILES_FATAL; + readonly kind: "invariant"; + readonly category: FilesInvariantCategory; + }; + +export const FILES_PROVIDER_UNAVAILABLE_MESSAGE = "Files provider is not installed"; +export const FILES_OPERATION_DENIED_MESSAGE = "Files provider does not support temporary-directory"; +export const FILES_INVARIANT_MESSAGE = "Files provider invariant failed"; + +/** No Files provider is installed, and there is no host to fall back to. */ +export class FilesProviderUnavailableError extends Error { + readonly data: FilesFatalData; + + constructor() { + super(FILES_PROVIDER_UNAVAILABLE_MESSAGE); + this.name = "FilesProviderUnavailableError"; + this.data = Object.freeze({ type: FILES_FATAL, kind: "provider-unavailable" }); + } +} + +/** The installed provider does not implement this operation at all. */ +export class FilesOperationDeniedError extends Error { + readonly data: FilesFatalData; + + constructor(operation: FilesDeniableOperation) { + super(FILES_OPERATION_DENIED_MESSAGE); + this.name = "FilesOperationDeniedError"; + const denied = deniableOperation(operation); + if (denied === undefined) { + throw new FilesInvariantError("protocol"); + } + this.data = Object.freeze({ type: FILES_FATAL, kind: "operation-denied", operation: denied }); + } +} + +/** A provider broke its own contract. */ +export class FilesInvariantError extends Error { + readonly data: FilesFatalData; + + constructor(category: FilesInvariantCategory) { + super(FILES_INVARIANT_MESSAGE); + this.name = "FilesInvariantError"; + const parsed = invariantCategory(category); + if (parsed === undefined) { + throw new Error(FILES_INVARIANT_MESSAGE); + } + this.data = Object.freeze({ type: FILES_FATAL, kind: "invariant", category: parsed }); + } +} + +/** An infrastructure failure, recognized structurally rather than by class. */ +export interface FilesFatalFailure extends Error { + readonly data: FilesFatalData; +} + +/** + * Everything below reads values it did not create. + * + * A thrown value is whatever a provider threw, and a provider is as free to + * hand back a Proxy with a throwing `get` trap, an accessor that fails, or an + * object whose key enumeration explodes as it is to hand back a plain record. + * These parsers run from `fatalCause`, which every generic catch in the engine + * consults — so one of them throwing would replace the failure being classified + * with a failure *about classifying it*, at the exact moment the engine is + * deciding whether the execution may continue. + * + * So reading is total: every access that can fail goes through a helper that + * answers `undefined` instead, and each exported parser is additionally wrapped + * so that a shape nobody anticipated is simply not recognized. + */ +function attempt(read: () => T): T | undefined { + try { + return read(); + } catch { + return undefined; + } +} + +function isRecord(value: unknown): value is Record { + return typeof value === "object" && value !== null && !Array.isArray(value); +} + +/** Whether this is an Error, without trusting its prototype chain. */ +function isError(value: unknown): value is Error { + return attempt(() => value instanceof Error) === true; +} + +/** One property, read through a trap that may refuse or fail. */ +function property(target: object, name: string): unknown { + return attempt(() => Reflect.get(target, name)); +} + +/** How many own enumerable keys, when the object will say. */ +function keyCount(value: object): number | undefined { + return attempt(() => Object.keys(value).length); +} + +function isFrozen(value: object): boolean { + return attempt(() => Object.isFrozen(value)) === true; +} + +function dataOf(error: unknown): Record | undefined { + if (!isError(error)) { + return undefined; + } + const data = property(error, "data"); + return isRecord(data) ? data : undefined; +} + +/** The one diagnostic each kind of infrastructure failure carries. */ +const FATAL_DIAGNOSTICS: ReadonlyMap = new Map([ + ["provider-unavailable", FILES_PROVIDER_UNAVAILABLE_MESSAGE], + ["operation-denied", FILES_OPERATION_DENIED_MESSAGE], + ["invariant", FILES_INVARIANT_MESSAGE], +]); + +/** The one class name each kind of infrastructure failure carries. */ +const FATAL_NAMES: ReadonlyMap = new Map([ + ["provider-unavailable", "FilesProviderUnavailableError"], + ["operation-denied", "FilesOperationDeniedError"], + ["invariant", "FilesInvariantError"], +]); + +/** + * Everything a constructor here puts on the Error itself, and nothing else. + * + * `message` and `stack` are non-enumerable own properties of every Error, so + * what remains enumerable is exactly what a constructor assigned. Anything more + * is payload the contract does not describe — and since recognition hands the + * object onward by identity, payload travels with it. + */ +const FATAL_MEMBERS: readonly string[] = ["data", "name"]; + +/** + * Whether the Error carries only the members its constructor assigns. + * + * Symbols are checked as well as string keys: a symbol-keyed enumerable + * property survives spreading and appears in `Object.assign`'d copies, so + * leaving it unexamined would let a path ride along through exactly the + * mechanisms a consumer uses to inspect a failure. + */ +function hasOnlyContractMembers(error: Error): boolean { + const keys = attempt(() => [...Object.keys(error)].sort()); + if (keys === undefined || keys.length !== FATAL_MEMBERS.length) { + return false; + } + if (!keys.every((key, index) => key === FATAL_MEMBERS[index])) { + return false; + } + const payload = attempt(() => + Object.getOwnPropertySymbols(error).filter( + (symbol) => Object.getOwnPropertyDescriptor(error, symbol)?.enumerable === true, + ), + ); + return payload !== undefined && payload.length === 0; +} + +function reasonOf(value: unknown): FilesReason | undefined { + return REASONS.find((reason) => reason === value); +} + +function operationOf(value: unknown): FilesOperation | undefined { + return OPERATIONS.find((operation) => operation === value); +} + +function phaseOf(value: unknown): FilesPhase | undefined { + return PHASES.find((phase) => phase === value); +} + +function invariantCategory(value: unknown): FilesInvariantCategory | undefined { + return INVARIANT_CATEGORIES.find((category) => category === value); +} + +function deniableOperation(value: unknown): FilesDeniableOperation | undefined { + return value === "temporary-directory" ? "temporary-directory" : undefined; +} + +/** + * The infrastructure failure data this Error carries, if it carries valid data. + * + * Every field is checked, the member count with them, and that the object is + * frozen: extra keys are not the shape this contract describes, and a mutable + * one is not the shape a constructor here produces. Accepting either would let + * a provider smuggle a path or a message through under a recognized tag. + */ +export function parseFilesFatal(error: unknown): FilesFatalData | undefined { + return attempt(() => { + const data = dataOf(error); + if (data === undefined || property(data, "type") !== FILES_FATAL || !isFrozen(data)) { + return undefined; + } + const members = keyCount(data); + const kind = property(data, "kind"); + if (kind === "provider-unavailable" && members === 2) { + return Object.freeze({ type: FILES_FATAL, kind: "provider-unavailable" }); + } + if (kind === "operation-denied" && members === 3) { + const operation = deniableOperation(property(data, "operation")); + if (operation !== undefined) { + return Object.freeze({ type: FILES_FATAL, kind: "operation-denied", operation }); + } + } + if (kind === "invariant" && members === 3) { + const category = invariantCategory(property(data, "category")); + if (category !== undefined) { + return Object.freeze({ type: FILES_FATAL, kind: "invariant", category }); + } + } + return undefined; + }); +} + +/** + * Whether this failure satisfies the whole public infrastructure-failure + * contract, not merely the tag. + * + * Recognition decides two different things at once, and the second is why this + * is stricter than `parseFilesFatal`. A recognized failure is **rethrown by + * identity** — the object that was thrown is the object a fail-stop records — + * so recognizing one is a decision to let that exact object travel onward. An + * Error that carries the right data but also a raw platform message, or a cause + * chain holding an errno and a path, would then carry all of that past the + * boundary the reason vocabulary exists to hold. + * + * So the whole object has to match what a constructor here produces: the fixed + * name and diagnostic for its kind, frozen structural data with exactly the + * fields the kind describes, no cause, and no other enumerable member — string + * or symbol. Anything else is a candidate that fails the contract, and + * `invokeFiles` replaces it with a fresh invariant rather than preserving it. + * + * Structural throughout, so a failure constructed by a separately loaded copy + * of this package is recognized on exactly the same terms as one constructed + * here — `instanceof` answers false across two copies, which is the case this + * has to survive. That is also why the `name` is checked rather than the class: + * a second copy's constructor is a different function producing the same name. + */ +export function isFilesFatal(error: unknown): error is FilesFatalFailure { + return ( + attempt(() => { + const data = parseFilesFatal(error); + if (data === undefined || !isError(error)) { + return false; + } + if (property(error, "name") !== FATAL_NAMES.get(data.kind)) { + return false; + } + if (property(error, "message") !== FATAL_DIAGNOSTICS.get(data.kind)) { + return false; + } + if (property(error, "cause") !== undefined) { + return false; + } + return hasOnlyContractMembers(error); + }) === true + ); +} + +/** + * The infrastructure failure this one is, by identity. + * + * The original object comes back rather than a replacement, because a fail-stop + * that records "the first error" has to record the one that was thrown. + */ +export function asFilesFatal(error: unknown): FilesFatalFailure | undefined { + return isFilesFatal(error) ? error : undefined; +} + +/** + * What a write phase is allowed to say, which is the whole validity rule. + * + * The phase decides the target claim: nothing selects `committed` except a + * cleanup that failed after the commit returned, and nothing claims + * `commit-unknown` except a commit that threw. Keeping the table here rather + * than at each construction site makes an invalid combination unconstructable + * instead of merely unwritten. + */ +interface WritePhaseRule { + readonly target: FileWriteTarget; + readonly reason: "required" | "absent"; + readonly cleanup: "required" | "optional" | "absent"; +} + +const WRITE_PHASES: ReadonlyMap = new Map< + FileWritePhase, + WritePhaseRule +>([ + ["lexical", { target: "unchanged", reason: "required", cleanup: "absent" }], + ["resolution", { target: "unchanged", reason: "required", cleanup: "absent" }], + ["target", { target: "unchanged", reason: "required", cleanup: "absent" }], + ["parents", { target: "unchanged", reason: "required", cleanup: "absent" }], + ["temporary", { target: "unchanged", reason: "required", cleanup: "optional" }], + ["commit", { target: "commit-unknown", reason: "required", cleanup: "optional" }], + ["cleanup", { target: "committed", reason: "absent", cleanup: "required" }], + ["transaction", { target: "rolled-back", reason: "required", cleanup: "absent" }], +]); + +function writePhaseOf(value: unknown): [FileWritePhase, WritePhaseRule] | undefined { + for (const entry of WRITE_PHASES) { + if (entry[0] === value) { + return entry; + } + } + return undefined; +} + +function violatesRule( + rule: WritePhaseRule, + reason: FilesReason | undefined, + cleanup: FilesReason | undefined, +): boolean { + if (rule.reason === "required" && reason === undefined) { + return true; + } + if (rule.reason === "absent" && reason !== undefined) { + return true; + } + if (rule.cleanup === "required" && cleanup === undefined) { + return true; + } + return rule.cleanup === "absent" && cleanup !== undefined; +} + +function writeData( + phase: FileWritePhase, + rule: WritePhaseRule, + reason: FilesReason | undefined, + cleanup: FilesReason | undefined, +): FileWriteFailureData { + return Object.freeze({ + type: FILES_ERROR, + operation: "write", + phase, + target: rule.target, + ...(reason === undefined ? {} : { reason }), + ...(cleanup === undefined ? {} : { cleanup }), + }); +} + +/** Build an ordinary non-write failure. */ +export function filesFailure(input: { + operation: FilesOperation; + phase: FilesPhase; + reason: FilesReason; +}): FilesError { + const operation = operationOf(input.operation); + const phase = phaseOf(input.phase); + const reason = reasonOf(input.reason); + if (operation === undefined || phase === undefined || reason === undefined) { + throw new FilesInvariantError("protocol"); + } + return new FilesError(Object.freeze({ type: FILES_ERROR, operation, phase, reason })); +} + +/** + * Build a write failure, refusing any combination a consumer could not read. + * + * A write's report is the only place a document learns what became of a file it + * asked to replace, so an invalid combination is a provider bug rather than a + * value to pass along and interpret later. + */ +export function fileWriteFailure(input: { + phase: FileWritePhase; + reason?: FilesReason; + cleanup?: FilesReason; +}): FilesError { + const found = writePhaseOf(input.phase); + if (found === undefined) { + throw new FilesInvariantError("protocol"); + } + const [phase, rule] = found; + + const reason = input.reason === undefined ? undefined : reasonOf(input.reason); + const cleanup = input.cleanup === undefined ? undefined : reasonOf(input.cleanup); + if ( + (input.reason !== undefined && reason === undefined) || + (input.cleanup !== undefined && cleanup === undefined) + ) { + throw new FilesInvariantError("protocol"); + } + if (violatesRule(rule, reason, cleanup)) { + throw new FilesInvariantError("protocol"); + } + + return new FilesError(writeData(phase, rule, reason, cleanup)); +} + +/** + * The non-write failure data this error carries, if it carries valid data. + * + * Malformed data is not fatal here — the consumer already has a sentence for + * "the operation failed" and nothing about a target is at stake — so this + * simply declines to recognize it. + */ +export function parseFilesFailure(error: unknown): FilesFailureData | undefined { + return attempt(() => { + const data = dataOf(error); + if (data === undefined || property(data, "type") !== FILES_ERROR || keyCount(data) !== 4) { + return undefined; + } + const operation = operationOf(property(data, "operation")); + const phase = phaseOf(property(data, "phase")); + const reason = reasonOf(property(data, "reason")); + if (operation === undefined || phase === undefined || reason === undefined) { + return undefined; + } + return Object.freeze({ type: FILES_ERROR, operation, phase, reason }); + }); +} + +/** + * The write failure data this error carries, if it carries valid data. + * + * Unlike a non-write failure, malformed data here has no safe reading: every + * sentence a consumer could print makes a claim about whether the file was + * replaced. A caller treats `undefined` from a write as a protocol invariant + * rather than inventing a commit state. + */ +export function parseFileWriteFailure(error: unknown): FileWriteFailureData | undefined { + return attempt(() => { + const data = dataOf(error); + if ( + data === undefined || + property(data, "type") !== FILES_ERROR || + property(data, "operation") !== "write" + ) { + return undefined; + } + const found = writePhaseOf(property(data, "phase")); + if (found === undefined) { + return undefined; + } + const [phase, rule] = found; + if (property(data, "target") !== rule.target) { + return undefined; + } + + const declared = property(data, "reason"); + const declaredCleanup = property(data, "cleanup"); + const reason = declared === undefined ? undefined : reasonOf(declared); + const cleanup = declaredCleanup === undefined ? undefined : reasonOf(declaredCleanup); + if ( + (declared !== undefined && reason === undefined) || + (declaredCleanup !== undefined && cleanup === undefined) || + violatesRule(rule, reason, cleanup) + ) { + return undefined; + } + + const members = 4 + (reason === undefined ? 0 : 1) + (cleanup === undefined ? 0 : 1); + if (keyCount(data) !== members) { + return undefined; + } + + return writeData(phase, rule, reason, cleanup); + }); +} + +/** A successful write's outcome. */ +export function fileWriteSuccess(publication: FileWriteSuccess["publication"]): FileWriteSuccess { + const parsed = parseFileWriteSuccess({ type: FILES_WRITE_SUCCESS, publication }); + if (parsed === undefined) { + throw new FilesInvariantError("protocol"); + } + return parsed; +} + +/** + * The write outcome this value is, if it is a valid one. + * + * A malformed success is as untrustworthy as a malformed failure: a provider + * that cannot describe what it did may not have done it, so a caller treats + * `undefined` here as a protocol invariant too. + */ +export function parseFileWriteSuccess(value: unknown): FileWriteSuccess | undefined { + return attempt(() => { + if ( + !isRecord(value) || + property(value, "type") !== FILES_WRITE_SUCCESS || + keyCount(value) !== 2 + ) { + return undefined; + } + const publication = property(value, "publication"); + if (publication === "host-committed") { + return Object.freeze({ type: FILES_WRITE_SUCCESS, publication: "host-committed" }); + } + if (publication === "transaction-staged") { + return Object.freeze({ type: FILES_WRITE_SUCCESS, publication: "transaction-staged" }); + } + return undefined; + }); +} + +/** + * The document filesystem Api. + * + * The terminal handler throws for every operation, including `checkFilePath`. + * A default that reached the host would make an uninstalled provider + * indistinguishable from an installed one, and the whole point of the boundary + * is that a workflow run cannot silently touch the caller's filesystem. + */ +export const Files: Api = createApi("executablemd.runtime.files", { + // deno-lint-ignore require-yield + *checkFilePath(_input: FilePathInput): Operation> { + throw new FilesProviderUnavailableError(); + }, + // deno-lint-ignore require-yield + *readTextFile(_input: FilePathInput): Operation> { + throw new FilesProviderUnavailableError(); + }, + // deno-lint-ignore require-yield + *writeTextFile(_input: FileWriteInput): Operation> { + throw new FilesProviderUnavailableError(); + }, + // deno-lint-ignore require-yield + *globFiles(_input: GlobInput): Operation> { + throw new FilesProviderUnavailableError(); + }, + // deno-lint-ignore require-yield + *temporaryDirectory(): Operation> { + throw new FilesProviderUnavailableError(); + }, +}); diff --git a/packages/runtime/host-files.ts b/packages/runtime/host-files.ts new file mode 100644 index 00000000..492c8bbc --- /dev/null +++ b/packages/runtime/host-files.ts @@ -0,0 +1,559 @@ +/** + * The host `API.Files` provider — document filesystem access in the caller's + * own filesystem. + * + * This is what `xmd run` installs. A document's relative path is resolved + * against the contextual working directory and used as an ordinary host path, + * so a document can hand a file to a tool the caller already has. Everything + * below is built on the low-level `API.Fs`, which is deliberate: a host that + * already wraps `API.Fs` to observe or sandbox the engine's own file access + * keeps seeing a document's access on the same terms. + * + * ## What containment means here + * + * Access is confined to the contextual directory, judged against the filesystem + * as this adapter observes it. An empty path, an absolute path, and a lexical + * `..` escape are refused without touching the filesystem at all; a symlink + * leading out is refused once resolution can see it. + * + * That is sound **while the host pathname namespace is stable**, and every + * guarantee here is stated on that basis. It is not a sandbox. Another process + * can replace a directory, symlink, junction, or reparse point between the + * moment this adapter observes a path and the moment it uses one, and nothing + * available on the shipped runtimes closes that window without a native + * dependency. What is contained is the document's own children — the case a + * document controls — because resolution is deferred until after they run. + * + * ## Writes + * + * A write goes through a sibling temporary file and a rename. The rename is the + * commit point: everything before it can fail or be cancelled with the previous + * file untouched, and once it begins the target holds the complete old file or + * the complete new one, never a partial write. It is a commit rather than a + * transaction — a rename that returned is not undone by a later cancellation. + * The temporary also closes the one hole resolution cannot: a dangling symlink + * has nothing to resolve, and `rename` replaces the link rather than following + * it wherever it points. + * + * ## What crosses the boundary + * + * Nothing from a caught platform error. An errno code *selects* a + * `FilesReason`, and the reason is all the consumer receives — no message, no + * code, no resolved path, no temporary path, and no symlink target. A platform + * error names the path it failed on, and for a write that path can be a + * temporary the document never chose. + */ + +import { ensure, Err, Ok, resource, scoped } from "effection"; +import type { Operation, Result } from "effection"; +import { basename, dirname, isAbsolute, join, relative, resolve, sep } from "node:path"; +import { randomUUID } from "node:crypto"; +import { mkdtempSync, realpathSync } from "node:fs"; +import { tmpdir } from "node:os"; +import { FsApi, rm } from "@effectionx/fs"; +import { API } from "./apis.ts"; +import { fileWriteFailure, fileWriteSuccess, filesFailure, FilesInvariantError } from "./files.ts"; +import type { + FilePathInput, + FilesHandler, + FilesOperation, + FilesPhase, + FilesReason, + FileWriteInput, + FileWritePhase, + FileWriteSuccess, + GlobInput, +} from "./files.ts"; + +/** + * A private step a host operation is about to take. + * + * Test-only. Production entrypoints install the adapter without one, so this is + * neither global state nor a capability: an observer can watch, and the point of + * watching is to replace part of the tree between an observation and the call + * that follows it, which is how the stable-namespace limitation is made + * observable rather than merely stated. + */ +export interface HostFilesEvent { + readonly operation: "read" | "write" | "glob"; + readonly phase: "target" | "access" | "parents" | "temporary" | "commit" | "cleanup" | "read-dir"; +} + +/** Synchronous, so nothing can run between the observation and the call it precedes. */ +export type HostFilesObserver = (event: HostFilesEvent) => void; + +export interface HostFilesOptions { + readonly observe?: HostFilesObserver; +} + +/** + * The errno codes this adapter recognizes, and the reason each selects. + * + * A `Map` rather than an object literal, because a lookup on one answers for + * inherited keys — `codes["toString"]` would hand back a function — and the code + * is chosen by whatever implements `API.Fs`. + */ +const REASON_BY_CODE: ReadonlyMap = new Map([ + ["ENOENT", "missing"], + ["ENOTDIR", "not-directory"], + ["EISDIR", "directory"], + ["ENOTEMPTY", "directory-not-empty"], + ["EACCES", "permission-denied"], + ["EPERM", "permission-denied"], + ["EROFS", "read-only"], + ["ELOOP", "too-many-symlinks"], + ["ENAMETOOLONG", "path-too-long"], + ["ENOSPC", "no-space"], + ["EDQUOT", "quota-exhausted"], + ["EXDEV", "cross-device"], + ["EBUSY", "busy"], + ["EMFILE", "too-many-open-files"], +]); + +/** + * The `errno` string a failed call carries, when it carries one. + * + * Read rather than asserted: `catch` gives back `unknown`, and what arrives + * there is only conventionally an `ErrnoException`. + */ +function errorCode(error: unknown): string | undefined { + if (typeof error !== "object" || error === null || !("code" in error)) { + return undefined; + } + const { code } = error; + return typeof code === "string" ? code : undefined; +} + +/** The reason a caught platform error selects, defaulting to the generic one. */ +function reasonOf(error: unknown): FilesReason { + const code = errorCode(error); + if (code === undefined) { + return "operation-failed"; + } + return REASON_BY_CODE.get(code) ?? "operation-failed"; +} + +/** + * Why an authored path is inadmissible, decided from arithmetic alone. + * + * `resolve` normalizes `..` lexically, so this holds against the contextual + * directory as given — canonicalizing it belongs to resolution and would only + * move the same comparison onto a different pair of strings. + */ +function inadmissible(input: FilePathInput): FilesReason | undefined { + if (input.path.length === 0) { + return "empty-path"; + } + if (isAbsolute(input.path)) { + return "absolute-path"; + } + if (!within(input.cwd, resolve(input.cwd, input.path))) { + return "lexical-escape"; + } + return undefined; +} + +/** + * Whether `path` names `base` or something inside it. + * + * The directory itself is contained — `.` is not an escape. That it is a + * directory is a question about the target, which the target check answers. + * + * Only a complete `..` segment leaves. A name that merely starts with two dots + * — `..notes.md` — is an ordinary file inside, and a prefix test would refuse it. + */ +function within(base: string, path: string): boolean { + const rel = relative(base, path); + if (isAbsolute(rel)) { + return false; + } + if (rel.length === 0) { + return true; + } + return rel !== ".." && !rel.startsWith(`..${sep}`); +} + +/** + * `path` with every symlink in its existing prefix resolved. + * + * `realpath` needs the whole path to exist, and a write commonly names one that + * does not yet, so the walk gives up one trailing segment at a time until + * something answers and then puts the segments back. The working directory + * always exists, so the loop terminates there at the latest. + */ +function* resolveExisting(path: string): Operation { + const trailing: string[] = []; + let current = path; + + while (true) { + const resolved = yield* API.Fs.operations.realpath(current); + if (resolved !== undefined) { + return trailing.length === 0 ? resolved : join(resolved, ...trailing); + } + const parent = dirname(current); + if (parent === current) { + return join(current, ...trailing); + } + trailing.unshift(basename(current)); + current = parent; + } +} + +/** What resolution produced, or why it could not. */ +type Destination = { readonly path: string } | { readonly reason: FilesReason }; + +/** + * The path as the filesystem currently has it. + * + * Resolves the part of the path that is already on disk — the file itself when + * it is there, the deepest existing ancestor when it is not — and re-checks the + * result, which is what catches a symlink pointing out. What comes back is that + * resolved path, so an internal symlink is followed to the file it names rather + * than replaced. + * + * Both sides of the comparison are canonical, so a working directory reached + * through a symlink — macOS's `/var` against `/private/var` — does not read as + * an escape. + */ +function* destination(input: FilePathInput): Operation { + try { + const base = (yield* API.Fs.operations.realpath(input.cwd)) ?? input.cwd; + const path = yield* resolveExisting(resolve(input.cwd, input.path)); + if (!within(base, path)) { + return { reason: "resolved-escape" }; + } + return { path }; + } catch (error) { + return { reason: reasonOf(error) }; + } +} + +function nonWriteFailure( + operation: FilesOperation, + phase: FilesPhase, + reason: FilesReason, +): Result { + return Err(filesFailure({ operation, phase, reason })); +} + +function writeFailure(input: { + phase: FileWritePhase; + reason?: FilesReason; + cleanup?: FilesReason; +}): Result { + return Err(fileWriteFailure(input)); +} + +function notify(observe: HostFilesObserver | undefined, event: HostFilesEvent): void { + observe?.(event); +} + +/** Code point order: what a document branches on must not depend on a locale. */ +function byCodePoint(left: string, right: string): number { + if (left < right) { + return -1; + } + if (left > right) { + return 1; + } + return 0; +} + +/** + * Build a host provider. + * + * Exported so a test can drive one operation directly; entrypoints install it + * with {@link useHostFiles}. + */ +export function hostFilesHandler(options: HostFilesOptions = {}): FilesHandler { + const observe = options.observe; + + function* checkFilePath(input: FilePathInput): Operation> { + const reason = inadmissible(input); + if (reason !== undefined) { + return nonWriteFailure("check-file-path", "lexical", reason); + } + return Ok(undefined); + } + + function* readTextFile(input: FilePathInput): Operation> { + const lexical = inadmissible(input); + if (lexical !== undefined) { + return nonWriteFailure("read", "lexical", lexical); + } + + const target = yield* destination(input); + if ("reason" in target) { + return nonWriteFailure("read", "resolution", target.reason); + } + + notify(observe, { operation: "read", phase: "target" }); + try { + const info = yield* API.Fs.operations.stat(target.path); + if (!info.exists) { + return nonWriteFailure("read", "target", "missing"); + } + if (info.isDirectory) { + return nonWriteFailure("read", "target", "directory"); + } + if (!info.isFile) { + return nonWriteFailure("read", "target", "special-file"); + } + } catch (error) { + return nonWriteFailure("read", "target", reasonOf(error)); + } + + notify(observe, { operation: "read", phase: "access" }); + try { + return Ok(yield* API.Fs.operations.readTextFile(target.path)); + } catch (error) { + return nonWriteFailure("read", "access", reasonOf(error)); + } + } + + /** + * Replace the target with exactly `content`. + * + * Admission is repeated here from the authored path and contextual directory + * rather than carried over from `checkFilePath`. The check answers whether + * children may expand; a child can change what a path means, and a + * destination resolved before they ran would not be the one this write lands + * on. + * + * Removal of the temporary is registered before it is written rather than + * after. The write is where an interruption is most likely to land, and a + * cleanup installed on the far side of it would not run for the one failure it + * exists to handle. `remove` is forced, so registering it for a file that was + * never created — or one the rename has already consumed — is a no-op. + * + * Both halves are collected rather than thrown. A destructor that threw would + * replace the failure it is unwinding, and a write's own failure must not hide + * the fact that a temporary was left behind. + */ + function* writeTextFile(input: FileWriteInput): Operation> { + const lexical = inadmissible(input); + if (lexical !== undefined) { + return writeFailure({ phase: "lexical", reason: lexical }); + } + + const target = yield* destination(input); + if ("reason" in target) { + return writeFailure({ phase: "resolution", reason: target.reason }); + } + + notify(observe, { operation: "write", phase: "target" }); + try { + const info = yield* API.Fs.operations.stat(target.path); + if (info.exists && !info.isFile) { + return writeFailure({ + phase: "target", + reason: info.isDirectory ? "directory" : "special-file", + }); + } + } catch (error) { + return writeFailure({ phase: "target", reason: reasonOf(error) }); + } + + notify(observe, { operation: "write", phase: "parents" }); + try { + yield* API.Fs.operations.ensureDir(dirname(target.path)); + } catch (error) { + return writeFailure({ phase: "parents", reason: reasonOf(error) }); + } + + let failed: FilesReason | undefined; + let cleanup: FilesReason | undefined; + // Which step the write reached, which is what decides what may be said + // about the target: everything before the rename leaves the previous file + // in place, and a rename that threw may have run or not. + let step: FileWritePhase = "temporary"; + // Whether the write reached its own end. Cleanup runs on every exit, and + // the two exits need different answers: one has a Result to compose with + // and the other does not. + let settled = false; + + yield* scoped(function* () { + const temporary = `${target.path}.xmd-${randomUUID().slice(0, 8)}.tmp`; + yield* ensure(function* () { + notify(observe, { operation: "write", phase: "cleanup" }); + try { + yield* API.Fs.operations.remove(temporary, { force: true }); + } catch (error) { + if (settled) { + cleanup = reasonOf(error); + return; + } + // Cancellation is unwinding, so there is no outcome to report this + // beside — and manufacturing one would turn a halt into a write + // result. It leaves the scope as an infrastructure failure instead, + // carrying neither the platform's error nor the generated temporary's + // name, and the engine's fatal discovery finds it there. + throw new FilesInvariantError("teardown"); + } + }); + try { + notify(observe, { operation: "write", phase: "temporary" }); + yield* API.Fs.operations.writeTextFile(temporary, input.content); + step = "commit"; + notify(observe, { operation: "write", phase: "commit" }); + yield* API.Fs.operations.rename(temporary, target.path); + } catch (error) { + failed = reasonOf(error); + } + settled = true; + }); + + if (failed !== undefined) { + return writeFailure({ phase: step, reason: failed, cleanup }); + } + if (cleanup !== undefined) { + return writeFailure({ phase: "cleanup", cleanup }); + } + return Ok(fileWriteSuccess("host-committed")); + } + + /** + * The regular files under `cwd` that `include` selects and `exclude` does not. + * + * Traversal is `API.Fs`'s: it reports directories and symbolic links too, and + * never follows one, which is what keeps the walk inside `cwd` and free of + * cycles. What this adds is the document-facing shape — regular files only, + * deduplicated, and sorted, so a document that branches on a listing branches + * the same way on every host. + */ + function* globFiles(input: GlobInput): Operation> { + try { + const info = yield* API.Fs.operations.stat(input.cwd); + if (!info.exists) { + return nonWriteFailure("glob", "target", "missing"); + } + if (!info.isDirectory) { + return nonWriteFailure("glob", "target", "not-directory"); + } + } catch (error) { + return nonWriteFailure("glob", "target", reasonOf(error)); + } + + try { + const matched = yield* traverse(input, observe); + const files = matched.filter((entry) => entry.isFile).map((entry) => entry.path); + return Ok([...new Set(files)].sort(byCodePoint)); + } catch (error) { + // The Api compiles patterns as it starts, so an unusable one — an + // unterminated character class — arrives as a `SyntaxError` from `RegExp` + // rather than as an errno. It is the one failure here a document can fix + // by editing what it wrote. + if (error instanceof SyntaxError) { + return nonWriteFailure("glob", "pattern", "invalid-pattern"); + } + return nonWriteFailure("glob", "traversal", reasonOf(error)); + } + } + + /** + * A directory this call created, named by its canonical path. + * + * Creation is synchronous so that nothing can suspend between it and the + * `ensure` that removes it. `until()` cannot cancel the promise it is waiting + * on, so an asynchronous `mkdtemp` halted mid-flight would go on to create a + * directory after the generator had already stopped — one nothing owns and + * nothing removes. Reading the canonical path does not suspend either, so the + * whole acquisition is a single uninterruptible step. + * + * `mkdtemp` names and creates at once, so the directory is never one an + * earlier run left behind. The path is then canonicalized: on macOS `tmpdir()` + * is a symlink (`/var/folders/…`) while a child process resolves it + * (`/private/var/…`), and canonicalizing is what makes the rendered path, the + * contextual directory, and a subprocess's own `cwd` the same string. + */ + function temporaryDirectory(): Operation> { + return resource(function* (provide) { + let created: string; + let canonical: string; + try { + created = mkdtempSync(join(tmpdir(), "xmd-tempdir-")); + canonical = realpathSync(created); + } catch (error) { + yield* provide(nonWriteFailure("temporary-directory", "acquire", reasonOf(error))); + return; + } + yield* ensure(() => discard(created)); + yield* provide(Ok(canonical)); + }); + } + + return { checkFilePath, readTextFile, writeTextFile, globFiles, temporaryDirectory }; +} + +/** + * Remove a temporary directory as its scope ends. + * + * A removal that fails during teardown is reported as an invariant rather than + * passed along: the failure it would otherwise carry names the generated + * directory, which the document never chose, and it can be unwinding a failure + * of its own that must not be replaced by platform text. + */ +function* discard(directory: string): Operation { + try { + yield* rm(directory, { recursive: true, force: true }); + } catch { + throw new FilesInvariantError("teardown"); + } +} + +/** + * Run the traversal, announcing each directory read when an observer is watching. + * + * The announcement is installed as `API.Fs` middleware for the duration of this + * one call rather than left in place, so an observer sees the reads this glob + * performs and nothing else. + */ +function traverse( + input: GlobInput, + observe: HostFilesObserver | undefined, +): Operation> { + const search = { patterns: input.include, root: input.cwd, exclude: input.exclude }; + if (observe === undefined) { + return API.Fs.operations.glob(search); + } + return scoped(function* () { + yield* FsApi.around({ + *readdirDirents([directory], next) { + observe({ operation: "glob", phase: "read-dir" }); + return yield* next(directory); + }, + }); + return yield* API.Fs.operations.glob(search); + }); +} + +/** + * Install the host provider beneath ordinary middleware. + * + * `at: "min"` is what lets a host wrap document filesystem access without + * replacing it — middleware installed later sees these operations and can + * delegate to them. + */ +export function useHostFiles(options: HostFilesOptions = {}): Operation { + const handler = hostFilesHandler(options); + return API.Files.around( + { + *checkFilePath([input]) { + return yield* handler.checkFilePath(input); + }, + *readTextFile([input]) { + return yield* handler.readTextFile(input); + }, + *writeTextFile([input]) { + return yield* handler.writeTextFile(input); + }, + *globFiles([input]) { + return yield* handler.globFiles(input); + }, + *temporaryDirectory() { + return yield* handler.temporaryDirectory(); + }, + }, + { at: "min" }, + ); +} diff --git a/packages/runtime/mod.ts b/packages/runtime/mod.ts index 8e468bee..4834d1ac 100644 --- a/packages/runtime/mod.ts +++ b/packages/runtime/mod.ts @@ -5,10 +5,12 @@ * `API` is available for middleware (`.around()`). * For normal calls, import operations directly. * - * Six domain APIs: + * Seven domain APIs: * - `API.Process` — subprocess execution (`exec`) - * - `API.Fs` — filesystem (`readTextFile`, `writeTextFile`, `stat`, `glob`, - * `realpath`, `ensureDir`, `rename`, `remove`) + * - `API.Fs` — the low-level host filesystem (`readTextFile`, `writeTextFile`, + * `stat`, `glob`, `realpath`, `ensureDir`, `rename`, `remove`) + * - `API.Files` — document filesystem access as whole semantic operations, + * with no host default. `useHostFiles()` installs the host provider. * - `API.Fetch` — HTTP requests (`fetch`) * - `API.Env` — the host: variables, platform info, the command that invokes * this xmd, and eval-block compilation @@ -64,3 +66,47 @@ export type { } from "./service.ts"; export { Config, timeout } from "./config.ts"; export type { ConfigApi } from "./config.ts"; +export { + asFilesFatal, + FILES_ERROR, + FILES_ERROR_MESSAGE, + FILES_FATAL, + FILES_INVARIANT_MESSAGE, + FILES_OPERATION_DENIED_MESSAGE, + FILES_PROVIDER_UNAVAILABLE_MESSAGE, + FILES_WRITE_SUCCESS, + Files, + FilesError, + FilesInvariantError, + FilesOperationDeniedError, + FilesProviderUnavailableError, + fileWriteFailure, + fileWriteSuccess, + filesFailure, + isFilesFatal, + parseFileWriteFailure, + parseFileWriteSuccess, + parseFilesFailure, + parseFilesFatal, +} from "./files.ts"; +export type { + FilePathInput, + FilesDeniableOperation, + FilesErrorData, + FilesFailureData, + FilesFatalData, + FilesFatalFailure, + FilesHandler, + FilesInvariantCategory, + FilesOperation, + FilesPhase, + FilesReason, + FileWriteFailureData, + FileWriteInput, + FileWritePhase, + FileWriteSuccess, + FileWriteTarget, + GlobInput, +} from "./files.ts"; +export { hostFilesHandler, useHostFiles } from "./host-files.ts"; +export type { HostFilesEvent, HostFilesObserver, HostFilesOptions } from "./host-files.ts"; diff --git a/packages/runtime/package.json b/packages/runtime/package.json index 371e4e4b..5282df66 100644 --- a/packages/runtime/package.json +++ b/packages/runtime/package.json @@ -5,6 +5,7 @@ "type": "module", "exports": { ".": "./mod.ts", + "./files": "./files.ts", "./test": "./test/mod.ts" }, "dependencies": { diff --git a/packages/runtime/tests/host-files.test.ts b/packages/runtime/tests/host-files.test.ts new file mode 100644 index 00000000..050c6cde --- /dev/null +++ b/packages/runtime/tests/host-files.test.ts @@ -0,0 +1,855 @@ +/** + * Tier HF — the host `API.Files` provider. + * + * These drive the provider directly rather than through a document, because + * what they assert is the contract a component cannot see: which phase a + * failure came from, that an ordinary condition is an `Err` and never a throw, + * that a missing provider throws and never falls back, and where the host + * guarantee actually stops. + * + * The replacement cases are the last of those. The host contract holds while + * the pathname namespace is stable, and the adapter's test-only observer is + * what makes "stable" falsifiable: it swaps part of the tree between the moment + * the adapter observes a path and the moment it uses one, synchronously, with + * no sleeping and no racing. What they prove is the documented weakness, not a + * containment claim. + * + * `mkdtemp`, `realpath`, `symlink`, `lstat`, and `readdir` have no + * `@effectionx/fs` equivalent; everything else goes through it. + */ + +import { describe, it } from "@executablemd/test-support/bdd"; +import { expect } from "@executablemd/test-support/expect"; +import { + ensure, + race, + resource, + scoped, + sleep, + spawn, + suspend, + until, + withResolvers, +} from "effection"; +import type { Operation, Result } from "effection"; +import { exists, readTextFile, rm, writeTextFile } from "@effectionx/fs"; +import { lstat, mkdir, mkdtemp, readdir, realpath, symlink } from "node:fs/promises"; +import { renameSync, symlinkSync, unlinkSync } from "node:fs"; +import { tmpdir } from "node:os"; +import { join } from "node:path"; +import process from "node:process"; +import { API } from "../apis.ts"; +import { hostFilesHandler, useHostFiles } from "../host-files.ts"; +import type { HostFilesEvent } from "../host-files.ts"; +import { + FILES_ERROR, + FILES_WRITE_SUCCESS, + Files, + FilesProviderUnavailableError, + parseFileWriteFailure, + parseFileWriteSuccess, + parseFilesFailure, + parseFilesFatal, +} from "../files.ts"; +import type { FilesHandler } from "../files.ts"; + +/** A workspace, and a directory beside it that is deliberately out of reach. */ +interface Fixture { + root: string; + workspace: string; + outside: string; +} + +function useFixture(): Operation { + return resource(function* (provide) { + const root = yield* until(realpath(yield* until(mkdtemp(join(tmpdir(), "hf-test-"))))); + yield* ensure(() => rm(root, { recursive: true, force: true })); + const workspace = join(root, "workspace"); + const outside = join(root, "outside"); + yield* until(mkdir(workspace)); + yield* until(mkdir(outside)); + yield* provide({ root, workspace, outside }); + }); +} + +/** + * A directory symlink, spelled the way the running platform accepts one. + * + * A junction is what Windows gives an unprivileged process, and it is the + * reparse point the guarantee statement names; everywhere else it is an + * ordinary directory symlink. + */ +const DIRECTORY_LINK = process.platform === "win32" ? "junction" : "dir"; + +function linkDirectory(target: string, path: string): Operation { + return until(symlink(target, path, DIRECTORY_LINK)); +} + +function handler(observe?: (event: HostFilesEvent) => void): FilesHandler { + return hostFilesHandler(observe === undefined ? {} : { observe }); +} + +function failed(result: Result): unknown { + if (result.ok) { + throw new Error("expected a failure, got a value"); + } + return result.error; +} + +function value(result: Result): T { + if (!result.ok) { + throw result.error; + } + return result.value; +} + +/** Everything in a directory, sorted, so an assertion names a set. */ +function* entries(directory: string): Operation { + return (yield* until(readdir(directory))).sort(); +} + +/** A platform error shaped like the ones that leak: a code, and a path. */ +const PLANTED = "/planted/absolute/path/secret.txt"; + +function planted(code: string): Error { + return Object.assign(new Error(`${code}: operation failed, at '${PLANTED}'`), { code }); +} + +/** + * The first Files infrastructure failure in a thrown graph. + * + * A destructor failure arrives wrapped in whatever Effection collected it with, + * so a test that only inspected the top of the graph would assert about the + * wrapper. Core owns the real traversal; this is the small local one a runtime + * test can have without importing the engine. + */ +function firstFilesFatal(error: unknown, seen = new Set()): unknown { + if (parseFilesFatal(error) !== undefined) { + return error; + } + if (typeof error !== "object" || error === null || seen.has(error)) { + return undefined; + } + seen.add(error); + const causes = + error instanceof AggregateError + ? error.errors + : error instanceof Error && error.cause !== undefined + ? [error.cause] + : []; + for (const cause of causes) { + const found = firstFilesFatal(cause, seen); + if (found !== undefined) { + return found; + } + } + return undefined; +} + +/** Every field a failure carries, as one string, for a leak assertion. */ +function inspected(error: unknown): string { + return JSON.stringify({ + message: error instanceof Error ? error.message : String(error), + data: error instanceof Error && "data" in error ? error.data : undefined, + }); +} + +describe("Tier HF — host Files provider", () => { + // HF1: the preliminary check is arithmetic. It has to be, because it runs + // before a write's children and its whole job is to cost nothing. + it("HF1: checkFilePath refuses inadmissible paths with no filesystem access", function* () { + const fixture = yield* useFixture(); + const touched: string[] = []; + + yield* scoped(function* () { + yield* API.Fs.around({ + *stat([path], next) { + touched.push(path); + return yield* next(path); + }, + *realpath([path], next) { + touched.push(path); + return yield* next(path); + }, + *readTextFile([path], next) { + touched.push(path); + return yield* next(path); + }, + }); + const files = handler(); + + for (const [path, reason] of [ + ["", "empty-path"], + [join(fixture.outside, "secret.txt"), "absolute-path"], + ["../outside/secret.txt", "lexical-escape"], + ]) { + const result = yield* files.checkFilePath({ cwd: fixture.workspace, path }); + expect(parseFilesFailure(failed(result))).toEqual({ + type: FILES_ERROR, + operation: "check-file-path", + phase: "lexical", + reason, + }); + } + + // And an admissible path answers with nothing usable. + const admitted = yield* files.checkFilePath({ cwd: fixture.workspace, path: "notes.md" }); + expect(admitted).toEqual({ ok: true, value: undefined }); + }); + + expect(touched).toEqual([]); + }); + + // HF2: the ordinary round trip, and the shape a success has. + it("HF2: a write commits and reads back, reporting a validated outcome", function* () { + const fixture = yield* useFixture(); + const files = handler(); + + const written = yield* files.writeTextFile({ + cwd: fixture.workspace, + path: "nested/notes.md", + content: "first", + }); + expect(parseFileWriteSuccess(value(written))).toEqual({ + type: FILES_WRITE_SUCCESS, + publication: "host-committed", + }); + + const read = yield* files.readTextFile({ cwd: fixture.workspace, path: "nested/notes.md" }); + expect(value(read)).toBe("first"); + // The commit consumed the temporary, so nothing is left beside the file. + expect(yield* entries(join(fixture.workspace, "nested"))).toEqual(["notes.md"]); + }); + + // HF3: the search's document-facing shape. Sorting and deduplication are the + // provider's, so a document that branches on a listing branches the same way + // wherever it runs. + it("HF3: globFiles returns sorted, deduplicated regular files", function* () { + const fixture = yield* useFixture(); + yield* writeTextFile(join(fixture.workspace, "zebra.md"), "z"); + yield* writeTextFile(join(fixture.workspace, "Beta.md"), "b"); + yield* until(mkdir(join(fixture.workspace, "one"))); + yield* writeTextFile(join(fixture.workspace, "one/middle.md"), "m"); + yield* until(symlink(join(fixture.workspace, "Beta.md"), join(fixture.workspace, "link.md"))); + + const found = yield* handler().globFiles({ + cwd: fixture.workspace, + include: ["**/*.md", "**/*.md"], + exclude: [], + }); + + // A symbolic link is a link rather than a file, so it is not a result. + expect(value(found)).toEqual(["Beta.md", "one/middle.md", "zebra.md"]); + }); + + // HF4: every ordinary failure is a Result. Nothing throws, and nothing the + // platform said reaches the caller — a code that is not recognized selects + // the generic reason rather than being carried across. + it("HF4: platform failures become structured Errs, and leak nothing", function* () { + const fixture = yield* useFixture(); + yield* writeTextFile(join(fixture.workspace, "notes.md"), "existing"); + + const cases: Array<{ + code: string; + install: () => Operation; + expected: { phase: string; reason: string }; + }> = [ + { + code: "ELOOP", + expected: { phase: "resolution", reason: "too-many-symlinks" }, + *install() { + yield* API.Fs.around({ + *realpath([path], next) { + if (path.endsWith("notes.md")) { + throw planted("ELOOP"); + } + return yield* next(path); + }, + }); + }, + }, + { + code: "EACCES", + expected: { phase: "target", reason: "permission-denied" }, + *install() { + yield* API.Fs.around({ + *stat([path], next) { + if (path.endsWith("notes.md")) { + throw planted("EACCES"); + } + return yield* next(path); + }, + }); + }, + }, + { + code: "ENOTAREALCODE", + expected: { phase: "access", reason: "operation-failed" }, + *install() { + yield* API.Fs.around({ + // deno-lint-ignore require-yield + *readTextFile() { + throw planted("ENOTAREALCODE"); + }, + }); + }, + }, + ]; + + for (const shape of cases) { + yield* scoped(function* () { + yield* shape.install(); + const result = yield* handler().readTextFile({ + cwd: fixture.workspace, + path: "notes.md", + }); + const error = failed(result); + expect(parseFilesFailure(error)).toEqual({ + type: FILES_ERROR, + operation: "read", + phase: shape.expected.phase, + reason: shape.expected.reason, + }); + const text = inspected(error); + expect(text).not.toContain(PLANTED); + expect(text).not.toContain(shape.code); + expect(text).not.toContain(fixture.workspace); + }); + } + }); + + // HF5: the write's phase table. Which phase a write stopped at is the only + // thing that decides what may be said about the target, so each one is + // produced from the outside and its whole structure checked. + it("HF5: each write phase reports a valid, phase-consistent outcome", function* () { + const fixture = yield* useFixture(); + + const cases: Array<{ + name: string; + install?: () => Operation; + path?: string; + prepare?: () => Operation; + expected: Record; + }> = [ + { + name: "lexical", + path: "../outside/planted.txt", + expected: { phase: "lexical", reason: "lexical-escape", target: "unchanged" }, + }, + { + name: "target", + path: "adirectory", + *prepare() { + yield* until(mkdir(join(fixture.workspace, "adirectory"))); + }, + expected: { phase: "target", reason: "directory", target: "unchanged" }, + }, + { + name: "parents", + expected: { phase: "parents", reason: "read-only", target: "unchanged" }, + *install() { + yield* API.Fs.around({ + // deno-lint-ignore require-yield + *ensureDir() { + throw planted("EROFS"); + }, + }); + }, + }, + { + name: "temporary", + expected: { phase: "temporary", reason: "no-space", target: "unchanged" }, + *install() { + yield* API.Fs.around({ + // deno-lint-ignore require-yield + *writeTextFile() { + throw planted("ENOSPC"); + }, + }); + }, + }, + { + name: "commit", + expected: { phase: "commit", reason: "cross-device", target: "commit-unknown" }, + *install() { + yield* API.Fs.around({ + // deno-lint-ignore require-yield + *rename() { + throw planted("EXDEV"); + }, + }); + }, + }, + { + name: "cleanup", + expected: { phase: "cleanup", cleanup: "permission-denied", target: "committed" }, + *install() { + yield* API.Fs.around({ + // deno-lint-ignore require-yield + *remove() { + throw planted("EPERM"); + }, + }); + }, + }, + { + name: "commit and cleanup together", + expected: { + phase: "commit", + reason: "cross-device", + cleanup: "permission-denied", + target: "commit-unknown", + }, + *install() { + yield* API.Fs.around({ + // deno-lint-ignore require-yield + *rename() { + throw planted("EXDEV"); + }, + // deno-lint-ignore require-yield + *remove() { + throw planted("EPERM"); + }, + }); + }, + }, + ]; + + for (const shape of cases) { + if (shape.prepare) { + yield* shape.prepare(); + } + yield* scoped(function* () { + if (shape.install) { + yield* shape.install(); + } + const result = yield* handler().writeTextFile({ + cwd: fixture.workspace, + path: shape.path ?? `${shape.name.replace(/ /g, "-")}.txt`, + content: "replacement", + }); + const parsed = parseFileWriteFailure(failed(result)); + expect(parsed).toEqual({ + type: FILES_ERROR, + operation: "write", + ...shape.expected, + }); + expect(inspected(failed(result))).not.toContain(PLANTED); + }); + } + }); + + // HF6: containment, judged against what the adapter can observe. A symlink + // out is refused at resolution for both forms, and the destination is never + // named — reporting where a link pointed would perform the escape. + it("HF6: a symlink leaving the working directory is refused, naming nothing", function* () { + const fixture = yield* useFixture(); + yield* writeTextFile(join(fixture.outside, "secret.txt"), "SECRET"); + yield* until(symlink(join(fixture.outside, "secret.txt"), join(fixture.workspace, "escape"))); + const files = handler(); + + const read = yield* files.readTextFile({ cwd: fixture.workspace, path: "escape" }); + expect(parseFilesFailure(failed(read))?.reason).toBe("resolved-escape"); + expect(inspected(failed(read))).not.toContain("SECRET"); + expect(inspected(failed(read))).not.toContain(fixture.outside); + + const written = yield* files.writeTextFile({ + cwd: fixture.workspace, + path: "escape", + content: "planted", + }); + expect(parseFileWriteFailure(failed(written))).toEqual({ + type: FILES_ERROR, + operation: "write", + phase: "resolution", + reason: "resolved-escape", + target: "unchanged", + }); + expect(yield* readTextFile(join(fixture.outside, "secret.txt"))).toBe("SECRET"); + }); + + // HF7: an internal link is followed rather than replaced, which is what makes + // reading back through either name agree. + it("HF7: an internal symlink is followed for a write", function* () { + const fixture = yield* useFixture(); + yield* until(mkdir(join(fixture.workspace, "real"))); + yield* writeTextFile(join(fixture.workspace, "real/target.txt"), "original"); + yield* until( + symlink(join(fixture.workspace, "real/target.txt"), join(fixture.workspace, "link.txt")), + ); + + yield* handler().writeTextFile({ + cwd: fixture.workspace, + path: "link.txt", + content: "replaced", + }); + + expect(yield* readTextFile(join(fixture.workspace, "real/target.txt"))).toBe("replaced"); + expect((yield* until(lstat(join(fixture.workspace, "link.txt")))).isSymbolicLink()).toBe(true); + }); + + // HF8: a dangling final link is the one hole resolution cannot close — there + // is nothing to resolve — and the temporary plus rename is what closes it. + // The link is replaced rather than followed to wherever it pointed. + it("HF8: a write replaces a dangling symlink instead of following it", function* () { + const fixture = yield* useFixture(); + yield* until(symlink(join(fixture.outside, "absent.txt"), join(fixture.workspace, "dangling"))); + + const written = yield* handler().writeTextFile({ + cwd: fixture.workspace, + path: "dangling", + content: "content", + }); + + expect(parseFileWriteSuccess(value(written))?.publication).toBe("host-committed"); + expect(yield* readTextFile(join(fixture.workspace, "dangling"))).toBe("content"); + expect((yield* until(lstat(join(fixture.workspace, "dangling")))).isSymbolicLink()).toBe(false); + expect(yield* exists(join(fixture.outside, "absent.txt"))).toBe(false); + }); + + // HF9: the observer's own contract. Production installs no observer, so this + // fixes what a test may rely on: the private phases, in order, once each. + it("HF9: the observer reports each private phase in order", function* () { + const fixture = yield* useFixture(); + const seen: string[] = []; + const files = handler((event) => seen.push(`${event.operation}.${event.phase}`)); + + yield* files.writeTextFile({ cwd: fixture.workspace, path: "a.txt", content: "a" }); + expect(seen).toEqual([ + "write.target", + "write.parents", + "write.temporary", + "write.commit", + "write.cleanup", + ]); + + seen.length = 0; + yield* files.readTextFile({ cwd: fixture.workspace, path: "a.txt" }); + expect(seen).toEqual(["read.target", "read.access"]); + + seen.length = 0; + yield* files.globFiles({ cwd: fixture.workspace, include: ["**/*"], exclude: [] }); + expect(seen).toEqual(["glob.read-dir"]); + }); + + // HF10: where the host guarantee stops, made observable. The parent is + // replaced by a link out of the workspace between the moment the adapter + // resolved the path and the moment it creates parents — synchronously, in one + // uninterrupted step, which is exactly what another process can do and what + // no shipped runtime lets this adapter prevent. + // + // What is asserted is the documented weakness. A test that expected refusal + // here would be asserting a containment claim `xmd run` does not make. + it("HF10: a parent replaced after resolution is observed by the later calls", function* () { + const fixture = yield* useFixture(); + yield* until(mkdir(join(fixture.workspace, "parent"))); + + let swapped = false; + const files = handler((event) => { + if (swapped || event.operation !== "write" || event.phase !== "parents") { + return; + } + swapped = true; + // Synchronous, so nothing runs between the observation and the use. + renameSync(join(fixture.workspace, "parent"), join(fixture.root, "moved")); + symlinkSync(fixture.outside, join(fixture.workspace, "parent"), DIRECTORY_LINK); + }); + + const written = yield* files.writeTextFile({ + cwd: fixture.workspace, + path: "parent/planted.txt", + content: "complete replacement", + }); + + expect(swapped).toBe(true); + // The write committed — through the replacement, which is the limitation. + expect(parseFileWriteSuccess(value(written))?.publication).toBe("host-committed"); + expect(yield* readTextFile(join(fixture.outside, "planted.txt"))).toBe("complete replacement"); + // Atomicity still holds: what landed is the whole file, never a fragment, + // and no temporary was left beside it. + expect(yield* entries(fixture.outside)).toEqual(["planted.txt"]); + }); + + // HF10b: the same seam on a read. A file swapped for a link out after + // resolution is read through, and the failure that would have been reported + // is not manufactured. + it("HF10b: a target replaced after resolution is read through", function* () { + const fixture = yield* useFixture(); + yield* writeTextFile(join(fixture.workspace, "notes.md"), "inside"); + yield* writeTextFile(join(fixture.outside, "secret.txt"), "SECRET"); + + let swapped = false; + const files = handler((event) => { + if (swapped || event.operation !== "read" || event.phase !== "access") { + return; + } + swapped = true; + unlinkSync(join(fixture.workspace, "notes.md")); + symlinkSync(join(fixture.outside, "secret.txt"), join(fixture.workspace, "notes.md")); + }); + + const read = yield* files.readTextFile({ cwd: fixture.workspace, path: "notes.md" }); + + expect(swapped).toBe(true); + expect(value(read)).toBe("SECRET"); + }); + + // HF11: rename is reached only as the commit phase, and a middleware fault on + // either side of it is the same event from where the adapter stands. Both + // report `commit-unknown`, and they have to: one of the two runs did commit. + it("HF11: a commit fault before and after next reports the same unknown outcome", function* () { + const fixture = yield* useFixture(); + + for (const side of ["before", "after"] as const) { + yield* writeTextFile(join(fixture.workspace, `${side}.md`), "first"); + const result = yield* scoped(function* () { + yield* API.Fs.around({ + *rename([from, to], next) { + if (side === "after") { + yield* next(from, to); + } + throw planted("EXDEV"); + }, + }); + return yield* handler().writeTextFile({ + cwd: fixture.workspace, + path: `${side}.md`, + content: "second", + }); + }); + + expect(parseFileWriteFailure(failed(result))).toEqual({ + type: FILES_ERROR, + operation: "write", + phase: "commit", + reason: "cross-device", + target: "commit-unknown", + }); + } + + // The claim is honest about both: one run kept the previous file and one + // replaced it, and the report was the same. + expect(yield* readTextFile(join(fixture.workspace, "before.md"))).toBe("first"); + expect(yield* readTextFile(join(fixture.workspace, "after.md"))).toBe("second"); + }); + + // HF12: cancellation is not a Result. It resumes the generator rather than + // throwing, so nothing here can convert it into a write outcome — and the + // temporary is still removed, because its cleanup was registered before the + // step most likely to be interrupted. + it("HF12: cancellation produces no Result and leaves no temporary", function* () { + const fixture = yield* useFixture(); + yield* writeTextFile(join(fixture.workspace, "notes.md"), "first"); + let settled: unknown = "not settled"; + + yield* race([ + scoped(function* () { + yield* API.Fs.around({ + *writeTextFile([path, content], next) { + yield* next(path, content); + yield* suspend(); + }, + }); + settled = yield* handler().writeTextFile({ + cwd: fixture.workspace, + path: "notes.md", + content: "second", + }); + }), + sleep(250), + ]); + + expect(settled).toBe("not settled"); + expect(yield* readTextFile(join(fixture.workspace, "notes.md"))).toBe("first"); + expect(yield* entries(fixture.workspace)).toEqual(["notes.md"]); + }); + + // HF12b: cleanup that fails *while cancellation is unwinding* is the one exit + // with no outcome to report beside it. Manufacturing one would turn a halt + // into a write result, and forwarding the platform's error would name a + // temporary the document never chose — so a fixed teardown invariant leaves + // the scope instead, and the engine's fatal discovery finds it there. + // + // Deterministic rather than raced: the write suspends after the temporary is + // created, the test waits to observe that it did, and only then halts. + it("HF12b: a cleanup failure during cancellation escapes as a teardown invariant", function* () { + const fixture = yield* useFixture(); + yield* writeTextFile(join(fixture.workspace, "notes.md"), "first"); + const suspended = withResolvers(); + let settled: unknown = "not settled"; + + const write = yield* spawn(function* () { + yield* scoped(function* () { + yield* API.Fs.around({ + *writeTextFile([path, content], next) { + yield* next(path, content); + suspended.resolve(); + yield* suspend(); + }, + // deno-lint-ignore require-yield + *remove() { + throw planted("EPERM"); + }, + }); + settled = yield* handler().writeTextFile({ + cwd: fixture.workspace, + path: "notes.md", + content: "second", + }); + }); + }); + + yield* suspended.operation; + let thrown: unknown; + try { + yield* write.halt(); + } catch (error) { + thrown = error; + } + + // No Result was produced: the halt is not a write outcome. + expect(settled).toBe("not settled"); + // And the failure that did leave is the fixed infrastructure one. + const teardown = firstFilesFatal(thrown); + expect(parseFilesFatal(teardown)).toEqual({ + type: "executablemd.runtime.files-fatal/v1", + kind: "invariant", + category: "teardown", + }); + expect(teardown instanceof Error ? teardown.message : "").toBe( + "Files provider invariant failed", + ); + const text = inspected(teardown); + expect(text).not.toContain(PLANTED); + expect(text).not.toContain(".tmp"); + expect(text).not.toContain("EPERM"); + // The category is structural control data for a consumer deciding what to + // fence — it belongs in `data`, and never in the message. + expect(teardown instanceof Error ? teardown.message : "").not.toContain("teardown"); + expect(teardown instanceof Error ? teardown.cause : "unset").toBeUndefined(); + expect(yield* readTextFile(join(fixture.workspace, "notes.md"))).toBe("first"); + }); + + // HF13: the temporary directory is a resource, so its lifetime is the + // acquiring scope's and a halt cannot land between creating it and owning its + // removal. + it("HF13: a temporary directory lives and dies with its acquiring scope", function* () { + const files = handler(); + let acquired = ""; + + yield* scoped(function* () { + acquired = value(yield* files.temporaryDirectory()); + expect(yield* exists(acquired)).toBe(true); + }); + expect(yield* exists(acquired)).toBe(false); + + // Halted before the acquiring task ran at all: an acquisition that + // suspended on a pending creation would finish afterwards and leave a + // directory nothing owns. + const early = yield* spawn(() => files.temporaryDirectory()); + yield* early.halt(); + yield* sleep(50); + const stale = (yield* entries(yield* until(realpath(tmpdir())))).filter((entry) => + entry.startsWith("xmd-tempdir-"), + ); + expect(stale.includes(acquired)).toBe(false); + }); + + // HF14: absence is fatal, for every operation. A default that reached the + // host would make an uninstalled provider indistinguishable from an installed + // one, which is the whole reason this Api has no host default. + it("HF14: with no provider installed every operation throws and touches nothing", function* () { + const fixture = yield* useFixture(); + const touched: string[] = []; + + yield* scoped(function* () { + yield* API.Fs.around({ + *stat([path], next) { + touched.push(path); + return yield* next(path); + }, + *realpath([path], next) { + touched.push(path); + return yield* next(path); + }, + *readTextFile([path], next) { + touched.push(path); + return yield* next(path); + }, + *writeTextFile([path, content], next) { + touched.push(path); + return yield* next(path, content); + }, + }); + + const calls: Array> = [ + Files.operations.checkFilePath({ cwd: fixture.workspace, path: "a.md" }), + Files.operations.readTextFile({ cwd: fixture.workspace, path: "a.md" }), + Files.operations.writeTextFile({ cwd: fixture.workspace, path: "a.md", content: "x" }), + Files.operations.globFiles({ cwd: fixture.workspace, include: ["*"], exclude: [] }), + Files.operations.temporaryDirectory(), + ]; + + for (const call of calls) { + let thrown: unknown; + try { + yield* call; + } catch (error) { + thrown = error; + } + expect(thrown).toBeInstanceOf(FilesProviderUnavailableError); + expect(parseFilesFatal(thrown)).toEqual({ + type: "executablemd.runtime.files-fatal/v1", + kind: "provider-unavailable", + }); + // Fixed diagnostics, and no cause to inspect. + expect(thrown instanceof Error ? thrown.message : "").toBe( + "Files provider is not installed", + ); + expect(thrown instanceof Error ? thrown.cause : "unset").toBeUndefined(); + } + }); + + expect(touched).toEqual([]); + }); + + // HF15: installation is what `useHostFiles` is for, and it goes beneath + // ordinary middleware so a host can still wrap document filesystem access. + it("HF15: useHostFiles installs beneath middleware that can wrap it", function* () { + const fixture = yield* useFixture(); + yield* writeTextFile(join(fixture.workspace, "notes.md"), "underneath"); + + const observed: string[] = []; + const read = yield* scoped(function* () { + yield* useHostFiles(); + yield* Files.around({ + *readTextFile([input], next) { + observed.push(input.path); + return yield* next(input); + }, + }); + return yield* Files.operations.readTextFile({ cwd: fixture.workspace, path: "notes.md" }); + }); + + expect(value(read)).toBe("underneath"); + expect(observed).toEqual(["notes.md"]); + }); + + // HF16: a junction is the Windows shape of the same limitation, and this is + // the row that names it. Elsewhere it is an ordinary directory symlink, so + // the case runs on every target rather than only where the reparse point + // exists. + it("HF16: a directory link is refused at resolution on every platform", function* () { + const fixture = yield* useFixture(); + yield* linkDirectory(fixture.outside, join(fixture.workspace, "escape")); + + const written = yield* handler().writeTextFile({ + cwd: fixture.workspace, + path: "escape/planted.txt", + content: "planted", + }); + + expect(parseFileWriteFailure(failed(written))?.reason).toBe("resolved-escape"); + expect(yield* entries(fixture.outside)).toEqual([]); + }); +}); diff --git a/scripts/files-contract-probe.ts b/scripts/files-contract-probe.ts new file mode 100644 index 00000000..8039e250 --- /dev/null +++ b/scripts/files-contract-probe.ts @@ -0,0 +1,175 @@ +/** + * The host `API.Files` contract, as an executable a release target can run. + * + * `packages/runtime/tests/host-files.test.ts` proves the same contract from + * source. This exists because the shipped artifact is a compiled binary, and + * the host adapter reaches `node:path`, `node:fs`, and `node:os` — the modules + * whose behavior a `deno compile` graph, and the platform it was compiled for, + * can change. A source suite that passes says nothing about that. + * + * So this is deliberately small and self-verifying: it asserts the contract's + * observable claims and exits non-zero if any does not hold. It prints every + * claim it checked, because a probe that passes silently is indistinguishable + * from one that checked nothing. + * + * Usage: + * deno run --allow-all scripts/files-contract-probe.ts + * deno compile --allow-all --output scripts/files-contract-probe.ts + */ + +import { ensure, exit, main, scoped } from "effection"; +import type { Result } from "effection"; +import { mkdirSync, mkdtempSync, realpathSync, rmSync, statSync, symlinkSync } from "node:fs"; +import { tmpdir } from "node:os"; +import { join } from "node:path"; +import process from "node:process"; +import { + Files, + parseFileWriteFailure, + parseFileWriteSuccess, + parseFilesFailure, + parseFilesFatal, + useHostFiles, +} from "@executablemd/runtime"; + +const checked: string[] = []; +const failures: string[] = []; + +function check(claim: string, held: boolean): void { + checked.push(claim); + if (!held) { + failures.push(claim); + } +} + +function reasonOf(result: Result): string | undefined { + return result.ok ? undefined : parseFilesFailure(result.error)?.reason; +} + +/** + * A write reports its own shape. Reading one with the non-write parser is how a + * probe silently checks nothing, so the two are separate here. + */ +function writeReasonOf(result: Result): string | undefined { + return result.ok ? undefined : parseFileWriteFailure(result.error)?.reason; +} + +function valueOf(result: Result): T | undefined { + return result.ok ? result.value : undefined; +} + +function exists(path: string): boolean { + try { + statSync(path); + return true; + } catch { + return false; + } +} + +/** A junction is what an unprivileged Windows process gets; elsewhere, a symlink. */ +const DIRECTORY_LINK = process.platform === "win32" ? "junction" : "dir"; + +await main(function* () { + const root = realpathSync(mkdtempSync(join(tmpdir(), "xmd-files-probe-"))); + yield* ensure(() => rmSync(root, { recursive: true, force: true })); + const workspace = join(root, "workspace"); + const outside = join(root, "outside"); + mkdirSync(workspace); + mkdirSync(outside); + + // Absence is checked before a provider exists, so the terminal handler is the + // one answering. + let absent: unknown; + try { + yield* Files.operations.readTextFile({ cwd: workspace, path: "notes.md" }); + } catch (error) { + absent = error; + } + check( + "an absent provider throws provider-unavailable", + parseFilesFatal(absent)?.kind === "provider-unavailable", + ); + + yield* useHostFiles(); + + check( + "an empty path is refused lexically", + reasonOf(yield* Files.operations.checkFilePath({ cwd: workspace, path: "" })) === "empty-path", + ); + check( + "an absolute path is refused lexically", + reasonOf( + yield* Files.operations.checkFilePath({ cwd: workspace, path: join(outside, "secret.txt") }), + ) === "absolute-path", + ); + check( + "a lexical escape is refused", + reasonOf( + yield* Files.operations.checkFilePath({ cwd: workspace, path: "../outside/secret.txt" }), + ) === "lexical-escape", + ); + check( + "an admissible path is admitted", + (yield* Files.operations.checkFilePath({ cwd: workspace, path: "notes.md" })).ok, + ); + + const written = yield* Files.operations.writeTextFile({ + cwd: workspace, + path: "nested/notes.md", + content: "probe content", + }); + check( + "a write commits to the host", + parseFileWriteSuccess(valueOf(written))?.publication === "host-committed", + ); + check( + "the write reads back", + valueOf(yield* Files.operations.readTextFile({ cwd: workspace, path: "nested/notes.md" })) === + "probe content", + ); + check("the commit left no temporary behind", !exists(join(workspace, "nested/notes.md.tmp"))); + + symlinkSync(outside, join(workspace, "escape"), DIRECTORY_LINK); + check( + "a directory link out of the working directory is refused", + writeReasonOf( + yield* Files.operations.writeTextFile({ + cwd: workspace, + path: "escape/planted.txt", + content: "planted", + }), + ) === "resolved-escape", + ); + check("nothing was written through the link", !exists(join(outside, "planted.txt"))); + + const found = yield* Files.operations.globFiles({ + cwd: workspace, + include: ["**/*.md"], + exclude: [], + }); + check( + "the search returns POSIX-relative files", + JSON.stringify(valueOf(found)) === '["nested/notes.md"]', + ); + + let temporary = ""; + yield* scoped(function* () { + temporary = valueOf(yield* Files.operations.temporaryDirectory()) ?? ""; + check("a temporary directory is acquired", temporary.length > 0 && exists(temporary)); + }); + check("a temporary directory is removed with its scope", !exists(temporary)); + + // Diagnostics rather than a result, so they go to stderr: this probe's whole + // output is the account of what it checked. + for (const claim of checked) { + console.error(`${failures.includes(claim) ? "FAIL" : "ok "} ${claim}`); + } + if (failures.length > 0) { + console.error(`files contract: ${failures.length} of ${checked.length} claims failed`); + yield* exit(1); + } + console.error( + `files contract: ${checked.length} claims hold on ${process.platform}/${process.arch}`, + ); +}); diff --git a/scripts/runtime-test-exclusions.ts b/scripts/runtime-test-exclusions.ts index 6f2d2333..9a63b7fd 100644 --- a/scripts/runtime-test-exclusions.ts +++ b/scripts/runtime-test-exclusions.ts @@ -114,6 +114,12 @@ const DENO_ONLY_TOOLING: RuntimeExclusion[] = [ "exercises Deno-private node:sqlite transaction identities and real SQLite savepoint failure behavior; node:sqlite remains behind --experimental-sqlite on Node 22", issue: "https://github.com/taras/executable.md/issues/365", }, + { + path: "packages/core/tests/loaded-copy-files.test.ts", + reason: + "builds the second copy of the runtime's Files module with `deno bundle`, which is Deno's; the structural recognition it proves is runtime-neutral and is also covered by fatal-cause.test.ts under all three", + issue: DERIVED_SCOPE, + }, ]; /** diff --git a/scripts/tests/filesystem-contract-workflow.test.ts b/scripts/tests/filesystem-contract-workflow.test.ts new file mode 100644 index 00000000..a65e599d --- /dev/null +++ b/scripts/tests/filesystem-contract-workflow.test.ts @@ -0,0 +1,152 @@ +/** + * The five-target filesystem contract job, held to the release matrix. + * + * The claim `xmd run` makes about containment is per-platform: path arithmetic + * and `realpath` are the host's, and Windows brings drive letters, UNC paths, + * junctions, and reparse points that POSIX does not. A job that covered four of + * the five triples would leave the fifth's claim asserted and unproven, and + * nothing about adding a release target would notice. + * + * So the matrix is held to `RELEASE_TARGETS` by set equality, the same way + * `publish-workflow-membership.test.ts` holds the release workflow. Adding a + * target without adding a row fails here. + */ + +import matter from "gray-matter"; +import { describe, it } from "@executablemd/test-support/bdd"; +import { expect } from "@executablemd/test-support/expect"; +import type { Operation } from "effection"; +import { readTextFile } from "@effectionx/fs"; +import { RELEASE_TARGETS } from "../lib/release-targets.ts"; + +const CI_WORKFLOW = new URL("../../.github/workflows/ci.yml", import.meta.url); +const JOB = "filesystem-contract"; +const PROBE = "scripts/files-contract-probe.ts"; +const SUITE = "packages/runtime/tests/host-files.test.ts"; + +function object(value: unknown, label: string): Record { + if (value === null || typeof value !== "object" || Array.isArray(value)) { + throw new Error(`${label} is not an object`); + } + return Object.fromEntries(Object.entries(value)); +} + +function string(value: unknown, label: string): string { + if (typeof value !== "string") { + throw new Error(`${label} is not a string`); + } + return value; +} + +interface Row { + runner: string; + target: string; +} + +interface ContractJob { + runsOn: string; + rows: Row[]; + commands: string[]; +} + +function* job(): Operation { + const source = yield* readTextFile(CI_WORKFLOW); + const document = object(matter(`---\n${source}\n---`).data, "workflow"); + const jobs = object(document.jobs, "workflow.jobs"); + const contract = object(jobs[JOB], `workflow.jobs.${JOB}`); + const strategy = object(contract.strategy, `${JOB}.strategy`); + const matrix = object(strategy.matrix, `${JOB}.strategy.matrix`); + const include = matrix.include; + if (!Array.isArray(include)) { + throw new Error(`${JOB}.strategy.matrix.include is not an array`); + } + const steps = contract.steps; + if (!Array.isArray(steps)) { + throw new Error(`${JOB}.steps is not an array`); + } + + return { + runsOn: string(contract["runs-on"], `${JOB}.runs-on`), + rows: include.map((entry, index) => { + const row = object(entry, `${JOB}.strategy.matrix.include[${index}]`); + return { + runner: string(row.runner, `include[${index}].runner`), + target: string(row.target, `include[${index}].target`), + }; + }), + commands: steps.flatMap((entry, index) => { + const step = object(entry, `${JOB}.steps[${index}]`); + return "run" in step ? [string(step.run, `${JOB}.steps[${index}].run`)] : []; + }), + }; +} + +describe("the filesystem contract matrix", () => { + it("covers exactly the release targets", function* () { + const contract = yield* job(); + expect([...contract.rows.map((row) => row.target)].sort()).toEqual( + Object.keys(RELEASE_TARGETS).sort(), + ); + }); + + it("gives every target its own runner", function* () { + const contract = yield* job(); + const runners = contract.rows.map((row) => row.runner); + expect(new Set(runners).size).toEqual(runners.length); + expect(contract.runsOn).toEqual("${{ matrix.runner }}"); + }); + + it("names a runner whose platform matches its target", function* () { + const platforms: Record = { + "macos-15": { os: "darwin", arch: "arm64" }, + "macos-15-intel": { os: "darwin", arch: "x64" }, + "ubuntu-24.04": { os: "linux", arch: "x64" }, + "ubuntu-24.04-arm": { os: "linux", arch: "arm64" }, + "windows-2025": { os: "win32", arch: "x64" }, + }; + + const contract = yield* job(); + for (const row of contract.rows) { + const runner = platforms[row.runner]; + if (runner === undefined) { + throw new Error(`no platform recorded for runner "${row.runner}"`); + } + expect(runner).toEqual(RELEASE_TARGETS[row.target]); + } + }); + + // Every row must run all four: the source suite under each runtime the + // project ships, and the compiled probe. A row that only ran one of them + // would report "the contract holds" for a shape it never executed. + it("runs the contract under Deno, Node, Bun, and a compiled binary", function* () { + const commands = (yield* job()).commands.join("\n"); + + expect(commands).toContain(`deno test --allow-all --frozen ${SUITE}`); + expect(commands).toContain(`tsx --tsconfig tsconfig.node.json --test ${SUITE}`); + expect(commands).toContain(`bun test --timeout=300000 ${SUITE}`); + expect(commands).toContain(`deno compile`); + expect(commands).toContain(PROBE); + }); + + // The Deno steps come before either package manager's install, because each + // rewrites `node_modules` into its own layout and a Deno run afterwards + // resolves through links the other pruned (#279). + it("runs every Deno step before a package manager rewrites node_modules", function* () { + const commands = (yield* job()).commands; + const lastDeno = commands.findLastIndex((command) => command.includes("deno ")); + const firstInstall = commands.findIndex( + (command) => command.trim() === "pnpm install" || command.trim() === "bun install", + ); + + expect(lastDeno).toBeGreaterThan(-1); + expect(firstInstall).toBeGreaterThan(lastDeno); + }); + + it("does not duplicate the whole corpus on every row", function* () { + const commands = (yield* job()).commands.join("\n"); + + expect(commands).not.toContain("deno task test\n"); + expect(commands).not.toContain("pnpm test:node"); + expect(commands).not.toContain("bun run test:bun"); + }); +}); diff --git a/scripts/tests/publish-workflow-membership.test.ts b/scripts/tests/publish-workflow-membership.test.ts index 14bafc37..c306be80 100644 --- a/scripts/tests/publish-workflow-membership.test.ts +++ b/scripts/tests/publish-workflow-membership.test.ts @@ -94,13 +94,20 @@ describe("release.yml binary compilation", () => { if (!commands.includes("deno compile")) { continue; } + // Every compile in the file, not only the first: a workflow may compile + // more than one entrypoint, and reading to a single named one would let + // the first invocation's flags answer for all of them. + // // A folded shell command: read from `deno compile` to the entrypoint that - // ends it, so line breaks and continuations do not matter. - const invocation = commands.slice(commands.indexOf("deno compile")); - const flags = invocation.slice(0, invocation.indexOf("packages/cli/src/compiled.ts")); - for (const flag of ["--node-modules-dir=none", "--cached-only", "--frozen"]) { - if (!flags.includes(flag)) { - compiles.push(`${entry} compiles without ${flag}`); + // ends it — the first `.ts` path after the flags — so line breaks and + // continuations do not matter. + for (const invocation of commands.split("deno compile").slice(1)) { + const entrypoint = invocation.search(/\S+\.ts\b/); + const flags = entrypoint === -1 ? invocation : invocation.slice(0, entrypoint); + for (const flag of ["--node-modules-dir=none", "--cached-only", "--frozen"]) { + if (!flags.includes(flag)) { + compiles.push(`${entry} compiles without ${flag}`); + } } } } diff --git a/specs/executable-mdx-spec.md b/specs/executable-mdx-spec.md index 97836724..5beb9e4b 100644 --- a/specs/executable-mdx-spec.md +++ b/specs/executable-mdx-spec.md @@ -116,12 +116,31 @@ paths relative to cwd, and the engine's own file access is written that way: component search directories (`["./components", "./"]`) are relative, and resolved paths in the journal (`"components/Greeting.md"`) are relative. -A component that resolves against the **contextual** working directory is the -exception, because a relative path would resolve against the process's -directory instead of the one it was given. `` (§6.13) resolves its -`path` prop against `Env.cwd` itself and hands the Fs Api the absolute result; -`` (§6.14) searches `Env.cwd` and returns relative paths. Nothing either -resolves reaches a printed error or the journal. +#### Document data and engine control plane + +Two kinds of filesystem access are separate boundaries, and the separation is +what lets one document mean the same thing in two environments. + +**Document data** — the files a document names in its own text — goes through +`API.Files`, a contextual Api of whole semantic operations. `` (§6.13), +`` (§6.14), and `` (§6.11) speak only that Api, hold no host +path, and never learn which provider answered. `xmd run` installs a host +provider that resolves those paths in the caller's filesystem; a workflow run +installs one whose paths name entries in a logical filesystem the run owns. +The Api has **no host default**: with no provider installed, every operation +fails the execution rather than reaching the host. + +**The engine's own control plane** — the root document, component search, +replay guards, the eval compiler, the diagnostic journal, and the test target +— reads host paths the caller selected, through the low-level `API.Fs`. Those +are not document-addressable, and they stay where they are. + +A path a document authored is always relative and always resolved by the +provider, against the contextual `Env.cwd` the component supplies with it. +Nothing the provider resolves reaches a printed error or the journal: what +crosses back is a reason from a fixed vocabulary (§6.13), never a resolved +path, a symlink target, a temporary name, an errno code, or a platform +message. #### The contextual working directory @@ -4493,6 +4512,26 @@ remove the directory through structured concurrency, and there is no `retain` prop and no retention after failure. A future execution-level inspection policy may keep scoped resources without changing this component's contract. +#### Where the directory comes from + +`` creates nothing itself. It asks the installed `API.Files` provider +(§1.2) for a temporary directory, and the provider owns creation, the canonical +path, and removal as one acquisition — so nothing can land between creating a +directory and owning its removal, and a cancellation arriving mid-acquisition +cannot leave one behind. + +A provider is allowed to have none to give. A run whose whole filesystem is a +database transaction has no temporary directories, and inventing a logical one +would put the content somewhere the run does not own — so that provider refuses +the operation outright, with the fixed diagnostic `Files provider does not +support temporary-directory`. The refusal is **fatal** on §6.13's terms: the +content does not run, nothing is rendered, and no later sibling expands. It is +never a printed error, because there is no directory for the document to work +in and carrying on would mean carrying on somewhere else. + +With no provider installed at all, `` fails the same way and for the +same reason (§6.13). + #### Why the path is canonical `` renders the directory's resolved path, not the one the host's @@ -4550,16 +4589,38 @@ something a document reports, and `ContentError` is a public type an author constructs and subclasses, so what one carries underneath is never taken as a guarantee that nothing fatal is inside it. -**A durability failure takes precedence over a documentation failure, whatever -the wrapper order.** A wrapper carries whatever failed together, in whatever -order the platform happened to collect it: an `AggregateError`'s members and an -`InvocationTeardownError`'s stage failures are both positional. Precedence is -therefore decided by kind rather than by position — the cause graph is searched -for a durability failure first, and only a graph without one reports a -documentation failure. Position-based discovery would let one ordering of the -same teardown report the document's failure instead, which `` would then -record as an ordinary `error` outcome onto a journal already known not to -describe the run. +**A Files infrastructure failure is fatal on the same terms.** A missing +document filesystem provider, an operation a provider refuses, and a provider +that broke its own contract (§6.13) are none of them things a document did or +can act on. Each is discovered through the same cycle-safe traversal, each +crosses a `ContentError` the way a durability failure does, and each is +recognized by its structural tag rather than by class — so a failure a +separately loaded copy of the runtime package constructed is found on the same +terms as one this copy did. Durability failures keep their own recognition +unchanged; only Files failures are recognized structurally, because only that +boundary is crossed by a second loaded copy. + +Recognition is strict, because a recognized failure travels onward as the exact +object that was thrown. It must carry the fixed diagnostic for its kind, frozen +data with no extra fields, and no cause. A failure that carries the right tag +and anything else — a raw platform message, an errno beneath it — is not +preserved: it is replaced by a fresh invariant carrying none of it. + +**Precedence is decided by kind rather than by position.** A wrapper carries +whatever failed together, in whatever order the platform happened to collect +it: an `AggregateError`'s members and an `InvocationTeardownError`'s stage +failures are both positional. The cause graph is therefore searched for a +durability failure first, then for a Files infrastructure failure, and only a +graph with neither reports a documentation failure. Position-based discovery +would let one ordering of the same teardown report the document's failure +instead, which `` would then record as an ordinary `error` outcome onto a +journal already known not to describe the run. + +Whichever is selected comes back **by identity** — the object that was thrown, +not a replacement — because a fail-stop that records "the first error" has to +record the one that happened. Only an output-mode `DocumentationError` is a +decision a printing boundary may still act on; a durability failure and a Files +infrastructure failure never are. This is a limitation of the current durable model, not of the component. The `import_component` entry recording that `` resolved to core's @@ -4757,12 +4818,27 @@ reachable beneath it, carrying the same error segments the document reported — reporting the component's account costs nothing that a host inspecting the failure needs. +#### The provider boundary + +`` makes no filesystem call of its own. It calls `API.Files` (§1.2), a +contextual Api of whole semantic operations, and what "the filesystem" means +belongs to whichever provider is installed. `xmd run` installs a host provider; +a workflow run installs one whose paths name entries in a logical filesystem +the run owns. The component holds no host path, learns no resolved name, and +receives no handle or capability — the two forms it performs are `readTextFile` +and `writeTextFile`, and each is one call. + +What the component owns is **order**, and it owns it because ordering is what +containment depends on. + #### Containment Everything `` touches stays inside `Env.cwd`, checked in two stages that -answer different questions and therefore run at different times. +answer different questions and therefore run at different times. Both stages +are the provider's; the component decides when each happens. -The **lexical** stage is path arithmetic against `Env.cwd` and nothing else. +The **lexical** stage is `checkFilePath`: path arithmetic against `Env.cwd` and +nothing else. An empty path, an absolute path, and a `..` escape are all decided there, before any filesystem call — so the failure reveals nothing about what the path named. Only a complete `..` segment escapes: a name that merely begins @@ -4776,6 +4852,11 @@ For the write form this stage runs **before the children expand**. An unusable path costs nothing, and the printed error it produces is about the path rather than about whatever the children then did. +`checkFilePath` returns nothing usable — no path, no resolved name, no handle, +no capability. It answers one question, "may the children run?", and the answer +authorizes nothing else. A check that was skipped, replaced by middleware, or +answered by a different provider therefore cannot admit the write that follows. + A lexical check is not enough on its own, because a symlink inside the directory can point anywhere. The **resolving** stage takes the part of the path that already exists — the file itself when it is there, the deepest @@ -4784,24 +4865,37 @@ destination is still inside the directory is ordinary and is followed to the file it names; one that leaves is refused, before the content outside is read or changed. -For the write form this stage runs **after the children have finished**, and -immediately before the write. A child can change what a path means — replacing -a directory with a symlink out of the workspace — so a destination resolved -any earlier would not be the one the write lands on. - -Writes land through a sibling temporary file and a rename, which also closes -the one case resolution cannot: a **dangling** symlink has nothing to resolve, -and `rename` replaces the link rather than following it wherever it points. -Removal of the temporary is registered before it is written, so the write is -covered by it rather than the other way round, and removal is attempted on every -exit including cancellation. +For the write form this stage is inside `writeTextFile`, which runs **after the +children have finished**. A child can change what a path means — replacing a +directory with a symlink out of the workspace — so a destination resolved any +earlier would not be the one the write lands on. That one call **repeats +lexical admission** from the same authored path and contextual directory and +then owns every later step: resolution, target classification, parent creation, +and the commit. Nothing is handed between the two stages, which is why the +earlier check cannot be turned into authority for the later write. + +The read form is one call, `readTextFile`, and that call owns admission, +resolution, target classification, and the read together. + +A **host** provider lands writes through a sibling temporary file and a rename, +which also closes the one case resolution cannot: a dangling symlink has +nothing to resolve, and `rename` replaces the link rather than following it +wherever it points. Removal of the temporary is registered before it is +written, so the write is covered by it rather than the other way round, and +removal is attempted on every exit including cancellation. Reading a path that does not exist, or a directory, fails naming which it was. #### The commit point -The rename is the write's commit point, and the guarantees are stated around -it: +Every provider commits, but not every provider commits the same way, and the +difference is visible only when a write fails. A host provider's commit is a +rename; a transaction-bound provider's is a savepoint released into the +transaction that owns it. A successful write renders nothing under either, so +the distinction never reaches a document that succeeds. + +The rename is the **host** write's commit point, and the guarantees are stated +around it: - A failure or a cancellation **before** the rename leaves the previous target exactly as it was. Nothing has replaced it yet. @@ -4816,24 +4910,35 @@ write can be taken back. ##### What a failed write can say about the target -`rename` is an operation on the contextual Fs Api, and an `around` handler may +A failed write reports **where it stopped**, and where it stopped is what +decides what may be said about the target. The provider names the phase; the +component chooses the sentence. No other combination exists, and a provider +that reports one is a provider that broke its contract (below). + +A rename is an operation on the contextual Fs Api, and an `around` handler may do work on both sides of `next()`. So a rename that **throws** may have thrown -before the underlying rename ran, or after it succeeded, and the component -cannot tell which. The three outcomes it reports are exactly what it can -observe: +before the underlying rename ran, or after it succeeded, and no provider can +tell which. | Where the write stopped | What is reported | |---|---| +| Admission, resolution, target, or parent creation | no outcome sentence — nothing was attempted on the target | | Preparation — writing the temporary | `The previous file is unchanged.` | -| The rename threw | `Whether the replacement committed is unknown: the target holds either the complete previous content or the complete replacement, never a partial write.` | -| The rename returned | `The file was written.` | +| The commit threw | `Whether the replacement committed is unknown: the target holds either the complete previous content or the complete replacement, never a partial write.` | +| The commit returned, cleanup failed | `The file was written.` | +| A transaction rolled the change back | `The Workspace change was rolled back.` | + +The unknown row is the honest answer rather than a missing one: atomicity still +holds, so it is one of two whole files, but which one is not knowable from +there. Reporting that the previous file survived would be a guess, and wrong in +exactly the case where a handler failed after committing. -Only the first and third are conclusions. The middle one is the honest answer: -atomicity still holds, so it is one of two whole files, but which one is not -knowable from here. Reporting that the previous file survived would be a guess, -and wrong in exactly the case where a handler failed after committing. +The last row belongs to a transaction-bound provider and cannot appear from a +host one; equally, no host rename or temporary-leftover wording appears from a +transaction-bound one. A rolled-back change is a conclusion — the write did not +happen, and nothing is left over. -A failed cleanup is orthogonal and composes with any of the three, appending: +A failed cleanup is orthogonal to all of them and appends: ```text A temporary file beside it may remain. @@ -4849,17 +4954,88 @@ refusal exists to prevent. A platform error carries the path it failed on — `ENOTDIR: not a directory, stat '/private/var/…'` — so forwarding one would leak exactly what the rest of -this withholds. Every filesystem call is wrapped, and nothing from the error it -caught is reproduced: the errno code **selects** a phrase from a fixed -allowlist, and an unrecognized code selects `the filesystem operation failed`. -The code itself is never emitted. It is supplied by whatever implements the Fs -Api, so it can hold a path, a newline, or a comment terminator as easily as -`ENOENT` can. +this withholds. Nothing from one crosses the provider boundary. What a provider +returns is a **reason** drawn from a fixed vocabulary, and the reason *selects* +a phrase; an unrecognized reason, and a condition the provider could not +classify, both select `the filesystem operation failed`. + +The reasons are: + +```text +empty-path, absolute-path, lexical-escape, resolved-escape, missing, +directory, special-file, not-directory, permission-denied, read-only, +too-many-symlinks, path-too-long, no-space, quota-exhausted, cross-device, +busy, too-many-open-files, directory-not-empty, invalid-pattern, +operation-failed +``` + +A failure carries that reason and the phase it came from, as a plain frozen +object under a stable tag, and it is **parsed** before any field is read. Data +that does not validate is treated as absent rather than trusted: for a non-write +operation that means the generic phrase, and for a write it is a +provider-contract failure (below), because every sentence a write could print +makes a claim about whether the file was replaced. + +The outcome a provider returns is checked the same way, and the distinction +matters more than it looks. An outcome that **will not say** what it is — one +that does not report whether it succeeded, or whose success value or failure +cannot be read at all — has described nothing, and is a provider-contract +failure. An outcome that reads perfectly well but carries a failure the +vocabulary does not recognize *has* described something, just not in terms this +version knows: for a non-write operation that is the generic sentence and the +document carries on. + +One operation qualifies that. Admitting a path succeeds with no value at all, so +an outcome that carries none is its ordinary success; what is refused there is a +value that cannot be read, and one that is present but is something other than +nothing. Every other operation's success carries a value and requires it to be +readable. + +Nothing a provider returned is passed onward. What reaches the component is +rebuilt from the fields that validated — including a search's list of paths, +which is copied, so what a document binds is not something the provider can +still change. The error's class carries no authority either. A `FileAccessError` arriving -from a wrapped call is replaced like any other, because a class says nothing -about whether a message is safe to show — trusting one would let an Fs -implementation choose the text of a printed error by choosing what to throw. +from a provider call is replaced like any other, because a class says nothing +about whether a message is safe to show — trusting one would let a provider +choose the text of a printed error by choosing what to throw. Recognition is by +structural tag rather than by `instanceof` for the same reason two copies of the +runtime package can be loaded at once, and `instanceof` answers false across +them. + +#### When the provider is the problem + +An ordinary filesystem condition is something the document did and can act on. +Three things are not, and none of them becomes a printed error: + +- **No provider is installed.** Every operation fails with the fixed diagnostic + `Files provider is not installed`. The write form reaches it at + `checkFilePath`, so it lands before the children; the read form, ``, + and `` reach it at their first call. Nothing falls back to the host. +- **The provider refuses the operation.** `Files provider does not support + temporary-directory` is the one such refusal (§6.11). +- **The provider broke its contract** — stale authority, a failed rollback, a + handler that threw, or result data that does not validate. All report + `Files provider invariant failed`, and which contract broke is structural + data for a consumer deciding what to fence rather than text: the category is + never interpolated into the message. + +All three **end the execution** (§6.11's rules for a durability failure apply +here too): no printed error, no `` output, no root or child `Close`, and +no later sibling runs. A missing provider is an installation fault, and a +document that carried on after one would run every step after the file work as +though the file work had happened. + +Precedence among fatal failures is by kind rather than position: a durability +failure first, then a Files infrastructure failure, then a documentation +failure, wherever each sits in the cause graph. + +Cancellation is none of these. Halting resumes a generator rather than throwing, +so no Result is manufactured and no printed error is created. A host provider's +cleanup still runs; a cleanup that fails while cancellation is unwinding is +reported as a sanitized teardown invariant rather than turning the cancellation +into a write outcome. #### When cleanup fails @@ -4894,19 +5070,31 @@ to remove. #### Threat model -Containment is judged against the filesystem as `` observes it. That is -sound while the filesystem is stable, and every guarantee above is stated on -that basis. +Containment is the installed provider's claim, and the two providers make +different ones. -It is not a sandbox. Nothing prevents another process from replacing a -directory with a symlink between the moment a path is validated and the moment -it is used. Resolving a write's destination immediately before writing narrows -that window and closes it for the case a document controls — its own children — -but check-then-use does not become atomic by being ordered more carefully. +**`xmd run`** resolves document paths in the caller's own filesystem, and +judges containment against that filesystem as it observes it. That is sound +**while the host pathname namespace is stable**, and every guarantee above is +stated on that basis. -Containment that does not depend on observed filesystem state — directory -handles, `openat`-style resolution, or platform-enforced sandboxing — is issue -#227. +It is not a sandbox. Nothing prevents another process from replacing a +directory, symlink, junction, or reparse point between the moment a path is +observed and the moment it is used. Resolving a write's destination immediately +before writing narrows that window and closes it for the case a document +controls — its own children — but check-then-use does not become atomic by being +ordered more carefully, and no capability the shipped runtimes expose closes it +without a native dependency. + +**A workflow run** resolves document paths in a logical filesystem the run +owns. A document path never becomes a host path there, so there is no host +namespace for another process to replace: lookup, symlink resolution, and +traversal are all the provider's own, and a symlink target that looks like a +host absolute path is an ordinary logical name. + +Neither claim covers a native command a document runs. A subprocess receives +the contextual working directory and the caller's filesystem, and containing +what it then does is not this component's boundary. #### Scope @@ -4968,9 +5156,22 @@ which is not an order at all. Finding nothing is a result. An empty array succeeds and the document carries on; it is not a failure and not a printed error. +#### Who searches + +`` validates the shape of what the document wrote — a pattern that is +empty, absolute, or begins by leaving cannot match anything a search produces — +and then makes exactly one `API.Files` call (§1.2). The provider compiles the +patterns, walks the tree, and returns the deduplicated, sorted, POSIX-relative +files. No directory path, no partial listing, and no host path crosses back. + +An absolute pattern is judged by the pattern's own grammar rather than the +running platform's: patterns match POSIX-relative paths everywhere, so a +leading `/` is absolute wherever the document runs, and so is a drive-letter +prefix. Deciding it from the host would make one document mean two things. + #### The pattern dialect -Patterns are the filesystem glob library's own, and `` adds no syntax: +Patterns are the provider's dialect, and `` adds no syntax: - `*` matches within one path segment; - `**` crosses segments, and `**/` matches no directories as readily as many — @@ -5013,10 +5214,9 @@ and a link to a directory is not descended into. That last rule is what keeps a search inside `Env.cwd` without judging any destination: traversal only ever follows real directories, so it cannot leave the -working directory and cannot cycle. The filesystem library exposes symlink -following, but nothing in it confines a resolved destination to the root or -detects a traversal cycle, so following one cannot be offered safely. A later -implementation may, if the library guarantees both. +working directory and cannot cycle. Following one cannot be offered safely +without confining a resolved destination to the root and detecting a traversal +cycle, and no provider guarantees both today. A later one may. #### Failures @@ -5039,9 +5239,9 @@ files". Only a whole leading `..` segment leaves: `..notes.md` is an ordinary name, and a `..` further along — `docs/../*.md` — is a path a search never produces, so it matches nothing for the ordinary reason. -A pattern that cannot be compiled arrives as a `RegExp` error about a translated -expression the author never wrote, and which pattern it was is not recoverable -from it. The candidates are listed rather than one being named; they are the +A pattern that cannot be compiled is reported by the provider as an +`invalid-pattern` failure, and which pattern it was does not survive that +boundary. The candidates are listed rather than one being named; they are the document's own text. #### Printed errors @@ -5051,20 +5251,24 @@ traversal failure names **no path at all**: what failed is a directory under `Env.cwd` that the document never wrote, and §1.2 keeps absolute paths out of printed errors. -As in §6.13, nothing from a caught platform error is reproduced. The errno code -**selects** a phrase from the fixed allowlist the filesystem components share, -and an unrecognized code selects `the filesystem operation failed`. The code -itself is never emitted, and the error's class carries no authority either — a -`GlobError` arriving from a wrapped call is replaced like any other, because a -class says nothing about whether a message is safe to show. +As in §6.13, nothing from a caught platform error is reproduced. The provider +returns a reason from the shared vocabulary, the reason **selects** a phrase, +and an unrecognized one selects `the filesystem operation failed`. The error's +class carries no authority either — a `GlobError` arriving from a provider call +is replaced like any other, because a class says nothing about whether a +message is safe to show. + +A provider that is absent or that broke its contract is not a search failure at +all. It ends the execution on §6.13's terms, and `` binds nothing. #### Threat model As with ``, the guarantee is about traversal rather than about the -filesystem being stable. `` never follows a symlink, so nothing it reads -is chosen by one; but a directory that is real when it is read could be replaced -afterwards, and this is not a sandbox. Containment that does not depend on -observed filesystem state is issue #227. +filesystem being stable. No provider follows a symlink, so nothing a search +reads is chosen by one. Under `xmd run` a directory that is real when it is read +could still be replaced afterwards: the host claim holds while the host pathname +namespace is stable, and this is not a sandbox. A workflow run's traversal walks +logical entries that no other process can replace. #### Scope @@ -6443,6 +6647,8 @@ visible warning blocks, gather into a separate error report). | TD14 | Ordinary failures are unchanged | A failing block inside a `` still renders a printed error and the following sibling still runs | | TD15 | Cancelled acquisition | Cancelling while the directory is live, and before the acquiring task runs, both leave nothing behind | | TD16 | Replayed component import | A nested component's journaled import is the other effect a `` can consume; it fails the execution the same way | +| TD18 | Provider-backed acquisition | Creation, the canonical path, and removal are one provider acquisition; with no provider installed the component fails the execution before rendering anything | +| TD19 | A refused operation | A provider that denies `temporary-directory` fails the execution with the fixed diagnostic, renders no content, and lets no later sibling run | | TD17 | Colocated document | `xmd test packages/core/src/components/TempDir.test.md` narrates the lifetime — ordinary cwd, live directory inside, removed and restored after, a captured directory live for a sibling, and the bare form's path — with no search path and no JavaScript | ### Tier PC — `` and `` @@ -6502,6 +6708,12 @@ visible warning blocks, gather into a separate error report). | FL24 | Regular file as a path component | `parent/child.txt` with `parent` a file fails for both forms without naming the resolved path | | FL25 | The working directory itself | `.` and a path normalizing to it are contained, and fail as a directory rather than as an escape | | FL26 | Adversarial error shapes | A `code` holding an absolute path, markup and a newline, an inherited key (`toString`), a planted path in both message and code, and an externally thrown `FileAccessError` all produce the generic phrase; nothing planted reaches the document and the printed error stays one line | +| FL28 | The check authorizes nothing | A provider whose `checkFilePath` refuses expands no children and receives no second call; the sibling after the component still runs | +| FL29 | The write repeats admission | The semantic write is one call that re-admits the authored path and owns resolution, target, parents, and the commit — proven by FL18's child swapping the parent between the two | +| FL30 | Every write phase | Admission, resolution, target, parents, temporary, commit, cleanup, and a rolled-back transaction each produce their own outcome sentence, and no other combination is constructable | +| FL31 | Provider absence | With no provider installed the write fails the execution before its children, writes nothing, renders nothing, reaches no low-level `API.Fs` call, and stops the sibling after it | +| FL32 | Malformed provider data | A write failure or success whose data does not validate is a provider-contract failure that ends the execution; a malformed non-write failure is the generic printed error and the document carries on | +| FL33 | A handler that throws | An arbitrary throw becomes a fixed `protocol` invariant with no cause, message, errno text, or host value; an existing durability or Files failure beneath it is rethrown by identity instead | | FL27 | Colocated document | `xmd test packages/core/src/components/File.test.md` covers both forms, `as` capture, nested parents, replacement, exact content for both authoring shapes, a leading-dots name, and isolation between temporary directories — with no search path and no JavaScript | ### Tier FA — Fatal error discovery @@ -6524,6 +6736,65 @@ visible warning blocks, gather into a separate error report). | FA17 | No resurrection | A `DocumentationError` a component recovered from is not reported as the outward failure, while the same one reached without crossing a content failure still is | | FA18 | Precedence behind a content failure | A durability failure beneath a recovered content failure outranks a documentation failure, in either wrapper order | | FA19 | Cycles through a content failure | A self-caused content failure and one whose cause points back at the wrapper holding it both terminate, and the durability failure is still found | +| FA20 | Every Files infrastructure failure | Provider-unavailable, operation-denied, and each invariant category is discovered as fatal, bare and wrapped, and none is a durability failure | +| FA21 | Discovery through every wrapper | A Files failure is found inside a teardown aggregate, an `AggregateError`, an ordinary `cause`, and all three at once | +| FA22 | Files outranks documentation | In either aggregate order and either teardown order, for every kind | +| FA23 | Durability outranks Files | In either order, for every durability kind | +| FA24 | All three at once | Every ordering of a durability, a Files, and a documentation failure reports the durability one; a graph with the last two reports the Files one; nesting changes neither | +| FA25 | A content failure hides no Files failure | Found beneath a `ContentError` set by subclass and by assignment, and preferred over a documentation failure the boundary would otherwise stop at | +| FA26 | Cycles carrying a Files failure | A cyclic teardown graph and a self-caused content failure both terminate and still find it | +| FA27 | A separately loaded runtime copy | A failure with no shared class identity is recognized by its structural tag; an object carrying a different tag is not | +| FA28 | Decided by output | Only an output-mode `DocumentationError` is; a `throw` decision, every durability kind, and every Files kind are not | + +### Tier HF — The host Files provider + +Driven directly rather than through a document: what these assert is the +contract a component cannot see. Every row runs on all five release targets +(`filesystem-contract` in CI), because path arithmetic and `realpath` are the +platform's. + +| # | Test | Verify | +|---|------|--------| +| HF1 | The check touches nothing | Empty, absolute, and lexically escaping paths are each refused with their own reason and no filesystem call at all; an admissible one answers `Ok(undefined)` | +| HF2 | The round trip | A write commits, reports `host-committed`, reads back, and leaves no temporary beside the file | +| HF3 | The search's shape | Sorted, deduplicated, POSIX-relative regular files; a symbolic link is not a result | +| HF4 | Platform failures are Results | `realpath`, `stat`, and the read each fail with the right phase and reason, and neither the message, the code, nor the workspace path survives | +| HF5 | Every write phase | Each of the eight phases produces data whose target claim matches it, and nothing planted survives | +| HF6 | Escapes | A link out is refused at resolution for both forms, the destination is not named, and the outside file is unchanged | +| HF7 | Internal links | Followed to the file they name; the link stays a link | +| HF8 | Dangling links | Replaced rather than followed, and nothing is created where the link pointed | +| HF9 | The observer contract | The private phases are announced in order, once each, for write, read, and search | +| HF10, HF10b | Where the guarantee stops | A parent replaced synchronously between resolution and use is written through, and a target replaced between resolution and access is read through — the documented weakness, with atomicity still holding | +| HF11 | The commit is one event | A fault before and after `next()` report the same unknown outcome, and one of the two runs really did commit | +| HF12 | Cancellation | No Result is produced and no temporary is left | +| HF12b | Cleanup failing as cancellation unwinds | There is no outcome to report it beside, so a fixed teardown invariant leaves the scope instead of a manufactured Result, carrying neither the platform's error nor the generated temporary's name | +| HF13 | Temporary directories | Live and die with the acquiring scope; a halt before acquisition leaves nothing | +| HF14 | Absence | Every operation throws provider-unavailable with the fixed diagnostic and no cause, and no low-level call is made | +| HF15 | Installation | `useHostFiles()` installs beneath ordinary middleware, which can still wrap it | +| HF16 | Directory links on every platform | A junction on Windows and a directory symlink elsewhere are refused on the same terms | +| HF17 | The compiled artifact | `scripts/files-contract-probe.ts`, compiled and run on each of the five targets, asserts the contract's observable claims and prints every one it checked | + +### Tier FF — Files infrastructure failure + +| # | Test | Verify | +|---|------|--------| +| FF1 | Absence before children | A write with no provider fails the execution, expands no children, renders nothing, reaches no `API.Fs` call, and stops the following sibling | +| FF2 | Every other form | Read, ``, and `` each stop at their first provider call, and nothing after them expands | +| FF3 | A refused operation | Denial is fatal, carries its own fixed diagnostic, and renders no content | +| FF4 | The check authorizes nothing | A refused check makes exactly one provider call; the children never run and the document carries on | +| FF5 | Malformed write data | A phase and target that contradict each other, a reason outside the vocabulary, and an undescribable success are all fatal `protocol` invariants, and the category is not interpolated | +| FF6 | Malformed non-write data | Becomes the generic printed error, leaks nothing, and the document carries on | +| FF7 | An arbitrary throw | Replaced by a fixed invariant with no cause and no host value | +| FF8 | Identity is preserved | A nested durability failure and a nested Files failure are each rethrown as the same object | +| FF9 | Precedence at the wrapper | A durability failure beneath a Files invariant is the one preserved | +| FF10 | Ordinary failures | A missing file is still a printed error and the sibling still runs | +| FF11 | Hostile-shape inspection is total | A throwing `data` accessor, fields and key enumeration that refuse, an unreadable prototype, a throwing `cause`, unreadable or non-list aggregate members, and unreadable teardown causes are each declined rather than allowed to throw — none of them is a valid structural failure — and a real failure beneath one is still found | +| FF12 | An unsafe tagged candidate is replaced, not preserved | The right tag plus a raw message, a cause chain, mutable data, an extra data field, a path-bearing `name`, an extra Error-level property, an enumerable symbol payload, or hostile enumeration each fail the identity contract; the planted value survives neither stringification nor enumeration, and the real constructors stay recognizable | +| FF13 | Cancellation cleanup is discovered as fatal | A host cleanup that fails while cancellation unwinds is selected by the engine's own fatal discovery, by identity, with nothing of the platform's failure in it | +| FF14 | A hostile outcome never reaches a component | A settlement that is absent, unreadable, or not a boolean is a provider-contract failure; so is a selected failure that is absent or unreadable, and a success value that is absent or unreadable for the operations that carry one — path admission's succeeds without a value (FF14c). A readable but invalid success payload is a contract failure too, while a readable but unrecognized non-write failure takes FF15's printable path. Where the outcome is a contract failure no child expands, no later sibling runs, and nothing planted escapes | +| FF14b | Search results are recognized and copied totally | A refusing array brand, `length` or element is a fatal protocol violation; the walk never consults the iterator; and the array a document binds is its own copy | +| FF14c | A payload-free success is accepted however it is spelled | Effection's `Unit` carries no `value` member, so an absent one is the ordinary path-admission success — while a present but unreadable one, and a present one that is not `undefined`, are fatal | +| FF15 | Readable malformed failure data stays printable | A non-write failure whose data does not validate renders the generic sentence and the document carries on, with nothing the provider put there reaching it | ### Tier OM — The `output` error mode diff --git a/tsconfig.node.json b/tsconfig.node.json index ff774562..7fbaf1af 100644 --- a/tsconfig.node.json +++ b/tsconfig.node.json @@ -34,6 +34,9 @@ "@executablemd/runtime": [ "./packages/runtime/mod.ts" ], + "@executablemd/runtime/files": [ + "./packages/runtime/files.ts" + ], "@executablemd/runtime/test": [ "./packages/runtime/test/mod.ts" ],