diff --git a/SECURITY.md b/SECURITY.md index 965d9a450..ec498d781 100644 --- a/SECURITY.md +++ b/SECURITY.md @@ -544,15 +544,22 @@ interpolation). There is **no command allowlist** — the OS sandbox (`srt`, see the supervisor) is the structural boundary, and a per-session **permission mode** governs the surface (`protocol/mode.ts`): -- `accept-edits` (default): only read-only/inspection commands auto-run - (`permissions.ts` `isReadOnlyCommand`); a mutating/executing command **pauses - for a supervised Allow/Deny approval** before it runs. The gate is the AI - SDK's native `needsApproval` on the tool (`tools/run-command.ts`), wired from - the session mode at `workspace-agent-bindings.ts` (`needs_approval = -!isReadOnlyCommand` in `accept-edits`, absent in `auto`). The gate is the - tool's, NOT the backend's: by the time the command backend's `execute` runs, - the call is already cleared (auto, or user-approved), so the backend cannot - re-gate on mode without refusing an approved command. +- `accept-edits` (default): read-only/inspection commands auto-run + (`permissions.ts` `isReadOnlyCommand`). The only mutating exception is a + plain, two-path `cp` or `mv` whose existing regular-file source and absent + destination both canonicalize inside the same session scratch root; flags, + symlinks, overwrites, missing parents, or any path outside scratch fail closed + to the ordinary supervised Allow/Deny approval. This exception lets the agent + organize pre-authorized ephemeral working bytes without treating promotion + into the workspace as pre-authorized. Every other mutating/executing command + **pauses for approval** before it runs. The gate is the AI SDK's native + `needsApproval` on the tool (`tools/run-command.ts`), wired from the session + mode at `workspace-agent-bindings.ts` (`needs_approval = !isReadOnlyCommand && +!isScratchLocalCopyOrMove` in `accept-edits`, absent in `auto`). The gate is + the tool's, NOT the backend's: by the time the command backend's `execute` + runs, the call is already cleared (auto, pre-authorized scratch-local + operation, or user-approved), so the backend cannot re-gate on mode without + refusing an approved command. - `auto`: every command runs; the OS sandbox is the sole guard. The semantic safety classifier that would judge intent is **deferred** — `auto` is an opt-in, informed-consent posture. @@ -613,13 +620,36 @@ output is converted to `output-error` before queue recovery: the host cannot prove whether its side effect completed before the crash, so it must not replay the approved operation. -Three structural checks hold regardless of mode: the -cwd-must-be-inside-an-opened-workspace check, the in-process secret-arg -containment check (below), and a no-clobber protected-path guard on the -fs-edit tools (`fs/scope.ts`: `.git`, rc/env files, lockfiles, agent config). -The OS-level outer sandbox confines the _whole_ sidecar; a per-command fs/net -sub-policy that would constrain each spawned child (the kernel-level finish of -the secret-dir guard below) does not exist yet and is the deferred hardening. +Three structural checks hold regardless of mode: cwd must be inside the exact +current workspace or the current session's scratch (never another registered +workspace or session), command args receive a defense-in-depth secret-root +check, and fs-edit tools retain their no-clobber protected-path guard +(`fs/scope.ts`: `.git`, rc/env files, lockfiles, agent config). + +The long-lived sidecar's outer sandbox is only a coarse backstop. It does not +raw-spawn model commands: `@grida/agent` calls a host-injected `ShellExecutor`, +the sidecar sends one bounded `command.request` over its inherited private +channel, and Electron main independently canonicalizes the exact workspace, +session scratch, cwd, and host-owned scratch base. Main then asks `srt` for a +fresh kernel profile for that finite command. The profile denies the entire +shared scratch base and the daemon's secret `userData`, re-allows only this +session's scratch, and grants writes only to the exact workspace, exact +scratch, and a private per-command temp directory. It also denies SRT's shared +compatibility temp/log write defaults and gives the command no direct network +destination or local-bind authority. An interpreter that computes a sibling +scratch or `userData` path at runtime is therefore denied by the kernel, not +merely by argv inspection. + +- **Abort is a cleanup barrier, not just a UI edge.** A turn abort propagates + through the AI SDK tool signal as `command.abort`. Main keeps that command + active until the executor has terminated the ordinary process group and + `AgentCommandHost` has removed its private temp and released SRT bookkeeping; + only then does it return the terminal `command.aborted` acknowledgement. + The agent runtime tracks both that command promise and the model-stream pump + in the turn's settlement barrier, so replacement admission and session + deletion remain HTTP 409 until the acknowledgement has been consumed and + the aborted pump has settled. This orders cleanup for the authority the host + actually owns; it does not strengthen the macOS `setsid(2)` limitation below. - **Network (allow-only, enumerated).** `srt` denies all outbound except a host-set domain allowlist and **forbids `*` / broad patterns by design** — @@ -636,48 +666,35 @@ the secret-dir guard below) does not exist yet and is the deferred hardening. the package's allowlisted defaults unless they explicitly choose the same strict construction mode. -- **Fail-closed exposure (no sandbox ⇒ no shell).** The shell tool is not - registered at all unless the host affirms containment. The decision is - computed once at the tenant boundary (`createAgentTenant`, - `packages/grida-ai-agent/src/server.ts`) as - `sandbox_enforced || allow_unsandboxed_shell` and threaded to the tool - registry; the default is off. The desktop supervisor sets `sandbox_enforced` - true only when it actually wrapped the sidecar spawn with `srt`, so on - platforms where Desktop does not enable the wrapper (Windows today) the agent - gets fs/todos/skills but **no** `run_command` and no external ACP agent. The - `grida-agent` CLI — a local, user-invoked tool - with no OS sandbox — sets the explicit `allow_unsandboxed_shell` opt-in - instead, which logs a warning. New privileged tools added later inherit the - same gate: a capability that needs containment is born behind this switch, - so the system's default posture is "no containment, no capability." - -- **Secret-dir containment (in-process).** The daemon's own secret dir — +- **Fail-closed exposure (no executor ⇒ no shell).** `sandbox_enforced` is an + attestation about the coarse sidecar boundary, not command authority. The + tenant registers `run_command` only when its host injects a `ShellExecutor`; + omission is the default. Desktop injects the private main-owned executor only + on platforms where SRT is enabled, so Windows gets fs/todos/skills but **no** + `run_command` and no external ACP agent. The `grida-agent` CLI — a local, + user-invoked tool with no OS sandbox — uses the separately named + `allow_unsandboxed_shell` opt-in, which explicitly injects the raw runner and + logs a warning. A boolean claim alone can never cause raw execution. + +- **Secret-dir containment (per command).** The daemon's own secret dir — its `userData`, where BYOK `auth.json`, `workspaces.json`, `recent.json`, and the sessions db live — is deliberately **not** in the `srt` - `deny_read` policy, because the host process itself must read `auth.json` - for provider calls. Denying it at the kernel level would break host auth. - Instead the shell _child_ is kept out of it in-process: `validateShellRequest` - rejects any command arg that resolves (after realpath of the nearest - existing ancestor, mirroring the cwd discipline so a symlink can't bypass it) - inside that protected root. HOME secrets (`~/.ssh`, `~/.aws`, shell rc files) - remain denied for the entire tree by the `srt` policy, where the host has no - legitimate read. This ownership split is the responsibility-and-reconciliation - rule: `srt` owns HOME secrets, the in-process runner owns the host's own - `userData`. **Caveat (`auto`):** the in-process arg check only inspects - top-level argv, so an interpreter or shell (`bash -c`, `python3 -c`) reachable - in `auto` can read `userData` by a computed path. Closing that for the shell - _child_ needs the kernel-level per-call `deny_read` (the deferred per-command - sub-policy). Desktop's empty direct external allowlist blocks network - exfiltration from that child, but does not make the in-process read itself - acceptable. The fs-edit tools (`read_file`) remain workspace-scoped and never - serve `userData`. + **outer** policy, because the sidecar itself must read `auth.json` for + provider calls. Electron main does not need that authority to execute a + command, so every finite-command profile adds a kernel `deny_read` and + `deny_write` for `userData`. `validateShellRequest` still rejects an explicit + arg resolving there as defense in depth, but computed interpreter paths are + covered as well. HOME secrets (`~/.ssh`, `~/.aws`, shell rc files) remain + denied in both outer and command profiles. The fs-edit tools (`read_file`) + remain workspace/scratch-scoped and never serve `userData`. - **`auto` is informed-consent.** `auto` removes command-identity gating; the sandbox still bounds the blast radius (writes confined to writable roots, direct external network denied), but it does not judge _intent_ — an injected or confused agent can read broadly and run anything within those bounds. Restoring intent judgment is the classifier/watchdog layer, named and deferred. `auto` is - opt-in; the default `accept-edits` keeps a read-only-only shell. + opt-in; the default `accept-edits` requires approval for mutations except + the narrow scratch-local copy/move operation described above. **Human terminal (deliberate contrast to the agent shell).** The workbench's Terminal pane (`bridge.terminal.*`) is a real, **unsandboxed** @@ -778,10 +795,11 @@ registry writes nothing but the directory, so the earlier manifest-injection surface is removed outright rather than field-constrained — whatever document the workspace eventually holds is created by the AGENT through its own already-bound (and separately-gated) fs write capability, not by this route. -The sidecar's own `fs` writes and every child process are inside the same coarse -whole-sidecar `srt` profile; no narrower per-command filesystem profile exists -yet. A created project becomes an in-process workspace root for structured -tools, while shell cwd authorization is checked separately by the runner. +The sidecar's structured `fs` writes remain inside its coarse outer `srt` +profile. A created project becomes an in-process workspace root for structured +tools; a shell command receives that exact canonical root again through the +main-owned per-command SRT profile rather than inheriting the union of every +opened workspace. `workspaces.create.test.ts` pins traversal-name containment, that the created project is empty (an unexpected `seed` body is inert), and the no-managed-root refusal. @@ -821,10 +839,10 @@ Today: - [packages/grida-ai-agent/src/runtime/index.ts](packages/grida-ai-agent/src/runtime/index.ts) — agent run orchestration; owns run / stream / abort behavior and binds a consumed human-input result to the exact resumed run through terminal recorder settlement. - [packages/grida-ai-agent/src/runtime/stream-registry.ts](packages/grida-ai-agent/src/runtime/stream-registry.ts) — in-flight run replay/abort registry; async model producers append and finish only through their exact `StreamEntry` generation, so a late response or error from aborted turn A cannot mutate queued replacement B under the same session id. Explicit human abort remains session-keyed so it targets whichever turn is current. - [packages/grida-ai-agent/src/runtime/session-scheduler.ts](packages/grida-ai-agent/src/runtime/session-scheduler.ts), [status-sse.ts](packages/grida-ai-agent/src/runtime/status-sse.ts), and [the scheduler contract tests](packages/grida-ai-agent/src/runtime/session-scheduler.test.ts) — authoritative per-session run-state machine and its observation channel; classifies persisted approvals/questions, projects explicit waiting states after restart, and pauses/rechecks queue drain so an ordinary queued turn cannot run ahead of unresolved human input. Status-SSE hydration is deliberately read-only: only trusted lifecycle/mutation edges, host-start recovery, and provider-ready retries can schedule a queued turn. -- [packages/grida-ai-agent/src/runtime/command-backend.ts](packages/grida-ai-agent/src/runtime/command-backend.ts) — agent `run_command` adapter through shell policy (structural gates only; the supervised mode gate is the tool's `needsApproval`). +- [packages/grida-ai-agent/src/runtime/command-backend.ts](packages/grida-ai-agent/src/runtime/command-backend.ts) — agent `run_command` adapter: validates cwd against only the current canonical workspace/own scratch, flushes structured writes, and delegates the exact immutable scope to a host-injected executor. It never raw-spawns. - [packages/grida-ai-agent/src/tools/run-command.ts](packages/grida-ai-agent/src/tools/run-command.ts) — the supervised-approval gate itself: the AI SDK `needsApproval` predicate that pauses a mutating command before `execute` in `accept-edits` (absent in `auto`). The decision lives on the tool, not the backend. -- [packages/grida-ai-agent/src/runtime/workspace-agent-bindings.ts](packages/grida-ai-agent/src/runtime/workspace-agent-bindings.ts) — opened workspace to agent fs/todos/command bindings; wires the `accept-edits` supervised-approval predicate. The session scratch dir is wired as an additional sanctioned root for BOTH surfaces from one source (`deps.scratch_dir`): the shell's allowed cwd roots AND the fs backend's reachable roots (so `view_image`/`read_file`/`write_file` reach scratch, not just the shell). Containment is preserved per root — a path under no reachable root falls back contained to the workspace, and the secrets root is never a reachable root. Also builds the `generate_image` binding: it reads BYOK keys via `SecretsStore` to call the image provider in-process and returns the saved scratch path + metadata + base64 `data` (the bytes are for the CLIENT to render; `AgentGen.toModelOutput` is text-only, so they are NEVER lowered to the model — no context bloat, no perception claim). The complementary `view_image` perception path DOES deliver bytes to the model, but only ones already read under the agent's existing fs read capability: `agent/hoist-tool-result-images.ts` (wired at `agent/index.ts` `prepareStep`, #923) relocates an image tool-result into a synthetic user-message image part so the model can actually see it on the openai-compatible wire — a model-view lowering that moves bytes already inside the prompt, never persisted, with no new read, no new egress, and no boundary change. The key never leaves the host, and the call omits `providerOptions.grida` so it is BYOK-paid, never Grida-billed (mirrors the `/images/generate` route). -- [packages/grida-ai-agent/src/session/scratch.ts](packages/grida-ai-agent/src/session/scratch.ts) — per-session ephemeral scratch dir (WG `scratch.md`): asserts the shell-writable scratch tree sits OUTSIDE `userData` (the secret root), creates it owner-only (`0700`), and reclaims it (per-session delete + synchronous host-start sweep). `writeScratchFile` lands produced bytes (e.g. `generate_image`) owner-only (`0600`) within the session tree, rejecting any filename that is not a single safe path segment AND opening `O_NOFOLLOW` so a symlink planted at the basename (e.g. by an auto-approved scratch-cwd `run_command`) fails the write instead of redirecting it outside the tree — closing the lexical-check TOCTOU. +- [packages/grida-ai-agent/src/runtime/workspace-agent-bindings.ts](packages/grida-ai-agent/src/runtime/workspace-agent-bindings.ts) — opened workspace to agent fs/todos/command bindings; wires the `accept-edits` supervised-approval predicate. The session scratch dir is wired into BOTH surfaces from one source (`deps.scratch_dir`): the shell executor's exact scope and the fs backend's reachable roots (so `view_image`/`read_file`/`write_file` reach scratch, not just the shell). Containment is preserved per root — a path under no reachable root falls back contained to the workspace, and the secrets root is never a reachable root. Also builds the `generate_image` binding: it reads BYOK keys via `SecretsStore` to call the image provider in-process and returns the saved scratch path + metadata + base64 `data` (the bytes are for the CLIENT to render; `AgentGen.toModelOutput` is text-only, so they are NEVER lowered to the model — no context bloat, no perception claim). The complementary `view_image` perception path DOES deliver bytes to the model, but only ones already read under the agent's existing fs read capability: `agent/hoist-tool-result-images.ts` (wired at `agent/index.ts` `prepareStep`, #923) relocates an image tool-result into a synthetic user-message image part so the model can actually see it on the openai-compatible wire — a model-view lowering that moves bytes already inside the prompt, never persisted, with no new read, no new egress, and no boundary change. The key never leaves the host, and the call omits `providerOptions.grida` so it is BYOK-paid, never Grida-billed (mirrors the `/images/generate` route). +- [packages/grida-ai-agent/src/session/scratch.ts](packages/grida-ai-agent/src/session/scratch.ts) — per-session ephemeral scratch dir (WG `scratch.md`): derives a host-namespaced base under a host-injected temp root and rejects lexical or physical overlap with `userData` in either direction before mutation. Authority creation is non-recursive; every predictable base/session level must be a non-symlink current-uid-owned directory and is tightened/verified to `0700` on POSIX, while an unsafe parent fails closed. Reclamation holds session admission, revalidates the authority before listing, and unlinks child symlinks rather than following them (per-session delete + synchronous host-start sweep). Desktop resolves that temp root in main before SRT can replace the sidecar's `TMPDIR` with a shared compatibility directory. `writeScratchFile` lands produced bytes (e.g. `generate_image`) owner-only (`0600`) within the session tree, rejecting any filename that is not a single safe path segment AND opening `O_NOFOLLOW` so a symlink planted at the basename (e.g. by an auto-approved scratch-cwd `run_command`) fails the write instead of redirecting it outside the tree — closing the lexical-check TOCTOU. - [packages/grida-daemon/src/path-contains.ts](packages/grida-daemon/src/path-contains.ts) — shared `path.sep`-prefix containment used by the shell runner's workspace/secret-root gates, the scratch containment assert, and `createProject`'s managed-root assert (one source so the discipline can't drift). - [packages/grida-ai-agent/src/runtime/run-input.ts](packages/grida-ai-agent/src/runtime/run-input.ts) and [tools/human-input-result.ts](packages/grida-ai-agent/src/tools/human-input-result.ts) — wire-message normalization + `coerceApprovalAnswer`/`applyApprovalAnswer` (shape-gates the explicit `approval_answer` body field and atomically binds a valid answer to its exact consuming run), plus exact correlation and canonical output-schema validation before a renderer-authored question/design-search result can consume a pending interaction. - [packages/grida-ai-agent/src/protocol/context.ts](packages/grida-ai-agent/src/protocol/context.ts) — renderer-safe, persistable directory-reference descriptor vocabulary; the virtual path and read-only access are fixed by the host contract, while the descriptor itself carries no authority. @@ -841,9 +859,10 @@ Today: - `desktop/src/window.ts` — blocks exposed desktop windows from navigating outside `/desktop/*`; injects non-secret preload arguments. - `desktop/src/agent-sidecar.ts` — sidecar entrypoint; constructs the composed agent daemon (`createAgentDaemon`) in socketless mode and accepts only main-transferred daemon sockets. - `desktop/src/agent-sidecar-daemon-sockets.ts` — injects only validated, already-connected socket capabilities into the unbound HTTP server; it exposes no listen, bind, connect, or target-selection operation. -- `desktop/src/agent-sidecar-channel.ts`, `agent-network-policy.ts`, and `agent-sidecar-network.ts` — strict private stdio framing, destination/header policy, and the sidecar's explicit provider/provider-asset transport client. +- `desktop/src/agent-sidecar-channel.ts`, `agent-network-policy.ts`, and `agent-sidecar-network.ts` — strict private stdio framing, destination/header policy, the sidecar's explicit provider/provider-asset transport client, and the bounded finite-command request/result client. Command output is sequenced and capped; abort remains pending until main returns `command.aborted` after worker cleanup; malformed or unknown frames fail the channel. - `desktop/src/main/agent-daemon-socket-host.ts` — owns the exact loopback listener, pauses accepted sockets, rejects non-loopback peers, and transfers the bounded connected capability over per-spawn Node IPC. -- `desktop/src/main/agent-network-host.ts` and `agent-network-authority.ts` — main-owned Chromium network execution, bounded response streaming, redirect/route reauthorization, and per-spawn built-in/custom grant state. +- `desktop/src/main/agent-network-host.ts` and `agent-network-authority.ts` — main-owned Chromium network execution, bounded response streaming, redirect/route reauthorization, per-spawn built-in/custom grant state, and the capped private dispatch seam into the command host. +- `desktop/src/main/agent-command-host.ts` and `main/sandbox/manager.ts` — main-owned finite-command execution: canonical exact-root validation, shared-scratch/secret denies, own-scratch/workspace/private-temp grants, a fixed no-network policy, fresh per-command SRT wrapping, raw spawn only after the wrapper succeeds, and terminal cleanup before abort acknowledgement. - `desktop/src/main/agent-sandbox-policy.ts` — binds Desktop's strict sandbox posture: empty direct external egress, host-routed provider HTTP, and no generic local bind/connect authority. - `desktop/src/main/agent-sidecar-supervisor.ts` — generates the per-spawn password; spawns/supervises the daemon sidecar; initializes the OS sandbox wrapper when supported; owns both private channels and removes direct provider hosts from the sidecar policy (Desktop deliberately withholds srt's alpha Windows backend pending a supported lifecycle). - `desktop/src/main/desktop-entry-window.ts` — owns the exact bridge-attached entry window and admits auxiliary native windows only while the authenticated main role is active; the Grida-account transition is additionally bound by GRIDA-SEC-005. diff --git a/desktop/README.md b/desktop/README.md index 6de3fc210..ee26bcc5a 100644 --- a/desktop/README.md +++ b/desktop/README.md @@ -56,10 +56,20 @@ flag is never consulted again. On macOS and Linux, the sidecar runs under `srt` with no direct external destinations and `allow_local_binding: false`; Electron main supplies the two -explicit capabilities above. Windows currently runs the sidecar without that -outer wrapper: shell and external ACP are withheld, while structured local file -tools remain available and no kernel egress fence exists. That is a documented -nonconformance rather than a sandbox claim. +explicit capabilities above. A model-selected command is a third private +sidecar-to-main capability: main gives each finite worker a fresh SRT profile +containing only its exact workspace, own session scratch, and private command +temp write roots. The coarse sidecar profile is not used as proof of +cross-session shell isolation. Windows currently runs the sidecar without the +outer wrapper: shell and external ACP are withheld, while structured local +file tools remain available and no kernel egress fence exists. That is a +documented nonconformance rather than a sandbox claim. + +Windows still accepts per-session scratch staging. Raster inputs remain +operable through provider perception/`view_image`, and structured text +(including SVG) remains operable through filesystem tools. Scratch-only binary +inputs such as PDF/archives are withheld because the confined binary command +tool is unavailable; the renderer does not create an inert attachment path. If a bug reproduces in files, workspaces, BYOK providers, sessions, or agent execution, add the first test in diff --git a/desktop/docs/agent-authority.md b/desktop/docs/agent-authority.md index 4dca70252..7c04447c6 100644 --- a/desktop/docs/agent-authority.md +++ b/desktop/docs/agent-authority.md @@ -4,8 +4,9 @@ This document binds the language-agnostic [execution-authority model](https://github.com/gridaco/grida/blob/main/docs/wg/ai/agent/execution-authority.md) to Grida Desktop. Issue [#974](https://github.com/gridaco/grida/issues/974) lands the native provider -transport slice described below; supervisor-owned raw execution workers remain -the target architecture, not a claim about the current sidecar. +transport slice described below; issue +[#916](https://github.com/gridaco/grida/issues/916) binds finite shell workers +to supervisor-owned per-command confinement. On macOS and Linux, the implementation wraps the entire AgentSidecar with `@anthropic-ai/sandbox-runtime` (`srt`) 0.0.65 and one global destination @@ -14,9 +15,12 @@ policy. Its direct external destination set is empty and transfers only already-accepted connected sockets to the socketless sidecar, and routes trusted provider HTTP through a separate private stdio channel. BYOK/GG hosts are absent from the outer policy. The wrap remains the shipping -filesystem/process boundary until every raw-worker replacement gate in this -document exists. Removing it first would expose raw shell without an equivalent -kernel boundary; Desktop already withholds the external ACP agent. +backstop for the sidecar's structured capabilities and in-process state. +Model-selected finite commands no longer spawn inside that coarse authority: +the sidecar requests them over the same inherited capability channel and main +creates a fresh SRT filesystem profile for each command. Removing the outer +wrap would still expose the sidecar itself and is not implied by this narrower +worker boundary; Desktop continues to withhold the external ACP agent. Windows currently starts AgentSidecar unwrapped. Shell and external ACP are withheld, but @@ -24,6 +28,10 @@ structured local filesystem capabilities remain available; that is a known nonconformance with the target fail-closed posture below, not a sandbox claim. The package-level empty destination intent is not a Windows kernel egress fence; an AgentSidecar compromise there retains ambient process networking. +Base64 scratch staging remains available there for raster perception and +structured text/SVG filesystem access. The renderer refuses a scratch-only +arbitrary binary file when no provider route exists, because Windows withholds +the confined binary command capability and a bare path would be inoperable. ## Decision @@ -212,9 +220,12 @@ updates carry monotonically increasing revisions and main does not report them published until the sidecar acknowledges application; the host re-authorizes a completed upload against its current grant snapshot immediately before I/O. Unknown, stale, out-of-order, oversized, or grant-mismatched messages terminate -the channel. A bounded cancellation tombstone accepts only the late response -frames that an in-flight abort can legitimately race. Channel failure exits the -sidecar so supervision restarts a fresh pair. The renderer can invoke the +the channel. A bounded network-cancellation tombstone accepts only the late +response frames that an in-flight abort can legitimately race. A finite command +instead has a terminal handshake: `command.abort` keeps its tool promise pending +until main has joined the worker and completed per-command cleanup, then +`command.aborted` releases the turn's settlement barrier. Channel failure exits +the sidecar so supervision restarts a fresh pair. The renderer can invoke the existing typed product bridge, but never receives the channel or a service credential. @@ -249,32 +260,67 @@ capabilities: - generic local bind/connect authority is denied; daemon access arrives only as a main-accepted connected-socket capability; - model-selected commands reach process creation only through the structured - `run_command` and approval path; external ACP is absent; and + `run_command` and approval path; the sidecar delegates each finite command + back to Electron main for a fresh exact-root SRT profile; external ACP is + absent; and - fixed helpers, if any, have host-fixed executable and argument shapes. -This boundary cannot hide sidecar-owned BYOK/session data from a compromise in -the same process, and it cannot distinguish two workspace roots held by one -sidecar. A capability that must resist that compromise belongs in its own -worker or must receive capability-safe handles rather than ambient paths. +This outer boundary cannot hide sidecar-owned BYOK/session data from a +compromise in the same process, and by itself cannot distinguish roots held by +one sidecar. The per-command host closes the model-selected ambient-path gap +for scratch and secrets: main denies the shared scratch parent and `userData`, +then re-allows only the request's own scratch and private temp. Workspace reads +retain SRT's default broad-read posture; the exact workspace is the command's +cwd/write grant, not its exclusive readable tree. + +The sidecar runtime—not the model—supplies those roots, but the current private +frame still names them as paths. Main canonicalizes their shape and overlap; it +does not authenticate them against a fully compromised sidecar. A capability +that must resist that threat belongs in its own worker or must receive a +main-issued opaque workspace/session grant rather than caller-named paths. ## Raw execution and extensions -The runtime never supplies `sandbox_enforced: true` as evidence of authority. -The capability is the supervisor binding itself: a workload request plus an -opaque, host-issued grant. The supervisor independently canonicalizes the -executable, arguments, environment, working directory, roots, and requested -network subset before launch. +The runtime never treats `sandbox_enforced: true` as command authority. The +capability is the injected `ShellExecutor`: the sidecar emits a bounded private +request, and main independently canonicalizes the working directory, named +workspace, session-shaped scratch, shared scratch authority root, host secret +root, and command temp before launch. The command and arguments remain +structured until main shell-quotes the complete argv solely for SRT's wrapper; +the resulting wrapper is spawned with `shell: false`. One-shot shell, long-lived ACP stdio, and managed MCP/extension lifecycle are separate behavioral contracts. A private launcher may serve all three, but no public shared API is promoted until these consumers have proved the common shape. -The installed srt `0.0.65` manager has process-global proxy and network-policy -state. Per-command custom configuration does not provide per-command network -identity. Desktop must therefore use one manager-owning worker per concurrent -authority domain, add an attributable mediator, or replace the enforcer. It -must not mutate one global allowlist as grants come and go. +`runShell` owns a fresh POSIX process group and terminates that group before +main removes command temp or releases SRT's per-command bookkeeping. This +reclaims ordinary background descendants. It is not a macOS kernel job object: +model-selected code that deliberately creates a new session with `setsid(2)` +can leave the original group while retaining its inherited Seatbelt profile. +The current command surface therefore does not claim hard process revocation +against deliberate daemonization on macOS. Any workload requiring that +guarantee needs a dedicated worker/job mechanism; Linux Bubblewrap's PID +namespace is stronger, but its packaged end-to-end evidence remains pending. + +Turn cancellation observes the same ordering. The sidecar forwards the AI SDK +tool abort as `command.abort`; main does not acknowledge it until the executor +returns and the cleanup above has finished. The agent runtime keeps the session +occupied through both that acknowledgement and the model pump consuming the +aborted tool result. A replacement run or session deletion therefore remains +busy rather than overlapping a command whose authority is still being +reclaimed. This does not make a deliberately detached macOS process revocable; +it only closes the lifecycle gap for the process group and SRT worker the host +still owns. + +The installed srt `0.0.65` manager has process-global proxy state. Desktop +therefore gives every finite command the same empty external-domain set and +never mutates a global command allowlist as requests come and go. Per-command +custom filesystem configuration supplies exact scratch/workspace grants. A +future feature that needs distinct command network identities must use one +manager-owning worker per concurrent authority domain, add an attributable +mediator, or replace the enforcer. Raw networking follows these postures: @@ -377,7 +423,8 @@ The implementation must prove at least: - a structured file-tool validation defect still hits the outer filesystem boundary; - shell and ACP workers cannot read sidecar state or write outside their grant; -- two simultaneous workers cannot use each other's roots or destinations; +- two simultaneous workers cannot write through each other's grants or use + each other's scratch/network destinations; - unsupported route/trust clients fail with a specific diagnostic; - Windows exposes no falsely labeled sandboxed capability; and - GG mint preserves the renderer-transient scoped-token handoff without diff --git a/desktop/package.json b/desktop/package.json index 83341d350..cdd7d4bc2 100644 --- a/desktop/package.json +++ b/desktop/package.json @@ -1,6 +1,6 @@ { "name": "desktop", - "version": "0.0.16", + "version": "0.0.17", "private": true, "description": "Grida Desktop App", "keywords": [], diff --git a/desktop/src/agent-sidecar-channel.test.ts b/desktop/src/agent-sidecar-channel.test.ts index 12164a246..98a1f18c7 100644 --- a/desktop/src/agent-sidecar-channel.test.ts +++ b/desktop/src/agent-sidecar-channel.test.ts @@ -88,6 +88,53 @@ const frames: readonly AgentSidecarChannel.Frame[] = [ message: "request failed", }, { v: 1, type: "response.credit", requestId: "req_1", bytes: 65_536 }, + { + v: 1, + type: "command.request", + requestId: "cmd_1", + command: "node", + args: ["script.js", "--name=Grida"], + workdir: "/workspace", + timeoutMs: 30_000, + workspaceRoot: "/workspace", + scratchDir: "/scratch/session-1", + }, + { + v: 1, + type: "command.abort", + requestId: "cmd_cancelled", + reason: "caller aborted", + }, + { + v: 1, + type: "command.aborted", + requestId: "cmd_cancelled", + }, + { + v: 1, + type: "command.output", + requestId: "cmd_1", + stream: "stdout", + sequence: 0, + data: "hello, 세계\n", + }, + { + v: 1, + type: "command.end", + requestId: "cmd_1", + sequence: 1, + exitCode: 0, + signal: null, + timedOut: false, + truncated: false, + durationMs: 12, + }, + { + v: 1, + type: "command.error", + requestId: "cmd_2", + message: "command scope denied", + }, { v: 1, type: "shutdown" }, ]; @@ -221,6 +268,118 @@ describe("AgentSidecarChannel.parse", () => { }) ).toThrow(/bounded base64 chunk/); }); + + it("keeps command requests and UTF-8 output strictly bounded", () => { + const request = frames.find( + (frame) => frame.type === "command.request" + ) as AgentSidecarChannel.CommandRequestFrame; + + expect(() => + AgentSidecarChannel.parse({ + ...request, + args: Array.from( + { length: AgentSidecarChannel.MAX_COMMAND_ARGS + 1 }, + () => "x" + ), + }) + ).toThrow(/args must be an array/); + expect(() => + AgentSidecarChannel.parse({ + ...request, + command: "node\0--inspect", + }) + ).toThrow(/null byte/); + expect( + AgentSidecarChannel.parse({ + ...request, + timeoutMs: 60_001, + }) + ).toMatchObject({ timeoutMs: 60_001 }); + expect(() => + AgentSidecarChannel.parse({ + ...request, + timeoutMs: 0, + }) + ).toThrow(/timeoutMs must be an integer/); + expect(() => + AgentSidecarChannel.parse({ + ...request, + scratchDir: "", + }) + ).toThrow(/scratchDir must be a string/); + expect(() => + AgentSidecarChannel.parse({ + v: 1, + type: "command.output", + requestId: "cmd", + stream: "stdout", + sequence: 0, + data: "🌍".repeat( + AgentSidecarChannel.MAX_COMMAND_OUTPUT_CHUNK_BYTES / 4 + 1 + ), + }) + ).toThrow(/bounded UTF-8 chunk/); + }); + + it("rejects malformed command completion frames", () => { + expect(() => + AgentSidecarChannel.parse({ + v: 1, + type: "command.output", + requestId: "cmd", + stream: "stdin", + sequence: 0, + data: "nope", + }) + ).toThrow(/output stream/); + expect( + AgentSidecarChannel.parse({ + v: 1, + type: "command.end", + requestId: "cmd", + sequence: 0, + exitCode: 256, + signal: null, + timedOut: false, + truncated: false, + durationMs: 1, + }) + ).toMatchObject({ exitCode: 256 }); + expect( + AgentSidecarChannel.parse({ + v: 1, + type: "command.end", + requestId: "cmd", + sequence: 0, + exitCode: -1, + signal: null, + timedOut: false, + truncated: false, + durationMs: 1, + }) + ).toMatchObject({ exitCode: -1 }); + expect(() => + AgentSidecarChannel.parse({ + v: 1, + type: "command.end", + requestId: "cmd", + sequence: 0, + exitCode: Number.MAX_SAFE_INTEGER + 1, + signal: null, + timedOut: false, + truncated: false, + durationMs: 1, + }) + ).toThrow(/exitCode must be an integer/); + expect(() => + AgentSidecarChannel.parse({ + v: 1, + type: "command.error", + requestId: "cmd", + message: "", + }) + ).toThrow(/message must be a string/); + }); }); describe("AgentSidecarChannel.Decoder", () => { diff --git a/desktop/src/agent-sidecar-channel.ts b/desktop/src/agent-sidecar-channel.ts index 8d353bfad..f749237f3 100644 --- a/desktop/src/agent-sidecar-channel.ts +++ b/desktop/src/agent-sidecar-channel.ts @@ -1,4 +1,4 @@ -/** GRIDA-SEC-004 — private host/sidecar provider transport framing. */ +/** GRIDA-SEC-004 — private host/sidecar capability transport framing. */ import type { Writable } from "node:stream"; import { TextDecoder } from "node:util"; @@ -8,15 +8,19 @@ import { TextDecoder } from "node:util"; * frames, and stderr remains the sidecar's human-readable log stream. * * Framing is deliberately independent of Node IPC because the channels carry - * different authority. This stdio protocol carries provider/control frames; - * the per-spawn Node IPC descriptor carries only already-connected daemon - * sockets. Neither channel is a fallback for the other. + * different authority. This stdio protocol carries provider, finite-command, + * and control frames; the per-spawn Node IPC descriptor carries only + * already-connected daemon sockets. Neither channel is a fallback for the + * other. */ export namespace AgentSidecarChannel { export const VERSION = 1 as const; export const MAX_FRAME_BYTES = 256 * 1024; export const MAX_BINARY_CHUNK_BYTES = 64 * 1024; export const MAX_RESPONSE_CREDIT_BYTES = 16 * 1024 * 1024; + export const MAX_COMMAND_ARGS = 256; + export const MAX_COMMAND_OUTPUT_CHUNK_BYTES = 64 * 1024; + export const MAX_COMMAND_OUTPUT_BYTES = 1024 * 1024; export type Header = readonly [name: string, value: string]; @@ -143,6 +147,74 @@ export namespace AgentSidecarChannel { bytes: number; }>; + /** + * Ask Electron main to execute one command inside a tenant-derived + * filesystem scope. The model supplies only command/args/workdir; the + * sidecar runtime supplies the workspace and scratch roots. Main treats the + * path bytes as untrusted and independently canonicalizes their shape and + * containment before spawning. This frame does not authenticate those roots + * against a fully compromised sidecar. + */ + export type CommandRequestFrame = Readonly<{ + v: typeof VERSION; + type: "command.request"; + requestId: string; + command: string; + args: readonly string[]; + workdir: string; + timeoutMs?: number; + workspaceRoot: string; + scratchDir?: string; + }>; + + /** Cancel one still-active command request. */ + export type CommandAbortFrame = Readonly<{ + v: typeof VERSION; + type: "command.abort"; + requestId: string; + reason?: string; + }>; + + /** One bounded UTF-8 stdout or stderr fragment, ordered within its stream. */ + export type CommandOutputFrame = Readonly<{ + v: typeof VERSION; + type: "command.output"; + requestId: string; + stream: "stdout" | "stderr"; + sequence: number; + data: string; + }>; + + export type CommandEndFrame = Readonly<{ + v: typeof VERSION; + type: "command.end"; + requestId: string; + sequence: number; + exitCode: number | null; + signal: string | null; + timedOut: boolean; + truncated: boolean; + durationMs: number; + }>; + + /** + * Terminal acknowledgement for command.abort. Main emits this only after the + * confined executor has returned and released its per-command authority. + */ + export type CommandAbortedFrame = Readonly<{ + v: typeof VERSION; + type: "command.aborted"; + requestId: string; + }>; + + /** A host-side validation or launch failure, before a run result exists. */ + export type CommandErrorFrame = Readonly<{ + v: typeof VERSION; + type: "command.error"; + requestId: string; + message: string; + }>; + export type ShutdownFrame = Readonly<{ v: typeof VERSION; type: "shutdown"; @@ -155,6 +227,10 @@ export namespace AgentSidecarChannel { | ResponseChunkFrame | ResponseEndFrame | ResponseErrorFrame + | CommandOutputFrame + | CommandEndFrame + | CommandAbortedFrame + | CommandErrorFrame | ShutdownFrame; export type SidecarToHostFrame = @@ -164,7 +240,9 @@ export namespace AgentSidecarChannel { | RequestChunkFrame | RequestEndFrame | RequestAbortFrame - | ResponseCreditFrame; + | ResponseCreditFrame + | CommandRequestFrame + | CommandAbortFrame; export type Frame = HostToSidecarFrame | SidecarToHostFrame; @@ -276,6 +354,103 @@ export namespace AgentSidecarChannel { expectIdentifier(frame.requestId, "requestId"); expectInteger(frame.bytes, "bytes", 1, MAX_RESPONSE_CREDIT_BYTES); break; + case "command.request": + expectExactKeys( + frame, + [ + "v", + "type", + "requestId", + "command", + "args", + "workdir", + "workspaceRoot", + ], + ["timeoutMs", "scratchDir"] + ); + expectIdentifier(frame.requestId, "requestId"); + expectCommandString(frame.command, "command", false); + expectCommandArgs(frame.args); + expectCommandString(frame.workdir, "workdir", false); + if (frame.timeoutMs !== undefined) { + expectInteger( + frame.timeoutMs, + "timeoutMs", + 1, + Number.MAX_SAFE_INTEGER + ); + } + expectCommandString(frame.workspaceRoot, "workspaceRoot", false); + if (frame.scratchDir !== undefined) { + expectCommandString(frame.scratchDir, "scratchDir", false); + } + break; + case "command.abort": + expectExactKeys(frame, ["v", "type", "requestId"], ["reason"]); + expectIdentifier(frame.requestId, "requestId"); + if (frame.reason !== undefined) { + expectBoundedString(frame.reason, "reason", 1, 1024); + } + break; + case "command.output": + expectExactKeys(frame, [ + "v", + "type", + "requestId", + "stream", + "sequence", + "data", + ]); + expectIdentifier(frame.requestId, "requestId"); + if (frame.stream !== "stdout" && frame.stream !== "stderr") { + throw protocolError("invalid command output stream"); + } + expectSequence(frame.sequence); + expectUtf8Chunk(frame.data); + break; + case "command.end": + expectExactKeys(frame, [ + "v", + "type", + "requestId", + "sequence", + "exitCode", + "signal", + "timedOut", + "truncated", + "durationMs", + ]); + expectIdentifier(frame.requestId, "requestId"); + expectSequence(frame.sequence); + if (frame.exitCode !== null) { + expectInteger( + frame.exitCode, + "exitCode", + Number.MIN_SAFE_INTEGER, + Number.MAX_SAFE_INTEGER + ); + } + if (frame.signal !== null) { + expectBoundedString(frame.signal, "signal", 1, 64); + } + expectBoolean(frame.timedOut, "timedOut"); + expectBoolean(frame.truncated, "truncated"); + expectInteger( + frame.durationMs, + "durationMs", + 0, + Number.MAX_SAFE_INTEGER + ); + break; + case "command.aborted": + expectExactKeys(frame, ["v", "type", "requestId"]); + expectIdentifier(frame.requestId, "requestId"); + break; + case "command.error": + expectExactKeys(frame, ["v", "type", "requestId", "message"]); + expectIdentifier(frame.requestId, "requestId"); + expectBoundedString(frame.message, "message", 1, 2048); + break; case "shutdown": expectExactKeys(frame, ["v", "type"]); break; @@ -684,6 +859,40 @@ function expectBase64Chunk(value: unknown): asserts value is string { } } +function expectCommandString( + value: unknown, + field: string, + allowEmpty: boolean +): asserts value is string { + expectBoundedString(value, field, allowEmpty ? 0 : 1, 32 * 1024); + if (value.includes("\0")) { + throw protocolError(`${field} must not contain a null byte`); + } +} + +function expectCommandArgs(value: unknown): asserts value is string[] { + if ( + !Array.isArray(value) || + value.length > AgentSidecarChannel.MAX_COMMAND_ARGS + ) { + throw protocolError( + `args must be an array with at most ${AgentSidecarChannel.MAX_COMMAND_ARGS} entries` + ); + } + for (const arg of value) expectCommandString(arg, "arg", true); +} + +function expectUtf8Chunk(value: unknown): asserts value is string { + if ( + typeof value !== "string" || + value.length === 0 || + Buffer.byteLength(value, "utf8") > + AgentSidecarChannel.MAX_COMMAND_OUTPUT_CHUNK_BYTES + ) { + throw protocolError("data must be a non-empty bounded UTF-8 chunk"); + } +} + function expectGrants( value: unknown ): asserts value is AgentSidecarChannel.NetworkGrant[] { diff --git a/desktop/src/agent-sidecar-network.test.ts b/desktop/src/agent-sidecar-network.test.ts index 250b1c79d..73ae4198b 100644 --- a/desktop/src/agent-sidecar-network.test.ts +++ b/desktop/src/agent-sidecar-network.test.ts @@ -1,9 +1,297 @@ import { PassThrough, Transform } from "node:stream"; -import { describe, expect, it } from "vitest"; +import { describe, expect, it, vi } from "vitest"; import { AgentSidecarChannel } from "./agent-sidecar-channel"; import { AgentSidecarNetwork } from "./agent-sidecar-network"; describe("AgentSidecarNetwork", () => { + it("executes a host command and reconstructs the daemon shell result", async () => { + const harness = createHarness(); + await harness.bootstrap(); + const result = harness.network.shellExecutor( + { + cmd: "node", + args: ["script.js", "--mode=test"], + cwd: "/workspace/project", + timeout_ms: 12_345, + }, + { + workspace_root: "/workspace", + scratch_root: "/scratch/session-1", + scratch_base: "/scratch", + protected_read_roots: ["/agent-home"], + } + ); + + const request = await harness.untilFrame("command.request"); + expect(request).toEqual({ + v: 1, + type: "command.request", + requestId: request.requestId, + command: "node", + args: ["script.js", "--mode=test"], + workdir: "/workspace/project", + timeoutMs: 12_345, + workspaceRoot: "/workspace", + scratchDir: "/scratch/session-1", + }); + await harness.host.write({ + v: 1, + type: "command.output", + requestId: request.requestId, + stream: "stdout", + sequence: 0, + data: "hello, ", + }); + await harness.host.write({ + v: 1, + type: "command.output", + requestId: request.requestId, + stream: "stderr", + sequence: 1, + data: "warning\n", + }); + await harness.host.write({ + v: 1, + type: "command.output", + requestId: request.requestId, + stream: "stdout", + sequence: 2, + data: "세계\n", + }); + await harness.host.write({ + v: 1, + type: "command.end", + requestId: request.requestId, + sequence: 3, + exitCode: 0, + signal: null, + timedOut: false, + truncated: false, + durationMs: 42, + }); + + await expect(result).resolves.toEqual({ + cmd: "node", + args: ["script.js", "--mode=test"], + cwd: "/workspace/project", + exit_code: 0, + signal: null, + stdout: "hello, 세계\n", + stderr: "warning\n", + duration_ms: 42, + timed_out: false, + truncated: false, + }); + harness.network.close(); + }); + + it("omits absent optional command scope fields and propagates host errors", async () => { + const harness = createHarness(); + await harness.bootstrap(); + const result = harness.network.shellExecutor( + { cmd: "pwd", args: [], cwd: "/workspace" }, + { + workspace_root: "/workspace", + protected_read_roots: [], + } + ); + const request = await harness.untilFrame("command.request"); + expect(request).not.toHaveProperty("timeoutMs"); + expect(request).not.toHaveProperty("scratchDir"); + + const rejection = result.catch((error: unknown) => error); + await harness.host.write({ + v: 1, + type: "command.error", + requestId: request.requestId, + message: "host refused the command scope", + }); + expect(await rejection).toMatchObject({ + message: expect.stringMatching(/refused the command scope/), + }); + harness.network.close(); + }); + + it("sends command.abort and ignores a terminal response that races cancellation", async () => { + const harness = createHarness(); + await harness.bootstrap(); + const fatal = vi.fn<(error: Error) => void>(); + harness.network.onFatal(fatal); + const controller = new AbortController(); + const result = harness.network.shellExecutor( + { cmd: "sleep", args: ["60"], cwd: "/workspace" }, + { + workspace_root: "/workspace", + protected_read_roots: [], + }, + controller.signal + ); + const request = await harness.untilFrame("command.request"); + let settled = false; + void result.then( + () => { + settled = true; + }, + () => { + settled = true; + } + ); + + controller.abort("user stopped the turn"); + + await expect(harness.untilFrame("command.abort")).resolves.toEqual({ + v: 1, + type: "command.abort", + requestId: request.requestId, + reason: "caller aborted", + }); + await new Promise((resolve) => setImmediate(resolve)); + expect(settled).toBe(false); + + // Main may have already queued output before it observes command.abort. + // Those frames belong to the cancelled generation and are not a protocol + // violation. + await harness.host.write({ + v: 1, + type: "command.output", + requestId: request.requestId, + stream: "stdout", + sequence: 0, + data: "late", + }); + await harness.host.write({ + v: 1, + type: "command.end", + requestId: request.requestId, + sequence: 1, + exitCode: null, + signal: "SIGTERM", + timedOut: false, + truncated: false, + durationMs: 1, + }); + await harness.host.write({ + v: 1, + type: "command.error", + requestId: request.requestId, + message: "late host failure", + }); + await new Promise((resolve) => setImmediate(resolve)); + expect(fatal).not.toHaveBeenCalled(); + expect(settled).toBe(false); + + await harness.host.write({ + v: 1, + type: "command.aborted", + requestId: request.requestId, + }); + await expect(result).rejects.toMatchObject({ + name: "AbortError", + message: "user stopped the turn", + }); + expect(settled).toBe(true); + harness.network.close(); + }); + + it("fails the generation when command.request delivery is ambiguous", async () => { + const sidecarOutput = new NthWriteFailingTransform(1); + const harness = createHarness(sidecarOutput); + await harness.bootstrap(); + const fatal = new Promise((resolve) => { + harness.network.onFatal(resolve); + }); + + const result = harness.network.shellExecutor( + { cmd: "sleep", args: ["60"], cwd: "/workspace" }, + { + workspace_root: "/workspace", + protected_read_roots: [], + } + ); + const rejection = result.catch((error: unknown) => error); + + // The bytes reached the peer-facing stream before its completion callback + // failed, so main could already own a live worker. + await expect(harness.untilFrame("command.request")).resolves.toMatchObject({ + type: "command.request", + command: "sleep", + }); + expect(await rejection).toMatchObject({ + message: expect.stringMatching(/request delivery failed/), + }); + await expect(fatal).resolves.toMatchObject({ + message: expect.stringMatching(/request delivery failed/), + }); + }); + + it("fails the channel on an out-of-sequence command response", async () => { + const harness = createHarness(); + await harness.bootstrap(); + const fatal = new Promise((resolve) => { + harness.network.onFatal(resolve); + }); + const result = harness.network.shellExecutor( + { cmd: "pwd", args: [], cwd: "/workspace" }, + { + workspace_root: "/workspace", + protected_read_roots: [], + } + ); + const rejection = result.catch((error: unknown) => error); + const request = await harness.untilFrame("command.request"); + await harness.host.write({ + v: 1, + type: "command.output", + requestId: request.requestId, + stream: "stdout", + sequence: 1, + data: "out of order", + }); + + expect((await fatal).message).toMatch(/sequence mismatch/); + expect(await rejection).toMatchObject({ + message: expect.stringMatching(/sequence mismatch/), + }); + }); + + it("caps cumulative command output even when every frame is legal", async () => { + const harness = createHarness(); + await harness.bootstrap(); + const fatal = new Promise((resolve) => { + harness.network.onFatal(resolve); + }); + const result = harness.network.shellExecutor( + { cmd: "noisy", args: [], cwd: "/workspace" }, + { + workspace_root: "/workspace", + protected_read_roots: [], + } + ); + const rejection = result.catch((error: unknown) => error); + const request = await harness.untilFrame("command.request"); + const chunk = "x".repeat( + AgentSidecarChannel.MAX_COMMAND_OUTPUT_CHUNK_BYTES + ); + const legalChunks = + AgentSidecarChannel.MAX_COMMAND_OUTPUT_BYTES / + AgentSidecarChannel.MAX_COMMAND_OUTPUT_CHUNK_BYTES; + for (let sequence = 0; sequence <= legalChunks; sequence += 1) { + await harness.host.write({ + v: 1, + type: "command.output", + requestId: request.requestId, + stream: "stdout", + sequence, + data: chunk, + }); + } + + expect((await fatal).message).toMatch(/exceeded the sidecar limit/); + expect(await rejection).toMatchObject({ + message: expect.stringMatching(/exceeded the sidecar limit/), + }); + }); + it("turns an injected provider fetch into a streamed, credited exchange", async () => { const harness = createHarness(); await harness.bootstrap(); @@ -425,3 +713,25 @@ class NthWriteBlockedTransform extends Transform { callback?.(); } } + +class NthWriteFailingTransform extends Transform { + private writes = 0; + + constructor(private readonly failedWrite: number) { + super(); + } + + override _transform( + chunk: Buffer, + _encoding: BufferEncoding, + callback: (error?: Error | null) => void + ): void { + this.push(chunk); + this.writes += 1; + callback( + this.writes === this.failedWrite + ? new Error("simulated ambiguous pipe failure") + : undefined + ); + } +} diff --git a/desktop/src/agent-sidecar-network.ts b/desktop/src/agent-sidecar-network.ts index b2aed1abc..abead0914 100644 --- a/desktop/src/agent-sidecar-network.ts +++ b/desktop/src/agent-sidecar-network.ts @@ -2,6 +2,12 @@ import crypto from "node:crypto"; import type { Readable, Writable } from "node:stream"; import type { ProviderHttpTransport } from "@grida/agent/server"; +import type { + ShellExecutionScope, + ShellExecutor, + ShellRunRequest, + ShellRunResult, +} from "@grida/daemon/server"; import { AgentNetworkPolicy } from "./agent-network-policy"; import { AgentSidecarChannel } from "./agent-sidecar-channel"; @@ -28,17 +34,32 @@ type PendingResponse = { abortCleanup: () => void; }; +type PendingCommand = { + request: ShellRunRequest; + resolve: (result: ShellRunResult) => void; + reject: (error: Error) => void; + sequence: number; + outputBytes: number; + stdout: string[]; + stderr: string[]; + abortCleanup: () => void; + abortError: Error | null; +}; + /** - * Sidecar half of the private provider-HTTP channel. This is the only object - * handed to `@grida/agent`; model tools and spawned children receive neither - * the object nor a channel address/token in argv or environment. + * Sidecar half of the private provider-HTTP and host-command channel. These + * are the only capabilities handed to `@grida/agent`; model tools and spawned + * children receive neither the objects nor a channel address/token in argv or + * environment. */ export class AgentSidecarNetwork { readonly providerHttp: ProviderHttpTransport; + readonly shellExecutor: ShellExecutor; private readonly decoder = new AgentSidecarChannel.Decoder(); private readonly writer: AgentSidecarChannel.Writer; private readonly pending = new Map(); + private readonly pendingCommands = new Map(); private readonly cancelled = new Set(); private grants: AgentSidecarChannel.NetworkGrant[] = []; private revision = -1; @@ -59,6 +80,8 @@ export class AgentSidecarNetwork { request: (input, init) => this.fetch("provider", input, init), download: (input, init) => this.fetch("download", input, init), }); + this.shellExecutor = (request, scope, signal) => + this.executeCommand(request, scope, signal); input.on("data", (chunk: Buffer | string) => { try { @@ -120,6 +143,88 @@ export class AgentSidecarNetwork { this.fail(new Error(reason)); } + private async executeCommand( + request: ShellRunRequest, + scope: ShellExecutionScope, + signal?: AbortSignal + ): Promise { + if (!this.bootstrapped || this.closed) { + throw new Error("agent host command execution is not available"); + } + if (signal?.aborted) throw abortError(signal.reason); + + const requestSnapshot: ShellRunRequest = { + cmd: request.cmd, + args: [...request.args], + cwd: request.cwd, + ...(request.timeout_ms === undefined + ? {} + : { timeout_ms: request.timeout_ms }), + }; + const requestId = crypto.randomUUID(); + const result = new Promise((resolve, reject) => { + const abort = () => { + const pending = this.pendingCommands.get(requestId); + if (!pending || pending.abortError) return; + // Keep the tool promise pending until main confirms that the confined + // worker returned and AgentCommandHost's finally cleanup completed. + pending.abortError = abortError(signal?.reason); + void this.send({ + v: 1, + type: "command.abort", + requestId, + reason: "caller aborted", + }).catch((error) => this.fail(asError(error))); + }; + signal?.addEventListener("abort", abort, { once: true }); + this.pendingCommands.set(requestId, { + request: requestSnapshot, + resolve, + reject, + sequence: 0, + outputBytes: 0, + stdout: [], + stderr: [], + abortCleanup: () => signal?.removeEventListener("abort", abort), + abortError: null, + }); + }); + // A host can reject as soon as it decodes command.request, before the + // Writer's callback/backpressure promise resolves. Observe that result + // synchronously and return the original rejection below. + void result.catch(() => undefined); + + try { + await this.send({ + v: 1, + type: "command.request", + requestId, + command: requestSnapshot.cmd, + args: requestSnapshot.args, + workdir: requestSnapshot.cwd, + ...(requestSnapshot.timeout_ms === undefined + ? {} + : { timeoutMs: requestSnapshot.timeout_ms }), + workspaceRoot: scope.workspace_root, + ...(scope.scratch_root === undefined + ? {} + : { scratchDir: scope.scratch_root }), + }); + } catch (error) { + // A failed write is ambiguous: main may already have decoded the frame + // and launched a confined worker before the pipe reported failure. + // Only generation-fatal supervision can own that worker's cleanup; an + // ordinary tool rejection would let this runtime admit replacement work + // without the terminal cleanup acknowledgement. + this.fail( + new Error( + `host command request delivery failed: ${asError(error).message}` + ) + ); + } + return await result; + } + private async fetch( lane: AgentNetworkPolicy.Lane, input: string | URL | Request, @@ -287,6 +392,18 @@ export class AgentSidecarNetwork { if (this.ignoreCancelledResponse(frame)) return; this.onResponseError(frame); return; + case "command.output": + this.onCommandOutput(frame); + return; + case "command.end": + this.onCommandEnd(frame); + return; + case "command.aborted": + this.onCommandAborted(frame); + return; + case "command.error": + this.onCommandError(frame); + return; case "shutdown": this.shutdownRequested = true; this.deliverShutdown(); @@ -379,6 +496,78 @@ export class AgentSidecarNetwork { this.rejectPending(frame.requestId, error); } + private onCommandOutput(frame: AgentSidecarChannel.CommandOutputFrame): void { + const pending = this.requirePendingCommand(frame.requestId); + if (pending.abortError) return; + if (frame.sequence !== pending.sequence) { + throw new Error("command output sequence mismatch"); + } + const bytes = Buffer.byteLength(frame.data, "utf8"); + if ( + pending.outputBytes + bytes > + AgentSidecarChannel.MAX_COMMAND_OUTPUT_BYTES + ) { + throw new Error("command output exceeded the sidecar limit"); + } + pending.sequence += 1; + pending.outputBytes += bytes; + pending[frame.stream].push(frame.data); + } + + private onCommandEnd(frame: AgentSidecarChannel.CommandEndFrame): void { + const pending = this.requirePendingCommand(frame.requestId); + if (pending.abortError) return; + if (frame.sequence !== pending.sequence) { + throw new Error("command end sequence mismatch"); + } + this.pendingCommands.delete(frame.requestId); + pending.abortCleanup(); + pending.resolve({ + cmd: pending.request.cmd, + args: [...pending.request.args], + cwd: pending.request.cwd, + exit_code: frame.exitCode, + signal: frame.signal, + stdout: pending.stdout.join(""), + stderr: pending.stderr.join(""), + duration_ms: frame.durationMs, + timed_out: frame.timedOut, + truncated: frame.truncated, + }); + } + + private onCommandAborted( + frame: AgentSidecarChannel.CommandAbortedFrame + ): void { + const pending = this.requirePendingCommand(frame.requestId); + if (!pending.abortError) { + throw new Error("command.aborted arrived without a caller abort"); + } + this.pendingCommands.delete(frame.requestId); + pending.abortCleanup(); + pending.reject(pending.abortError); + } + + private onCommandError(frame: AgentSidecarChannel.CommandErrorFrame): void { + const pending = this.requirePendingCommand(frame.requestId); + if (pending.abortError) return; + this.rejectPendingCommand(frame.requestId, new Error(frame.message)); + } + + private requirePendingCommand(requestId: string): PendingCommand { + const pending = this.pendingCommands.get(requestId); + if (!pending) throw new Error("command response names an unknown request"); + return pending; + } + + private rejectPendingCommand(requestId: string, error: Error): void { + const pending = this.pendingCommands.get(requestId); + if (!pending) return; + this.pendingCommands.delete(requestId); + pending.abortCleanup(); + pending.reject(error); + } + private requirePending(requestId: string): PendingResponse { const pending = this.pending.get(requestId); if (!pending) throw new Error("response names an unknown request"); @@ -455,6 +644,9 @@ export class AgentSidecarNetwork { for (const requestId of this.pending.keys()) { this.rejectPending(requestId, error); } + for (const requestId of this.pendingCommands.keys()) { + this.rejectPendingCommand(requestId, error); + } this.cancelled.clear(); if (this.fatalHandler) this.fatalHandler(error); else this.fatalValue = error; diff --git a/desktop/src/agent-sidecar.ts b/desktop/src/agent-sidecar.ts index 412fe1043..f75661cb7 100644 --- a/desktop/src/agent-sidecar.ts +++ b/desktop/src/agent-sidecar.ts @@ -32,6 +32,9 @@ * the agent home dir (`~/.grida/agent`, resolved * via `@grida/home`). We can't import that (or * `electron`) here, so the supervisor forwards it. + * --scratch-base= + * Electron-main-resolved temp authority root. This is + * intentionally resolved before SRT rewrites TMPDIR. * * Stdout contract: framed sidecar→host control and provider requests only. * Human-readable logs always use stderr. @@ -96,6 +99,12 @@ if (!userDataPath) { process.exit(1); } const requiredUserDataPath = userDataPath; +const scratchBase = getCliArg("scratch-base"); +if (!scratchBase) { + console.error("[agent-sidecar] fatal: missing --scratch-base"); + process.exit(1); +} +const requiredScratchBase = scratchBase; const runtimeEditorBaseUrl = getCliArg("editor-base-url") ?? EDITOR_BASE_URL; // GRIDA-SEC-004 — the supervisor tells us whether it wrapped this spawn with // srt. Trusted: argv is set by the trusted main process, not the renderer. @@ -158,16 +167,18 @@ async function main() { const host = createAgentDaemon({ password, user_data_path: requiredUserDataPath, + scratch_base: requiredScratchBase, projects_root: projectsRoot, skills_root: skillsRoot, http_access: { allowed_origins: [editorOrigin], allowed_referer_paths: ["/desktop"], }, - // GRIDA-SEC-004 — fail-closed shell: `run_command` is exposed only when - // srt actually confines this process tree. On platforms srt can't wrap - // (Windows), this is false and the agent gets fs/todos/skills but no shell. + // GRIDA-SEC-004 — the boolean attests the coarse outer process wrap; the + // private callback below is the actual finite-command capability. Both are + // present only when main can enforce SRT on this platform. sandbox_enforced: sandboxEnforced, + shell_executor: sandboxEnforced ? network.shellExecutor : undefined, // External ACP owns a subprocess and network stack that cannot consume the // host-routed provider transport. Keep it unavailable in Desktop until it // has a separately confined, route-compatible authority domain. diff --git a/desktop/src/main/agent-command-host.srt.test.ts b/desktop/src/main/agent-command-host.srt.test.ts new file mode 100644 index 000000000..d8b45e8da --- /dev/null +++ b/desktop/src/main/agent-command-host.srt.test.ts @@ -0,0 +1,167 @@ +import fs from "node:fs/promises"; +import os from "node:os"; +import path from "node:path"; +import { afterEach, beforeEach, describe, expect, it } from "vitest"; +import { DesktopAgentSandboxPolicy } from "./agent-sandbox-policy"; +import { AgentCommandHost } from "./agent-command-host"; +import { dispose, ensureInitialized } from "./sandbox/manager"; + +const describeMacOS = process.platform === "darwin" ? describe : describe.skip; + +/** + * GRIDA-SEC-004 attack regression. + * + * Unit policy assertions prove what main asks SRT to enforce. This test proves + * the resulting Seatbelt process cannot recover a sibling session's bytes, + * including through an interpreter that computes the path at runtime. + */ +describeMacOS("AgentCommandHost per-command SRT boundary", () => { + let root: string; + let userData: string; + let scratchBase: string; + let workspace: string; + let scratchA: string; + let scratchB: string; + let host: AgentCommandHost; + + beforeEach(async () => { + root = await fs.mkdtemp(path.join(os.tmpdir(), "grida-command-srt-")); + userData = path.join(root, "user-data"); + scratchBase = path.join(root, "scratch-authority"); + workspace = path.join(root, "workspace"); + scratchA = path.join(scratchBase, "sessions", "ses_A", "scratch"); + scratchB = path.join(scratchBase, "sessions", "ses_B", "scratch"); + await Promise.all( + [userData, workspace, scratchA, scratchB].map((dir) => + fs.mkdir(dir, { recursive: true }) + ) + ); + await fs.writeFile(path.join(scratchA, "own.txt"), "session-a"); + await fs.writeFile(path.join(scratchB, "private.txt"), "session-b-secret"); + await fs.writeFile(path.join(userData, "auth.json"), "host-secret"); + + const policy = DesktopAgentSandboxPolicy.build({ + userData, + home: os.homedir(), + ggHost: "grida.co", + }); + await ensureInitialized({ + network: { + allowedDomains: policy.network.allowed_domains, + deniedDomains: policy.network.denied_domains, + allowLocalBinding: policy.network.allow_local_binding, + }, + filesystem: { + denyRead: policy.filesystem.deny_read, + allowRead: policy.filesystem.allow_read, + allowWrite: policy.filesystem.allow_write, + denyWrite: policy.filesystem.deny_write, + }, + }); + host = new AgentCommandHost({ + scratchBase, + userData, + home: os.homedir(), + filesystemPolicy: policy.filesystem, + }); + }); + + afterEach(async () => { + await dispose(); + await fs.rm(root, { recursive: true, force: true }); + }); + + it("allows its own scratch but denies sibling reads and writes", async () => { + const scope = { + workspace_root: workspace, + scratch_root: scratchA, + scratch_base: scratchBase, + protected_read_roots: [userData], + }; + + const own = await host.shellExecutor( + { + cmd: "cp", + args: ["own.txt", "own-copy.txt"], + cwd: scratchA, + }, + scope + ); + expect(own.exit_code).toBe(0); + await expect( + fs.readFile(path.join(scratchA, "own-copy.txt"), "utf8") + ).resolves.toBe("session-a"); + + const readAttack = await host.shellExecutor( + { + cmd: process.execPath, + args: [ + "-e", + "process.stdout.write(require('node:fs').readFileSync(process.argv[1], 'utf8'))", + path.join(scratchB, "private.txt"), + ], + cwd: workspace, + }, + scope + ); + expect(readAttack.exit_code).not.toBe(0); + expect(readAttack.stdout).not.toContain("session-b-secret"); + + const computedSecretAttack = await host.shellExecutor( + { + cmd: process.execPath, + args: [ + "-e", + `process.stdout.write(require('node:fs').readFileSync(${JSON.stringify(path.join(userData, "auth.json"))}, 'utf8'))`, + ], + cwd: workspace, + }, + scope + ); + expect(computedSecretAttack.exit_code).not.toBe(0); + expect(computedSecretAttack.stdout).not.toContain("host-secret"); + + const writeAttack = await host.shellExecutor( + { + cmd: process.execPath, + args: [ + "-e", + "require('node:fs').writeFileSync(process.argv[1], 'overwritten')", + path.join(scratchB, "private.txt"), + ], + cwd: workspace, + }, + scope + ); + expect(writeAttack.exit_code).not.toBe(0); + await expect( + fs.readFile(path.join(scratchB, "private.txt"), "utf8") + ).resolves.toBe("session-b-secret"); + }); + + it("kills a background descendant before releasing its SRT profile", async () => { + const scope = { + workspace_root: workspace, + scratch_root: scratchA, + scratch_base: scratchBase, + protected_read_roots: [userData], + }; + const result = await host.shellExecutor( + { + cmd: "/bin/sh", + args: [ + "-c", + "(sleep 0.4; printf escaped > late-write.txt) >/dev/null 2>&1 &", + ], + cwd: workspace, + }, + scope + ); + + expect(result.exit_code).toBe(0); + await new Promise((resolve) => setTimeout(resolve, 650)); + await expect( + fs.stat(path.join(workspace, "late-write.txt")) + ).rejects.toThrow(/ENOENT/); + }); +}); diff --git a/desktop/src/main/agent-command-host.test.ts b/desktop/src/main/agent-command-host.test.ts new file mode 100644 index 000000000..47fda7a62 --- /dev/null +++ b/desktop/src/main/agent-command-host.test.ts @@ -0,0 +1,264 @@ +import fs from "node:fs/promises"; +import os from "node:os"; +import path from "node:path"; +import { afterEach, beforeEach, describe, expect, it, vi } from "vitest"; +import type { + ShellExecutionScope, + ShellRunRequest, + ShellRunResult, +} from "@grida/daemon/server"; +import type { SandboxRuntimeConfig } from "@anthropic-ai/sandbox-runtime"; +import { AgentCommandHost, sweepAgentCommandTemps } from "./agent-command-host"; + +describe("AgentCommandHost", () => { + let root: string; + let userData: string; + let scratchBase: string; + let workspaceA: string; + let workspaceB: string; + let scratchA: string; + let scratchB: string; + + beforeEach(async () => { + root = await fs.mkdtemp(path.join(os.tmpdir(), "grida-command-host-")); + userData = path.join(root, "userdata"); + scratchBase = path.join(root, "scratch-base"); + workspaceA = path.join(root, "workspace-a"); + workspaceB = path.join(root, "workspace-b"); + scratchA = path.join(scratchBase, "sessions", "ses_A", "scratch"); + scratchB = path.join(scratchBase, "sessions", "ses_B", "scratch"); + await Promise.all( + [userData, workspaceA, workspaceB, scratchA, scratchB].map((dir) => + fs.mkdir(dir, { recursive: true }) + ) + ); + }); + + afterEach(async () => { + await fs.rm(root, { recursive: true, force: true }); + }); + + it("wraps a command with only its exact workspace and scratch grants", async () => { + let policy: Partial | undefined; + const wrap = vi.fn< + ( + command: string, + customConfig: Partial + ) => Promise<{ argv: string[]; env: NodeJS.ProcessEnv }> + >(async (_command: string, customConfig: Partial) => { + policy = customConfig; + return { argv: ["/wrapped", "--profile"], env: {} }; + }); + const run = vi.fn<(request: ShellRunRequest) => Promise>( + async (request: ShellRunRequest): Promise => ({ + cmd: request.cmd, + args: request.args, + cwd: request.cwd, + exit_code: 0, + signal: null, + stdout: "ok", + stderr: "", + duration_ms: 1, + timed_out: false, + truncated: false, + }) + ); + const host = commandHost({ wrap, run }); + + const result = await host.shellExecutor( + { + cmd: "cp", + args: ["input.png", "copy.png"], + cwd: scratchA, + }, + scopeA() + ); + + expect(result).toMatchObject({ + cmd: "cp", + args: ["input.png", "copy.png"], + cwd: await fs.realpath(scratchA), + stdout: "ok", + }); + expect(run).toHaveBeenCalledWith( + expect.objectContaining({ + cmd: "/wrapped", + args: ["--profile"], + cwd: await fs.realpath(scratchA), + }), + { signal: undefined } + ); + expect(policy?.network).toMatchObject({ + allowedDomains: [], + allowLocalBinding: false, + }); + expect(policy?.filesystem?.denyRead).toContain( + await fs.realpath(scratchBase) + ); + expect(policy?.filesystem?.allowRead).toEqual( + expect.arrayContaining([await fs.realpath(scratchA)]) + ); + expect(policy?.filesystem?.allowRead).not.toContain( + await fs.realpath(workspaceA) + ); + expect(policy?.filesystem?.allowRead).not.toContain( + await fs.realpath(scratchB) + ); + expect(policy?.filesystem?.allowWrite).not.toContain( + await fs.realpath(workspaceB) + ); + }); + + it("refuses a cwd in another registered session workspace", async () => { + const host = commandHost(); + + await expect( + host.shellExecutor({ cmd: "pwd", args: [], cwd: workspaceB }, scopeA()) + ).rejects.toThrow(/outside its session-bound roots/); + }); + + it("refuses a scratch grant outside the host-owned session tree", async () => { + const host = commandHost(); + const outsideScratch = path.join( + root, + "other-scratch-base", + "sessions", + "ses_B", + "scratch" + ); + await fs.mkdir(outsideScratch, { recursive: true }); + + await expect( + host.shellExecutor( + { cmd: "pwd", args: [], cwd: outsideScratch }, + { ...scopeA(), scratch_root: outsideScratch } + ) + ).rejects.toThrow(/invalid session scratch root/); + }); + + it("refuses a workspace whose write grant would cover the scratch base", async () => { + const nestedScratchBase = path.join(workspaceA, "scratch-authority"); + await fs.mkdir(nestedScratchBase, { recursive: true }); + const host = commandHost({ scratchBase: nestedScratchBase }); + + await expect( + host.shellExecutor( + { cmd: "true", args: [], cwd: workspaceA }, + { + workspace_root: workspaceA, + scratch_base: nestedScratchBase, + protected_read_roots: [userData], + } + ) + ).rejects.toThrow(/overlaps the scratch authority root/); + }); + + it("does not re-allow a workspace through a protected read root", async () => { + const protectedWorkspace = path.join(root, ".ssh", "workspace"); + await fs.mkdir(protectedWorkspace, { recursive: true }); + let policy: Partial | undefined; + const host = commandHost({ + wrap: async (_command, customConfig) => { + policy = customConfig; + return { argv: ["/wrapped"], env: {} }; + }, + }); + + await host.shellExecutor( + { cmd: "pwd", args: [], cwd: protectedWorkspace }, + { + workspace_root: protectedWorkspace, + protected_read_roots: [userData], + } + ); + + expect(policy?.filesystem?.denyRead).toContain(path.join(root, ".ssh")); + expect(policy?.filesystem?.allowRead).not.toContain( + await fs.realpath(protectedWorkspace) + ); + expect(policy?.filesystem?.allowWrite).toContain( + await fs.realpath(protectedWorkspace) + ); + }); + + it("does not double-clean SRT state when wrapping fails", async () => { + const cleanup = vi.fn<() => void>(); + const host = commandHost({ + wrap: async () => { + throw new Error("wrap failed"); + }, + cleanup, + }); + + await expect( + host.shellExecutor({ cmd: "pwd", args: [], cwd: workspaceA }, scopeA()) + ).rejects.toThrow("wrap failed"); + expect(cleanup).not.toHaveBeenCalled(); + }); + + it("sweeps command-temp remnants left by a prior crash", async () => { + const remnant = path.join(scratchBase, "commands", "cmd-old"); + await fs.mkdir(remnant, { recursive: true }); + await fs.writeFile(path.join(remnant, "payload.bin"), "bytes"); + + await sweepAgentCommandTemps(scratchBase); + + await expect(fs.lstat(path.join(scratchBase, "commands"))).rejects.toThrow( + /ENOENT/ + ); + }); + + it("unlinks a command-temp symlink without deleting its target", async () => { + if (process.platform === "win32") return; + const target = path.join(root, "command-target"); + await fs.mkdir(target); + await fs.writeFile(path.join(target, "keep.txt"), "keep"); + await fs.symlink(target, path.join(scratchBase, "commands")); + + await sweepAgentCommandTemps(scratchBase); + + await expect( + fs.readFile(path.join(target, "keep.txt"), "utf8") + ).resolves.toBe("keep"); + await expect(fs.lstat(path.join(scratchBase, "commands"))).rejects.toThrow( + /ENOENT/ + ); + }); + + function scopeA(): ShellExecutionScope { + return { + workspace_root: workspaceA, + scratch_root: scratchA, + scratch_base: scratchBase, + protected_read_roots: [userData], + }; + } + + function commandHost( + overrides: Partial[0]> = {} + ): AgentCommandHost { + return new AgentCommandHost({ + scratchBase, + userData, + home: root, + filesystemPolicy: { + deny_read: [path.join(root, ".ssh")], + deny_write: [path.join(root, ".ssh")], + }, + wrap: async () => ({ argv: ["/wrapped"], env: {} }), + run: async (request) => ({ + cmd: request.cmd, + args: request.args, + cwd: request.cwd, + exit_code: 0, + signal: null, + stdout: "", + stderr: "", + duration_ms: 1, + timed_out: false, + truncated: false, + }), + ...overrides, + }); + } +}); diff --git a/desktop/src/main/agent-command-host.ts b/desktop/src/main/agent-command-host.ts new file mode 100644 index 000000000..685f40d23 --- /dev/null +++ b/desktop/src/main/agent-command-host.ts @@ -0,0 +1,381 @@ +/** + * GRIDA-SEC-004 — supervisor-owned, per-command shell confinement. + * + * AgentSidecar owns the model loop and validates the tool request, but it does + * not spawn raw commands in Desktop. It asks Electron main over the inherited + * private channel. Main canonicalizes one exact tenant-supplied workspace and, + * when present, one exact session scratch root, then asks SRT for a fresh + * kernel profile for the command's original POSIX process group. + * + * The long-lived sidecar's outer profile is intentionally too coarse for this + * job: one sidecar serves many sessions, so a child that merely inherits that + * profile can read a sibling session's scratch. Per-command deny/allow carving + * is the security boundary; argv inspection remains only defense in depth. + */ +import fs from "node:fs/promises"; +import path from "node:path"; +import { + containsPath, + runShell, + type ShellExecutionScope, + type ShellExecutor, + type ShellRunOptions, + type ShellRunRequest, + type ShellRunResult, +} from "@grida/daemon/server"; +import type { SandboxRuntimeConfig } from "@anthropic-ai/sandbox-runtime"; +import { cleanupAfterCommand, wrapArgv } from "./sandbox/manager"; + +const PRIVATE_DIR_MODE = 0o700; +const SHARED_SRT_TMP_DIRS = ["/tmp/claude", "/private/tmp/claude"] as const; +const COMMANDS_DIRNAME = "commands"; + +type FilesystemPolicy = Readonly<{ + deny_read: readonly string[]; + deny_write: readonly string[]; +}>; + +type WrappedCommand = Readonly<{ + argv: string[]; + env: NodeJS.ProcessEnv; +}>; + +export type AgentCommandHostOptions = Readonly<{ + scratchBase: string; + userData: string; + home: string; + filesystemPolicy: FilesystemPolicy; + wrap?: ( + command: string, + customConfig: Partial, + abortSignal?: AbortSignal + ) => Promise; + run?: ( + request: ShellRunRequest, + options?: ShellRunOptions + ) => Promise; + cleanup?: () => void; +}>; + +/** + * Reclaim command-temp remnants from a prior crashed host. The caller must + * first establish the scratch base as a trusted, owner-only non-symlink + * authority with `prepareScratchAuthority`. + */ +export async function sweepAgentCommandTemps( + scratchBase: string +): Promise { + const commands = path.join(path.resolve(scratchBase), COMMANDS_DIRNAME); + let stat; + try { + stat = await fs.lstat(commands); + } catch (error) { + if ((error as NodeJS.ErrnoException).code === "ENOENT") return; + throw error; + } + if (stat.isSymbolicLink() || !stat.isDirectory()) { + await fs.unlink(commands); + return; + } + assertCurrentOwner(commands, stat); + await fs.rm(commands, { recursive: true, force: true }); +} + +/** + * The only raw-command executor Desktop injects into the agent tenant. + * + * The public function identity is stable so it can be handed directly to + * `createAgentDaemon({ shell_executor })`; all mutable state remains + * private to this main-process object. + */ +export class AgentCommandHost { + readonly shellExecutor: ShellExecutor; + + private readonly scratchBase: string; + private readonly userData: string; + private readonly home: string; + private readonly filesystemPolicy: FilesystemPolicy; + private readonly wrapCommand: NonNullable; + private readonly runCommand: NonNullable; + private readonly cleanupCommand: NonNullable< + AgentCommandHostOptions["cleanup"] + >; + + constructor(options: AgentCommandHostOptions) { + this.scratchBase = path.resolve(options.scratchBase); + this.userData = path.resolve(options.userData); + this.home = path.resolve(options.home); + this.filesystemPolicy = options.filesystemPolicy; + this.wrapCommand = options.wrap ?? wrapArgv; + this.runCommand = options.run ?? runShell; + this.cleanupCommand = options.cleanup ?? cleanupAfterCommand; + this.shellExecutor = async (request, scope, signal) => + await this.execute(request, scope, signal); + } + + private async execute( + request: ShellRunRequest, + scope: ShellExecutionScope, + signal?: AbortSignal + ): Promise { + const grant = await this.resolveGrant(request, scope); + const commandTempBase = path.join(this.scratchBase, COMMANDS_DIRNAME); + await ensurePrivateDirectory(commandTempBase); + const commandTemp = await fs.mkdtemp(path.join(commandTempBase, "cmd-")); + await fs.chmod(commandTemp, PRIVATE_DIR_MODE); + let wrappedSuccessfully = false; + + try { + const policy = this.commandPolicy(grant, commandTemp); + const command = shellJoin([ + "env", + `TMPDIR=${commandTemp}`, + `TMP=${commandTemp}`, + `TEMP=${commandTemp}`, + request.cmd, + ...request.args, + ]); + const wrapped = await this.wrapCommand(command, policy, signal); + // SRT owns failed-wrap cleanup internally. Only a successful wrap adds + // one live worker that this host must release after the process exits. + wrappedSuccessfully = true; + if (wrapped.argv.length === 0) { + throw new Error("sandbox wrapper returned no command"); + } + const result = await this.runCommand( + { + cmd: wrapped.argv[0], + args: wrapped.argv.slice(1), + cwd: grant.cwd, + timeout_ms: request.timeout_ms, + }, + { signal } + ); + // The wrapper argv/profile are host internals. Preserve the model-visible + // identity of the command the tool actually requested. + return { + ...result, + cmd: request.cmd, + args: [...request.args], + cwd: grant.cwd, + }; + } finally { + try { + await fs.rm(commandTemp, { recursive: true, force: true }); + } finally { + if (wrappedSuccessfully) this.cleanupCommand(); + } + } + } + + private async resolveGrant( + request: ShellRunRequest, + scope: ShellExecutionScope + ): Promise<{ + workspaceRoot: string; + scratchRoot?: string; + scratchBase: string; + userData: string; + cwd: string; + }> { + const expectedScratchBase = await realpathNearest(this.scratchBase); + if (scope.scratch_base) { + const claimed = await realpathNearest(scope.scratch_base); + if (claimed !== expectedScratchBase) { + throw new Error("command scope names an unexpected scratch base"); + } + } + + const workspaceRoot = await realDirectory( + scope.workspace_root, + "workspace" + ); + const userData = await realpathNearest(this.userData); + if ( + containsPath(userData, workspaceRoot) || + containsPath(workspaceRoot, userData) + ) { + throw new Error("command workspace overlaps the agent secret root"); + } + if ( + containsPath(userData, expectedScratchBase) || + containsPath(expectedScratchBase, userData) + ) { + throw new Error("scratch authority overlaps the agent secret root"); + } + // A workspace that contains the shared scratch base would make the + // workspace write grant cover every session. The inverse is equally + // invalid: scratch is not a workspace. + if ( + containsPath(expectedScratchBase, workspaceRoot) || + containsPath(workspaceRoot, expectedScratchBase) + ) { + throw new Error("command workspace overlaps the scratch authority root"); + } + + let scratchRoot: string | undefined; + if (scope.scratch_root) { + scratchRoot = await realDirectory(scope.scratch_root, "scratch"); + const sessionDir = path.dirname(scratchRoot); + const sessionsDir = path.dirname(sessionDir); + if ( + path.basename(scratchRoot) !== "scratch" || + path.dirname(sessionsDir) !== expectedScratchBase || + path.basename(sessionsDir) !== "sessions" || + !/^[A-Za-z0-9_-]+$/.test(path.basename(sessionDir)) + ) { + throw new Error("command scope names an invalid session scratch root"); + } + } + + const cwd = await realDirectory(request.cwd, "cwd"); + if ( + !containsPath(workspaceRoot, cwd) && + !(scratchRoot && containsPath(scratchRoot, cwd)) + ) { + throw new Error("command cwd is outside its session-bound roots"); + } + + return { + workspaceRoot, + scratchRoot, + scratchBase: expectedScratchBase, + userData, + cwd, + }; + } + + private commandPolicy( + grant: { + workspaceRoot: string; + scratchRoot?: string; + scratchBase: string; + userData: string; + }, + commandTemp: string + ): Partial { + const sharedWriteDefaults = [ + ...SHARED_SRT_TMP_DIRS, + path.join(this.home, ".npm", "_logs"), + path.join(this.home, ".claude", "debug"), + ]; + const allowedRead = [ + ...(grant.scratchRoot ? [grant.scratchRoot] : []), + commandTemp, + ]; + const allowedWrite = [ + grant.workspaceRoot, + ...(grant.scratchRoot ? [grant.scratchRoot] : []), + commandTemp, + ]; + return { + // Desktop's process-global SRT manager is initialized with an empty + // destination set. SRT 0.0.65 authorizes proxy requests against that + // global set, not this per-call copy; any future outer widening must use + // a separate manager/proxy for finite workers rather than assuming this + // field narrows it again. + network: { + allowedDomains: [], + deniedDomains: [], + allowLocalBinding: false, + }, + filesystem: { + // Read is deny-then-allow in SRT. Deny the whole shared scratch base, + // then carve back only this session's root and this command's private + // temp. `userData` is denied at the kernel here even though the + // long-lived host itself must read it. + denyRead: uniquePaths([ + ...this.filesystemPolicy.deny_read, + grant.userData, + grant.scratchBase, + ...sharedWriteDefaults, + ]), + // Workspace reads are already allowed by SRT's default-read model. + // Do not carve the workspace back through an outer deny (for example, + // when a user opens ~/.ssh as a workspace). + allowRead: uniquePaths(allowedRead), + // Write is allow-then-deny. The scratch base deliberately lives + // outside SRT's unconditional shared temp defaults, so these exact + // grants do not cover sibling sessions. + allowWrite: uniquePaths(allowedWrite), + denyWrite: uniquePaths([ + ...this.filesystemPolicy.deny_write, + grant.userData, + ...sharedWriteDefaults, + ]), + }, + }; + } +} + +async function realDirectory(value: string, label: string): Promise { + if (!path.isAbsolute(value)) { + throw new Error(`${label} must be an absolute path`); + } + const real = await fs.realpath(value); + const stat = await fs.stat(real); + if (!stat.isDirectory()) throw new Error(`${label} is not a directory`); + return real; +} + +async function realpathNearest(value: string): Promise { + let current = path.resolve(value); + const tail: string[] = []; + for (;;) { + try { + const real = await fs.realpath(current); + return tail.length ? path.join(real, ...tail.reverse()) : real; + } catch { + const parent = path.dirname(current); + if (parent === current) return path.resolve(value); + tail.push(path.basename(current)); + current = parent; + } + } +} + +function uniquePaths(values: readonly string[]): string[] { + return [...new Set(values.map((value) => path.resolve(value)))]; +} + +async function ensurePrivateDirectory(target: string): Promise { + try { + await fs.mkdir(target, { recursive: false, mode: PRIVATE_DIR_MODE }); + } catch (error) { + if ((error as NodeJS.ErrnoException).code !== "EEXIST") throw error; + } + let stat = await fs.lstat(target); + if (stat.isSymbolicLink() || !stat.isDirectory()) { + throw new Error(`command temp authority is not a directory: ${target}`); + } + assertCurrentOwner(target, stat); + if (process.platform !== "win32") { + await fs.chmod(target, PRIVATE_DIR_MODE); + stat = await fs.lstat(target); + assertCurrentOwner(target, stat); + if ((stat.mode & 0o777) !== PRIVATE_DIR_MODE) { + throw new Error(`command temp authority is not owner-only: ${target}`); + } + } +} + +function assertCurrentOwner(target: string, stat: import("node:fs").Stats) { + if ( + process.platform !== "win32" && + typeof process.getuid === "function" && + stat.uid !== process.getuid() + ) { + throw new Error( + `command temp authority is not owned by this user: ${target}` + ); + } +} + +function shellJoin(values: readonly string[]): string { + return values.map(shellQuote).join(" "); +} + +function shellQuote(value: string): string { + if (value.length === 0) return "''"; + return `'${value.replaceAll("'", `'\\''`)}'`; +} diff --git a/desktop/src/main/agent-network-host.test.ts b/desktop/src/main/agent-network-host.test.ts index 26f2d159d..fc304970c 100644 --- a/desktop/src/main/agent-network-host.test.ts +++ b/desktop/src/main/agent-network-host.test.ts @@ -2,6 +2,7 @@ import { EventEmitter } from "node:events"; import { PassThrough, Readable, Writable } from "node:stream"; import { beforeEach, describe, expect, it, vi } from "vitest"; import { net, session } from "electron"; +import type { ShellExecutor } from "@grida/daemon/server"; vi.mock("electron", () => ({ session: { @@ -111,6 +112,292 @@ describe("AgentNetworkHost", () => { } }); + it("executes a finite command through the injected host capability", async () => { + const commandExecutor = vi.fn(async (request, _scope) => ({ + cmd: request.cmd, + args: request.args, + cwd: request.cwd, + exit_code: 0, + signal: null, + stdout: "hello, 세계\n", + stderr: "warning\n", + duration_ms: 12, + timed_out: false, + truncated: false, + })); + const harness = createHarness({ + fetch: async () => new Response(), + commandExecutor, + }); + await harness.start(); + + await harness.sidecar.write({ + v: 1, + type: "command.request", + requestId: "cmd_1", + command: "node", + args: ["script.js"], + workdir: "/workspace/project", + timeoutMs: 2_000, + workspaceRoot: "/workspace", + scratchDir: "/scratch/sessions/ses_A/scratch", + }); + + const end = await harness.untilFrame("command.end"); + expect(commandExecutor).toHaveBeenCalledWith( + { + cmd: "node", + args: ["script.js"], + cwd: "/workspace/project", + timeout_ms: 2_000, + }, + { + workspace_root: "/workspace", + scratch_root: "/scratch/sessions/ses_A/scratch", + protected_read_roots: [], + }, + expect.objectContaining({ aborted: false }) + ); + expect( + harness.frames.filter((frame) => frame.type === "command.output") + ).toEqual([ + { + v: 1, + type: "command.output", + requestId: "cmd_1", + stream: "stdout", + sequence: 0, + data: "hello, 세계\n", + }, + { + v: 1, + type: "command.output", + requestId: "cmd_1", + stream: "stderr", + sequence: 1, + data: "warning\n", + }, + ]); + expect(end).toEqual({ + v: 1, + type: "command.end", + requestId: "cmd_1", + sequence: 2, + exitCode: 0, + signal: null, + timedOut: false, + truncated: false, + durationMs: 12, + }); + harness.host.close(); + }); + + it("fails closed when no finite-command capability was injected", async () => { + const harness = createHarness({ + fetch: async () => new Response(), + }); + await harness.start(); + + await harness.sidecar.write({ + v: 1, + type: "command.request", + requestId: "cmd_unavailable", + command: "pwd", + args: [], + workdir: "/workspace", + workspaceRoot: "/workspace", + }); + + await expect(harness.untilFrame("command.error")).resolves.toMatchObject({ + requestId: "cmd_unavailable", + message: expect.stringMatching(/unavailable/), + }); + harness.host.close(); + }); + + it("aborts finite commands when their sidecar generation closes", async () => { + let observedSignal: AbortSignal | undefined; + let startedResolve: (() => void) | undefined; + const started = new Promise((resolve) => { + startedResolve = resolve; + }); + const commandExecutor: ShellExecutor = async (request, _scope, signal) => { + observedSignal = signal; + startedResolve?.(); + await new Promise((resolve) => + signal?.addEventListener("abort", () => resolve(), { once: true }) + ); + return { + cmd: request.cmd, + args: request.args, + cwd: request.cwd, + exit_code: null, + signal: "SIGTERM", + stdout: "", + stderr: "", + duration_ms: 1, + timed_out: false, + truncated: false, + }; + }; + const harness = createHarness({ + fetch: async () => new Response(), + commandExecutor, + }); + await harness.start(); + await harness.sidecar.write({ + v: 1, + type: "command.request", + requestId: "cmd_lifecycle", + command: "sleep", + args: ["60"], + workdir: "/workspace", + workspaceRoot: "/workspace", + }); + await started; + + harness.host.close(); + + expect(observedSignal?.aborted).toBe(true); + }); + + it("revokes a finite command when the sidecar cancels its tool call", async () => { + let observedSignal: AbortSignal | undefined; + let startedResolve: (() => void) | undefined; + let cleanupResolve: (() => void) | undefined; + const started = new Promise((resolve) => { + startedResolve = resolve; + }); + const cleanup = new Promise((resolve) => { + cleanupResolve = resolve; + }); + const commandExecutor: ShellExecutor = async (request, _scope, signal) => { + observedSignal = signal; + startedResolve?.(); + await new Promise((resolve) => + signal?.addEventListener("abort", () => resolve(), { once: true }) + ); + await cleanup; + return { + cmd: request.cmd, + args: request.args, + cwd: request.cwd, + exit_code: null, + signal: "SIGTERM", + stdout: "", + stderr: "", + duration_ms: 1, + timed_out: false, + truncated: false, + }; + }; + const harness = createHarness({ + fetch: async () => new Response(), + commandExecutor, + }); + await harness.start(); + await harness.sidecar.write({ + v: 1, + type: "command.request", + requestId: "cmd_cancelled", + command: "sleep", + args: ["60"], + workdir: "/workspace", + workspaceRoot: "/workspace", + }); + await started; + + await harness.sidecar.write({ + v: 1, + type: "command.abort", + requestId: "cmd_cancelled", + reason: "caller aborted", + }); + await new Promise((resolve) => setImmediate(resolve)); + + expect(observedSignal?.aborted).toBe(true); + expect( + harness.frames.some( + (frame) => + frame.type === "command.aborted" && + frame.requestId === "cmd_cancelled" + ) + ).toBe(false); + cleanupResolve?.(); + await expect(harness.untilFrame("command.aborted")).resolves.toEqual({ + v: 1, + type: "command.aborted", + requestId: "cmd_cancelled", + }); + expect( + harness.frames.some( + (frame) => + "requestId" in frame && + frame.requestId === "cmd_cancelled" && + (frame.type === "command.output" || + frame.type === "command.end" || + frame.type === "command.error") + ) + ).toBe(false); + expect(harness.fatal).not.toHaveBeenCalled(); + harness.host.close(); + }); + + it("acknowledges abort that races a rejecting command's error write", async () => { + let errorWriteStartedResolve: (() => void) | undefined; + let releaseErrorWrite: (() => void) | undefined; + const errorWriteStarted = new Promise((resolve) => { + errorWriteStartedResolve = resolve; + }); + const errorWrite = new Promise((resolve) => { + releaseErrorWrite = resolve; + }); + const harness = createHarness({ + fetch: async () => new Response(), + commandExecutor: async () => { + throw new Error("worker rejected"); + }, + }); + const commandError = vi.spyOn( + harness.host as unknown as { + commandError(requestId: string, message: string): Promise; + }, + "commandError" + ); + commandError.mockImplementation(async () => { + errorWriteStartedResolve?.(); + await errorWrite; + }); + await harness.start(); + await harness.sidecar.write({ + v: 1, + type: "command.request", + requestId: "cmd_error_abort_race", + command: "false", + args: [], + workdir: "/workspace", + workspaceRoot: "/workspace", + }); + await errorWriteStarted; + + await harness.sidecar.write({ + v: 1, + type: "command.abort", + requestId: "cmd_error_abort_race", + reason: "caller aborted", + }); + releaseErrorWrite?.(); + + await expect(harness.untilFrame("command.aborted")).resolves.toEqual({ + v: 1, + type: "command.aborted", + requestId: "cmd_error_abort_race", + }); + expect(commandError).toHaveBeenCalledTimes(1); + expect(harness.fatal).not.toHaveBeenCalled(); + harness.host.close(); + }); + it("aborts and drains a late response after interactive proxy auth", async () => { const request = Object.assign(new EventEmitter(), { abort: vi.fn<() => void>(), @@ -831,6 +1118,7 @@ describe("AgentNetworkHost", () => { function createHarness(adapter: { fetch: (url: string, init: RequestInit) => Promise; maxBufferedRequestBodyBytes?: number; + commandExecutor?: ShellExecutor; }) { const sidecarOutput = new PassThrough(); const sidecarInput = new PassThrough(); @@ -842,7 +1130,8 @@ function createHarness(adapter: { authority, adapter, fatal, - adapter.maxBufferedRequestBodyBytes + adapter.maxBufferedRequestBodyBytes, + adapter.commandExecutor ); const sidecar = new AgentSidecarChannel.Writer(sidecarOutput); const decoder = new AgentSidecarChannel.Decoder(); diff --git a/desktop/src/main/agent-network-host.ts b/desktop/src/main/agent-network-host.ts index 7cf63cce4..d0f65bb34 100644 --- a/desktop/src/main/agent-network-host.ts +++ b/desktop/src/main/agent-network-host.ts @@ -1,6 +1,11 @@ import { isIP } from "node:net"; import { Readable, type Writable } from "node:stream"; import { net, session, type IncomingMessage, type Session } from "electron"; +import type { + ShellExecutionScope, + ShellExecutor, + ShellRunResult, +} from "@grida/daemon/server"; import { AgentNetworkPolicy } from "../agent-network-policy"; import { AgentSidecarChannel } from "../agent-sidecar-channel"; import { AgentNetworkAuthority } from "./agent-network-authority"; @@ -16,6 +21,8 @@ const MAX_RESPONSE_BODY_BYTES = 512 * 1024 * 1024; const RESPONSE_CHUNK_BYTES = 48 * 1024; const MAX_REDIRECTS = 5; const MAX_TERMINAL_RESPONSE_TOMBSTONES = 128; +const MAX_CONCURRENT_COMMANDS = 4; +const MAX_TERMINAL_COMMAND_TOMBSTONES = 32; type NetworkAdapter = Readonly<{ fetch: (url: string, init: RequestInit) => Promise; @@ -44,12 +51,14 @@ type ActiveResponse = { }; /** - * GRIDA-SEC-004 — Electron-main implementation of trusted provider HTTP. + * GRIDA-SEC-004 — Electron-main implementation of trusted sidecar + * capabilities. * - * It owns the system-network stack and grant validation while the sidecar - * remains inside its whole-process SRT sandbox. There is no listener, renderer - * IPC method, environment token, or general-purpose proxy: only the inherited - * stdio pair can speak the strict provider protocol. + * It owns provider HTTP grant validation and delegates finite commands to the + * per-command sandbox host while the sidecar remains inside its coarse outer + * SRT sandbox. There is no listener, renderer IPC method, environment token, + * or general-purpose proxy: only the inherited stdio pair can speak the strict + * capability protocol. */ export class AgentNetworkHost { private readonly decoder = new AgentSidecarChannel.Decoder(); @@ -58,6 +67,9 @@ export class AgentNetworkHost { private readonly active = new Map(); private readonly requestControllers = new Map(); private readonly terminalResponses = new Set(); + private readonly activeCommands = new Map(); + private readonly abortingCommands = new Set(); + private readonly terminalCommands = new Set(); private bufferedRequestBodyBytes = 0; // Together with `incoming.size`, this gives O(1) accepted/discarded // admission accounting. Both classes are independently capped, so hostile @@ -81,7 +93,8 @@ export class AgentNetworkHost { private readonly authority: AgentNetworkAuthority, private readonly adapter: NetworkAdapter, private readonly onFatal: (error: Error) => void, - private readonly maxBufferedRequestBodyBytes = MAX_BUFFERED_REQUEST_BODY_BYTES + private readonly maxBufferedRequestBodyBytes = MAX_BUFFERED_REQUEST_BODY_BYTES, + private readonly commandExecutor?: ShellExecutor ) { this.writer = new AgentSidecarChannel.Writer(output); input.on("data", (chunk: Buffer | string) => { @@ -103,6 +116,7 @@ export class AgentNetworkHost { output: Writable; authority: AgentNetworkAuthority; onFatal: (error: Error) => void; + commandExecutor?: ShellExecutor; }): Promise { const networkSession = session.fromPartition( // No `persist:` prefix: one app-lifetime in-memory BrowserContext reused @@ -117,7 +131,9 @@ export class AgentNetworkHost { args.output, args.authority, AgentNetworkHost.electronAdapter(networkSession), - args.onFatal + args.onFatal, + MAX_BUFFERED_REQUEST_BODY_BYTES, + args.commandExecutor ); } @@ -266,11 +282,184 @@ export class AgentNetworkHost { case "response.credit": this.onResponseCredit(frame); return; + case "command.request": + this.onCommandRequest(frame); + return; + case "command.abort": + this.onCommandAbort(frame.requestId); + return; default: throw new Error(`unexpected sidecar frame: ${frame.type}`); } } + private onCommandRequest( + frame: AgentSidecarChannel.CommandRequestFrame + ): void { + if ( + this.activeCommands.has(frame.requestId) || + this.terminalCommands.has(frame.requestId) + ) { + throw new Error("duplicate host command request"); + } + if (!this.commandExecutor) { + this.rememberTerminalCommand(frame.requestId); + void this.commandError( + frame.requestId, + "host command execution is unavailable" + ).catch((error) => this.fail(asError(error))); + return; + } + if (this.activeCommands.size >= MAX_CONCURRENT_COMMANDS) { + this.rememberTerminalCommand(frame.requestId); + void this.commandError( + frame.requestId, + "host command execution capacity is temporarily exhausted" + ).catch((error) => this.fail(asError(error))); + return; + } + + const controller = new AbortController(); + this.activeCommands.set(frame.requestId, controller); + const scope: ShellExecutionScope = Object.freeze({ + workspace_root: frame.workspaceRoot, + scratch_root: frame.scratchDir, + // These roots are host facts and therefore are deliberately absent from + // the untrusted frame. AgentCommandHost binds them from constructor state. + protected_read_roots: [], + }); + void this.executeCommand(frame, scope, controller).catch((error) => + this.fail(asError(error)) + ); + } + + private onCommandAbort(requestId: string): void { + const controller = this.activeCommands.get(requestId); + if (!controller) { + if (this.terminalCommands.has(requestId)) { + // The result won the race, so cleanup is already complete. Acknowledge + // the caller's abort immediately instead of leaving its promise open. + void this.commandAborted(requestId).catch((error) => + this.fail(asError(error)) + ); + return; + } + throw new Error("command abort names an unknown request"); + } + if (this.abortingCommands.has(requestId)) { + throw new Error("duplicate command abort"); + } + // Keep the request active until the executor returns: for AgentCommandHost, + // that means its finally block has removed command temp and released SRT's + // per-command bookkeeping. Only then may command.aborted settle the tool. + this.abortingCommands.add(requestId); + controller.abort(); + } + + private async executeCommand( + frame: AgentSidecarChannel.CommandRequestFrame, + scope: ShellExecutionScope, + controller: AbortController + ): Promise { + try { + let result: ShellRunResult; + try { + result = await this.commandExecutor!( + { + cmd: frame.command, + args: [...frame.args], + cwd: frame.workdir, + timeout_ms: frame.timeoutMs, + }, + scope, + controller.signal + ); + } catch { + if (this.closed) return; + if (this.abortingCommands.has(frame.requestId)) { + await this.commandAborted(frame.requestId); + return; + } + await this.commandError( + frame.requestId, + "host command execution was denied or failed" + ); + // command.abort can arrive while the terminal error write is + // backpressured. The sidecar then ignores that raced error and waits + // for the authoritative post-cleanup acknowledgement. + if (this.abortingCommands.has(frame.requestId)) { + await this.commandAborted(frame.requestId); + } + return; + } + if (this.closed) return; + if (this.abortingCommands.has(frame.requestId)) { + await this.commandAborted(frame.requestId); + return; + } + await this.sendCommandResult(frame.requestId, result); + if (this.closed) return; + if (this.abortingCommands.has(frame.requestId)) { + await this.commandAborted(frame.requestId); + } + } finally { + this.activeCommands.delete(frame.requestId); + this.abortingCommands.delete(frame.requestId); + this.rememberTerminalCommand(frame.requestId); + } + } + + private async sendCommandResult( + requestId: string, + result: ShellRunResult + ): Promise { + let sequence = 0; + let remainingBytes = AgentSidecarChannel.MAX_COMMAND_OUTPUT_BYTES; + let hostTruncated = false; + for (const [stream, value] of [ + ["stdout", result.stdout], + ["stderr", result.stderr], + ] as const) { + const bounded = utf8Prefix(value, remainingBytes); + remainingBytes -= bounded.bytes; + hostTruncated ||= bounded.truncated; + for (const data of utf8Chunks( + bounded.value, + AgentSidecarChannel.MAX_COMMAND_OUTPUT_CHUNK_BYTES + )) { + if (this.abortingCommands.has(requestId)) return; + await this.send({ + v: 1, + type: "command.output", + requestId, + stream, + sequence, + data, + }); + sequence += 1; + } + } + if (this.abortingCommands.has(requestId)) return; + await this.send({ + v: 1, + type: "command.end", + requestId, + sequence, + exitCode: + result.exit_code === null || Number.isSafeInteger(result.exit_code) + ? result.exit_code + : -1, + signal: + result.signal === null ? null : result.signal.slice(0, 64) || null, + timedOut: result.timed_out, + truncated: result.truncated || hostTruncated, + durationMs: + Number.isSafeInteger(result.duration_ms) && result.duration_ms >= 0 + ? result.duration_ms + : 0, + }); + } + private onRequestStart(frame: AgentSidecarChannel.RequestStartFrame): void { if ( this.incoming.has(frame.requestId) || @@ -774,6 +963,28 @@ export class AgentNetworkHost { await this.send({ v: 1, type: "response.error", requestId, code, message }); } + private async commandError( + requestId: string, + message: string + ): Promise { + await this.send({ v: 1, type: "command.error", requestId, message }); + } + + private async commandAborted(requestId: string): Promise { + await this.send({ v: 1, type: "command.aborted", requestId }); + } + + private rememberTerminalCommand(requestId: string): void { + this.terminalCommands.add(requestId); + while (this.terminalCommands.size > MAX_TERMINAL_COMMAND_TOMBSTONES) { + const oldest = this.terminalCommands.values().next().value as + | string + | undefined; + if (oldest === undefined) break; + this.terminalCommands.delete(oldest); + } + } + private async send( frame: AgentSidecarChannel.HostToSidecarFrame ): Promise { @@ -792,6 +1003,7 @@ export class AgentNetworkHost { for (const response of this.active.values()) { void response.reader.cancel().catch(() => undefined); } + for (const controller of this.activeCommands.values()) controller.abort(); for (const request of this.incoming.values()) clearTimeout(request.timeout); this.bufferedRequestBodyBytes = 0; this.discardedIncomingCount = 0; @@ -799,6 +1011,9 @@ export class AgentNetworkHost { this.active.clear(); this.requestControllers.clear(); this.terminalResponses.clear(); + this.activeCommands.clear(); + this.abortingCommands.clear(); + this.terminalCommands.clear(); for (const pending of this.grantAcks.values()) { clearTimeout(pending.timeout); pending.reject(error); @@ -808,6 +1023,48 @@ export class AgentNetworkHost { } } +function utf8Prefix( + value: string, + maxBytes: number +): { value: string; bytes: number; truncated: boolean } { + if (maxBytes <= 0) { + return { value: "", bytes: 0, truncated: value.length > 0 }; + } + let bytes = 0; + let end = 0; + for (const scalar of value) { + const scalarBytes = Buffer.byteLength(scalar, "utf8"); + if (bytes + scalarBytes > maxBytes) { + return { + value: value.slice(0, end), + bytes, + truncated: true, + }; + } + bytes += scalarBytes; + end += scalar.length; + } + return { value, bytes, truncated: false }; +} + +function utf8Chunks(value: string, maxBytes: number): string[] { + const chunks: string[] = []; + let chunk = ""; + let bytes = 0; + for (const scalar of value) { + const scalarBytes = Buffer.byteLength(scalar, "utf8"); + if (bytes > 0 && bytes + scalarBytes > maxBytes) { + chunks.push(chunk); + chunk = ""; + bytes = 0; + } + chunk += scalar; + bytes += scalarBytes; + } + if (chunk.length > 0) chunks.push(chunk); + return chunks; +} + export function requestThroughSession( networkSession: Session, url: string, diff --git a/desktop/src/main/agent-sidecar-supervisor.ts b/desktop/src/main/agent-sidecar-supervisor.ts index 700fe0c2d..d12482215 100644 --- a/desktop/src/main/agent-sidecar-supervisor.ts +++ b/desktop/src/main/agent-sidecar-supervisor.ts @@ -48,6 +48,10 @@ import crypto from "node:crypto"; import fs from "node:fs"; import path from "node:path"; import { home } from "@grida/home"; +import { + defaultScratchBase, + prepareScratchAuthority, +} from "@grida/agent/server"; import { ensureInitialized, wrap, @@ -61,6 +65,7 @@ import { AgentNetworkAuthority } from "./agent-network-authority"; import { AgentNetworkHost } from "./agent-network-host"; import { AgentDaemonSocketHost } from "./agent-daemon-socket-host"; import { DesktopAgentSandboxPolicy } from "./agent-sandbox-policy"; +import { AgentCommandHost, sweepAgentCommandTemps } from "./agent-command-host"; export type AgentSidecarInfo = { /** Electron-main-owned TCP listener port (exact 127.0.0.1, ephemeral). */ @@ -85,6 +90,8 @@ export class AgentSidecarSupervisor { private isShuttingDown = false; private restartTimer: NodeJS.Timeout | null = null; private sandboxReady = false; + private scratchBase: string | null = null; + private commandHost: AgentCommandHost | null = null; private readonly networkAuthority = new AgentNetworkAuthority( new URL(EDITOR_BASE_URL).origin ); @@ -237,6 +244,19 @@ export class AgentSidecarSupervisor { */ private async initSandbox(): Promise { if (this.sandboxReady) return; + // Resolve temp in trusted Electron main before SRT rewrites the sidecar's + // TMPDIR to its shared compatibility directory. Session scratch must live + // outside those unconditional write defaults so a per-command deny of the + // shared base can carve back exactly one session. + this.scratchBase = defaultScratchBase( + this.user_data_path, + app.getPath("temp") + ); + // Establish the predictable temp authority before any listing/removal. + // This rejects a foreign-owned or symlinked base, then safely reclaims + // command-temp remnants left by a prior crash. + prepareScratchAuthority(this.scratchBase, this.user_data_path); + await sweepAgentCommandTemps(this.scratchBase); if (!isSupportedPlatform()) { // Windows or another unsupported platform. Log loudly and let // the spawn proceed unwrapped — the alternative is refusing @@ -282,6 +302,12 @@ export class AgentSidecarSupervisor { denyWrite: policy.filesystem.deny_write, }, }); + this.commandHost = new AgentCommandHost({ + scratchBase: this.scratchBase, + userData: this.user_data_path, + home: app.getPath("home"), + filesystemPolicy: policy.filesystem, + }); this.sandboxReady = true; } @@ -335,9 +361,13 @@ export class AgentSidecarSupervisor { const supportedSandbox = isSupportedPlatform(); const skillsRoot = this.skillsRootPath(); + if (!this.scratchBase) { + throw new Error("agent scratch authority was not initialized"); + } const args = [ scriptPath, `--user-data=${this.user_data_path}`, + `--scratch-base=${this.scratchBase}`, // Host-bundled skills dir (repo-root `skills/`) — the built-in skills the // agent advertises + loads on demand. Read-only; omitted if unresolved. ...(skillsRoot ? [`--skills-root=${skillsRoot}`] : []), @@ -347,9 +377,9 @@ export class AgentSidecarSupervisor { // ready app, which holds at spawn time (post `app.whenReady`). `--projects-root=${path.join(app.getPath("documents"), "Grida")}`, `--editor-base-url=${EDITOR_BASE_URL}`, - // GRIDA-SEC-004 — tell the sidecar whether srt wraps this spawn. Only - // then does it expose the `run_command` shell tool (fail-closed). On - // platforms srt can't wrap, this is "0" and the agent gets no shell. + // GRIDA-SEC-004 — attest the coarse outer SRT wrap. The sidecar exposes + // `run_command` only when this is true AND main injected the private + // per-command executor above. On unsupported platforms both are absent. `--sandbox-enforced=${supportedSandbox ? "1" : "0"}`, ]; let wrappedCmd: string | null = null; @@ -478,7 +508,8 @@ export class AgentSidecarSupervisor { input: child.stdout!, output: child.stdin!, authority: this.networkAuthority, - onFatal: (error) => failSpawnChannel("provider channel", error), + onFatal: (error) => failSpawnChannel("capability channel", error), + commandExecutor: this.commandHost?.shellExecutor, }); spawnNetworkHost = host; if (!this.isCurrentGeneration(child, generation) || resolved) { @@ -502,7 +533,7 @@ export class AgentSidecarSupervisor { resolve(this.info); })().catch((error) => { failSpawnChannel( - "provider channel startup", + "capability channel startup", error instanceof Error ? error : new Error(String(error)) ); }); diff --git a/desktop/src/main/sandbox/manager.ts b/desktop/src/main/sandbox/manager.ts index 1cc184237..2285e5568 100644 --- a/desktop/src/main/sandbox/manager.ts +++ b/desktop/src/main/sandbox/manager.ts @@ -69,6 +69,40 @@ export async function wrap(command: string): Promise { return await SandboxManager.wrapWithSandbox(command); } +/** + * Wrap one supervisor-owned raw command with a call-specific policy and return + * an argv descriptor suitable for `spawn(..., { shell: false })`. + * + * Unlike {@link wrap}, this is used for finite agent shell workers, not the + * long-lived sidecar. The caller supplies the exact workspace/session roots in + * `customConfig`; SRT therefore emits a fresh kernel profile for every command + * instead of inheriting the sidecar's coarse, all-session filesystem view. + */ +export async function wrapArgv( + command: string, + customConfig: Partial, + abortSignal?: AbortSignal +): Promise<{ argv: string[]; env: NodeJS.ProcessEnv }> { + if (!initialized) { + throw new Error( + "[agent-sidecar:srt] wrapArgv() called before ensureInitialized() - refusing to run an unwrapped command" + ); + } + return await SandboxManager.wrapWithSandboxArgv( + command, + undefined, + customConfig, + abortSignal + ); +} + +/** Release per-command Linux bind-mount placeholders after a wrapped worker + * exits. Safe and intentionally a no-op on macOS. */ +export function cleanupAfterCommand(): void { + if (!initialized) return; + SandboxManager.cleanupAfterCommand(); +} + /** * `before-quit` cleanup. Tears down srt's proxy servers; safe to * call even if `ensureInitialized` was never called (no-ops). diff --git a/desktop/src/preload-contract.test.ts b/desktop/src/preload-contract.test.ts index 8754f949b..c9558fa66 100644 --- a/desktop/src/preload-contract.test.ts +++ b/desktop/src/preload-contract.test.ts @@ -20,6 +20,9 @@ describe("Desktop preload agent seam", () => { expect(preloadSource).toContain("protocol: DESKTOP_BRIDGE_PROTOCOL"); expect(preloadSource).toContain("native:"); expect(preloadSource).toContain("scratch_seed_base64: true"); + expect(preloadSource).toContain( + 'scratch_binary_tools: process.platform !== "win32"' + ); expect(preloadSource).not.toContain("agentServer:"); }); diff --git a/desktop/src/preload.ts b/desktop/src/preload.ts index 7f259c7e1..4f15d4dc0 100644 --- a/desktop/src/preload.ts +++ b/desktop/src/preload.ts @@ -303,6 +303,9 @@ const bridge: DesktopBridge = { agent: { // This host accepts `{ path, base64 }` scratch seeds on agent runs. scratch_seed_base64: true, + // Windows intentionally withholds confined run_command; a scratch-only + // PDF/archive would therefore be a path the agent cannot operate on. + scratch_binary_tools: process.platform !== "win32", }, native: { host_apps: true, diff --git a/docs/wg/ai/agent/compositor.md b/docs/wg/ai/agent/compositor.md index d764ec08f..271d05b48 100644 --- a/docs/wg/ai/agent/compositor.md +++ b/docs/wg/ai/agent/compositor.md @@ -106,9 +106,18 @@ separate inputs: The generating rule is: **describe the available representations, evaluate the configured preferences in order, reject infeasible routes, -then materialize exactly one typed route**. User-interface cards are a -view of that result; their incidental fields MUST NOT be inspected later -to rediscover whether the resource was an attachment or a reference. +then materialize exactly one typed route**. A route may explicitly be +composite when one ingress body must serve two distinct consumers. For +example, a byte-backed raster can pair provider-native perception with a +byte-exact scratch copy for file operations. That is one declared +attachment route, not an attachment plus a source reference: scratch is +host materialization and grants no authority over the source location. +The two legs MUST carry an unambiguous shared identity or provider-part +index so duplicate names, transcoding, and asymmetric fallback cannot detach +the perceived image from its operable path. +User-interface cards are a view of the result; their incidental fields MUST +NOT be inspected later to rediscover whether the resource was an attachment +or a reference. Preference is configurable because different agent surfaces have different useful defaults. A design surface may prefer provider-native @@ -135,10 +144,21 @@ The following constraints override every preference configuration: - Scratch delivery is admitted against the final turn's aggregate byte, file, and path budget. Every seed source merged into that turn reserves capacity; an implementation SHOULD reject an impossible batch before reading all of - its bodies and MUST keep the user's draft when final preflight fails. -- One resource has one primary route. A compositor MUST NOT silently - duplicate it as both attachment and reference, or silently switch to a - semantically different route after materialization fails. + its bodies and MUST keep the user's draft when final preflight fails. Under + scarce capacity, resources with no surviving provider leg reserve scratch + before optional scratch twins that can still fall back to perception only. +- A compositor that retains raw operable twins before submit MUST enforce a + separate bounded draft-memory budget. Capacity under that stable bound may be + reallocated when draft resources or turn reservations change; once it is + exhausted, later rasters use an explicit provider-only fallback rather than + accumulating unbounded retained raw byte bodies. +- One resource has one primary route. A compositor MUST NOT silently duplicate + it as both attachment and reference, or silently switch to a semantically + different route after materialization fails. A declared provider-and-scratch + raster route counts as one route. When its scratch leg is unavailable or + cannot fit the aggregate budget, provider-only is the fallback; when provider + perception is unavailable, scratch-only is the fallback. The selected + fallback MUST remain explicit in the routing result. This separation keeps source provenance useful without making source labels magical. A drop can carry evidence that a trusted host may mint a @@ -441,15 +461,15 @@ entity into the compositor. The compositor MUST handle, at minimum, the following attachment classes: -| Class | Default treatment | -| -------------- | ------------------------------------------------------------------------------------------------------------------- | -| Text | Inlined as text part if small (< host threshold); path-ref otherwise. | -| Image | Multi-modal part if the provider supports it; descriptor + a [`view_image`](./vision.md) perception path otherwise. | -| PDF | Multi-modal part if supported; PDF-to-text tool route otherwise. | -| Audio | Multi-modal where supported; transcription tool route otherwise. | -| Video | Multi-modal where supported; descriptor + frame-extractor tool otherwise. | -| Binary unknown | Always a path-ref or descriptor, never inlined. The model sees a name, mime, size. | -| Directory | Always a `directory-ref`; never recursively copied, archived, or staged into scratch by implication. | +| Class | Default treatment | +| -------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------- | +| Text | Inlined as text part if small (< host threshold); path-ref otherwise. | +| Image | Multi-modal part if supported, plus a live scratch descriptor when tool-visible scratch and its budget permit; [`view_image`](./vision.md) otherwise. | +| PDF | Multi-modal part if supported; PDF-to-text tool route otherwise. | +| Audio | Multi-modal where supported; transcription tool route otherwise. | +| Video | Multi-modal where supported; descriptor + frame-extractor tool otherwise. | +| Binary unknown | Always a path-ref or descriptor, never inlined. The model sees a name, mime, size. | +| Directory | Always a `directory-ref`; never recursively copied, archived, or staged into scratch by implication. | The agent always has a fallback path. Any attachment the model cannot read directly SHOULD still be reachable through `read` or @@ -503,7 +523,7 @@ Per part type: | `text` | Inline text | `{ type: "text", text }` | Text, verbatim. | | `file-ref` | Chip / link with file name and optional `:lines` | `{ type: "file-ref", ref }` | A tool-addressable descriptor by default. A range-carrying ref lowers only the selected range; a deliberately resolved image MAY become a provider-native image block. The model never sees the literal `@path`. | | `directory-ref` | Folder chip / link | `{ type: "directory-ref", ref }` | A compact descriptor naming the tool-addressable directory scope. Descendant contents are never inlined by lowering. | -| `file-attachment` | Thumbnail or file chip | `{ type: "file-attachment", data?, url?, mime, name, … }` | Provider-native multi-modal block when the delivery encoder and provider declare support for the MIME and representation; descriptor placeholder otherwise. | +| `file-attachment` | Thumbnail or file chip | `{ type: "file-attachment", data?, url?, mime, name, … }` | The declared route's provider-native block, tool-addressable staged-copy descriptor, or explicit composite of both. Either leg may appear alone only when the routing result selected that fallback. | | `command` | Resolved chip / palette result | `{ type: "command", id, args }` | **Nothing** when the command is host-action-only. Otherwise the command's result lowered as text/file parts (e.g. `/read foo.ts` → the file's contents). The literal `/foo` is never sent. | | `mention` (skill) | Chip / pill in the input | `{ type: "mention", target }` | **Nothing** in the user message; the [skill body](./skills.md) loads via the normal `skill` tool flow. | | `mention` (file) | Same | Same | Lowered as a `file-ref` (and from there per the file-ref row). | @@ -615,9 +635,10 @@ A conforming compositor MUST: documents, recent actions) as `editor-context` parts rather than inlining it into the user's text. - Carry a schema version on user message metadata. -- Lower attachments to provider-native multimodal blocks at the - provider boundary; fall back to descriptor parts when the model - has no native support. +- Lower attachments according to the declared route: a provider-native + multimodal block, a tool-addressable staged-copy descriptor, or an explicit + composite of both. Apply only capability-governed fallbacks recorded by the + routing result, and correlate the two legs of a composite explicitly. - Persist user messages **before** they reach the model. ## What this guide does not specify diff --git a/docs/wg/ai/agent/scratch.md b/docs/wg/ai/agent/scratch.md index 752f19cde..de81a1c8f 100644 --- a/docs/wg/ai/agent/scratch.md +++ b/docs/wg/ai/agent/scratch.md @@ -98,6 +98,13 @@ scratch lives; it receives a handle the same way it receives its working root. A consequence: scratch isolation is structural. One session cannot reach another's working files because each is handed only its own. +Giving the model only one path is not sufficient when an ambient-path tool +(such as a shell or interpreter) runs inside a host that can see the shared +scratch parent. That tool's execution boundary MUST deny the shared parent and +grant back only the current session's subtree, or provide an equivalent +capability-safe mount/handle. Session isolation is an enforced authority fact, +not prompt secrecy or cwd validation. + ### S2 — Ephemeral; durability only by promotion Scratch MUST have a **bounded lifetime** and the agent MUST NOT rely on it as @@ -140,6 +147,24 @@ authority over a workspace file. The edit operation's own current-content match still applies wherever that operation is used; it is intrinsic conflict detection, not a workspace-authorization rule. +## Input staging + +An attachment-routing policy MAY select a route that gives tool operability to +byte-backed input with no authorized live reference. Once selected, the host +MUST materialize a working copy in scratch or select an explicit fallback +allowed by the routing result. This is ingress staging, not promotion: the +resulting descriptor names the host-owned scratch copy and grants no authority +over any source path. A provider-native representation of the same attachment +MAY coexist for immediate perception; the +[compositor's routing policy](./compositor.md#resource-routing-policy) owns that +choice and its fallbacks. + +The host MUST admit staged inputs against the turn's aggregate byte, file, and +path budgets before materialization. The staged copy has scratch's ephemeral +lifetime; if durable replay depends on those bytes, the host MUST retain or +rematerialize them outside scratch rather than treating the descriptor as +durable storage. + ## Lifecycle - **Creation** is on demand — at session start or first use. An idle session @@ -148,6 +173,10 @@ detection, not a workspace-authorization rule. retention window, or on a coarser sweep. The invariant is only that the lifetime is bounded and that S2 holds — nothing of value is lost, because value lives outside scratch by promotion. +- **Cleanup authority** is verified before traversal. A predictable shared + base MUST fail closed when it is a symlink, is owned by another local + principal, or can be replaced through an unsafe parent. A sweep MUST NOT + follow session-entry symlinks into another tree; it removes the link itself. - **Promotion** is explicit and agent-driven: there is no implicit "scratch is saved" step. To keep an artifact, the agent moves it. diff --git a/docs/wg/ai/agent/vision.md b/docs/wg/ai/agent/vision.md index dcb643a8c..a98e38bc1 100644 --- a/docs/wg/ai/agent/vision.md +++ b/docs/wg/ai/agent/vision.md @@ -107,9 +107,10 @@ injects, not a new tool and not a renamed one. > attachments the model _cannot read at all_ (a `.psd`, a `.zip`) and the > routes that make them useful. Visual perception is about sources the model > _could_ read as text but where the agent wants the **rendering** instead -> (an svg, a screenshot). A raster bitmap is the overlap: binary treats a -> pasted image as a native-multimodal attachment; this page is how the agent -> reaches one that lives at a path, by choice, through a tool. +> (an svg, a screenshot). A raster bitmap is the overlap: binary can deliver +> it natively for immediate perception, while this page defines how the agent +> re-perceives the same image when ingress also supplied or materialized a +> live reference. ## Result-to-image lowering @@ -173,15 +174,19 @@ policy bounds this. ### Asymmetry: only re-viewable perceptions are auto-evicted -A tool-produced perception is **re-viewable** — there is a reference and a -tool to call again. An inline image the user pasted into a message is -**not**: there is no path to re-fetch it, so eliding it is lossy and -irreversible. Retention therefore auto-evicts re-viewable perceptions but -leaves user-attached images in place (they are already bounded by the -attachment-storage policy in [`compositor`](./compositor.md#attachment-storage)). -The unifying rule is **evict only what perception can restore** — not "evict -all images." A host that later gives pasted images a re-view reference can -bring them under the same policy. +A tool-produced perception is **re-viewable** when there is still a live +reference and a tool that can resolve it. A user-attached image has the same +status only when the host materialized its bytes at ingress and paired the +inline media with that reference. A pathless attachment is not re-viewable, so +eliding it is lossy and irreversible. + +Retention therefore auto-evicts only images whose references remain live. A +session-scratch path qualifies only while the host can prove, at lowering time, +that it remains live and correlated with the attachment; a declared retention +window alone is not proof. When that liveness cannot be established, the host +must either rematerialize the attachment or treat it as non-re-viewable. The +unifying rule is **evict only what perception can restore** — not "evict all +images." ## Implementor checklist @@ -199,8 +204,8 @@ A conforming implementation SHOULD: Anthropic-native); stage-and-reattach on Chat Completions / openai-compatible. Verify perception **end-to-end through a real provider**, not just the media-block shape. -- Evict stale **re-viewable** perceptions to a naming descriptor; leave - non-re-viewable images (pasted attachments) intact. +- Evict stale **re-viewable** perceptions to a naming descriptor; leave any + image without a live, correlated re-view reference intact. - Declare only a read capability for bitmap perception; gate the rendered (svg / text) path behind a render capability when it lands. diff --git a/docs/wg/desktop/agent-security.md b/docs/wg/desktop/agent-security.md index d70b12269..83986a4ce 100644 --- a/docs/wg/desktop/agent-security.md +++ b/docs/wg/desktop/agent-security.md @@ -142,9 +142,13 @@ The `run_command` agent tool spawns child processes through the shell runner (`packages/grida-daemon/src/shell/runner.ts`) with `shell: false`. There is **no command allowlist** — a per-session **permission mode** (`protocol/mode.ts`) governs the surface: `accept-edits` (default — read-only -inspection commands auto-run; a mutating/executing command **pauses for a -supervised Allow/Deny prompt** and runs only on approval) or `auto` (every -command runs; the OS sandbox is the guard, the semantic classifier deferred). +inspection commands auto-run; so does a narrow, non-overwriting two-path +`cp`/`mv` whose source and destination stay canonically inside session scratch; +every other mutating/executing command **pauses for a supervised Allow/Deny +prompt** and runs only on approval) or `auto` (every command runs; the OS +sandbox is the guard, the semantic classifier deferred). Scratch-local +pre-authorization rejects flags, symlinks, overwrites, and paths outside +scratch, so promotion into the workspace remains supervised. The supervised gate is the AI SDK's native `needsApproval` on the tool (`tools/run-command.ts`), wired from the mode at `workspace-agent-bindings.ts` — not the command backend, which only runs an already-cleared call. The @@ -153,13 +157,29 @@ explicit `approval_answer` body field (the host owns message state, so the answe is not smuggled in a client-mutated message) and `store.answerApproval` (via `run-input.ts` `applyApprovalAnswer`) flips a part to `approval-responded` only when it was a real pending approval, so the renderer can answer but never forge a -call. Two -structural gates hold in every mode: the cwd-must-be-inside-an-opened-workspace -check and the in-process secret-arg containment check (below); the fs-edit tools -additionally refuse no-clobber paths (`fs/scope.ts`). The OS sandbox confines the -whole sidecar; the full per-command fs/net sub-policy that would constrain each -spawned child (the kernel-level finish of the secret-dir guard) is deferred — see -[Desktop authority binding / raw execution and extensions](https://github.com/gridaco/grida/blob/main/desktop/docs/agent-authority.md#raw-execution-and-extensions). +call. Two structural gates hold in every mode: cwd must be inside this +session's exact workspace or scratch root, and explicit secret-root args are +rejected before host execution; the fs-edit tools additionally refuse +no-clobber paths (`fs/scope.ts`). + +The sidecar never raw-spawns a model command. It sends one bounded request over +its inherited private capability channel; Electron main canonicalizes the +exact roots and asks SRT for a fresh kernel profile for that process tree. The +profile denies the shared scratch parent and host-secret directory, re-allows +only this session's scratch, grants writes only to that scratch, the exact +workspace, and a private command temp directory, and grants no direct network +destination or local binding. The whole-sidecar profile remains a coarse +backstop; per-command confinement is the command boundary. See the +[Desktop authority binding](https://github.com/gridaco/grida/blob/main/desktop/docs/agent-authority.md). + +Cancellation is also host-acknowledged. `command.abort` does not settle the +tool immediately: main first waits for the confined executor to return, remove +its private temp, and release per-command SRT state, then replies +`command.aborted`. The agent runtime holds turn settlement through that +acknowledgement and through the model pump consuming the aborted tool result, +so another turn or session deletion cannot overlap cleanup. POSIX process-group +termination reclaims ordinary descendants; a macOS process that deliberately +escapes with `setsid(2)` remains outside that hard-revocation claim. **Secret-dir containment — the srt / in-process split.** There are two classes of secret on disk, owned by two different gates: @@ -169,23 +189,14 @@ classes of secret on disk, owned by two different gates: there, so a kernel-level deny is safe. - **The agent host's own secret dir** — its `userData`, where BYOK `auth.json`, `workspaces.json`, `recent.json`, and the sessions db live — - is **not** in srt `deny_read`. srt confines the whole sidecar including the - host process, and the host process must read `auth.json` for provider - calls. Denying it at the kernel level would break host auth. Instead the - shell _child_ is kept out of it in-process: `validateShellRequest` rejects - any command arg that resolves (after realpath of the nearest existing - ancestor, mirroring the cwd discipline so a symlink can't bypass it) inside - that protected root, threaded down from the runtime. - -This is the responsibility-and-reconciliation rule for secret reads: srt owns -HOME secrets at the kernel; the in-process runner owns the host's own -`userData`. **Caveat (`auto`):** the in-process arg check only inspects -top-level argv, so an interpreter/shell reachable in `auto` (`bash -c`, -`python3 -c`) can read `userData` by a computed path. Closing that for the -shell child needs the kernel-level per-call `deny_read` (the deferred -sub-policy). Desktop's empty direct external allowlist prevents that child -from exfiltrating it over the network; local disclosure within the contained -process tree remains the gap. + is **not** in the whole-sidecar `deny_read`, because that process must read + provider credentials. Electron main executes finite commands separately, so + their per-command profiles do deny `userData` at the kernel. The argv check + remains defense in depth; an interpreter-computed path is denied as well. + +The responsibility split is therefore: the outer profile protects HOME +secrets from the entire sidecar, while each finite-command profile additionally +protects the sidecar-owned secret directory and every sibling scratch root. **`auto` is informed-consent, not a guarantee.** `auto` removes command-identity gating; the sandbox still bounds the blast radius (writable @@ -193,7 +204,8 @@ roots and no direct external networking) but does not judge intent — an injected or confused agent can read broadly and run anything within those bounds. Intent judgment is the [watchdog](../ai/agent/foundations.md#watchdog) layer, deferred. -`auto` is opt-in; the default `accept-edits` keeps a read-only-only shell. +`auto` is opt-in; the default `accept-edits` requires approval for mutations +except the narrow scratch-local copy/move operation described above. ## Secrets discipline diff --git a/editor/kits/agent-chat/group-parts.ts b/editor/kits/agent-chat/group-parts.ts index 412b16280..ec1b90c5f 100644 --- a/editor/kits/agent-chat/group-parts.ts +++ b/editor/kits/agent-chat/group-parts.ts @@ -8,12 +8,13 @@ import type { ChatMessage, ToolCallEntry } from "@/lib/agent-chat"; * dropped. Reasoning tokens remain transport/session data, but they are not * user-facing transcript content. * - * Note on `file` parts: user-message images render inline via a dedicated - * branch in `message.tsx` (perceive-only attachments), NOT through this - * grouping. Assistant `file` parts are intentionally dropped here — the agent - * produces none in the current scope. If assistant-emitted files ever need - * rendering, promote `file` to a first-class `RenderGroup` rather than adding - * another ad-hoc branch. + * Note on `file` parts: provider-native user-message images render inline via + * a dedicated branch in `message.tsx`, NOT through this grouping. Their + * optional operable scratch descriptors remain ordinary context parts. + * Assistant `file` parts are intentionally dropped here — the agent produces + * none in the current scope. If assistant-emitted files ever need rendering, + * promote `file` to a first-class `RenderGroup` rather than adding another + * ad-hoc branch. */ export type RenderGroup = | { type: "text"; key: string; text: string } diff --git a/editor/kits/agent-chat/message.tsx b/editor/kits/agent-chat/message.tsx index 3e315e5b4..676a1f061 100644 --- a/editor/kits/agent-chat/message.tsx +++ b/editor/kits/agent-chat/message.tsx @@ -157,9 +157,10 @@ export function ChatMessageView({ .filter(isTextUIPart) .map((part) => part.text) .join(""); - // Inline image attachments the user pasted/dropped (perceive-only `file` - // parts). Rendered as thumbnails in the bubble so the sent message mirrors - // what the model received. + // Provider-native image parts the user pasted/dropped. Rendered as + // thumbnails in the bubble so the sent message mirrors what the model + // perceived; an operable scratch twin, when present, is a separate context + // part. const images = message.parts.filter( (part): part is FileUIPart => isFileUIPart(part) && part.mediaType.startsWith("image/") diff --git a/editor/lib/agent-chat/build-agent-send.test.ts b/editor/lib/agent-chat/build-agent-send.test.ts index a3fddb0dd..25abf3bce 100644 --- a/editor/lib/agent-chat/build-agent-send.test.ts +++ b/editor/lib/agent-chat/build-agent-send.test.ts @@ -215,35 +215,48 @@ describe("buildAgentSend — context token parts (WG compositor.md §templating) contexts: buildTemplateContext({ title: "Pitch", slides: 4 }), }); - send("inspect it", undefined, { - scratchSeed: [{ path: "upload-report.pdf", base64: "AQID" }], - contexts: [ + send( + "inspect it", + [ { - type: USER_FILE_ATTACHMENTS, - data: { - location: "scratch", - files: [ - { - name: "Report.pdf", - mime: "application/pdf", - size: 3, - path: "upload-report.pdf", - }, - ], - }, + type: "file", + filename: "preview.png", + mediaType: "image/png", + url: "data:image/png;base64,AAAA", }, ], - }); + { + scratchSeed: [{ path: "upload-report.pdf", base64: "AQID" }], + contexts: [ + { + type: USER_FILE_ATTACHMENTS, + data: { + location: "scratch", + files: [ + { + name: "Report.pdf", + mime: "application/pdf", + size: 3, + path: "upload-report.pdf", + }, + ], + }, + }, + ], + } + ); const message = sendMessage.mock.calls[0][0] as { role: string; - parts: Array<{ type: string }>; + parts: Array<{ type: string; filename?: string }>; }; expect(message.parts.map((part) => part.type)).toEqual([ "text", + "file", USER_TEMPLATE_SELECTION, USER_FILE_ATTACHMENTS, ]); + expect(message.parts[1]).toMatchObject({ filename: "preview.png" }); expect(sendMessage.mock.calls[0][1]?.body?.scratch_seed).toEqual([ { path: "template.canvas", text: "{}" }, { path: "upload-report.pdf", base64: "AQID" }, diff --git a/editor/lib/agent-chat/build-agent-send.ts b/editor/lib/agent-chat/build-agent-send.ts index 9e6ff5dba..e59305c48 100644 --- a/editor/lib/agent-chat/build-agent-send.ts +++ b/editor/lib/agent-chat/build-agent-send.ts @@ -4,8 +4,8 @@ * * Both surfaces (the workspace `agent-pane.tsx` and the standalone-doc * `ai-sidebar/chat.tsx`) had near-identical inline send closures. Centralizing - * it keeps the two in lockstep and is the one spot that threads inline image - * `files` (perceive-only `file` parts) AND registered context token parts (a + * it keeps the two in lockstep and is the one spot that threads provider-native + * `files` AND registered context token parts (an operable upload descriptor, a * picked template, …) onto the message. (Skills are no longer per-send: the * agent discovers them from disk and advertises them itself.) */ diff --git a/editor/lib/agent-chat/file-attachment.test.ts b/editor/lib/agent-chat/file-attachment.test.ts index ff0179518..bca170241 100644 --- a/editor/lib/agent-chat/file-attachment.test.ts +++ b/editor/lib/agent-chat/file-attachment.test.ts @@ -91,6 +91,23 @@ describe("lowerOperableFiles", () => { expect(files[0].name).toBe("../a.pdf"); }); + it("derives distinct paths for the same file across composer lifetimes", () => { + const lower = (id: string) => + lowerOperableFiles([ + { + id, + name: "screen.png", + mime: "image/png", + size: 3, + base64: "AAAA", + }, + ]).scratchSeed[0].path; + + expect(lower("drop-00000000-0000-4000-8000-000000000001")).not.toBe( + lower("drop-00000000-0000-4000-8000-000000000002") + ); + }); + it("preserves a typed zero-byte file", () => { expect( lowerOperableFiles([ diff --git a/editor/lib/agent-chat/file-attachment.ts b/editor/lib/agent-chat/file-attachment.ts index 027a8ae15..4a8696668 100644 --- a/editor/lib/agent-chat/file-attachment.ts +++ b/editor/lib/agent-chat/file-attachment.ts @@ -1,12 +1,11 @@ /** - * Operable (non-image) file attachments for the agent composer. + * Operable file representations for the agent composer. * - * The sibling of {@link ./image-attachment}, for the OTHER half of "+ upload": - * an arbitrary file (PDF, zip, docx, code, …) the model cannot perceive as - * pixels. Instead of an inline `file` part, the bytes ride `scratch_seed` into - * the session scratch dir (WG `scratch.md` / `binary.md`) and the agent reads or - * extracts them there BY PATH via its shell — so it works for every file type, - * not just what a provider natively decodes. + * An arbitrary upload (PDF, zip, docx, code, image, …) can ride `scratch_seed` + * into the session scratch dir (WG `scratch.md` / `binary.md`) so the agent can + * read, convert, or extract it there BY PATH. Raster images additionally keep + * the provider-native representation from {@link ./image-attachment}; the + * resource router deliberately delivers both whenever scratch is available. * * Two pure steps, framework-free and unit-tested: * - {@link readFileAsBase64} — a `File` → base64 (no downscale; a raw byte @@ -63,6 +62,12 @@ export type EncodedOperableFile = { export type EncodedOperableResource = EncodedOperableFile & { /** Stable within the composer message; used to derive a collision-safe path. */ id: string; + /** + * Zero-based index among provider-native `file` parts in the same user + * message. Present when this scratch body is the operable twin of an inline + * provider image. + */ + providerFileIndex?: number; }; /** @@ -150,6 +155,9 @@ export function lowerOperableFiles( mime: file.mime || "application/octet-stream", size: file.size, path, + ...(file.providerFileIndex !== undefined + ? { provider_file_index: file.providerFileIndex } + : {}), }); } if (scratchSeed.length === 0) return { scratchSeed: [], context: null }; @@ -165,10 +173,10 @@ export function lowerOperableFiles( /** * Split a composer message's `file-attachment` parts into scratch-upload entries * + the context part that names them for the model. Only parts carrying - * `payload.base64` (operable files) are claimed; perceive images (inline `url`) - * are left for `toFileUiParts`. Scratch paths are sanitized to one safe segment - * and deduped within the batch. Compatibility adapter; new code should use - * {@link lowerOperableFiles}. Pure. + * `payload.base64` (operable files) are claimed; legacy inline-only images + * (`url` with no raw payload) are left for `toFileUiParts`. Scratch paths are + * sanitized to one safe segment and deduped within the batch. Compatibility + * adapter; new code should use {@link lowerOperableFiles}. Pure. */ export function extractOperableFiles( parts: readonly AttachmentPartLike[] diff --git a/editor/lib/agent-chat/image-attachment.ts b/editor/lib/agent-chat/image-attachment.ts index aa0038bb7..dd0e824a1 100644 --- a/editor/lib/agent-chat/image-attachment.ts +++ b/editor/lib/agent-chat/image-attachment.ts @@ -3,9 +3,9 @@ * * Turns a user pasted/dropped image `File` into an inline base64 data-URL * attachment the model can actually SEE — a provider-native multimodal `file` - * part. Perceive-only (Claude-Code-style): no path is surfaced to the agent, - * so the model sees pixels, not a file it can operate on. This is the - * `file-attachment` shape from `docs/wg/ai/agent/compositor.md`. + * part. This module owns only that bounded perception representation. The + * resource router separately preserves the original bytes in session scratch, + * yielding both pixels and an operable path when scratch is available. * * The pure helpers (policy, dimension math, byte accounting, part mapping) are * framework-free and unit-tested. `encodeImageFile` is the single DOM/canvas diff --git a/editor/lib/agent-chat/input-resource-policy.test.ts b/editor/lib/agent-chat/input-resource-policy.test.ts index 4f6d282e3..710cf5233 100644 --- a/editor/lib/agent-chat/input-resource-policy.test.ts +++ b/editor/lib/agent-chat/input-resource-policy.test.ts @@ -107,7 +107,7 @@ describe("InputResourcePolicy.CURRENT", () => { }), route: { kind: "attachment", - via: "provider", + via: "provider-and-scratch", from: "bytes", representation: "inline-bytes", }, @@ -133,7 +133,7 @@ describe("InputResourcePolicy.CURRENT", () => { expect(decision.route).toEqual(route); }); - it("rejects image bytes when the model cannot perceive their MIME", () => { + it("keeps image bytes operable when the model cannot perceive their MIME", () => { const decision = InputResourcePolicy.decide( resource({ id: "image", @@ -150,10 +150,218 @@ describe("InputResourcePolicy.CURRENT", () => { }, } ); + expect(decision).toMatchObject({ + status: "accept", + ruleId: "byte-image", + route: { kind: "attachment", via: "scratch", from: "bytes" }, + trace: [ + { + preference: "provider-and-scratch-bytes-attachment", + available: false, + reason: "provider-capability-unavailable", + }, + { + preference: "provider-bytes-attachment", + available: false, + reason: "provider-capability-unavailable", + }, + { preference: "scratch-attachment", available: true }, + ], + }); + }); + + it.each([ + { name: "notes.txt", mimeType: "text/plain", media: "other" as const }, + { + name: "diagram.svg", + mimeType: "image/svg+xml", + media: "other" as const, + }, + { + name: "photo.png", + mimeType: "image/png", + media: "raster-image" as const, + }, + ])( + "keeps $name scratch-operable without binary command tools", + ({ name, mimeType, media }) => { + const decision = InputResourcePolicy.decide( + resource({ + id: name, + name, + mimeType, + media, + available: + media === "raster-image" + ? providerBytes("image/png") + : { bytes: true }, + }), + { + ...capable, + attachment: { + provider: { + ...capable.attachment.provider, + inlineMimes: [], + }, + scratch: { + ...capable.attachment.scratch!, + binaryTools: false, + }, + }, + } + ); + + expect(decision).toMatchObject({ + status: "accept", + route: { kind: "attachment", via: "scratch", from: "bytes" }, + }); + } + ); + + it("rejects an inert binary scratch-only file without binary command tools", () => { + const decision = InputResourcePolicy.decide( + resource({ + id: "archive", + name: "archive.zip", + mimeType: "application/zip", + available: { bytes: true }, + }), + { + ...capable, + attachment: { + ...capable.attachment, + scratch: { + ...capable.attachment.scratch!, + binaryTools: false, + }, + }, + } + ); + expect(decision).toMatchObject({ status: "reject", + reason: "scratch-binary-tools-required", + }); + }); + + it.each([ + { name: "segment.ts", mimeType: "video/mp2t" }, + { name: "report.txt", mimeType: "application/pdf" }, + ])( + "does not override explicit binary MIME $mimeType from $name", + ({ name, mimeType }) => { + const decision = InputResourcePolicy.decide( + resource({ + id: mimeType, + name, + mimeType, + available: { bytes: true }, + }), + { + ...capable, + attachment: { + ...capable.attachment, + scratch: { + ...capable.attachment.scratch!, + binaryTools: false, + }, + }, + } + ); + + expect(decision).toMatchObject({ + status: "reject", + reason: "scratch-binary-tools-required", + }); + } + ); + + it.each([undefined, "application/typescript", "text/typescript"])( + "accepts .ts as structured text with MIME %s", + (mimeType) => { + const decision = InputResourcePolicy.decide( + resource({ + id: mimeType ?? "bare-ts", + name: "source.ts", + mimeType, + available: { bytes: true }, + }), + { + ...capable, + attachment: { + ...capable.attachment, + scratch: { + ...capable.attachment.scratch!, + binaryTools: false, + }, + }, + } + ); + + expect(decision).toMatchObject({ + status: "accept", + route: { kind: "attachment", via: "scratch", from: "bytes" }, + }); + } + ); + + it("falls back to provider-only image delivery without session scratch", () => { + const decision = InputResourcePolicy.decide( + resource({ + id: "image", + name: "image.png", + media: "raster-image", + mimeType: "image/png", + available: providerBytes("image/png"), + }), + { + ...capable, + attachment: { provider: capable.attachment.provider }, + } + ); + expect(decision).toMatchObject({ + status: "accept", ruleId: "byte-image", - reason: "provider-capability-unavailable", + route: { + kind: "attachment", + via: "provider", + from: "bytes", + representation: "inline-bytes", + }, + trace: [ + { + preference: "provider-and-scratch-bytes-attachment", + available: false, + reason: "scratch-unavailable", + }, + { preference: "provider-bytes-attachment", available: true }, + ], + }); + }); + + it("falls back to provider-only image delivery above the scratch file cap", () => { + const decision = InputResourcePolicy.decide( + resource({ + id: "large-image", + name: "large.png", + media: "raster-image", + mimeType: "image/png", + size: 9 * 1024 * 1024, + available: providerBytes("image/png"), + }), + capable + ); + expect(decision).toMatchObject({ + status: "accept", + route: { kind: "attachment", via: "provider", from: "bytes" }, + trace: [ + { + preference: "provider-and-scratch-bytes-attachment", + available: false, + reason: "file-too-large", + }, + { preference: "provider-bytes-attachment", available: true }, + ], }); }); @@ -202,17 +410,30 @@ describe("InputResourcePolicy.CURRENT", () => { mimeType: "image/webp", available: providerBytes("image/webp", "image/png"), }); + const providerOnly: InputResourcePolicy.Config = { + id: "provider-only", + rules: [ + { + id: "bytes", + when: () => true, + prefer: ["provider-bytes-attachment"], + }, + ], + }; expect( - InputResourcePolicy.decide(input, { - ...capable, - attachment: { - ...capable.attachment, - provider: { - ...capable.attachment.provider, - inlineMimes: ["image/png"], + InputResourcePolicy.decide( + input, + { + ...capable, + attachment: { + provider: { + ...capable.attachment.provider, + inlineMimes: ["image/png"], + }, }, }, - }) + providerOnly + ) ).toMatchObject({ status: "accept", route: { kind: "attachment", via: "provider", from: "bytes" }, @@ -223,13 +444,13 @@ describe("InputResourcePolicy.CURRENT", () => { { ...capable, attachment: { - ...capable.attachment, provider: { ...capable.attachment.provider, inlineMimes: ["image/png"], }, }, - } + }, + providerOnly ) ).toMatchObject({ status: "reject", @@ -320,7 +541,7 @@ describe("InputResourcePolicy.REFERENCE_FIRST", () => { status: "accept", route: { kind: "attachment", - via: "provider", + via: "provider-and-scratch", from: "bytes", representation: "inline-bytes", }, @@ -413,7 +634,11 @@ describe("InputResourcePolicy invariants", () => { { id: "directory-as-bytes", when: () => true, - prefer: ["scratch-attachment", "provider-bytes-attachment"], + prefer: [ + "scratch-attachment", + "provider-and-scratch-bytes-attachment", + "provider-bytes-attachment", + ], }, ], }; diff --git a/editor/lib/agent-chat/input-resource-policy.ts b/editor/lib/agent-chat/input-resource-policy.ts index 4598dc85c..9e01a4d73 100644 --- a/editor/lib/agent-chat/input-resource-policy.ts +++ b/editor/lib/agent-chat/input-resource-policy.ts @@ -69,7 +69,15 @@ export namespace InputResourcePolicy { remoteUrlMimes: readonly string[]; }; /** Absent when the surface has no tool-visible scratch. */ - scratch?: ScratchSeedBudget.Limits; + scratch?: ScratchSeedBudget.Limits & { + /** + * Whether scratch has a confined byte-oriented tool such as + * `run_command`. False still permits raster images (`view_image`) and + * structured text (`read_file`); arbitrary binary paths would be + * inoperable and are rejected. + */ + binaryTools?: boolean; + }; }; }; @@ -79,6 +87,7 @@ export namespace InputResourcePolicy { | "host-scope-reference" | "provider-url-inline-attachment" | "provider-url-remote-attachment" + | "provider-and-scratch-bytes-attachment" | "provider-bytes-attachment" | "scratch-attachment"; @@ -100,6 +109,12 @@ export namespace InputResourcePolicy { from: "url" | "bytes"; representation: "inline-bytes" | "remote-url"; } + | { + kind: "attachment"; + via: "provider-and-scratch"; + from: "bytes"; + representation: "inline-bytes"; + } | { kind: "attachment"; via: "scratch"; from: "bytes" }; export type UnavailableReason = @@ -107,9 +122,11 @@ export namespace InputResourcePolicy { | "reference-capability-unavailable" | "provider-capability-unavailable" | "scratch-unavailable" + | "scratch-binary-tools-required" | "file-too-large" | "scratch-file-count-exceeded" | "scratch-budget-exceeded" + | "draft-operable-copy-budget-exceeded" | "directory-cannot-be-attached" | "directory-reference-required"; @@ -148,7 +165,8 @@ export namespace InputResourcePolicy { /** Today's product behavior, expressed centrally rather than in React: * existing paths stay references; directories become host scopes; Library - * images and byte images become provider parts; other bytes go to scratch. */ + * images become provider parts; byte images use provider-native perception + * plus a byte-exact scratch copy; other bytes go to scratch. */ export const CURRENT: Config = { id: "current", rules: [ @@ -179,7 +197,11 @@ export namespace InputResourcePolicy { resource.kind === "file" && resource.media === "raster-image" && resource.available.bytes === true, - prefer: ["provider-bytes-attachment"], + prefer: [ + "provider-and-scratch-bytes-attachment", + "provider-bytes-attachment", + "scratch-attachment", + ], }, { id: "byte-file", @@ -223,7 +245,11 @@ export namespace InputResourcePolicy { resource.kind === "file" && resource.media === "raster-image" && resource.available.bytes === true, - prefer: ["provider-bytes-attachment"], + prefer: [ + "provider-and-scratch-bytes-attachment", + "provider-bytes-attachment", + "scratch-attachment", + ], }, { id: "byte-file", @@ -337,30 +363,55 @@ export namespace InputResourcePolicy { return providerRoute("url", "inline-bytes", resource, capabilities); case "provider-url-remote-attachment": return providerRoute("url", "remote-url", resource, capabilities); - case "provider-bytes-attachment": - return providerRoute("bytes", "inline-bytes", resource, capabilities); - case "scratch-attachment": { - if (resource.kind === "directory") { - return { reason: "directory-cannot-be-attached" }; - } - if (!resource.available.bytes) { - return { reason: "representation-unavailable" }; - } - const scratch = capabilities.attachment.scratch; - if (!scratch) return { reason: "scratch-unavailable" }; - if ( - resource.size !== undefined && - resource.size > scratch.maxFileBytes - ) { - return { reason: "file-too-large" }; - } + case "provider-and-scratch-bytes-attachment": { + const provider = providerRoute( + "bytes", + "inline-bytes", + resource, + capabilities + ); + if (!provider.route) return { reason: provider.reason }; + const scratch = scratchRoute(resource, capabilities); + if (!scratch.route) return { reason: scratch.reason }; return { - route: { kind: "attachment", via: "scratch", from: "bytes" }, + route: { + kind: "attachment", + via: "provider-and-scratch", + from: "bytes", + representation: "inline-bytes", + }, }; } + case "provider-bytes-attachment": + return providerRoute("bytes", "inline-bytes", resource, capabilities); + case "scratch-attachment": + return scratchRoute(resource, capabilities); } } + function scratchRoute( + resource: Readonly, + capabilities: Readonly + ): { route?: Route; reason?: UnavailableReason } { + if (resource.kind === "directory") { + return { reason: "directory-cannot-be-attached" }; + } + if (!resource.available.bytes) { + return { reason: "representation-unavailable" }; + } + const scratch = capabilities.attachment.scratch; + if (!scratch) return { reason: "scratch-unavailable" }; + if (scratch.binaryTools === false && requiresScratchBinaryTools(resource)) { + return { reason: "scratch-binary-tools-required" }; + } + if (resource.size !== undefined && resource.size > scratch.maxFileBytes) { + return { reason: "file-too-large" }; + } + return { + route: { kind: "attachment", via: "scratch", from: "bytes" }, + }; + } + function providerRoute( from: "url" | "bytes", representation: "inline-bytes" | "remote-url", @@ -406,5 +457,35 @@ export namespace InputResourcePolicy { }, }; } + + function requiresScratchBinaryTools( + resource: Readonly + ): boolean { + if (resource.media === "raster-image") return false; + + const mime = resource.mimeType?.split(";", 1)[0]?.trim().toLowerCase(); + if ( + mime?.startsWith("text/") || + mime === "image/svg+xml" || + mime === "application/json" || + mime?.endsWith("+json") || + mime === "application/xml" || + mime?.endsWith("+xml") || + mime === "application/javascript" || + mime === "application/typescript" || + mime === "application/yaml" || + mime === "application/x-yaml" || + mime === "application/toml" || + mime === "application/sql" + ) { + return false; + } + + if (mime) return true; + + return !/\.(?:c|cc|cpp|css|csv|go|h|hpp|html?|ini|java|js|jsx|json|jsonl|md|mdx|mjs|py|rb|rs|sh|sql|svg|toml|ts|tsx|txt|xml|ya?ml)$/i.test( + resource.name + ); + } } import type { ScratchSeedBudget } from "./scratch-seed-budget"; diff --git a/editor/lib/agent-chat/input-resource-router.test.ts b/editor/lib/agent-chat/input-resource-router.test.ts index 0e3e437e2..c9b00ad35 100644 --- a/editor/lib/agent-chat/input-resource-router.test.ts +++ b/editor/lib/agent-chat/input-resource-router.test.ts @@ -16,12 +16,18 @@ const directory = { access: "read", } as const; -function file(input: { name: string; type?: string; size?: number }): File { +function file(input: { + name: string; + type?: string; + size?: number; + bytes?: readonly number[]; +}): File { + const bytes = Uint8Array.from(input.bytes ?? [0, 0, 0]); return { name: input.name, type: input.type ?? "", - size: input.size ?? 3, - arrayBuffer: async () => new Uint8Array([0, 0, 0]).buffer, + size: input.size ?? bytes.byteLength, + arrayBuffer: async () => bytes.buffer, } as File; } @@ -85,7 +91,11 @@ describe("InputResourceRouter.prepare", () => { kind: "browser-file", id: "paste-1", source: "paste", - file: file({ name: "paste.png", type: "image/png" }), + file: file({ + name: "paste.png", + type: "image/png", + bytes: [1, 2, 3], + }), }, environment({ effects: { encodeProviderFile } }) ); @@ -94,14 +104,37 @@ describe("InputResourceRouter.prepare", () => { status: "accept", decision: { ruleId: "byte-image", - route: { kind: "attachment", via: "provider", from: "bytes" }, + route: { + kind: "attachment", + via: "provider-and-scratch", + from: "bytes", + }, + }, + materializedRoute: { + kind: "attachment", + via: "provider-and-scratch", + from: "bytes", + }, + resource: { + kind: "provider-and-scratch-file", + source: "paste", + name: "paste.png", + mimeType: "image/png", + size: 3, + base64: "AQID", + provider: { + name: "paste.png", + mimeType: "image/png", + size: 3, + url: "data:image/png;base64,AAAA", + representation: "inline-bytes", + }, }, - resource: { kind: "provider-file", source: "paste" }, }); expect(encodeProviderFile).toHaveBeenCalledOnce(); }); - it("rejects an encoder output MIME outside the active provider capability", async () => { + it("keeps the scratch fallback when provider output violates capability", async () => { const encodeProviderFile = vi.fn< InputResourceRouter.Effects["encodeProviderFile"] >(async () => ({ @@ -126,14 +159,128 @@ describe("InputResourceRouter.prepare", () => { ); expect(result).toMatchObject({ - status: "reject", - reason: "preparation-failed", + status: "accept", + decision: { route: { via: "provider-and-scratch" } }, + materializedRoute: { + kind: "attachment", + via: "scratch", + from: "bytes", + }, + resource: { + kind: "scratch-file", + name: "source.webp", + mimeType: "image/webp", + size: 3, + base64: "AAAA", + }, }); expect(encodeProviderFile).toHaveBeenCalledWith(expect.anything(), { outputMimes: ["image/png"], }); }); + it.each(["null", "throw"] as const)( + "keeps the scratch fallback when provider preparation returns %s", + async (failure) => { + const encodeProviderFile = vi.fn< + InputResourceRouter.Effects["encodeProviderFile"] + >(async () => { + if (failure === "throw") throw new Error("provider decode failed"); + return null; + }); + const encodeOperableFile = vi.fn< + InputResourceRouter.Effects["encodeOperableFile"] + >(async () => ({ + name: "original.gif", + mime: "image/gif", + size: 3, + base64: "AQID", + })); + const result = await InputResourceRouter.prepare( + { + kind: "browser-file", + id: "gif-1", + source: "drop", + file: file({ + name: "original.gif", + type: "image/gif", + bytes: [1, 2, 3], + }), + }, + environment({ + attachment: { + provider: { inlineMimes: ["image/png"], remoteUrlMimes: [] }, + }, + effects: { encodeProviderFile, encodeOperableFile }, + }) + ); + + expect(result).toMatchObject({ + status: "accept", + decision: { route: { via: "provider-and-scratch" } }, + materializedRoute: { kind: "attachment", via: "scratch" }, + resource: { + kind: "scratch-file", + name: "original.gif", + base64: "AQID", + }, + }); + } + ); + + it.each(["null", "throw"] as const)( + "keeps provider perception when scratch preparation returns %s", + async (failure) => { + const encodeProviderFile = vi.fn< + InputResourceRouter.Effects["encodeProviderFile"] + >(async () => ({ + name: "preview.png", + mime: "image/png", + size: 3, + url: "data:image/png;base64,AAAA", + })); + const encodeOperableFile = vi.fn< + InputResourceRouter.Effects["encodeOperableFile"] + >(async () => { + if (failure === "throw") throw new Error("raw read failed"); + return null; + }); + const result = await InputResourceRouter.prepare( + { + kind: "browser-file", + id: "gif-1", + source: "drop", + file: file({ + name: "original.gif", + type: "image/gif", + bytes: [1, 2, 3], + }), + }, + environment({ + attachment: { + provider: { inlineMimes: ["image/png"], remoteUrlMimes: [] }, + }, + effects: { encodeProviderFile, encodeOperableFile }, + }) + ); + + expect(result).toMatchObject({ + status: "accept", + decision: { route: { via: "provider-and-scratch" } }, + materializedRoute: { + kind: "attachment", + via: "provider", + from: "bytes", + }, + resource: { + kind: "provider-file", + name: "preview.png", + mimeType: "image/png", + }, + }); + } + ); + it("rejects an oversized scratch file before reading it", async () => { const encodeOperableFile = vi.fn(); @@ -177,6 +324,36 @@ describe("InputResourceRouter.prepare", () => { expect(encodeOperableFile).not.toHaveBeenCalled(); }); + it("does not read opaque bytes when scratch lacks binary command tools", async () => { + const encodeOperableFile = + vi.fn(); + const result = await InputResourceRouter.prepare( + { + kind: "browser-file", + id: "transport-stream-1", + source: "picker", + file: file({ name: "segment.ts", type: "video/mp2t" }), + }, + environment({ + attachment: { + scratch: { + maxFileBytes: 8 * 1024 * 1024, + maxFiles: 64, + maxTotalBytes: 8 * 1024 * 1024, + binaryTools: false, + }, + }, + effects: { encodeOperableFile }, + }) + ); + + expect(result).toMatchObject({ + status: "reject", + reason: "scratch-binary-tools-required", + }); + expect(encodeOperableFile).not.toHaveBeenCalled(); + }); + it("rejects scratch output whose declared size differs from its bytes", async () => { const encodeOperableFile = vi.fn< InputResourceRouter.Effects["encodeOperableFile"] @@ -379,6 +556,287 @@ describe("InputResourceRouter.prepare", () => { }); }); + it("bounds raw draft twins independently of final scratch capacity", async () => { + const encodeProviderFile = vi.fn< + InputResourceRouter.Effects["encodeProviderFile"] + >(async (source) => ({ + name: source.name, + mime: source.type, + size: 3, + url: `data:${source.type};base64,AAAA`, + })); + const encodeOperableFile = vi.fn< + InputResourceRouter.Effects["encodeOperableFile"] + >(async (source) => ({ + name: source.name, + mime: source.type, + size: source.size, + base64: "AQIDBAU=", + })); + const results = await InputResourceRouter.prepareBatch( + ["one", "two"].map( + (name) => + ({ + kind: "browser-file", + id: name, + source: "drop", + file: file({ + name: `${name}.png`, + type: "image/png", + bytes: [1, 2, 3, 4, 5], + }), + }) as const + ), + environment({ + attachment: { + operableTwinRetention: { maxFiles: 64, maxTotalBytes: 8 }, + }, + effects: { encodeProviderFile, encodeOperableFile }, + }) + ); + + expect(results).toMatchObject([ + { + status: "accept", + decision: { route: { via: "provider-and-scratch" } }, + resource: { kind: "provider-and-scratch-file" }, + }, + { + status: "accept", + decision: { + route: { via: "provider" }, + trace: [ + { + preference: "provider-and-scratch-bytes-attachment", + available: false, + reason: "draft-operable-copy-budget-exceeded", + }, + { preference: "provider-bytes-attachment", available: true }, + ], + }, + materializedRoute: { via: "provider" }, + resource: { kind: "provider-file" }, + }, + ]); + expect(encodeProviderFile).toHaveBeenCalledTimes(2); + expect(encodeOperableFile).toHaveBeenCalledOnce(); + }); + + it("retains admitted raster twins until final lowering can allocate current capacity", async () => { + const encodeProviderFile = vi.fn< + InputResourceRouter.Effects["encodeProviderFile"] + >(async (source) => ({ + name: source.name, + mime: source.type, + size: 3, + url: `data:${source.type};base64,AAAA`, + })); + const encodeOperableFile = vi.fn< + InputResourceRouter.Effects["encodeOperableFile"] + >(async (source) => ({ + name: source.name, + mime: source.type, + size: source.size, + base64: "AQIDBAU=", + })); + const results = await InputResourceRouter.prepareBatch( + ["one", "two"].map( + (name) => + ({ + kind: "browser-file", + id: name, + source: "drop", + file: file({ + name: `${name}.png`, + type: "image/png", + bytes: [1, 2, 3, 4, 5], + }), + }) as const + ), + environment({ + attachment: { + scratch: { maxFileBytes: 8, maxFiles: 64, maxTotalBytes: 8 }, + }, + effects: { encodeProviderFile, encodeOperableFile }, + }) + ); + + expect(results).toMatchObject([ + { + status: "accept", + decision: { route: { via: "provider-and-scratch" } }, + resource: { kind: "provider-and-scratch-file", base64: "AQIDBAU=" }, + }, + { + status: "accept", + decision: { route: { via: "provider-and-scratch" } }, + resource: { kind: "provider-and-scratch-file", base64: "AQIDBAU=" }, + }, + ]); + expect(encodeProviderFile).toHaveBeenCalledTimes(2); + expect(encodeOperableFile).toHaveBeenCalledTimes(2); + + const bound = results.flatMap((result) => + result.status === "accept" + ? [ + { + attachmentId: result.resource.sourceId, + resource: result.resource, + }, + ] + : [] + ); + const both = InputResourceRouter.lower(bound, { + provider: provider(["image/png"]), + scratch: { maxFileBytes: 8, maxFiles: 64, maxTotalBytes: 8 }, + }); + expect(both.files).toHaveLength(2); + expect(both.extras).toMatchObject({ + scratchSeed: [{ path: "upload-one-one.png", base64: "AQIDBAU=" }], + contexts: [ + { + type: USER_FILE_ATTACHMENTS, + data: { + files: [{ name: "one.png", provider_file_index: 0 }], + }, + }, + ], + }); + expect(both.rejected).toEqual([]); + expect(both.routes).toMatchObject([ + { + attachmentId: "one", + route: { via: "provider-and-scratch" }, + }, + { + attachmentId: "two", + route: { via: "provider" }, + }, + ]); + + // Capacity is a submit-time fact. Removing the first attachment lets the + // retained raw twin of the second image become operable immediately. + const afterRemoval = InputResourceRouter.lower(bound.slice(1), { + provider: provider(["image/png"]), + scratch: { maxFileBytes: 8, maxFiles: 64, maxTotalBytes: 8 }, + }); + expect(afterRemoval.extras).toMatchObject({ + scratchSeed: [{ path: "upload-two-two.png", base64: "AQIDBAU=" }], + contexts: [ + { + type: USER_FILE_ATTACHMENTS, + data: { + files: [{ name: "two.png", provider_file_index: 0 }], + }, + }, + ], + }); + expect(afterRemoval.routes).toMatchObject([ + { + attachmentId: "two", + route: { via: "provider-and-scratch" }, + }, + ]); + }); + + it("allocates mandatory scratch-only files before optional raster twins", async () => { + const encodeProviderFile = vi.fn< + InputResourceRouter.Effects["encodeProviderFile"] + >(async (source) => ({ + name: source.name, + mime: source.type, + size: 3, + url: `data:${source.type};base64,AAAA`, + })); + const encodeOperableFile = vi.fn< + InputResourceRouter.Effects["encodeOperableFile"] + >(async (source) => ({ + name: source.name, + mime: source.type || "application/octet-stream", + size: source.size, + base64: "AQIDBAU=", + })); + const results = await InputResourceRouter.prepareBatch( + [ + { + kind: "browser-file", + id: "image", + source: "drop", + file: file({ + name: "image.png", + type: "image/png", + bytes: [1, 2, 3, 4, 5], + }), + }, + { + kind: "browser-file", + id: "archive", + source: "drop", + file: file({ + name: "archive.zip", + type: "application/zip", + bytes: [1, 2, 3, 4, 5], + }), + }, + ], + environment({ + attachment: { + scratch: { maxFileBytes: 8, maxFiles: 64, maxTotalBytes: 8 }, + }, + effects: { encodeProviderFile, encodeOperableFile }, + }) + ); + + expect(results).toMatchObject([ + { + status: "accept", + decision: { route: { via: "provider-and-scratch" } }, + resource: { kind: "provider-and-scratch-file" }, + }, + { + status: "accept", + decision: { route: { via: "scratch" } }, + resource: { kind: "scratch-file", base64: "AQIDBAU=" }, + }, + ]); + expect(encodeProviderFile).toHaveBeenCalledOnce(); + expect(encodeOperableFile).toHaveBeenCalledTimes(2); + expect(encodeOperableFile).toHaveBeenCalledWith( + expect.objectContaining({ name: "archive.zip" }), + { maxBytes: 8 } + ); + + const bound = results.flatMap((result) => + result.status === "accept" + ? [ + { + attachmentId: result.resource.sourceId, + resource: result.resource, + }, + ] + : [] + ); + const lowered = InputResourceRouter.lower(bound, { + provider: provider(["image/png"]), + scratch: { maxFileBytes: 8, maxFiles: 64, maxTotalBytes: 8 }, + }); + expect(lowered.files).toHaveLength(1); + expect(lowered.extras).toMatchObject({ + scratchSeed: [{ path: "upload-archive-archive.zip", base64: "AQIDBAU=" }], + contexts: [ + { + type: USER_FILE_ATTACHMENTS, + data: { files: [{ name: "archive.zip" }] }, + }, + ], + }); + expect(lowered.rejected).toEqual([]); + expect(lowered.routes).toMatchObject([ + { attachmentId: "image", route: { via: "provider" } }, + { attachmentId: "archive", route: { via: "scratch" } }, + ]); + }); + it("rejects an over-budget scratch batch before reading any bytes", async () => { const encodeOperableFile = vi.fn(); @@ -474,12 +932,42 @@ describe("InputResourceRouter.card", () => { expect(card).toEqual({ kind: "file", + id: "file-1", name: "brief.pdf", mime: "application/pdf", size: 3, }); expect(card).not.toHaveProperty("payload"); }); + + it("shows one preview card for a dual-delivery image without exposing bytes", () => { + const card = InputResourceRouter.card({ + kind: "provider-and-scratch-file", + source: "paste", + sourceId: "paste-1", + name: "original.gif", + mimeType: "image/gif", + size: 5, + base64: "AQIDBAU=", + provider: { + name: "original.png", + mimeType: "image/png", + size: 3, + url: "data:image/png;base64,AAAA", + representation: "inline-bytes", + }, + }); + + expect(card).toEqual({ + kind: "file", + id: "paste-1", + name: "original.gif", + mime: "image/gif", + size: 5, + url: "data:image/png;base64,AAAA", + }); + expect(card).not.toHaveProperty("base64"); + }); }); describe("InputResourceRouter.lower", () => { @@ -489,13 +977,20 @@ describe("InputResourceRouter.lower", () => { { attachmentId: "image-1", resource: { - kind: "provider-file", - source: "library", - sourceId: "pin-1", - name: "hero.png", - mimeType: "image/png", - url: "data:image/png;base64,AAAA", - representation: "inline-bytes", + kind: "provider-and-scratch-file", + source: "paste", + sourceId: "paste-1", + name: "original.gif", + mimeType: "image/gif", + size: 5, + base64: "AQIDBAU=", + provider: { + name: "original.png", + mimeType: "image/png", + size: 3, + url: "data:image/png;base64,AAAA", + representation: "inline-bytes", + }, }, }, { @@ -547,7 +1042,7 @@ describe("InputResourceRouter.lower", () => { type: "file", url: "data:image/png;base64,AAAA", mediaType: "image/png", - filename: "hero.png", + filename: "original.png", }, ]); expect(lowered.references).toEqual([ @@ -558,15 +1053,120 @@ describe("InputResourceRouter.lower", () => { }, ]); expect(lowered.extras).toMatchObject({ - scratchSeed: [{ path: "upload-file-1-brief.pdf", base64: "AAAA" }], + scratchSeed: [ + { path: "upload-image-1-original.gif", base64: "AQIDBAU=" }, + { path: "upload-file-1-brief.pdf", base64: "AAAA" }, + ], contexts: [ - { type: USER_FILE_ATTACHMENTS }, + { + type: USER_FILE_ATTACHMENTS, + data: { + location: "scratch", + files: [ + { + name: "original.gif", + mime: "image/gif", + size: 5, + path: "upload-image-1-original.gif", + provider_file_index: 0, + }, + { + name: "brief.pdf", + mime: "application/pdf", + size: 3, + path: "upload-file-1-brief.pdf", + }, + ], + }, + }, { type: USER_DIRECTORY_REFERENCES }, ], }); expect(lowered.rejected).toEqual([]); }); + it("correlates same-named scratch twins to exact provider file indices", () => { + const dual = ( + attachmentId: string, + sourceId: string, + providerName: string, + base64: string + ): InputResourceRouter.BoundResource => ({ + attachmentId, + resource: { + kind: "provider-and-scratch-file", + source: "drop", + sourceId, + name: "duplicate.gif", + mimeType: "image/gif", + size: 3, + base64, + provider: { + name: providerName, + mimeType: "image/png", + size: 3, + url: "data:image/png;base64,AAAA", + representation: "inline-bytes", + }, + }, + }); + const lowered = InputResourceRouter.lower( + [ + { + attachmentId: "preview-only", + resource: { + kind: "provider-file", + source: "library", + sourceId: "library-1", + name: "duplicate.gif", + mimeType: "image/png", + size: 3, + url: "data:image/png;base64,AAAA", + representation: "inline-bytes", + }, + }, + dual("dual-a", "drop-a", "preview-a.png", "AQID"), + dual("dual-b", "drop-b", "preview-b.png", "BAUG"), + { + attachmentId: "notes", + resource: { + kind: "scratch-file", + source: "drop", + sourceId: "drop-notes", + name: "duplicate.gif", + mimeType: "application/octet-stream", + size: 3, + base64: "BwgJ", + }, + }, + ], + { + provider: provider(["image/png"]), + scratch: { maxFileBytes: 8, maxFiles: 8, maxTotalBytes: 16 }, + } + ); + + expect(lowered.files.map((part) => part.filename)).toEqual([ + "duplicate.gif", + "preview-a.png", + "preview-b.png", + ]); + expect(lowered.extras).toMatchObject({ + contexts: [ + { + type: USER_FILE_ATTACHMENTS, + data: { + files: [ + { path: "upload-dual-a-duplicate.gif", provider_file_index: 1 }, + { path: "upload-dual-b-duplicate.gif", provider_file_index: 2 }, + { path: "upload-notes-duplicate.gif" }, + ], + }, + }, + ], + }); + }); + it("rejects a provider file if the active model changed", () => { const lowered = InputResourceRouter.lower( [ @@ -595,6 +1195,147 @@ describe("InputResourceRouter.lower", () => { ]); }); + it("keeps a dual image operable when the active model loses vision support", () => { + const lowered = InputResourceRouter.lower( + [ + { + attachmentId: "image-1", + resource: { + kind: "provider-and-scratch-file", + source: "paste", + sourceId: "paste-1", + name: "paste.png", + mimeType: "image/png", + size: 3, + base64: "AQID", + provider: { + name: "paste.png", + mimeType: "image/png", + size: 3, + url: "data:image/png;base64,AAAA", + representation: "inline-bytes", + }, + }, + }, + ], + { + provider: provider(), + scratch: { maxFileBytes: 8, maxFiles: 1, maxTotalBytes: 8 }, + } + ); + + expect(lowered.files).toEqual([]); + expect(lowered.extras).toMatchObject({ + scratchSeed: [{ path: "upload-image-1-paste.png", base64: "AQID" }], + contexts: [{ type: USER_FILE_ATTACHMENTS }], + }); + expect(lowered.rejected).toEqual([]); + }); + + it("reserves scratch for a provider-failed twin before optional twins", () => { + const dual = ( + attachmentId: string, + providerMime: string, + base64: string + ): InputResourceRouter.BoundResource => ({ + attachmentId, + resource: { + kind: "provider-and-scratch-file", + source: "drop", + sourceId: attachmentId, + name: `${attachmentId}.gif`, + mimeType: "image/gif", + size: 3, + base64, + provider: { + name: `${attachmentId}.preview`, + mimeType: providerMime, + size: 3, + url: `data:${providerMime};base64,AAAA`, + representation: "inline-bytes", + }, + }, + }); + const lowered = InputResourceRouter.lower( + [ + dual("provider-ok", "image/png", "AQID"), + dual("scratch-required", "image/jpeg", "BAUG"), + ], + { + provider: provider(["image/png"]), + scratch: { maxFileBytes: 3, maxFiles: 1, maxTotalBytes: 3 }, + } + ); + + expect(lowered.files.map((part) => part.filename)).toEqual([ + "provider-ok.preview", + ]); + expect(lowered.extras).toMatchObject({ + scratchSeed: [ + { + path: "upload-scratch-required-scratch-required.gif", + base64: "BAUG", + }, + ], + contexts: [ + { + type: USER_FILE_ATTACHMENTS, + data: { + files: [ + { + name: "scratch-required.gif", + path: "upload-scratch-required-scratch-required.gif", + }, + ], + }, + }, + ], + }); + expect(lowered.routes).toMatchObject([ + { attachmentId: "provider-ok", route: { via: "provider" } }, + { attachmentId: "scratch-required", route: { via: "scratch" } }, + ]); + expect(lowered.rejected).toEqual([]); + }); + + it("keeps provider perception when a dual image loses scratch support", () => { + const lowered = InputResourceRouter.lower( + [ + { + attachmentId: "image-1", + resource: { + kind: "provider-and-scratch-file", + source: "drop", + sourceId: "drop-1", + name: "original.gif", + mimeType: "image/gif", + size: 5, + base64: "AQIDBAU=", + provider: { + name: "preview.png", + mimeType: "image/png", + size: 3, + url: "data:image/png;base64,AAAA", + representation: "inline-bytes", + }, + }, + }, + ], + { provider: provider(["image/png"]) } + ); + + expect(lowered.files).toEqual([ + { + type: "file", + url: "data:image/png;base64,AAAA", + mediaType: "image/png", + filename: "preview.png", + }, + ]); + expect(lowered.extras).toBeUndefined(); + expect(lowered.rejected).toEqual([]); + }); + it("rejects an already-prepared scratch file when the host capability is absent", () => { const lowered = InputResourceRouter.lower( [ diff --git a/editor/lib/agent-chat/input-resource-router.ts b/editor/lib/agent-chat/input-resource-router.ts index 6c4f2c636..def1cc542 100644 --- a/editor/lib/agent-chat/input-resource-router.ts +++ b/editor/lib/agent-chat/input-resource-router.ts @@ -3,9 +3,11 @@ * * Source adapters describe what they actually hold (browser bytes, a Library * URL, an agent-visible path, or a host-mintable directory handle). The pure - * policy selects one legal route; this module executes exactly that route and - * returns a typed prepared resource. Composer cards receive display data only - * and never become the hidden source of delivery semantics. + * policy selects one legal route; this module executes that route and returns + * the representation it actually materialized. A composite route may degrade + * to either declared single-leg fallback when only one encoder succeeds. + * Composer cards receive display data only and never become the hidden source + * of delivery semantics. */ import type { FileUIPart } from "ai"; @@ -15,6 +17,7 @@ import { lowerOperableFiles, readFileAsBase64, type EncodedOperableFile, + type EncodedOperableResource, } from "./file-attachment"; import { IMAGE_ATTACHMENT_POLICY, @@ -81,6 +84,15 @@ export namespace InputResourceRouter { /** Present only when this chat has a tool-visible scratch binding. */ scratch?: ScratchSeedBudget.Limits & { reservation?: ScratchSeedBudget.Reservation; + binaryTools?: boolean; + }; + /** + * Renderer-memory admission for raw operable twins retained in a draft. + * Independent of the smaller, submit-time scratch seed budget. + */ + operableTwinRetention?: { + maxFiles: number; + maxTotalBytes: number; }; }; /** Test/host injection points. Production uses the existing encoders. */ @@ -127,6 +139,21 @@ export namespace InputResourceRouter { url: string; representation: "inline-bytes" | "remote-url"; }) + | (PreparedBase & { + kind: "provider-and-scratch-file"; + /** Original upload metadata and bytes, preserved byte-for-byte. */ + mimeType: string; + size: number; + base64: string; + /** Provider-processed representation used for immediate perception. */ + provider: { + name: string; + mimeType: string; + size: number; + url: string; + representation: "inline-bytes"; + }; + }) | (PreparedBase & { kind: "scratch-file"; mimeType: string; @@ -151,6 +178,8 @@ export namespace InputResourceRouter { export type Card = | { kind: "file"; + /** Stable prepared-resource identity retained by ComposerCore. */ + id: string; name: string; mime?: string; size?: number; @@ -159,6 +188,8 @@ export namespace InputResourceRouter { } | { kind: "directory"; + /** Stable prepared-resource identity retained by ComposerCore. */ + id: string; name: string; ref: DirectoryScopeDescriptor; }; @@ -172,6 +203,8 @@ export namespace InputResourceRouter { | { status: "accept"; decision: Extract; + /** The route actually materialized after effect-level fallback. */ + materializedRoute: InputResourcePolicy.Route; resource: PreparedResource; } | { @@ -199,6 +232,11 @@ export namespace InputResourceRouter { files: FileUIPart[]; extras?: SendExtras; references: Reference[]; + /** Final route selected for each successfully lowered resource. */ + routes: Array<{ + attachmentId: string; + route: InputResourcePolicy.Route; + }>; rejected: Array<{ attachmentId: string; reason: InputResourcePolicy.UnavailableReason; @@ -221,6 +259,17 @@ export namespace InputResourceRouter { encodeOperableFile: readFileAsBase64, }; + /** + * Raw twins are base64 strings retained with the composer draft. Bound that + * memory independently of the one-turn scratch budget: resources admitted + * here can still be reallocated dynamically if other chips/reservations + * change before submit; later rasters use the declared provider-only fallback. + */ + export const OPERABLE_TWIN_RETENTION_LIMITS = { + maxFiles: 16, + maxTotalBytes: 32 * 1024 * 1024, + } as const; + export function capabilities( environment: Readonly ): InputResourcePolicy.Capabilities { @@ -251,9 +300,11 @@ export namespace InputResourceRouter { } /** - * Plan a gesture as one atomic batch before any byte-backed scratch input is - * read. Existing prepared resources and non-composer reservations participate - * in the same budget, while provider/reference routes remain independent. + * Preflight mandatory scratch-only members as one atomic batch before their + * bytes are read. Composite rasters retain both per-file representations; + * their optional scratch legs are allocated against the current aggregate + * only at final lowering, so draft removal or reservation changes can restore + * operability without rereading the user's source. */ export async function prepareBatch( inputs: readonly Readonly[], @@ -270,25 +321,53 @@ export namespace InputResourceRouter { decision: InputResourcePolicy.decide(facts, available, config), }; }); - const existingScratch = existing.flatMap((resource) => + const existingRequiredScratch = existing.flatMap((resource) => resource.kind === "scratch-file" ? [{ size: resource.size }] : [] ); - const incomingScratch = plans.flatMap(({ decision, facts }) => - isScratchDecision(decision) ? [{ size: facts.size ?? 0 }] : [] + const incomingRequiredScratch = plans.flatMap(({ decision, facts }) => + isScratchOnlyDecision(decision) ? [{ size: facts.size ?? 0 }] : [] ); - const scratchRejection = scratchBatchRejection( - [...existingScratch, ...incomingScratch], + const requiredScratchRejection = scratchBatchRejection( + [...existingRequiredScratch, ...incomingRequiredScratch], environment.attachment.scratch ); - + const retainedTwins = existing.flatMap((resource) => + resource.kind === "provider-and-scratch-file" + ? [{ size: resource.size }] + : [] + ); + const retentionLimits = + environment.attachment.operableTwinRetention ?? + OPERABLE_TWIN_RETENTION_LIMITS; + for (const plan of plans) { + if (!isProviderAndScratchDecision(plan.decision)) continue; + const rejected = retainedTwinBatchRejection( + [...retainedTwins, { size: plan.facts.size ?? 0 }], + retentionLimits + ); + if (rejected) { + plan.decision = fallbackWithoutScratch( + plan.facts, + available, + config, + rejected + ); + } else { + retainedTwins.push({ size: plan.facts.size ?? 0 }); + } + } const effects = { ...DEFAULT_EFFECTS, ...environment.effects }; return Promise.all( plans.map(async ({ input, decision }): Promise => { if (decision.status === "reject") { return { status: "reject", reason: decision.reason, decision }; } - if (scratchRejection && isScratchDecision(decision)) { - return { status: "reject", decision, reason: scratchRejection }; + if (requiredScratchRejection && isScratchOnlyDecision(decision)) { + return { + status: "reject", + decision, + reason: requiredScratchRejection, + }; } try { const resource = await execute( @@ -298,7 +377,12 @@ export namespace InputResourceRouter { effects ); return resource - ? { status: "accept", decision, resource } + ? { + status: "accept", + decision, + materializedRoute: routeForPrepared(resource), + resource, + } : { status: "reject", decision, @@ -320,14 +404,25 @@ export namespace InputResourceRouter { case "provider-file": return { kind: "file", + id: resource.sourceId, name: resource.name, mime: resource.mimeType, size: resource.size, url: resource.url, }; + case "provider-and-scratch-file": + return { + kind: "file", + id: resource.sourceId, + name: resource.name, + mime: resource.mimeType, + size: resource.size, + url: resource.provider.url, + }; case "scratch-file": return { kind: "file", + id: resource.sourceId, name: resource.name, mime: resource.mimeType, size: resource.size, @@ -335,12 +430,14 @@ export namespace InputResourceRouter { case "directory-reference": return { kind: "directory", + id: resource.sourceId, name: resource.name, ref: resource.ref, }; case "path-reference": return { kind: "file", + id: resource.sourceId, name: resource.name, mime: resource.mimeType, size: resource.size, @@ -349,6 +446,7 @@ export namespace InputResourceRouter { case "url-reference": return { kind: "file", + id: resource.sourceId, name: resource.name, mime: resource.mimeType, size: resource.size, @@ -370,43 +468,65 @@ export namespace InputResourceRouter { } ): Lowered { const files: FileUIPart[] = []; - const operable: Array = []; + const operable: EncodedOperableResource[] = []; const scratchCandidates: Array<{ attachmentId: string; - resource: Extract; + resource: { + name: string; + mimeType: string; + size: number; + base64: string; + }; + required: boolean; + providerDelivered: boolean; + /** Zero-based index among provider-native file parts in this message. */ + providerFileIndex?: number; }> = []; const directories: DirectoryScopeDescriptor[] = []; const references: Reference[] = []; + const routeByAttachmentId = new Map(); const rejected: Lowered["rejected"] = []; for (const { attachmentId, resource } of bound) { switch (resource.kind) { - case "provider-file": - if ( - !( - resource.representation === "inline-bytes" - ? input.provider.inlineMimes - : input.provider.remoteUrlMimes - ).includes(resource.mimeType) - ) { + case "provider-file": { + if (appendProviderFile(files, resource, input.provider)) { + routeByAttachmentId.set(attachmentId, routeForPrepared(resource)); + } else { rejected.push({ attachmentId, reason: "provider-capability-unavailable", }); - break; } - files.push({ - type: "file", - url: resource.url, - mediaType: resource.mimeType, - filename: resource.name, + break; + } + case "provider-and-scratch-file": { + const providerFileIndex = files.length; + const providerDelivered = appendProviderFile( + files, + resource.provider, + input.provider + ); + scratchCandidates.push({ + attachmentId, + resource, + required: !providerDelivered, + providerDelivered, + ...(providerDelivered ? { providerFileIndex } : {}), }); break; + } case "scratch-file": - scratchCandidates.push({ attachmentId, resource }); + scratchCandidates.push({ + attachmentId, + resource, + required: true, + providerDelivered: false, + }); break; case "directory-reference": directories.push(resource.ref); + routeByAttachmentId.set(attachmentId, routeForPrepared(resource)); break; case "path-reference": references.push({ @@ -415,6 +535,7 @@ export namespace InputResourceRouter { path: resource.path, space: resource.space, }); + routeByAttachmentId.set(attachmentId, routeForPrepared(resource)); break; case "url-reference": references.push({ @@ -422,28 +543,96 @@ export namespace InputResourceRouter { name: resource.name, url: resource.url, }); + routeByAttachmentId.set(attachmentId, routeForPrepared(resource)); break; } } - const scratchRejection = scratchBatchRejection( - scratchCandidates.map(({ resource }) => resource), + const requiredScratch = scratchCandidates.filter( + (candidate) => candidate.required + ); + const optionalScratch = scratchCandidates.filter( + (candidate) => !candidate.required + ); + const requiredScratchRejection = scratchBatchRejection( + requiredScratch.map(({ resource }) => resource), input.scratch ); - if (scratchRejection) { - for (const { attachmentId } of scratchCandidates) { - rejected.push({ attachmentId, reason: scratchRejection }); + const acceptedScratch: typeof scratchCandidates = []; + const omittedScratch: Array<{ + candidate: (typeof scratchCandidates)[number]; + reason: InputResourcePolicy.UnavailableReason; + }> = []; + + if (requiredScratchRejection) { + for (const candidate of scratchCandidates) { + omittedScratch.push({ + candidate, + reason: requiredScratchRejection, + }); } } else { - for (const { attachmentId, resource } of scratchCandidates) { + acceptedScratch.push(...requiredScratch); + for (const candidate of optionalScratch) { + const rejection = scratchBatchRejection( + [ + ...acceptedScratch.map(({ resource }) => resource), + candidate.resource, + ], + input.scratch + ); + if (rejection) { + omittedScratch.push({ candidate, reason: rejection }); + } else { + acceptedScratch.push(candidate); + } + } + } + + const acceptedScratchSet = new Set(acceptedScratch); + for (const candidate of scratchCandidates) { + const scratchDelivered = acceptedScratchSet.has(candidate); + if (scratchDelivered) { + const { attachmentId, resource } = candidate; operable.push({ id: attachmentId, name: resource.name, mime: resource.mimeType, size: resource.size, base64: resource.base64, + ...(candidate.providerFileIndex !== undefined + ? { providerFileIndex: candidate.providerFileIndex } + : {}), }); } + const route = + scratchDelivered && candidate.providerDelivered + ? ({ + kind: "attachment", + via: "provider-and-scratch", + from: "bytes", + representation: "inline-bytes", + } satisfies InputResourcePolicy.Route) + : scratchDelivered + ? ({ + kind: "attachment", + via: "scratch", + from: "bytes", + } satisfies InputResourcePolicy.Route) + : candidate.providerDelivered + ? ({ + kind: "attachment", + via: "provider", + from: "bytes", + representation: "inline-bytes", + } satisfies InputResourcePolicy.Route) + : undefined; + if (route) routeByAttachmentId.set(candidate.attachmentId, route); + } + for (const { candidate, reason } of omittedScratch) { + if (candidate.required || !candidate.providerDelivered) { + rejected.push({ attachmentId: candidate.attachmentId, reason }); + } } const upload = lowerOperableFiles(operable, { @@ -464,7 +653,11 @@ export namespace InputResourceRouter { } : undefined; - return { files, extras, references, rejected }; + const routes = bound.flatMap(({ attachmentId }) => { + const route = routeByAttachmentId.get(attachmentId); + return route ? [{ attachmentId, route }] : []; + }); + return { files, extras, references, routes, rejected }; } export function describe( @@ -555,6 +748,81 @@ export namespace InputResourceRouter { ): Promise { const base = preparedBase(input); if (route.kind === "attachment") { + if ( + route.via === "provider-and-scratch" && + route.from === "bytes" && + route.representation === "inline-bytes" + ) { + if (input.kind !== "browser-file") return null; + const scratch = environment.attachment.scratch; + if (!scratch) return null; + const [providerResult, operableResult] = await Promise.allSettled([ + effects.encodeProviderFile(input.file, { + outputMimes: environment.attachment.provider.inlineMimes, + }), + effects.encodeOperableFile(input.file, { + maxBytes: scratch.maxFileBytes, + }), + ]); + const provider = + providerResult.status === "fulfilled" && + providerResult.value && + isValidEncodedProviderFile( + providerResult.value, + environment.attachment.provider.inlineMimes + ) + ? providerResult.value + : null; + const operable = + operableResult.status === "fulfilled" && + operableResult.value && + isValidEncodedOperableFile( + operableResult.value, + input.file.size, + scratch.maxFileBytes + ) + ? operableResult.value + : null; + if (provider && operable) { + return { + ...base, + kind: "provider-and-scratch-file", + name: operable.name, + mimeType: operable.mime, + size: operable.size, + base64: operable.base64, + provider: { + name: provider.name, + mimeType: provider.mime, + size: provider.size, + url: provider.url, + representation: "inline-bytes", + }, + }; + } + if (provider) { + return { + ...base, + kind: "provider-file", + name: provider.name, + mimeType: provider.mime, + size: provider.size, + url: provider.url, + representation: "inline-bytes", + }; + } + if (operable) { + return { + ...base, + kind: "scratch-file", + name: operable.name, + mimeType: operable.mime, + size: operable.size, + base64: operable.base64, + }; + } + return null; + } if ( route.via === "provider" && route.from === "bytes" && @@ -679,6 +947,30 @@ export namespace InputResourceRouter { return null; } + function appendProviderFile( + files: FileUIPart[], + resource: Readonly<{ + name: string; + mimeType: string; + url: string; + representation: "inline-bytes" | "remote-url"; + }>, + capability: Readonly + ): boolean { + const supportedMimes = + resource.representation === "inline-bytes" + ? capability.inlineMimes + : capability.remoteUrlMimes; + if (!supportedMimes.includes(resource.mimeType)) return false; + files.push({ + type: "file", + url: resource.url, + mediaType: resource.mimeType, + filename: resource.name, + }); + return true; + } + /** * Inline provider delivery has one honest wire shape: a bounded data URL * whose declared MIME and decoded byte count match the prepared metadata. @@ -760,7 +1052,7 @@ export namespace InputResourceRouter { : "preparation-failed"; } - function isScratchDecision( + function isScratchOnlyDecision( decision: InputResourcePolicy.Decision ): decision is Extract & { route: { kind: "attachment"; via: "scratch"; from: "bytes" }; @@ -772,6 +1064,92 @@ export namespace InputResourceRouter { ); } + function isProviderAndScratchDecision( + decision: InputResourcePolicy.Decision + ): decision is Extract & { + route: { + kind: "attachment"; + via: "provider-and-scratch"; + from: "bytes"; + representation: "inline-bytes"; + }; + } { + return ( + decision.status === "accept" && + decision.route.kind === "attachment" && + decision.route.via === "provider-and-scratch" + ); + } + + function fallbackWithoutScratch( + facts: Readonly, + available: Readonly, + config: InputResourcePolicy.Config, + reason: InputResourcePolicy.UnavailableReason + ): InputResourcePolicy.Decision { + const fallback = InputResourcePolicy.decide( + facts, + { + reference: available.reference, + attachment: { provider: available.attachment.provider }, + }, + config + ); + const trace = fallback.trace.map((entry) => + entry.preference === "provider-and-scratch-bytes-attachment" && + entry.reason === "scratch-unavailable" + ? { ...entry, reason } + : entry + ); + return fallback.status === "accept" + ? { ...fallback, trace } + : { + ...fallback, + reason: + fallback.reason === "scratch-unavailable" + ? reason + : fallback.reason, + trace, + }; + } + + function routeForPrepared( + resource: Readonly + ): InputResourcePolicy.Route { + switch (resource.kind) { + case "provider-file": + return { + kind: "attachment", + via: "provider", + from: resource.source === "library" ? "url" : "bytes", + representation: resource.representation, + }; + case "provider-and-scratch-file": + return { + kind: "attachment", + via: "provider-and-scratch", + from: "bytes", + representation: "inline-bytes", + }; + case "scratch-file": + return { kind: "attachment", via: "scratch", from: "bytes" }; + case "directory-reference": + return { + kind: "reference", + via: "host-scope", + resource: "directory", + }; + case "path-reference": + return { + kind: "reference", + via: "path", + space: resource.space, + }; + case "url-reference": + return { kind: "reference", via: "url" }; + } + } + function scratchBatchRejection( resources: readonly { size: number }[], limits: @@ -794,6 +1172,24 @@ export namespace InputResourceRouter { return totalBytes > limits.maxTotalBytes ? "scratch-budget-exceeded" : null; } + function retainedTwinBatchRejection( + resources: readonly { size: number }[], + limits: Readonly< + NonNullable + > + ): InputResourcePolicy.UnavailableReason | null { + if (resources.length > limits.maxFiles) { + return "draft-operable-copy-budget-exceeded"; + } + const totalBytes = resources.reduce( + (sum, resource) => sum + resource.size, + 0 + ); + return totalBytes > limits.maxTotalBytes + ? "draft-operable-copy-budget-exceeded" + : null; + } + function preparedBase(input: Readonly): PreparedBase { switch (input.kind) { case "browser-file": diff --git a/editor/lib/agent-chat/use-turn-queue-controller.ts b/editor/lib/agent-chat/use-turn-queue-controller.ts index 702d12a39..d10110d77 100644 --- a/editor/lib/agent-chat/use-turn-queue-controller.ts +++ b/editor/lib/agent-chat/use-turn-queue-controller.ts @@ -46,9 +46,9 @@ export type UseTurnQueueControllerArgs = { /** * Start a brand-new turn NOW (the session is idle). The surface owns the * request body (model, skills, session id). Called by {@link submit} when - * the session is not busy. `files` carries inline image attachments - * (perceive-only) and `extras` carries operable "+"-uploads (scratch bytes + - * their marker) — both on the immediate-send path only. + * the session is not busy. `files` carries provider-native attachments and + * `extras` carries operable scratch copies + their marker — a raster upload + * may intentionally appear in both. Both use the immediate-send path only. */ send: ( text: string, @@ -71,8 +71,8 @@ export type UseTurnQueueControllerResult = { * the transcript (atomic move, no server delete). */ drop: (messageId: string) => void; /** The composer's submit handler: enqueue while busy/human-blocked, else send. - * `files` (inline images) and `extras` (operable uploads) only flow on the - * send-now path — the queue is text-only. */ + * `files` (provider-native media) and `extras` (operable scratch copies) + * only flow on the send-now path — the queue is text-only. */ submit: ( text: string, files?: FileUIPart[], diff --git a/editor/lib/desktop/bridge.test.ts b/editor/lib/desktop/bridge.test.ts index 7e15ff45b..ed53b0cc8 100644 --- a/editor/lib/desktop/bridge.test.ts +++ b/editor/lib/desktop/bridge.test.ts @@ -88,6 +88,7 @@ describe("desktop bridge client contract", () => { }); expect(ai.supportsScratchSeedBase64(getDesktopBridge())).toBe(false); + expect(ai.supportsScratchBinaryTools(getDesktopBridge())).toBe(false); }); it("accepts base64 scratch seeds only from an explicit host capability", () => { @@ -97,12 +98,16 @@ describe("desktop bridge client contract", () => { app: { version: "0.0.0", platform: "darwin" }, caps: { native: {}, - agent: { scratch_seed_base64: true }, + agent: { + scratch_seed_base64: true, + scratch_binary_tools: true, + }, }, } as unknown as DesktopBridge, }); expect(ai.supportsScratchSeedBase64(getDesktopBridge())).toBe(true); + expect(ai.supportsScratchBinaryTools(getDesktopBridge())).toBe(true); }); it("refuses base64 scratch seeds before calling an old host", async () => { diff --git a/editor/lib/desktop/bridge.ts b/editor/lib/desktop/bridge.ts index 9713ff236..90b98a73a 100644 --- a/editor/lib/desktop/bridge.ts +++ b/editor/lib/desktop/bridge.ts @@ -648,6 +648,17 @@ export namespace ai { return bridge?.caps.agent?.scratch_seed_base64 === true; } + /** + * Whether scratch-backed arbitrary bytes have an actual operability tool. + * A base64 staging transport alone is insufficient: without this capability + * a PDF/archive would become only an inert path in the prompt. + */ + export function supportsScratchBinaryTools( + bridge: DesktopBridge | null + ): boolean { + return bridge?.caps.agent?.scratch_binary_tools === true; + } + /** * Start an agent run. Resolves with `{streamId, done}` as * soon as the SSE connection is established; `onChunk` is invoked diff --git a/editor/scaffolds/desktop/shared/agent-composer-input.tsx b/editor/scaffolds/desktop/shared/agent-composer-input.tsx index 38823f62c..cc5dcbf09 100644 --- a/editor/scaffolds/desktop/shared/agent-composer-input.tsx +++ b/editor/scaffolds/desktop/shared/agent-composer-input.tsx @@ -61,6 +61,7 @@ import { } from "@/lib/agent-chat"; import { ai as desktopAi, useDesktopBridge } from "@/lib/desktop/bridge"; import { AgentLibraryAttachmentPicker } from "./agent-library-attachment-picker"; +import { AgentComposerResourceId } from "./agent-composer-resource-id"; import { AgentComposerQueueSubmitGuard, isSameComposerDraft, @@ -88,10 +89,10 @@ export type AgentComposerInputProps = { * of the catalog's own commands, and intercepted on submit. */ commandActions?: ComposerCommandAction[]; /** - * Receives the lowered prompt text, any inlined image attachments as AI-SDK - * `file` parts (perceive-only), and `extras` — operable file uploads (scratch - * bytes + their marker). Empty submissions (no text AND no attachments) are - * filtered unless `allowEmptySubmit` is set. + * Receives the lowered prompt text, provider-native media as AI-SDK `file` + * parts, and `extras` — operable scratch copies + their marker. One raster + * upload may intentionally appear in both. Empty submissions (no text AND no + * attachments) are filtered unless `allowEmptySubmit` is set. */ onSubmit: ( text: string, @@ -252,8 +253,8 @@ function AgentComposerInner({ return () => queueSubmitGuard.unmount(); }, [queueSubmitGuard]); - // Minimal, neutral inline feedback for the two cases that would otherwise do - // nothing visible: a non-vision model, or attempting to queue images. + // Minimal, neutral inline feedback for cases that would otherwise do nothing + // visible: a provider-capability miss or a busy out-of-band attachment send. const [notice, setNotice] = useState(null); const noticeTimer = useRef | null>(null); const notify = useCallback((msg: string | null) => { @@ -270,11 +271,6 @@ function AgentComposerInner({ const preparedResources = useRef(new PreparedResourceLedger()); const preparationTail = useRef>(Promise.resolve()); - const resourceSequence = useRef(0); - const nextResourceId = useCallback((source: string) => { - resourceSequence.current += 1; - return `${source}-${resourceSequence.current}`; - }, []); // Feasibility is dynamic and separate from the selected preference policy. // In particular, a File from paste/drop/picker is bytes-only until a trusted @@ -284,6 +280,8 @@ function AgentComposerInner({ : undefined; const scratchSeedBase64Supported = desktopAi.supportsScratchSeedBase64(desktopBridge); + const scratchBinaryToolsSupported = + desktopAi.supportsScratchBinaryTools(desktopBridge); const resourceEnvironment = useMemo( () => ({ // The runtime has path-aware filesystem tools but no general URL reader. @@ -300,6 +298,7 @@ function AgentComposerInner({ maxFiles: OPERABLE_FILE_POLICY.maxFiles, maxTotalBytes: OPERABLE_FILE_POLICY.maxTotalBytes, reservation: scratchReservation, + binaryTools: scratchBinaryToolsSupported, }, } : {}), @@ -307,6 +306,7 @@ function AgentComposerInner({ }), [ attachDirectory, + scratchBinaryToolsSupported, scratchSeedBase64Supported, operableFiles, providerFileMimes, @@ -398,14 +398,14 @@ function AgentComposerInner({ if (resource.kind === "file") { inputs.push({ kind: "browser-file", - id: nextResourceId(event.source), + id: AgentComposerResourceId.create(event.source), source: event.source, file: resource.file, }); } else if (event.source === "drop") { inputs.push({ kind: "browser-directory", - id: nextResourceId(event.source), + id: AgentComposerResourceId.create(event.source), source: event.source, directory: resource.file, }); @@ -413,7 +413,7 @@ function AgentComposerInner({ } void prepareResources(inputs); }, - [nextResourceId, prepareResources] + [prepareResources] ); // "+" upload feeds the same router with explicit picker provenance. The input @@ -432,14 +432,14 @@ function AgentComposerInner({ void prepareResources( files.map((file) => ({ kind: "browser-file", - id: nextResourceId("picker"), + id: AgentComposerResourceId.create("picker"), source: "picker", file, })) ); } }, - [nextResourceId, prepareResources] + [prepareResources] ); const [libraryOpen, setLibraryOpen] = useState(false); @@ -469,8 +469,9 @@ function AgentComposerInner({ return; } // No blanket `isStreaming` early-return: submitting WHILE a turn streams is - // how a TEXT message gets queued (RFC `queue`). Images are the exception — - // the turn queue persists text only, so image sends need an idle session. + // how a TEXT message gets queued (RFC `queue`). Out-of-band attachments are + // the exception — the turn queue persists text only, so they need an idle + // session. // `allow_empty` mirrors the later `allowEmptySubmit` guard: without it the // composer core returns null for a blank editor and we'd bail here, before // that guard — so a picked-template start or host-owned empty action would @@ -807,6 +808,11 @@ function resourcePreparationNotice( ? "Some files weren't added because this chat has no scratch space." : "This chat can only attach images."; } + if (reasons.size === 1 && reasons.has("scratch-binary-tools-required")) { + return some + ? "Some files weren't added because their formats need binary tools that aren't available in this chat." + : "This file's format needs binary tools that aren't available in this chat."; + } if ( reasons.size === 1 && reasons.has("reference-capability-unavailable") && @@ -873,6 +879,9 @@ function resourceLoweringNotice( if (reasons.has("scratch-unavailable")) { return "These files need scratch space, which isn't available in this chat."; } + if (reasons.has("scratch-binary-tools-required")) { + return "These files need binary tools, which aren't available in this chat."; + } if (reasons.has("file-too-large")) { return "An attached file is too large — remove it and try again."; } diff --git a/editor/scaffolds/desktop/shared/agent-composer-resource-id.test.ts b/editor/scaffolds/desktop/shared/agent-composer-resource-id.test.ts new file mode 100644 index 000000000..0b730f3f9 --- /dev/null +++ b/editor/scaffolds/desktop/shared/agent-composer-resource-id.test.ts @@ -0,0 +1,28 @@ +import { afterEach, describe, expect, it, vi } from "vitest"; +import { ComposerCore } from "@/kits/composer/composer-core"; +import { AgentComposerResourceId } from "./agent-composer-resource-id"; + +describe("AgentComposerResourceId", () => { + afterEach(() => { + vi.restoreAllMocks(); + }); + + it("does not reuse ids across remount-equivalent allocations", () => { + vi.spyOn(globalThis.crypto, "randomUUID") + .mockReturnValueOnce("00000000-0000-4000-8000-000000000001") + .mockReturnValueOnce("00000000-0000-4000-8000-000000000002"); + + const beforeRemount = new ComposerCore().addAttachment({ + id: AgentComposerResourceId.create("drop"), + name: "screen.png", + }); + const afterRemount = new ComposerCore().addAttachment({ + id: AgentComposerResourceId.create("drop"), + name: "screen.png", + }); + + expect(beforeRemount?.id).toBe("drop-00000000-0000-4000-8000-000000000001"); + expect(afterRemount?.id).toBe("drop-00000000-0000-4000-8000-000000000002"); + expect(afterRemount?.id).not.toBe(beforeRemount?.id); + }); +}); diff --git a/editor/scaffolds/desktop/shared/agent-composer-resource-id.ts b/editor/scaffolds/desktop/shared/agent-composer-resource-id.ts new file mode 100644 index 000000000..781c6e5ef --- /dev/null +++ b/editor/scaffolds/desktop/shared/agent-composer-resource-id.ts @@ -0,0 +1,14 @@ +import type { InputResourceRouter } from "@/lib/agent-chat"; + +/** + * Resource identity must outlive a particular composer mount because it also + * becomes part of the scratch filename. Browser UUIDs avoid reusing an id when + * the composer is unmounted and later recreated in the same chat session. + */ +export namespace AgentComposerResourceId { + export function create( + source: InputResourceRouter.BrowserFileSource + ): string { + return `${source}-${globalThis.crypto.randomUUID()}`; + } +} diff --git a/packages/grida-ai-agent/README.md b/packages/grida-ai-agent/README.md index beccb10b7..b387ef5a8 100644 --- a/packages/grida-ai-agent/README.md +++ b/packages/grida-ai-agent/README.md @@ -104,6 +104,25 @@ orthogonal: it remains enabled by default for compatibility, and a host with a listener-independent request transport can pass `allow_local_binding: false`. The outbound default is `"allowlisted"`, preserving CLI behavior. +## Finite command execution + +`run_command` is exposed only when a host injects `shell_executor`. Each call +receives the validated command plus an immutable scope naming the current +workspace, optional own-session scratch, shared scratch base, and protected +read roots. The executor is the authority boundary: Desktop sends the request +to Electron main and creates a fresh OS-sandbox profile for that finite process. +The boolean `sandbox_enforced` attests only the coarse process tree (used by the +external-agent disposition); it cannot expose raw shell by itself. + +The tool abort signal is part of the executor contract. Desktop acknowledges an +aborted command only after the worker has returned and its per-command +authority has been cleaned up; the runtime keeps the owning session occupied +until that acknowledgement and the aborted model pump settle. + +A standalone host that deliberately accepts ambient filesystem authority may +set `allow_unsandboxed_shell`; this injects the package's raw runner and logs +the weaker posture. Omission of both options withholds the tool. + ## Anti-goals The perimeter that keeps this package small. A feature request that diff --git a/packages/grida-ai-agent/src/__public-api__.test.ts b/packages/grida-ai-agent/src/__public-api__.test.ts index 0c0eb9f2c..d2c758f10 100644 --- a/packages/grida-ai-agent/src/__public-api__.test.ts +++ b/packages/grida-ai-agent/src/__public-api__.test.ts @@ -50,6 +50,8 @@ import { AGENT_DAEMON_DEFAULT_CAPABILITIES, createAgentDaemon, createAgentTenant, + defaultScratchBase, + prepareScratchAuthority, Daemon, DaemonServer, DAEMON_PROTOCOL, @@ -60,6 +62,9 @@ import { type DaemonHandshakeResponse, type DaemonHttpAccess, type DaemonTenant, + type ShellExecutionScope, + type ShellExecutor, + type ShellRunOptions, } from "./server"; import { AcpAgentAdapter, @@ -256,12 +261,34 @@ describe("@grida/agent public API", () => { request: globalThis.fetch, download: globalThis.fetch, }; + const shellExecutor: ShellExecutor = async (request) => ({ + ...request, + exit_code: 0, + signal: null, + stdout: "", + stderr: "", + duration_ms: 0, + timed_out: false, + truncated: false, + }); const tenantOpts: AgentTenantOptions = { interactive: true, provider_http: providerHttp, + shell_executor: shellExecutor, + }; + const scope: ShellExecutionScope = { + workspace_root: "/workspace", + protected_read_roots: [], }; + const runOptions: ShellRunOptions = {}; const tenant: DaemonTenant = createAgentTenant(tenantOpts); expect(typeof tenant.register).toBe("function"); + expect(scope.workspace_root).toBe("/workspace"); + expect(runOptions).toEqual({}); + expect(defaultScratchBase("/agent", "/native-temp")).toMatch( + /^[/\\]native-temp[/\\]grida-agent-[0-9a-f]{16}$/ + ); + expect(typeof prepareScratchAuthority).toBe("function"); expect(tenant.sse_query_token_paths?.length).toBe(2); expect(typeof createAgentDaemon).toBe("function"); // Host HTTP stays behind the Node-only server entry. Neither the raw diff --git a/packages/grida-ai-agent/src/agent/image-gen-capability.test.ts b/packages/grida-ai-agent/src/agent/image-gen-capability.test.ts index ac8da8021..d4063bbce 100644 --- a/packages/grida-ai-agent/src/agent/image-gen-capability.test.ts +++ b/packages/grida-ai-agent/src/agent/image-gen-capability.test.ts @@ -8,16 +8,6 @@ import { describe, expect, it } from "vitest"; import { buildCapabilityHints, type CreateAgentOptions } from "./index"; -const NOOP_BACKEND: NonNullable< - CreateAgentOptions["command"] ->["backend"] = async () => ({ - stdout: "", - stderr: "", - exit_code: 0, - timed_out: false, - truncated: false, -}); - const SCRATCH = "/tmp/grida-agent/sessions/ses_x/scratch"; const FAKE_GENERATOR: NonNullable = { @@ -35,15 +25,11 @@ function imageGenHint(hints: string[]): string | undefined { } describe("buildCapabilityHints — image generation", () => { - it("advertises generate_image when a generator and scratch path are wired", () => { + it("advertises generate_image when a generator and standalone scratch path are wired", () => { const hints = buildCapabilityHints({ model_factory: modelFactory, image_gen: FAKE_GENERATOR, - command: { - backend: NOOP_BACKEND, - default_workdir: "/work", - scratch_dir: SCRATCH, - }, + scratch_dir: SCRATCH, }); const hint = imageGenHint(hints); expect(hint).toBeDefined(); @@ -58,11 +44,7 @@ describe("buildCapabilityHints — image generation", () => { it("omits the hint when no generator is wired (no provider key)", () => { const hints = buildCapabilityHints({ model_factory: modelFactory, - command: { - backend: NOOP_BACKEND, - default_workdir: "/work", - scratch_dir: SCRATCH, - }, + scratch_dir: SCRATCH, }); expect(imageGenHint(hints)).toBeUndefined(); }); @@ -71,7 +53,6 @@ describe("buildCapabilityHints — image generation", () => { const hints = buildCapabilityHints({ model_factory: modelFactory, image_gen: FAKE_GENERATOR, - command: { backend: NOOP_BACKEND, default_workdir: "/work" }, }); expect(imageGenHint(hints)).toBeUndefined(); }); diff --git a/packages/grida-ai-agent/src/agent/index.ts b/packages/grida-ai-agent/src/agent/index.ts index 55a7a4470..878b4cd99 100644 --- a/packages/grida-ai-agent/src/agent/index.ts +++ b/packages/grida-ai-agent/src/agent/index.ts @@ -16,6 +16,10 @@ * capability hint. Without it the agent can't * even see command execution exists (LLM-level * safety, not just runtime gating). + * - `scratch_dir` — optional. The host-provisioned session scratch + * root already reachable through the injected + * filesystem bindings. This advertises its address; + * it grants no filesystem capability by itself. * - `onStepFinish` — optional diagnostic hook (per-step logger). * * The fs / todos tools are always present — they're the baseline. The @@ -111,6 +115,14 @@ export type CreateAgentOptions = { /** Command-execution capability. Without it, the `run_command` tool is * not registered and the LLM cannot call it. */ command?: ToolsetCapabilities["command"]; + /** + * Host-provisioned per-session scratch root already reachable through the + * injected filesystem/vision bindings. This is prompt metadata only: passing + * it grants no reach, so hosts MUST omit it unless those bindings can resolve + * the exact root. Kept separate from `command` because a sandbox-withheld + * shell must not hide scratch from the structured filesystem tools. + */ + scratch_dir?: string; /** * Discovered skills (RFC `skills`). When provided, their descriptions * are advertised in the system prompt and the locked `skill` tool joins @@ -274,11 +286,11 @@ export function gridaAttribution( /** * Free-form capability hints appended to the composed prompt. * - * Today: command execution + the session scratch dir (gated on the command, - * since scratch reach rides the shell) + vision. The blurb tells the LLM the - * tool exists, what defaults apply, and what enforcement to expect — kept to a - * few lines so it doesn't crowd the per-skill blocks. Exported (module-level, - * not at the package root) so the gating is unit-pinned without driving a model. + * Today: command execution + the independently provisioned session scratch dir + * + vision. The blurb tells the LLM what exists, what defaults apply, and what + * enforcement to expect — kept to a few lines so it doesn't crowd the + * per-skill blocks. Exported (module-level, not at the package root) so the + * gating is unit-pinned without driving a model. */ export function buildCapabilityHints(opts: CreateAgentOptions): string[] { // Surface presentation is locked into every toolset. The same generic hint @@ -297,17 +309,21 @@ export function buildCapabilityHints(opts: CreateAgentOptions): string[] { opts.command.default_workdir ) ); - // Scratch reach rides the shell, so it is advertised only alongside the - // command capability. Promote to a standalone hint if structured-fs or - // perception reach for scratch lands later (WG `scratch.md`). - if (opts.command.scratch_dir) { - hints.push( - prompts.scratch_capability( - RUN_COMMAND_TOOL_NAME, - opts.command.scratch_dir - ) - ); - } + } + // `scratch_dir` is the canonical host attestation. Keep the command-carried + // field as a compatibility fallback for callers predating the standalone + // seam; neither arm derives or widens filesystem authority. + const scratchDir = opts.scratch_dir ?? opts.command?.scratch_dir; + if (scratchDir) { + hints.push( + prompts.scratch_capability(scratchDir, { + // Only the standalone field attests that the injected structured + // filesystem resolves this root. The legacy command-carried fallback + // attests shell reach only. + filesystem: opts.scratch_dir !== undefined, + run_command_name: opts.command ? RUN_COMMAND_TOOL_NAME : undefined, + }) + ); } if (opts.vision) { hints.push(prompts.vision_capability(AgentVision.TOOL_NAMES.view_image)); @@ -315,11 +331,11 @@ export function buildCapabilityHints(opts: CreateAgentOptions): string[] { // Image generation rides scratch (its produced files sink there), so it is // advertised only when the generator AND a scratch path are wired — the same // gating that builds the generator binding in the first place. - if (opts.image_gen && opts.command?.scratch_dir) { + if (opts.image_gen && scratchDir) { hints.push( prompts.image_gen_capability( AgentGen.TOOL_NAMES.generate_image, - opts.command.scratch_dir + scratchDir ) ); } diff --git a/packages/grida-ai-agent/src/agent/scratch-capability.test.ts b/packages/grida-ai-agent/src/agent/scratch-capability.test.ts index 021994104..49ee4f65e 100644 --- a/packages/grida-ai-agent/src/agent/scratch-capability.test.ts +++ b/packages/grida-ai-agent/src/agent/scratch-capability.test.ts @@ -1,11 +1,12 @@ /** * Scratch capability injection (WG `scratch.md` S1: the agent is *told* its - * scratch location). Pins the gating in `buildCapabilityHints`: when a command - * binding carries a `scratch_dir`, a scratch hint is emitted advertising the - * path; with no scratch_dir (or no command at all) the hint is absent. Scratch - * reach rides the shell, so the hint is coupled to the command capability. + * scratch location). Pins the gating in `buildCapabilityHints`: the + * host-provisioned `scratch_dir` advertises the exact root independently of + * command execution, so structured filesystem reach remains discoverable when + * the fail-closed host posture withholds the shell. */ import { describe, expect, it } from "vitest"; +import { AgentFs } from "../fs"; import { buildCapabilityHints, type CreateAgentOptions } from "./index"; const NOOP_BACKEND: NonNullable< @@ -30,7 +31,26 @@ const modelFactory: CreateAgentOptions["model_factory"] = () => { }; describe("buildCapabilityHints — scratch", () => { - it("advertises the scratch path when the command binding carries scratch_dir (S1)", () => { + it("advertises structured-fs scratch without exposing command execution (S1)", () => { + const hints = buildCapabilityHints({ + model_factory: modelFactory, + fs: new AgentFs(new AgentFs.MemoryBackend()), + scratch_dir: SCRATCH, + }); + const hint = scratchHint(hints); + expect(hint).toBeDefined(); + expect(hint).toContain(SCRATCH); + expect(hint!.toLowerCase()).toContain("filesystem"); + expect(hint).not.toContain("run_command"); + expect(hints.some((h) => h.includes(''))).toBe( + false + ); + // The promotion + ephemerality guidance is the load-bearing part of S2. + expect(hint!.toLowerCase()).toContain("ephemeral"); + expect(hint!.toLowerCase()).toContain("promote"); + }); + + it("keeps the command-carried path as a compatibility fallback", () => { const hints = buildCapabilityHints({ model_factory: modelFactory, command: { @@ -40,14 +60,12 @@ describe("buildCapabilityHints — scratch", () => { }, }); const hint = scratchHint(hints); - expect(hint).toBeDefined(); expect(hint).toContain(SCRATCH); - // The promotion + ephemerality guidance is the load-bearing part of S2. - expect(hint!.toLowerCase()).toContain("ephemeral"); - expect(hint!.toLowerCase()).toContain("promote"); + expect(hint).toContain("run_command"); + expect(hint!.toLowerCase()).not.toContain("filesystem tools"); }); - it("omits the scratch hint when the command binding has no scratch_dir", () => { + it("omits the scratch hint when no root was provisioned", () => { const hints = buildCapabilityHints({ model_factory: modelFactory, command: { backend: NOOP_BACKEND, default_workdir: "/work" }, @@ -59,7 +77,7 @@ describe("buildCapabilityHints — scratch", () => { ); }); - it("omits the scratch hint when there is no command capability at all", () => { + it("omits the scratch hint when neither scratch nor command is present", () => { const hints = buildCapabilityHints({ model_factory: modelFactory }); expect(scratchHint(hints)).toBeUndefined(); }); diff --git a/packages/grida-ai-agent/src/e2e-approval-resume.test.ts b/packages/grida-ai-agent/src/e2e-approval-resume.test.ts index 31eae7ab1..67e697de6 100644 --- a/packages/grida-ai-agent/src/e2e-approval-resume.test.ts +++ b/packages/grida-ai-agent/src/e2e-approval-resume.test.ts @@ -26,6 +26,7 @@ import { readUIMessageStream, type UIMessage, type UIMessageChunk } from "ai"; import { MockLanguageModelV3, simulateReadableStream } from "ai/test"; import { AuthStore, + runUnsandboxedShell, SecretsStore, WorkspaceRegistry, } from "@grida/daemon/server"; @@ -154,7 +155,7 @@ beforeEach(async () => { skill_discovery: { include_user_scoped: false, stop_at: workspaceRoot }, // Mirror the production sidecar: the host affirmed containment, so the // `run_command` tool is wired (GRIDA-SEC-004 fail-closed gate). - shell_execution_allowed: true, + shell_executor: runUnsandboxedShell, }); registerAgentRoutes(app, runtime); registerSessionsRoutes(app, { store, runtime }); diff --git a/packages/grida-ai-agent/src/http/routes/agent.test.ts b/packages/grida-ai-agent/src/http/routes/agent.test.ts index 67df97566..7af840f32 100644 --- a/packages/grida-ai-agent/src/http/routes/agent.test.ts +++ b/packages/grida-ai-agent/src/http/routes/agent.test.ts @@ -23,6 +23,11 @@ import { openSessionsDb, type OpenedSessionsDb } from "../../session/db"; import { SessionsStore } from "../../session/store"; import { createRecorderConsumer } from "../../session/recorder"; import { DirectoryScopeRegistry } from "../../session/directory-scopes"; +import { + ensureScratch, + scratchRootFor, + sweepScratch, +} from "../../session/scratch"; import { AGENT_SESSION_AGENT } from "../../protocol/run"; import { AgentRuntime } from "../../runtime"; import { @@ -221,7 +226,7 @@ describe("HTTP wire — agent routes (run/stream/abort)", () => { }); }); - it("stages byte-identical attachment bodies before persisting their descriptors", async () => { + it("stages exact image bytes while preserving provider-native perception", async () => { const workspaceDir = path.join(baseDir, "workspace"); await fs.mkdir(workspaceDir); const workspace = await workspaceRegistry.open(workspaceDir); @@ -242,16 +247,23 @@ describe("HTTP wire — agent routes (run/stream/abort)", () => { role: "user", parts: [ { type: "text", text: "inspect this" }, + { + type: "file", + mediaType: "image/png", + url: "data:image/png;base64,AAAA", + filename: "preview.png", + }, { type: "data-user_file_attachments", data: { location: "scratch", files: [ { - name: "opaque.bin", - mime: "application/octet-stream", + name: "original.gif", + mime: "image/gif", size: 4, - path: "upload.bin", + path: "upload.gif", + provider_file_index: 0, }, ], }, @@ -259,7 +271,7 @@ describe("HTTP wire — agent routes (run/stream/abort)", () => { ], }, ], - scratch_seed: [{ path: "upload.bin", base64: "AP+Afg==" }], + scratch_seed: [{ path: "upload.gif", base64: "AP+Afg==" }], }), }); expect(response.status).toBe(200); @@ -270,15 +282,146 @@ describe("HTTP wire — agent routes (run/stream/abort)", () => { "sessions", session.id, "scratch", - "upload.bin" + "upload.gif" ) ); expect([...staged]).toEqual([0, 255, 128, 126]); const persisted = await sessionsStore.listMessages(session.id); - expect(persisted[0].parts[1].type).toBe("data-user_file_attachments"); + expect(persisted[0].parts.map((part) => part.type)).toEqual([ + "text", + "file", + "data-user_file_attachments", + ]); await response.text(); }); + it("rolls back new seeds on collision without truncating the existing file", async () => { + const workspaceDir = path.join(baseDir, "collision-workspace"); + await fs.mkdir(workspaceDir); + const workspace = await workspaceRegistry.open(workspaceDir); + const session = await sessionsStore.create({ + agent: AGENT_SESSION_AGENT, + workspace_id: workspace.id, + workspace_root: workspace.root, + }); + const scratchDir = scratchRootFor( + path.join(baseDir, "scratch-base"), + session.id + ); + await ensureScratch(scratchDir); + const target = path.join(scratchDir, "existing.bin"); + const partial = path.join(scratchDir, "new.bin"); + const original = new Uint8Array([1, 2, 3]); + await fs.writeFile(target, original); + + const response = await app.request("/agent/run", { + method: "POST", + headers: { "content-type": "application/json" }, + body: JSON.stringify({ + session_id: session.id, + workspace_id: workspace.id, + messages: [ + { + id: "u-collision", + role: "user", + parts: [ + { + type: "data-user_file_attachments", + data: { + location: "scratch", + files: [ + { + name: "new.bin", + mime: "application/octet-stream", + size: 2, + path: "new.bin", + }, + { + name: "existing.bin", + mime: "application/octet-stream", + size: 1, + path: "existing.bin", + }, + ], + }, + }, + ], + }, + ], + scratch_seed: [ + { path: "new.bin", base64: "BAU=" }, + { path: "existing.bin", base64: "CQ==" }, + ], + }), + }); + + expect(response.status).toBe(500); + expect(await response.json()).toMatchObject({ + code: "scratch-seed-failed", + session_id: session.id, + }); + expect(new Uint8Array(await fs.readFile(target))).toEqual(original); + await expect(fs.readFile(partial)).rejects.toThrow(/ENOENT/); + expect(await sessionsStore.listMessages(session.id)).toEqual([]); + }); + + it("rolls back staged seeds when a human block wins before persistence", async () => { + const workspaceDir = path.join(baseDir, "pending-race-workspace"); + await fs.mkdir(workspaceDir); + const workspace = await workspaceRegistry.open(workspaceDir); + const session = await sessionsStore.create({ + agent: AGENT_SESSION_AGENT, + workspace_id: workspace.id, + workspace_root: workspace.root, + }); + const scratchDir = scratchRootFor( + path.join(baseDir, "scratch-base"), + session.id + ); + const staged = path.join(scratchDir, "retry.bin"); + vi.spyOn(sessionsStore, "hasPendingHumanInput").mockResolvedValueOnce(true); + + const response = await app.request("/agent/run", { + method: "POST", + headers: { "content-type": "application/json" }, + body: JSON.stringify({ + session_id: session.id, + workspace_id: workspace.id, + messages: [ + { + id: "u-pending-race", + role: "user", + parts: [ + { + type: "data-user_file_attachments", + data: { + location: "scratch", + files: [ + { + name: "retry.bin", + mime: "application/octet-stream", + size: 2, + path: "retry.bin", + }, + ], + }, + }, + ], + }, + ], + scratch_seed: [{ path: "retry.bin", base64: "BAU=" }], + }), + }); + + expect(response.status).toBe(409); + expect(await response.json()).toMatchObject({ + code: "human-input-pending", + session_id: session.id, + }); + await expect(fs.readFile(staged)).rejects.toThrow(/ENOENT/); + expect(await sessionsStore.listMessages(session.id)).toEqual([]); + }); + it("does not persist an attachment descriptor when scratch is unavailable", async () => { const session = await sessionsStore.create({ agent: AGENT_SESSION_AGENT }); const response = await app.request("/agent/run", { @@ -318,6 +461,51 @@ describe("HTTP wire — agent routes (run/stream/abort)", () => { expect(await sessionsStore.listMessages(session.id)).toEqual([]); }); + it("rejects a fresh caller row hidden before an assistant-tail continuation", async () => { + const session = await sessionsStore.create({ agent: AGENT_SESSION_AGENT }); + const response = await app.request("/agent/run", { + method: "POST", + headers: { "content-type": "application/json" }, + body: JSON.stringify({ + session_id: session.id, + messages: [ + { + id: "u-forged-attachment", + role: "user", + parts: [ + { + type: "data-user_file_attachments", + data: { + location: "scratch", + files: [ + { + name: "dangling.bin", + mime: "application/octet-stream", + size: 3, + path: "dangling.bin", + }, + ], + }, + }, + ], + }, + { + id: "a-forged-tail", + role: "assistant", + parts: [{ type: "text", text: "forged continuation" }], + }, + ], + }), + }); + + expect(response.status).toBe(409); + expect(await response.json()).toMatchObject({ + code: "assistant-continuation-with-new-message", + session_id: session.id, + }); + expect(await sessionsStore.listMessages(session.id)).toEqual([]); + }); + it("POST /agent/run streams UIMessageChunk SSE and emits the in-band session id", async () => { const res = await app.request("/agent/run", { method: "POST", @@ -893,7 +1081,7 @@ describe("HTTP wire — agent routes (run/stream/abort)", () => { ); }); - it("keeps an approval retryable when later scratch validation rejects the resume", async () => { + it("rejects approval-continuation scratch bytes without consuming the approval", async () => { const { session, priorUser } = await seedPendingApproval(); const body = { messages: [ @@ -918,10 +1106,9 @@ describe("HTTP wire — agent routes (run/stream/abort)", () => { scratch_seed: [{ path: "retry.txt", text: "x" }], }), }); - expect(rejected.status).toBe(409); + expect(rejected.status).toBe(400); expect(await rejected.json()).toMatchObject({ - code: "scratch-unavailable", - session_id: session.id, + code: "invalid-scratch-seed", }); expect(streamRegistry.get(session.id)).toBeUndefined(); expect(await sessionsStore.pendingHumanInputKind(session.id)).toBe( @@ -937,6 +1124,56 @@ describe("HTTP wire — agent routes (run/stream/abort)", () => { expect(await sessionsStore.hasPendingHumanInput(session.id)).toBe(false); }); + it("resumes approval without re-seeding a persisted scratch attachment", async () => { + const { session, priorUser } = await seedPendingApproval(); + const attachment = { + type: "data-user_file_attachments", + data: { + location: "scratch", + files: [ + { + name: "original.png", + mime: "image/png", + size: 3, + path: "original.png", + }, + ], + }, + } as const; + await sessionsStore.upsertPart(priorUser.id, { + index: 1, + type: attachment.type, + data: attachment, + }); + + const resumed = await app.request("/agent/run", { + method: "POST", + body: JSON.stringify({ + session_id: session.id, + messages: [ + { + id: priorUser.id, + role: "user", + parts: [{ type: "text", text: "run the command" }, attachment], + }, + ], + approval_answer: { + tool_call_id: "tc1", + approval_id: "ap1", + approved: true, + }, + }), + }); + + expect(resumed.status).toBe(200); + await resumed.text(); + expect(await sessionsStore.hasPendingHumanInput(session.id)).toBe(false); + const persisted = await sessionsStore.listMessages(session.id); + expect( + persisted.find((message) => message.id === priorUser.id)?.parts + ).toHaveLength(2); + }); + it("rolls an approval answer back when synchronous stream reservation fails", async () => { const { session, priorUser } = await seedPendingApproval(); const body = { @@ -1758,7 +1995,7 @@ describe("HTTP wire — agent routes (run/stream/abort)", () => { ); }); - it("keeps a question retryable when later scratch validation rejects the resume", async () => { + it("rejects question-continuation scratch bytes without consuming the answer", async () => { const { session, priorUser, assistant } = await seedPendingQuestion(); const messages = [ { @@ -1789,10 +2026,9 @@ describe("HTTP wire — agent routes (run/stream/abort)", () => { scratch_seed: [{ path: "retry.txt", text: "x" }], }), }); - expect(rejected.status).toBe(409); + expect(rejected.status).toBe(400); expect(await rejected.json()).toMatchObject({ - code: "scratch-unavailable", - session_id: session.id, + code: "invalid-scratch-seed", }); expect(streamRegistry.get(session.id)).toBeUndefined(); expect(await sessionsStore.pendingHumanInputKind(session.id)).toBe( @@ -2424,6 +2660,114 @@ describe("HTTP wire — agent routes (run/stream/abort)", () => { } }); + it("rejects replacement until an aborted command and its owning pump settle", async () => { + runtime.dispose(); + streamRegistry = new StreamRegistry(); + + let releaseCommandCleanup!: () => void; + const commandCleanup = new Promise((resolve) => { + releaseCommandCleanup = resolve; + }); + let releasePump!: () => void; + const pumpGate = new Promise((resolve) => { + releasePump = resolve; + }); + let commandStarted!: () => void; + const commandStartedGate = new Promise((resolve) => { + commandStarted = resolve; + }); + let runCount = 0; + runtime = new AgentRuntime({ + secrets, + workspace_registry: workspaceRegistry, + sessions_store: sessionsStore, + streams: streamRegistry, + run_agent: async (_provider, _request, deps) => { + runCount += 1; + if (runCount === 1) { + // This task models the sidecar command promise: abort does not settle + // it until main has observed worker exit and released its authority. + deps.track_command_execution?.(commandCleanup); + commandStarted(); + // Model an AI SDK pump that consumes the command's terminal abort only + // after host cleanup has acknowledged it. + await pumpGate; + } + return await fakeRunAgent(); + }, + scratch_base: path.join(baseDir, "scratch-base"), + external_agent_execution: "sandboxed", + drain_cooldown_ms: 20, + }); + app = new Hono(); + registerAgentRoutes(app, runtime); + + const session = await sessionsStore.create({ + agent: AGENT_SESSION_AGENT, + }); + let firstBody: Promise | undefined; + try { + const firstResponse = await app.request("/agent/run", { + method: "POST", + body: JSON.stringify({ + session_id: session.id, + messages: [ + { id: "command-turn", role: "user", content: "run a command" }, + ], + }), + }); + expect(firstResponse.status).toBe(200); + firstBody = firstResponse.text(); + await commandStartedGate; + + const aborted = await app.request("/agent/abort", { + method: "POST", + body: JSON.stringify({ session_id: session.id }), + }); + expect(aborted.status).toBe(200); + await firstBody; + + const replacementBody = JSON.stringify({ + session_id: session.id, + messages: [ + { id: "replacement-turn", role: "user", content: "replacement" }, + ], + }); + const beforeCleanup = await app.request("/agent/run", { + method: "POST", + body: replacementBody, + }); + expect(beforeCleanup.status).toBe(409); + + releaseCommandCleanup(); + await commandCleanup; + // Host cleanup has acknowledged abort, but the owning pump has not yet + // consumed that terminal result. Admission must remain closed across + // this smaller lifecycle gap too. + const beforePumpSettlement = await app.request("/agent/run", { + method: "POST", + body: replacementBody, + }); + expect(beforePumpSettlement.status).toBe(409); + + releasePump(); + await vi.waitFor(() => + expect(streamRegistry.isOccupied(session.id)).toBe(false) + ); + + const replacement = await app.request("/agent/run", { + method: "POST", + body: replacementBody, + }); + expect(replacement.status).toBe(200); + await replacement.text(); + } finally { + releaseCommandCleanup(); + releasePump(); + await firstBody?.catch(() => undefined); + } + }); + it.each(["late-response", "late-error"] as const)( "does not let an aborted turn's %s contaminate its queued replacement", async (lateOutcome) => { @@ -2731,9 +3075,11 @@ describe("HTTP wire — session-scoped directory references", () => { }); }); -describe("HTTP wire — inline image attachments (perceive-only)", () => { +describe("HTTP wire — provider-native image attachments", () => { let baseDir: string; + let scratchBase: string; let sessionsStore: SessionsStore; + let workspaceRegistry: WorkspaceRegistry; let streamRegistry: StreamRegistry; let runtime: AgentRuntime; let app: Hono; @@ -2766,7 +3112,8 @@ describe("HTTP wire — inline image attachments (perceive-only)", () => { await secrets.set("openrouter", "sk-test"); const db = openSessionsDb({ user_data_path: baseDir }); sessionsStore = new SessionsStore(db); - const workspaceRegistry = new WorkspaceRegistry(baseDir); + workspaceRegistry = new WorkspaceRegistry(baseDir); + scratchBase = path.join(baseDir, "scratch-base"); streamRegistry = new StreamRegistry(); capturedRuns = []; app = new Hono(); @@ -2776,6 +3123,7 @@ describe("HTTP wire — inline image attachments (perceive-only)", () => { sessions_store: sessionsStore, streams: streamRegistry, run_agent: capturingRunAgent as never, + scratch_base: scratchBase, drain_cooldown_ms: 20, }); registerAgentRoutes(app, runtime); @@ -2798,6 +3146,21 @@ describe("HTTP wire — inline image attachments (perceive-only)", () => { return out; } + function attachmentMarker(messages: unknown[]): string | undefined { + for (const message of messages as Array<{ parts?: unknown[] }>) { + for (const part of message.parts ?? []) { + const candidate = part as { type?: string; text?: string }; + if ( + candidate.type === "text" && + candidate.text?.includes("") + ) { + return candidate.text; + } + } + } + return undefined; + } + it("forwards an inline image file part to the model on the turn it is sent", async () => { const res = await app.request("/agent/run", { method: "POST", @@ -2900,4 +3263,114 @@ describe("HTTP wire — inline image attachments (perceive-only)", () => { expect.objectContaining({ type: "file", url: PNG_DATA_URL }) ); }); + + it("keeps structured scratch live across turns and marks it unavailable after a sweep", async () => { + const workspaceDir = path.join(baseDir, "workspace"); + await fs.mkdir(workspaceDir); + const workspace = await workspaceRegistry.open(workspaceDir); + + const first = await app.request("/agent/run", { + method: "POST", + body: JSON.stringify({ + workspace_id: workspace.id, + messages: [ + { + id: "u-dual-1", + role: "user", + parts: [ + { type: "text", text: "remember this image" }, + { + type: "file", + mediaType: "image/png", + url: PNG_DATA_URL, + filename: "shot.png", + }, + { + type: "data-user_file_attachments", + data: { + location: "scratch", + files: [ + { + name: "shot.png", + mime: "image/png", + size: 8, + path: "shot.png", + provider_file_index: 0, + }, + ], + }, + }, + ], + }, + ], + scratch_seed: [{ path: "shot.png", base64: "iVBORw0KGgo=" }], + }), + }); + expect(first.status).toBe(200); + const sessionId = sessionIdFromSse(await first.text()); + expect(attachmentMarker(capturedRuns[0].messages)).toContain( + '"available": true' + ); + + await vi.waitFor(() => { + const entry = streamRegistry.get(sessionId); + expect(entry === undefined || entry.status === "ended").toBe(true); + }); + + const second = await app.request("/agent/run", { + method: "POST", + body: JSON.stringify({ + session_id: sessionId, + workspace_id: workspace.id, + messages: [ + { + id: "u-dual-2", + role: "user", + parts: [{ type: "text", text: "use the exact bytes again" }], + }, + ], + }), + }); + expect(second.status).toBe(200); + await second.text(); + const beforeSweep = capturedRuns.at(-1); + expect(beforeSweep).toBeDefined(); + expect(attachmentMarker(beforeSweep?.messages ?? [])).toContain( + '"available": true' + ); + + await vi.waitFor(() => { + const entry = streamRegistry.get(sessionId); + expect(entry === undefined || entry.status === "ended").toBe(true); + }); + sweepScratch(scratchBase); + + const third = await app.request("/agent/run", { + method: "POST", + body: JSON.stringify({ + session_id: sessionId, + workspace_id: workspace.id, + messages: [ + { + id: "u-dual-3", + role: "user", + parts: [{ type: "text", text: "what was in the image?" }], + }, + ], + }), + }); + expect(third.status).toBe(200); + await third.text(); + const afterSweep = capturedRuns.at(-1); + expect(afterSweep).toBeDefined(); + expect(attachmentMarker(afterSweep?.messages ?? [])).toContain( + '"available": false' + ); + expect(fileParts(afterSweep?.messages ?? [])).toContainEqual( + expect.objectContaining({ type: "file", url: PNG_DATA_URL }) + ); + + const persisted = await sessionsStore.listMessages(sessionId); + expect(JSON.stringify(persisted)).not.toContain('"available"'); + }); }); diff --git a/packages/grida-ai-agent/src/http/routes/sessions-lifecycle.test.ts b/packages/grida-ai-agent/src/http/routes/sessions-lifecycle.test.ts index a8d323cb6..233eab1ab 100644 --- a/packages/grida-ai-agent/src/http/routes/sessions-lifecycle.test.ts +++ b/packages/grida-ai-agent/src/http/routes/sessions-lifecycle.test.ts @@ -284,6 +284,93 @@ describe("HTTP wire — session lifecycle (rewind/fork/compact)", () => { expect(res.status).toBe(409); }); + it("DELETE refuses while a run is in flight and preserves the session", async () => { + const { id } = await seed(1); + streams.create(id); + + const res = await app.request(`/sessions/${id}`, { method: "DELETE" }); + + expect(res.status).toBe(409); + expect(((await res.json()) as { code?: string }).code).toBe( + "run_in_flight" + ); + expect(await store.get(id)).not.toBeNull(); + }); + + it("DELETE holds admission across the DB delete and scratch cleanup", async () => { + const { id } = await seed(1); + const originalDelete = store.delete.bind(store); + let enteredDelete!: () => void; + const insideDelete = new Promise((resolve) => { + enteredDelete = resolve; + }); + let releaseDelete!: () => void; + const deleteGate = new Promise((resolve) => { + releaseDelete = resolve; + }); + let enteredCleanup!: () => void; + const insideCleanup = new Promise((resolve) => { + enteredCleanup = resolve; + }); + let releaseCleanup!: () => void; + const cleanupGate = new Promise((resolve) => { + releaseCleanup = resolve; + }); + const deleteSpy = vi + .spyOn(store, "delete") + .mockImplementation(async (sid) => { + if (sid === id) { + enteredDelete(); + await deleteGate; + } + await originalDelete(sid); + }); + const cleanupSpy = vi + .spyOn(runtime, "removeSessionScratch") + .mockImplementation(async (sid) => { + if (sid === id) { + enteredCleanup(); + await cleanupGate; + } + }); + + const expectRunRejected = async (messageId: string) => { + const loser = await app.request("/agent/run", { + method: "POST", + body: JSON.stringify({ + session_id: id, + messages: [{ id: messageId, role: "user", content: "race" }], + }), + }); + expect(loser.status).toBe(409); + expect(((await loser.json()) as { code?: string }).code).toBe( + "run_in_flight" + ); + expect(await store.getMessage(messageId)).toBeNull(); + }; + + try { + const deletion = app.request(`/sessions/${id}`, { method: "DELETE" }); + await insideDelete; + await expectRunRejected("user-during-session-delete"); + + releaseDelete(); + await insideCleanup; + expect(await store.get(id)).toBeNull(); + await expectRunRejected("user-during-scratch-cleanup"); + + releaseCleanup(); + const res = await deletion; + expect(res.status).toBe(200); + expect(await res.json()).toEqual({ ok: true }); + } finally { + releaseDelete(); + releaseCleanup(); + deleteSpy.mockRestore(); + cleanupSpy.mockRestore(); + } + }); + it("rewind 404s for an unknown session", async () => { const res = await app.request(`/sessions/ses_nope/rewind`, { method: "POST", diff --git a/packages/grida-ai-agent/src/http/routes/sessions.ts b/packages/grida-ai-agent/src/http/routes/sessions.ts index 0d5140d64..df106db31 100644 --- a/packages/grida-ai-agent/src/http/routes/sessions.ts +++ b/packages/grida-ai-agent/src/http/routes/sessions.ts @@ -90,13 +90,9 @@ export function registerSessionsRoutes( app.delete("/sessions/:id", async (c: Context) => { const id = c.req.param("id"); + if (runtime) return runtime.deleteSession(id); + await store.delete(id); - // Drop any cached session-static context (skill index + body cache). - runtime?.forgetSession(id); - // Reclaim the session's scratch subtree (WG `scratch.md` S2). Best-effort - // and non-throwing inside the runtime, so a cleanup hiccup never fails the - // delete. - await runtime?.removeSessionScratch(id); return c.json({ ok: true }); }); diff --git a/packages/grida-ai-agent/src/prompts.ts b/packages/grida-ai-agent/src/prompts.ts index 87afdb5f3..5a8936aa0 100644 --- a/packages/grida-ai-agent/src/prompts.ts +++ b/packages/grida-ai-agent/src/prompts.ts @@ -169,27 +169,43 @@ Rules: [ '', `You have command execution access via the \`${run_command_name}\` tool.`, - `The default workdir is \`${default_workdir}\`. The agent host enforces`, - "an allowlist on the command and checks that the workdir is inside the", - "workspace.", + `The default workdir is \`${default_workdir}\`. The agent host validates`, + "the workdir and confines the process to its host-authorized scope.", "", ].join("\n"), /** * Session-scratch capability hint (WG `scratch.md`). Built per-run (the * scratch path is per-session runtime state) and appended by - * `buildCapabilityHints` (`agent/index.ts`) only when scratch is wired - * alongside command execution — the agent reaches scratch through the shell. + * `buildCapabilityHints` (`agent/index.ts`) when the host attests that scratch + * is wired. Structured filesystem reach remains discoverable when command + * execution is withheld; the optional command sentence is emitted only when + * that tool is actually present. */ - scratch_capability: (run_command_name: string, scratch_dir: string): string => + scratch_capability: ( + scratch_dir: string, + options: { + filesystem: boolean; + run_command_name?: string; + } + ): string => [ '', `You have a scratch directory at \`${scratch_dir}\`. It is an ephemeral,`, "system-managed working area, separate from the user's workspace.", - "It is readable and writable. Use `view_image` on its absolute paths to", - "SEE images you produced there, and the", - `\`${run_command_name}\` tool to read, list, move, copy, or extract files`, - "(you may `cd` into it).", + ...(options.filesystem + ? [ + "It is readable and writable through your filesystem tools. Use", + "absolute paths under this exact root to read or write files there,", + "and use `view_image` on an image's absolute path to SEE it.", + ] + : []), + ...(options.run_command_name + ? [ + `The \`${options.run_command_name}\` tool can read, list, move, or`, + "copy files there (you may `cd` into it).", + ] + : []), "It is the default place for files you PRODUCE or for intermediates", "(extracted archives, downloads, conversions): keep throwaway output out", "of the user's project.", diff --git a/packages/grida-ai-agent/src/protocol/context.ts b/packages/grida-ai-agent/src/protocol/context.ts index 6eaa0ae37..056083832 100644 --- a/packages/grida-ai-agent/src/protocol/context.ts +++ b/packages/grida-ai-agent/src/protocol/context.ts @@ -41,6 +41,12 @@ export type UserFileAttachmentDescriptor = { size: number; /** Flat scratch-relative path used by filesystem and shell tools. */ path: string; + /** + * Zero-based index among provider-native `file` parts in this user message. + * Present only when that provider part and this scratch path are two + * representations of the same attached resource. + */ + provider_file_index?: number; }; /** Persisted payload of {@link USER_FILE_ATTACHMENTS}. File order is semantic. */ @@ -51,14 +57,15 @@ export type UserFileAttachmentsData = { /** * The user attached one or more files to the turn. The part's `.data` payload - * carries LEAN facts — `{ location: "scratch", files: [{ name, mime, size, path }] }` - * — NOT instructions. `name` is display metadata; `path` is the stable, - * scratch-relative address. The file BYTES ride `scratch_seed` into session - * scratch separately and MUST be staged before this part is persisted; the - * agent reads or extracts them there via filesystem or shell tools (WG - * `scratch.md` / `binary.md`). - * Non-image files land here; an inline raster image rides the perceive-only - * `file` part instead (it needs no tool call to be seen). + * carries LEAN facts — `{ location: "scratch", files: [{ name, mime, size, + * path, provider_file_index? }] }` — NOT instructions. `name` is display + * metadata; `path` is the stable, scratch-relative address. The file BYTES ride + * `scratch_seed` into session scratch separately and MUST be staged before this + * part is persisted; the agent reads or extracts them there via filesystem or + * shell tools (WG `scratch.md` / `binary.md`). A raster upload can also ride a + * provider-native `file` part in the same message: the inline part supplies + * immediate perception while this descriptor names the byte-exact operable + * copy, and `provider_file_index` correlates the two representations. */ export const USER_FILE_ATTACHMENTS = "data-user_file_attachments"; diff --git a/packages/grida-ai-agent/src/runtime/command-backend.test.ts b/packages/grida-ai-agent/src/runtime/command-backend.test.ts index 9d4705441..35f91f521 100644 --- a/packages/grida-ai-agent/src/runtime/command-backend.test.ts +++ b/packages/grida-ai-agent/src/runtime/command-backend.test.ts @@ -2,12 +2,12 @@ /** * Contract pins — Permissions (GRIDA-SEC-004). * - * The command backend applies the STRUCTURAL gates (cwd-in-workspace + - * secret-arg containment), then runs through the host shell runner. Every - * refusal surfaces as a structured `{ ok: false, code, message }` tool result. + * The command backend applies exact session-root + secret-arg validation, then + * delegates to a host-injected executor. Every refusal surfaces as a + * structured `{ ok: false, code, message }` tool result. * The gates pinned here: * - * - cwd must be inside an opened workspace (every mode). + * - cwd must be inside this session's exact workspace or scratch (every mode). * - no arg may resolve inside the protected secret root (every mode). * * The supervised mode gate (RFC `permission modes`) is NOT in the backend — @@ -19,12 +19,18 @@ * command. The read-only-vs-mutating categorization that drives `needsApproval` * is unit-pinned in `permissions.test.ts`. */ -import { afterEach, beforeEach, describe, expect, it } from "vitest"; +import { afterEach, beforeEach, describe, expect, it, vi } from "vitest"; import fs from "node:fs/promises"; import os from "node:os"; import path from "node:path"; import { createAgentCommandBackend } from "./command-backend"; -import { WorkspaceRegistry } from "@grida/daemon/server"; +import { + runUnsandboxedShell, + type ShellExecutionScope, + type ShellExecutor, + type ShellRunRequest, + type ShellRunResult, +} from "@grida/daemon/server"; type Backend = ReturnType; type DenyResult = { ok: false; code: string; message: string }; @@ -35,20 +41,26 @@ function isDeny(r: Awaited>): r is DenyResult { describe("Permissions", () => { let workspaceRoot: string; - let registry: WorkspaceRegistry; + let otherWorkspaceRoot: string; let backend: Backend; let baseDir: string; beforeEach(async () => { baseDir = await fs.mkdtemp(path.join(os.tmpdir(), "grida-perms-")); const workspaceDir = path.join(baseDir, "workspace"); + const otherWorkspaceDir = path.join(baseDir, "other-workspace"); const userDataDir = path.join(baseDir, "userdata"); await fs.mkdir(workspaceDir); + await fs.mkdir(otherWorkspaceDir); await fs.mkdir(userDataDir); workspaceRoot = await fs.realpath(workspaceDir); - registry = new WorkspaceRegistry(userDataDir); - await registry.open(workspaceRoot); - backend = createAgentCommandBackend(registry); + otherWorkspaceRoot = await fs.realpath(otherWorkspaceDir); + // Raw execution is explicit in this local test. Production Desktop injects + // a confined host executor instead. + backend = createAgentCommandBackend({ + workspace_root: workspaceRoot, + executor: runUnsandboxedShell, + }); }); afterEach(async () => { await fs.rm(baseDir, { recursive: true, force: true }); @@ -71,11 +83,11 @@ describe("Permissions", () => { } }); - it("denies a command whose cwd is outside any opened workspace", async () => { + it("denies a command whose cwd is outside the exact workspace grant", async () => { const result = await backend({ command: "echo", args: ["hi"], - // The OS tmpdir is the *parent* of the registered workspace — + // The OS tmpdir is the *parent* of the granted workspace — // outside it, so the containment check rejects. workdir: os.tmpdir(), description: "echo outside the workspace", @@ -86,6 +98,22 @@ describe("Permissions", () => { } }); + it("denies a sibling workspace before invoking the executor", async () => { + const executor = vi.fn(runUnsandboxedShell); + const exact = createAgentCommandBackend({ + workspace_root: workspaceRoot, + executor, + }); + const result = await exact({ + command: "echo", + args: ["no"], + workdir: otherWorkspaceRoot, + description: "echo inside a sibling workspace", + }); + expect(isDeny(result)).toBe(true); + expect(executor).not.toHaveBeenCalled(); + }); + it("runs a command inside an opened workspace", async () => { const result = await backend({ command: "echo", @@ -107,7 +135,11 @@ describe("Permissions", () => { // structured tool result, not an execution. const secretsRoot = await fs.realpath(path.join(baseDir, "userdata")); await fs.writeFile(path.join(secretsRoot, "auth.json"), "{}"); - const guarded = createAgentCommandBackend(registry, [secretsRoot]); + const guarded = createAgentCommandBackend({ + workspace_root: workspaceRoot, + protected_read_roots: [secretsRoot], + executor: runUnsandboxedShell, + }); const result = await guarded({ command: "cat", args: [path.join(secretsRoot, "auth.json")], @@ -120,7 +152,125 @@ describe("Permissions", () => { } }); + it("passes the exact workspace, own scratch, base, and secrets scope to the host executor", async () => { + const scratchBase = path.join(baseDir, "scratch"); + const scratchRoot = path.join(scratchBase, "sessions", "a", "scratch"); + const secretsRoot = path.join(baseDir, "userdata"); + await fs.mkdir(scratchRoot, { recursive: true }); + let observedRequest: ShellRunRequest | undefined; + let observedScope: ShellExecutionScope | undefined; + let observedSignal: AbortSignal | undefined; + const executor: ShellExecutor = async (request, scope, signal) => { + observedRequest = request; + observedScope = scope; + observedSignal = signal; + return successfulResult(request); + }; + const scoped = createAgentCommandBackend({ + workspace_root: workspaceRoot, + scratch_root: scratchRoot, + scratch_base: scratchBase, + protected_read_roots: [secretsRoot], + executor, + }); + + const controller = new AbortController(); + const result = await scoped( + { + command: "echo", + args: ["ok"], + workdir: scratchRoot, + description: "work in own scratch", + }, + controller.signal + ); + + expect(isDeny(result)).toBe(false); + expect(observedRequest).toEqual({ + cmd: "echo", + args: ["ok"], + cwd: await fs.realpath(scratchRoot), + timeout_ms: undefined, + }); + expect(observedScope).toEqual({ + workspace_root: workspaceRoot, + scratch_root: scratchRoot, + scratch_base: scratchBase, + protected_read_roots: [secretsRoot], + }); + expect(observedSignal).toBe(controller.signal); + }); + + it("registers the exact command promise with the turn settlement barrier", async () => { + let releaseExecutor!: () => void; + const executorGate = new Promise((resolve) => { + releaseExecutor = resolve; + }); + const tracked: Promise[] = []; + const executor: ShellExecutor = async (request) => { + await executorGate; + return successfulResult(request); + }; + const scoped = createAgentCommandBackend({ + workspace_root: workspaceRoot, + executor, + track_execution: (task) => tracked.push(task), + }); + + const commandTask = scoped({ + command: "echo", + args: ["ok"], + workdir: workspaceRoot, + description: "wait for host cleanup", + }); + + expect(tracked).toEqual([commandTask]); + releaseExecutor(); + await expect(commandTask).resolves.toMatchObject({ exit_code: 0 }); + }); + + it("rejects a sibling session scratch cwd before invoking the executor", async () => { + const scratchBase = path.join(baseDir, "scratch"); + const ownScratch = path.join(scratchBase, "sessions", "a", "scratch"); + const siblingScratch = path.join(scratchBase, "sessions", "b", "scratch"); + await fs.mkdir(ownScratch, { recursive: true }); + await fs.mkdir(siblingScratch, { recursive: true }); + const executor = vi.fn(runUnsandboxedShell); + const scoped = createAgentCommandBackend({ + workspace_root: workspaceRoot, + scratch_root: ownScratch, + scratch_base: scratchBase, + executor, + }); + + const result = await scoped({ + command: "cat", + args: ["file.txt"], + workdir: siblingScratch, + description: "read a sibling session scratch", + }); + + expect(result).toMatchObject({ + ok: false, + code: "cwd-not-in-workspace", + }); + expect(executor).not.toHaveBeenCalled(); + }); + // Phase B+ coverage target once the layered permission ruleset exists: // manifest deny is not overridable by session allow; most-specific // matching rule wins; headless evaluator treats ask as deny. }); + +function successfulResult(request: ShellRunRequest): ShellRunResult { + return { + ...request, + exit_code: 0, + signal: null, + stdout: "", + stderr: "", + duration_ms: 0, + timed_out: false, + truncated: false, + }; +} diff --git a/packages/grida-ai-agent/src/runtime/command-backend.ts b/packages/grida-ai-agent/src/runtime/command-backend.ts index 9425ee73a..81f64a82a 100644 --- a/packages/grida-ai-agent/src/runtime/command-backend.ts +++ b/packages/grida-ai-agent/src/runtime/command-backend.ts @@ -1,9 +1,9 @@ /** * GRIDA-SEC-004 — agent command backend. * - * Bridges the agent's `run_command` tool to the agent-host shell - * policy. This file is the whole command execution adapter: validate - * workdir + secret-arg, then run through the host shell runner. + * Bridges the agent's `run_command` tool to a host-owned finite-command + * capability. This file validates the exact session roots, flushes pending fs + * writes, then delegates execution. It never raw-spawns on its own. * * The supervised mode gate (RFC `permission modes`) is NOT here — it lives in * the tool's `needsApproval` (`createRunCommandTool`), wired from the session @@ -16,51 +16,65 @@ */ import type { RunCommandBackend } from "../agent"; -import type { WorkspaceRegistry } from "@grida/daemon/server"; import { validateShellRequest, - runShell, + type ShellExecutionScope, + type ShellExecutor, type ProtectedReadRoots, - type AdditionalAllowedRoots, type ShellRunError, } from "@grida/daemon/server"; -/** - * @param protectedReadRoots Secret roots (the agent host's `userData`) the - * shell child must not read through an arg (GRIDA-SEC-004). Threaded down - * from the runtime; empty for the no-bindings/standalone path. - * @param additionalAllowedRoots Roots — beyond the registered workspaces — a - * cwd may sit inside (the session scratch dir, WG `scratch.md`). Empty when no - * scratch is wired. - * @param beforeRun Optional hook awaited just before a command spawns — used to - * flush the agent fs's pending (debounced) writes to disk, so a command that - * reads the workspace sees files the agent just wrote via the fs tools. - * @param runExclusive Optional shared workspace-operation FIFO. When present, - * the whole command (not only its pre-run flush) is serialized with - * server-bound read/write/edit tool calls. - */ +export type AgentCommandBackendOptions = Readonly<{ + /** Exact real workspace root granted to this session. */ + workspace_root: string; + /** Exact real scratch root granted to this session, when present. */ + scratch_root?: string; + /** Shared scratch base. The host uses it to deny sibling session roots. */ + scratch_base?: string; + /** Host secret roots denied to finite command workers. */ + protected_read_roots?: ProtectedReadRoots; + /** Host-injected execution boundary. */ + executor: ShellExecutor; + /** Flush pending AgentFs writes before execution. */ + before_run?: () => Promise; + /** Shared workspace-operation FIFO. */ + run_exclusive?: (action: () => Promise) => Promise; + /** + * Bind an executing command to its owning run's terminal-settlement barrier. + * Desktop uses this so abort cannot admit a replacement until main confirms + * the confined worker and its per-command authority are gone. + */ + track_execution?: (task: Promise) => void; +}>; + export function createAgentCommandBackend( - registry: WorkspaceRegistry, - protectedReadRoots: ProtectedReadRoots = [], - additionalAllowedRoots: AdditionalAllowedRoots = [], - beforeRun?: () => Promise, - runExclusive?: (action: () => Promise) => Promise + options: AgentCommandBackendOptions ): RunCommandBackend { - const execute: RunCommandBackend = async ({ - command, - args, - workdir, - timeout_ms: timeoutMs, - }) => { + const protectedReadRoots = Object.freeze([ + ...(options.protected_read_roots ?? []), + ]); + const allowedCwdRoots = Object.freeze([ + options.workspace_root, + ...(options.scratch_root ? [options.scratch_root] : []), + ]); + const scope: ShellExecutionScope = Object.freeze({ + workspace_root: options.workspace_root, + scratch_root: options.scratch_root, + scratch_base: options.scratch_base, + protected_read_roots: protectedReadRoots, + }); + const execute: RunCommandBackend = async ( + { command, args, workdir, timeout_ms: timeoutMs }, + signal + ) => { // Make the agent's just-written files visible on disk before the command // reads them (the fs tools flush on a debounce; a command bypasses the fs // and reads the backing store directly). - if (beforeRun) await beforeRun(); + if (options.before_run) await options.before_run(); const validation = await validateShellRequest( { cmd: command, args, cwd: workdir, timeout_ms: timeoutMs }, - registry, - protectedReadRoots, - additionalAllowedRoots + allowedCwdRoots, + protectedReadRoots ); if (!validation.ok) { return { @@ -69,7 +83,7 @@ export function createAgentCommandBackend( message: describeError(validation.error), }; } - const r = await runShell(validation.request); + const r = await options.executor(validation.request, scope, signal); return { stdout: r.truncated ? r.stdout + "\n[stdout truncated]" : r.stdout, stderr: r.truncated ? r.stderr + "\n[stderr truncated]" : r.stderr, @@ -80,8 +94,13 @@ export function createAgentCommandBackend( duration_ms: r.duration_ms, }; }; - return (input) => - runExclusive ? runExclusive(() => execute(input)) : execute(input); + return (input, signal) => { + const task = options.run_exclusive + ? options.run_exclusive(() => execute(input, signal)) + : execute(input, signal); + options.track_execution?.(task); + return task; + }; } function describeError(err: ShellRunError): string { diff --git a/packages/grida-ai-agent/src/runtime/image-gen.live.test.ts b/packages/grida-ai-agent/src/runtime/image-gen.live.test.ts index 9cab7c7ad..52e6c24d0 100644 --- a/packages/grida-ai-agent/src/runtime/image-gen.live.test.ts +++ b/packages/grida-ai-agent/src/runtime/image-gen.live.test.ts @@ -8,9 +8,10 @@ * result is the saved path + metadata, never the image bytes (a tool result * can't deliver pixels on the openai-compatible wire format — see AgentGen). * - * Mirrors the shipped macOS desktop: `image_gen_enabled` + `shell_execution_ - * allowed` TRUE, mode `auto`, `scratch_base` wired. The same BYOK key drives the - * text loop AND image generation (OpenRouter serves both /chat and /v1/images). + * Uses `image_gen_enabled` + the explicit raw shell executor, mode `auto`, and + * a wired `scratch_base`. The same BYOK key drives the text loop AND image + * generation (OpenRouter serves both /chat and /v1/images). Desktop confinement + * is not exercised here. * * Gated + excluded from CI. Run with a real BYOK key (source the gitignored env * file so process.env carries the key — vitest does NOT auto-load it): @@ -26,7 +27,7 @@ import { afterEach, beforeEach, describe, expect, it } from "vitest"; import { Hono } from "hono"; import { AuthStore } from "@grida/daemon/server"; import { SecretsStore } from "@grida/daemon/server"; -import { WorkspaceRegistry } from "@grida/daemon/server"; +import { runUnsandboxedShell, WorkspaceRegistry } from "@grida/daemon/server"; import { openSessionsDb } from "../session/db"; import { SessionsStore } from "../session/store"; import { AgentRuntime } from "."; @@ -133,7 +134,7 @@ function buildHost( streams: new StreamRegistry(), secrets_root: baseDir, scratch_base: scratchBase, - shell_execution_allowed: true, + shell_executor: runUnsandboxedShell, image_gen_enabled: true, image_model_id: IMAGE_MODEL_ID, drain_cooldown_ms: 20, diff --git a/packages/grida-ai-agent/src/runtime/index.ts b/packages/grida-ai-agent/src/runtime/index.ts index f60a0f1d2..398f1a41b 100644 --- a/packages/grida-ai-agent/src/runtime/index.ts +++ b/packages/grida-ai-agent/src/runtime/index.ts @@ -19,6 +19,7 @@ */ import crypto from "node:crypto"; +import { unlink } from "node:fs/promises"; import { AGENT_SESSION_AGENT } from "../protocol/run"; import { AGENT_DEFAULT_MODE } from "../protocol/mode"; import { @@ -64,7 +65,7 @@ import { import { discoverSkills } from "../skills/discovery"; import { discoverProjectInstructions } from "../skills/project-instructions"; import type { SkillBodyCache, SkillIndex } from "../skills/types"; -import type { WorkspaceRegistry } from "@grida/daemon/server"; +import type { ShellExecutor, WorkspaceRegistry } from "@grida/daemon/server"; import { RunInFlightError, StreamRegistry, @@ -94,6 +95,7 @@ import { runAgent, type AgentStepUsage } from "./run-agent"; import { scratchRootFor, ensureScratch, + listScratchFilePaths, removeScratch, writeScratchFile, } from "../session/scratch"; @@ -211,15 +213,11 @@ export type AgentRuntimeDeps = ResolveDeps & { */ secrets_root?: string; /** - * GRIDA-SEC-004 — whether the `run_command` shell tool may be exposed to - * the model. Default (undefined/false) is FAIL-CLOSED: no shell tool. The - * host sets this true only when the process tree is confined by an OS - * sandbox (srt), or when it has deliberately opted into an unsandboxed - * shell (the CLI). Computed once at the HTTP-server boundary from - * `sandbox_enforced || allow_unsandboxed_shell`; the gate itself lives in - * `createWorkspaceAgentBindings`. No sandbox (or no opt-in) ⇒ no shell. + * GRIDA-SEC-004 — host-owned finite-command capability. Omission is + * FAIL-CLOSED: no `run_command` tool. A Desktop host injects an OS-confined + * executor; explicit unsandboxed hosts inject the raw runner. */ - shell_execution_allowed?: boolean; + shell_executor?: ShellExecutor; /** * GRIDA-SEC-004 — whether the whole process tree is confined by an OS * sandbox. This is an attestation consumed by the external-agent @@ -380,18 +378,56 @@ async function prepareScratchForTurn( scratchDir: string, secretsRoot: string | undefined, scratchSeed: NonNullable -): Promise { +): Promise { await ensureScratch(scratchDir, secretsRoot); // Sequential by contract: duplicate paths are rejected at the run boundary, // and deterministic order keeps future seed-source extensions honest. - for (const file of scratchSeed) { - await writeScratchFile( - scratchDir, - file.path, - "text" in file - ? new TextEncoder().encode(file.text) - : Buffer.from(file.base64, "base64") - ); + const created: string[] = []; + try { + for (const file of scratchSeed) { + created.push( + await writeScratchFile( + scratchDir, + file.path, + "text" in file + ? new TextEncoder().encode(file.text) + : Buffer.from(file.base64, "base64"), + { overwrite: false } + ) + ); + } + } catch (cause) { + try { + await rollbackScratchSeeds(created); + } catch (rollbackCause) { + throw new AggregateError( + [cause, rollbackCause], + "scratch seed staging and rollback failed" + ); + } + throw cause; + } + return created; +} + +/** Remove only files this request created, never a pre-existing collision. */ +async function rollbackScratchSeeds(created: readonly string[]): Promise { + const cleanup = await Promise.allSettled( + created.map(async (file) => { + try { + await unlink(file); + } catch (err) { + if ((err as NodeJS.ErrnoException).code !== "ENOENT") throw err; + } + }) + ); + const cleanupFailures = cleanup + .filter( + (result): result is PromiseRejectedResult => result.status === "rejected" + ) + .map((result) => result.reason); + if (cleanupFailures.length > 0) { + throw new AggregateError(cleanupFailures, "scratch seed rollback failed"); } } @@ -1056,16 +1092,21 @@ export class AgentRuntime { // when scratch staging or incoming persistence rejects the request. const pendingHumanInputKind = await this.deps.sessions_store.pendingHumanInputKind(sessionId); + const assistantTail = messages.at(-1)?.role === "assistant"; + const continuationHasUnpersistedCallerHistory = + approvalAnswer !== undefined || + pendingHumanInputKind !== null || + assistantTail + ? await hasUnpersistedCallerMessage( + this.deps.sessions_store, + sessionId, + messages + ) + : false; let approvalContinuation: HumanInputContinuation | undefined; let incomingHumanInputResult: IncomingHumanInputResult | undefined; if (approvalAnswer) { - if ( - await hasUnpersistedCallerMessage( - this.deps.sessions_store, - sessionId, - messages - ) - ) { + if (continuationHasUnpersistedCallerHistory) { return Response.json( { error: @@ -1094,13 +1135,7 @@ export class AgentRuntime { ); } } else if (pendingHumanInputKind !== null) { - if ( - await hasUnpersistedCallerMessage( - this.deps.sessions_store, - sessionId, - messages - ) - ) { + if (continuationHasUnpersistedCallerHistory) { return humanInputPendingResponse(sessionId); } const matches = await findIncomingHumanInputResults( @@ -1121,6 +1156,21 @@ export class AgentRuntime { } incomingHumanInputResult = matches[0]; } + if ( + pendingHumanInputKind === null && + assistantTail && + continuationHasUnpersistedCallerHistory + ) { + return Response.json( + { + error: + "an assistant-tail continuation cannot add a new user or system message", + code: "assistant-continuation-with-new-message", + session_id: sessionId, + }, + { status: 409 } + ); + } // GRIDA-SEC-004 — a persisted `directory-ref` is NOT authority. Claim the // matching one-shot host grant for this exact session before mutating the @@ -1162,6 +1212,7 @@ export class AgentRuntime { req.scratch_seed.length > 0 ? scratchRootFor(this.deps.scratch_base, sessionId) : undefined; + let stagedScratchFiles: string[] = []; if (req.scratch_seed && req.scratch_seed.length > 0) { if (!scratchDir) { return Response.json( @@ -1175,7 +1226,7 @@ export class AgentRuntime { ); } try { - await prepareScratchForTurn( + stagedScratchFiles = await prepareScratchForTurn( scratchDir, this.deps.secrets_root, req.scratch_seed @@ -1200,19 +1251,40 @@ export class AgentRuntime { // turn can publish a late human block underneath the commit. A 409 from // this boundary is therefore mutation-free and safe for the renderer to // retry as a durable queued message. - if ( - pendingHumanInputKind === null && - (await this.deps.sessions_store.hasPendingHumanInput(sessionId)) - ) { - return humanInputPendingResponse(sessionId); - } + try { + if ( + pendingHumanInputKind === null && + (await this.deps.sessions_store.hasPendingHumanInput(sessionId)) + ) { + await rollbackScratchSeeds(stagedScratchFiles); + stagedScratchFiles = []; + return humanInputPendingResponse(sessionId); + } - // Persist only caller-owned non-assistant rows first. Assistant tool - // results are the continuation commit and deliberately remain untouched - // until every fallible preparation step above has succeeded. - await persistIncomingTail(this.deps.sessions_store, sessionId, messages, { - resolveAssistantToolResults: false, - }); + // Persist only caller-owned non-assistant rows first. Assistant tool + // results are the continuation commit and deliberately remain untouched + // until every fallible preparation step above has succeeded. + await persistIncomingTail( + this.deps.sessions_store, + sessionId, + messages, + { + resolveAssistantToolResults: false, + } + ); + // The durable descriptor now owns these bytes. Later turn-start failures + // leave both intact so the accepted message remains operable. + } catch (cause) { + try { + await rollbackScratchSeeds(stagedScratchFiles); + } catch (rollbackCause) { + throw new AggregateError( + [cause, rollbackCause], + "pre-persistence failure and scratch seed rollback failed" + ); + } + throw cause; + } // Fill every non-blocking client tool result first. Human-input results // are excluded as a class; for a question/design-search continuation, @@ -1569,30 +1641,50 @@ export class AgentRuntime { const { workspace_registry: workspaceRegistry, secrets_root: secretsRoot, - shell_execution_allowed: shellExecutionAllowed, + shell_executor: shellExecutor, scratch_base: scratchBase, } = this.deps; + let pumpTask: Promise | undefined; + let commandPumpTracked = false; // Per-session scratch dir (WG `scratch.md`). Derived (pure) here so it can // ride `runDeps`; the dir is created on disk just before the model turn - // (below). Scratch is the sink for the shell (an allowed cwd root) AND for - // `generate_image` (its output dir), so derive it when EITHER is enabled — - // an images-only host that keeps the shell off still needs it (#920 review). + // (below). Structured filesystem tools are the baseline operability path, + // independent of optional shell/image generation, so every workspace-bound + // Grida turn gets scratch when the host supplies a base. const scratchDir = preparedScratchDir ?? - (scratchBase && - workspaceRoot && - (shellExecutionAllowed || this.deps.image_gen_enabled === true) + (scratchBase && workspaceRoot ? scratchRootFor(scratchBase, sessionId) : undefined); // Bindings deps for the run. Typed (not an inline literal) so the - // GRIDA-SEC-004 `secrets_root` + `shell_execution_allowed` (and `scratch_dir`) + // GRIDA-SEC-004 `secrets_root` + `shell_executor` + exact scratch scope // thread through `runAgent`'s narrower `{ workspace_registry }` param into // `createWorkspaceAgentBindings`. const runDeps = { workspace_registry: workspaceRegistry, secrets_root: secretsRoot, - shell_execution_allowed: shellExecutionAllowed, + shell_executor: shellExecutor, + track_command_execution: (task: Promise) => { + // The command task ends only after the host's terminal abort ACK, which + // follows worker exit and per-command cleanup. The pump task closes the + // smaller gap between that ACK and the AI SDK consuming the aborted tool + // promise. Runs that never execute a command keep the existing policy: + // an uncooperative model promise does not block replacement forever. + this.streams.trackSettlementTask(entry, task); + if (!commandPumpTracked) { + if (!pumpTask) { + throw new Error( + "command execution started before pump registration" + ); + } + commandPumpTracked = this.streams.trackSettlementTask( + entry, + pumpTask + ); + } + }, scratch_dir: scratchDir, + scratch_base: scratchBase, // BYOK keys + the host's image-modality switch — together with scratchDir // they let `createWorkspaceAgentBindings` build the `generate_image` // binding (the produced bytes sink to scratch). @@ -1611,7 +1703,7 @@ export class AgentRuntime { // Pump: open the upstream model call, forward each SSE frame into the // registry. Doesn't block the caller; a client attaches as another // consumer (HTTP) or reconnects later (a core drain has no live consumer). - void (async () => { + pumpTask = (async () => { try { // Ordering invariant for the continuation prefix: the snapshot // completes before ANY of this turn's frames can reach the recorder. @@ -1625,6 +1717,9 @@ export class AgentRuntime { if (scratchDir && !preparedScratchDir) { await ensureScratch(scratchDir, secretsRoot); } + const availableScratchAttachmentPaths = scratchDir + ? await listScratchFilePaths(scratchDir) + : new Set(); // Agent-provider class (issue #813): the external agent owns the loop. // Skip compaction/model-factory/tool-injection entirely — just run one @@ -1639,6 +1734,7 @@ export class AgentRuntime { // External agents do not receive our structured fs binding. A // durable historical descriptor remains inspectable but inert. availableDirectoryScopeIds: new Set(), + availableScratchAttachmentPaths: new Set(), }); // Continuity (issue #813): resume the external agent's prior session // so it keeps the conversation. Read the id stored last turn, pass it @@ -1707,6 +1803,7 @@ export class AgentRuntime { : await sessionsStore.listVisibleMessages(sessionId); const preparedMessages = buildModelMessages(visible, { availableDirectoryScopeIds, + availableScratchAttachmentPaths, }); // Session-static skills + project instructions (discovered once). @@ -1804,6 +1901,7 @@ export class AgentRuntime { streams.finishEntry(entry, reason); } })(); + void pumpTask; return entry; } @@ -1878,6 +1976,25 @@ export class AgentRuntime { return Response.json({ error: "sessionId required" }, { status: 400 }); } + /** + * `DELETE /sessions/:id` — remove an idle session and its ephemeral + * runtime state. The admission lease spans the durable delete and scratch + * cleanup so a new run cannot resolve or recreate session state midway + * through teardown. + */ + async deleteSession(sessionId: string): Promise { + const admission = this.acquireIdleAdmission(sessionId); + if (admission instanceof Response) return admission; + try { + await this.deps.sessions_store.delete(sessionId); + this.forgetSession(sessionId); + await this.removeSessionScratch(sessionId); + return Response.json({ ok: true }); + } finally { + this.streams.releaseAdmission(admission); + } + } + /** * `POST /sessions/:id/rewind` — soft-truncate to a prior message (RFC * `session / rewinding`). `restore: true` un-rewinds (un-hides). Refuses @@ -2148,7 +2265,7 @@ export class AgentRuntime { const base = this.deps.scratch_base; if (!base) return; try { - await removeScratch(base, sessionId); + await removeScratch(base, sessionId, this.deps.secrets_root); } catch (err) { console.warn(`[agent] scratch cleanup failed for ${sessionId}:`, err); } diff --git a/packages/grida-ai-agent/src/runtime/message-view.test.ts b/packages/grida-ai-agent/src/runtime/message-view.test.ts index aaa75706d..e817d21e2 100644 --- a/packages/grida-ai-agent/src/runtime/message-view.test.ts +++ b/packages/grida-ai-agent/src/runtime/message-view.test.ts @@ -216,6 +216,44 @@ describe("buildModelMessages", () => { expect(JSON.stringify(rows)).not.toContain('"available"'); }); + it("annotates scratch attachments from live files without mutating persistence", () => { + const rows = [ + msg("m-attachments", "user", [ + part("data-user_file_attachments", { + type: "data-user_file_attachments", + data: { + location: "scratch", + files: [ + { + name: "live.bin", + mime: "application/octet-stream", + size: 3, + path: "live.bin", + }, + { + name: "expired.bin", + mime: "application/octet-stream", + size: 4, + path: "expired.bin", + }, + ], + }, + }), + ]), + ]; + + const out = buildModelMessages(rows, { + availableScratchAttachmentPaths: new Set(["live.bin"]), + }); + const marker = out[0].parts[0] as { text: string }; + expect(marker.text).toMatch(/"name": "live\.bin"[^}]*"available": true/); + expect(marker.text).toMatch( + /"name": "expired\.bin"[^}]*"available": false/ + ); + // Liveness is a turn-time model fact, never durable chat state. + expect(JSON.stringify(rows)).not.toContain('"available"'); + }); + it("resolves a bottom summary: drops the head, reorders the summary to the front", () => { // New model: the marker sorts LAST. The boundary is read from tail_start_id, // the head before it is dropped, and the summary leads. diff --git a/packages/grida-ai-agent/src/runtime/message-view.ts b/packages/grida-ai-agent/src/runtime/message-view.ts index a8398d96c..ddc44fcfd 100644 --- a/packages/grida-ai-agent/src/runtime/message-view.ts +++ b/packages/grida-ai-agent/src/runtime/message-view.ts @@ -38,6 +38,7 @@ import { AgentVision } from "../vision"; import { CONTEXT_MARKERS, USER_DIRECTORY_REFERENCES, + USER_FILE_ATTACHMENTS, } from "../protocol/context"; import { normalizeSdkToolPartFields } from "../protocol/tool-part-fields"; @@ -66,6 +67,9 @@ export function buildModelMessages( /** Live host grants for this session. Omitted by generic callers that only * need structural lowering; the runtime always supplies it. */ availableDirectoryScopeIds?: ReadonlySet; + /** Scratch-relative direct files live for this model turn. Omitted by + * generic callers; the runtime always supplies a snapshot. */ + availableScratchAttachmentPaths?: ReadonlySet; } = {} ): ModelUIMessage[] { const boundary = compactionBoundary(visible); @@ -108,6 +112,7 @@ export function buildModelMessages( const parts = lowerParts(m.parts, { elideImages: i < liveStart, availableDirectoryScopeIds: opts.availableDirectoryScopeIds, + availableScratchAttachmentPaths: opts.availableScratchAttachmentPaths, }); if (m.role === "user" && pendingSummary !== null) { parts.unshift({ @@ -173,6 +178,7 @@ function lowerParts( opts: { elideImages: boolean; availableDirectoryScopeIds?: ReadonlySet; + availableScratchAttachmentPaths?: ReadonlySet; } = { elideImages: false } ): unknown[] { const out: unknown[] = []; @@ -189,11 +195,12 @@ function lowerParts( type === "source-url" || type === "source-document" ) { - // Inline attachments are NOT auto-elided: unlike a `view_image` result, - // a pasted image has no re-view affordance (no path, no tool to re-call), - // so dropping it is lossy and irreversible. They stay durable across the - // rebuild (see agent.test.ts "DB-rebuild durability"); only re-viewable - // perceptions are evicted below. + // Inline attachments are NOT auto-elided here. A neighboring scratch + // descriptor may make an upload re-viewable while that path is live, but + // this model-view layer has no correlated liveness proof. Dropping pixels + // without one would be lossy, so attachments stay durable across the + // rebuild (see agent.test.ts "DB-rebuild durability"); only explicitly + // re-viewable perceptions are evicted below. out.push(data); continue; } @@ -265,38 +272,65 @@ function lowerParts( } /** - * Add model-only liveness to directory descriptors without mutating the - * durable message part. The transcript records intent; only the host registry - * can say whether that intent is currently operable for this session. Generic - * structural callers omit the availability set and retain the legacy payload. + * Add model-only liveness to host-backed resource descriptors without mutating + * the durable message part. The transcript records intent; only current host + * state can say whether that intent is operable for this turn. Generic + * structural callers omit the availability sets and retain the legacy payload. */ function lowerContextPayload( type: string, payload: unknown, - opts: { availableDirectoryScopeIds?: ReadonlySet } + opts: { + availableDirectoryScopeIds?: ReadonlySet; + availableScratchAttachmentPaths?: ReadonlySet; + } ): unknown { + if (payload == null || typeof payload !== "object") return payload; if ( - type !== USER_DIRECTORY_REFERENCES || - opts.availableDirectoryScopeIds === undefined || - payload == null || - typeof payload !== "object" + type === USER_DIRECTORY_REFERENCES && + opts.availableDirectoryScopeIds !== undefined ) { - return payload; + const availableDirectoryScopeIds = opts.availableDirectoryScopeIds; + const directories = (payload as { directories?: unknown }).directories; + if (!Array.isArray(directories)) return payload; + return { + ...(payload as Record), + directories: directories.map((directory) => { + if (directory == null || typeof directory !== "object") { + return directory; + } + const id = (directory as { id?: unknown }).id; + return { + ...(directory as Record), + available: + typeof id === "string" && availableDirectoryScopeIds.has(id), + }; + }), + }; } - const availableDirectoryScopeIds = opts.availableDirectoryScopeIds; - const directories = (payload as { directories?: unknown }).directories; - if (!Array.isArray(directories)) return payload; - return { - ...(payload as Record), - directories: directories.map((directory) => { - if (directory == null || typeof directory !== "object") return directory; - const id = (directory as { id?: unknown }).id; - return { - ...(directory as Record), - available: typeof id === "string" && availableDirectoryScopeIds.has(id), - }; - }), - }; + if ( + type === USER_FILE_ATTACHMENTS && + opts.availableScratchAttachmentPaths !== undefined + ) { + const availableScratchAttachmentPaths = + opts.availableScratchAttachmentPaths; + const files = (payload as { files?: unknown }).files; + if (!Array.isArray(files)) return payload; + return { + ...(payload as Record), + files: files.map((file) => { + if (file == null || typeof file !== "object") return file; + const path = (file as { path?: unknown }).path; + return { + ...(file as Record), + available: + typeof path === "string" && + availableScratchAttachmentPaths.has(path), + }; + }), + }; + } + return payload; } const DESIGN_SEARCH_PART_TYPE = `tool-${AgentDesignSearch.TOOL_NAME}`; diff --git a/packages/grida-ai-agent/src/runtime/run-agent.ts b/packages/grida-ai-agent/src/runtime/run-agent.ts index 907f1e05c..43dd64c81 100644 --- a/packages/grida-ai-agent/src/runtime/run-agent.ts +++ b/packages/grida-ai-agent/src/runtime/run-agent.ts @@ -192,6 +192,11 @@ export async function runAgent( library?: boolean; /** In-process provider HTTP. Used here only for AI SDK URL-part downloads. */ provider_http?: ProviderHttp; + /** + * Register a finite command with the owning runtime turn's terminal + * settlement. Omitted by callers that do not own a run registry. + */ + track_command_execution?: (task: Promise) => void; } ): Promise { // Wire bindings when the request carries workspace context OR host-authorized @@ -219,6 +224,11 @@ export async function runAgent( // host enabled it, a scratch sink exists, and a provider key is present. image_gen: bindings?.image_gen, command: bindings?.command, + // Scratch is a structured-filesystem capability even when the fail-closed + // host posture withholds shell execution. The binding exposes only the + // exact real root it already mounted into `fs`; this is prompt metadata, + // not an additional path grant. + scratch_dir: bindings?.scratch_dir, // RFC skills + project instructions are session-static context the // runtime discovered once and threads through every turn. When scratch is // wired (workspace path), the loader MATERIALIZES a loaded skill's tree into diff --git a/packages/grida-ai-agent/src/runtime/run-input.test.ts b/packages/grida-ai-agent/src/runtime/run-input.test.ts index 9a19501a9..a98bb0006 100644 --- a/packages/grida-ai-agent/src/runtime/run-input.test.ts +++ b/packages/grida-ai-agent/src/runtime/run-input.test.ts @@ -1,5 +1,5 @@ // GRIDA-SEC-008 — explicit native-provider input validation pins. -import { afterEach, beforeEach, describe, expect, it } from "vitest"; +import { afterEach, beforeEach, describe, expect, it, vi } from "vitest"; import fs from "node:fs/promises"; import os from "node:os"; import path from "node:path"; @@ -186,6 +186,42 @@ describe("persistIncomingTail", () => { expect((messages[0].parts[0].data as { text: string }).text).toBe("hello"); }); + it("rolls back the whole incoming message when a part write fails", async () => { + const s = await store.create({ agent: "grida" }); + const originalUpsertPart = store.upsertPart.bind(store); + let calls = 0; + const upsertSpy = vi + .spyOn(store, "upsertPart") + .mockImplementation(async (...args) => { + calls += 1; + if (calls === 2) throw new Error("injected part persistence failure"); + return await originalUpsertPart(...args); + }); + + try { + await expect( + persistIncomingTail(store, s.id, [ + { + id: "atomic-user", + role: "user", + parts: [ + { type: "text", text: "persist neither part" }, + { + type: "data-test-descriptor", + data: { location: "scratch", path: "/attachment.bin" }, + }, + ], + }, + ]) + ).rejects.toThrow("injected part persistence failure"); + expect(calls).toBe(2); + expect(await store.getMessage("atomic-user")).toBeNull(); + expect(await store.listMessageIds(s.id)).not.toContain("atomic-user"); + } finally { + upsertSpy.mockRestore(); + } + }); + it("skips assistant text/reasoning (the recorder owns those)", async () => { const s = await store.create({ agent: "grida" }); await persistIncomingTail(store, s.id, [ @@ -636,6 +672,66 @@ describe("parseRunBody", () => { expect(parsed.directory_scopes).toEqual([descriptor]); }); + it("does not re-fire durable resource facts on an assistant-tail continuation", async () => { + const directoryId = "dir_11111111-1111-4111-8111-111111111111"; + const parsed = await parseRunBody( + { + messages: [ + { + id: "u-resource", + role: "user", + parts: [ + { + type: "file", + mediaType: "image/png", + url: "data:image/png;base64,AAAA", + filename: "preview.png", + }, + { + type: "data-user_file_attachments", + data: { + location: "scratch", + files: [ + { + name: "original.gif", + mime: "image/gif", + size: 3, + path: "upload.gif", + }, + ], + }, + }, + { + type: "data-user_directory_references", + data: { + directories: [ + { + kind: "scope", + id: directoryId, + name: "references", + path: `/__references__/${directoryId}`, + access: "read", + }, + ], + }, + }, + ], + }, + { + id: "a-continuation", + role: "assistant", + parts: [clientTool("question", "q1", { answers: [["continue"]] })], + }, + ], + }, + deps as never + ); + + if (parsed instanceof Response) throw new Error("unexpected rejection"); + expect(parsed.scratch_seed).toBeUndefined(); + expect(parsed.directory_scopes).toBeUndefined(); + }); + it.each([ { id: "dir_11111111-1111-4111-8111-111111111111", @@ -796,6 +892,42 @@ describe("parseRunBody", () => { ]); }); + it.each([ + { + label: "an assistant-tail continuation", + messages: [ + { role: "user", parts: [{ type: "text", text: "earlier" }] }, + { role: "assistant", parts: [{ type: "text", text: "continuing" }] }, + ], + }, + { + label: "an explicit approval continuation", + messages: [ + { role: "user", parts: [{ type: "text", text: "run the command" }] }, + ], + approval_answer: { + tool_call_id: "tc1", + approval_id: "ap1", + approved: true, + }, + }, + ])("rejects scratch_seed on $label", async (body) => { + const parsed = await parseRunBody( + { + ...body, + scratch_seed: [{ path: "persisted-input.txt", text: "replacement" }], + }, + deps as never + ); + + expect(parsed).toBeInstanceOf(Response); + if (!(parsed instanceof Response)) return; + expect(parsed.status).toBe(400); + expect(await parsed.json()).toMatchObject({ + code: "invalid-scratch-seed", + }); + }); + it("accepts the canonical scratch_seed file-count limit and rejects one more", async () => { const messages = [{ role: "user", parts: [{ type: "text", text: "hi" }] }]; const entries = Array.from( @@ -907,16 +1039,23 @@ describe("parseRunBody", () => { role: "user", parts: [ { type: "text", text: "inspect these" }, + { + type: "file", + mediaType: "image/png", + url: "data:image/png;base64,AAAA", + filename: "preview.png", + }, { type: "data-user_file_attachments", data: { location: "scratch", files: [ { - name: "Document.pdf", - mime: "application/pdf", + name: "Original.gif", + mime: "image/gif", size: 3, - path: "upload-1.pdf", + path: "upload-1.gif", + provider_file_index: 0, }, { name: "Notes.txt", @@ -931,7 +1070,7 @@ describe("parseRunBody", () => { }, ], scratch_seed: [ - { path: "upload-1.pdf", base64: "AQID" }, + { path: "upload-1.gif", base64: "AQID" }, { path: "upload-2.txt", text: "hello" }, ], }, @@ -941,16 +1080,79 @@ describe("parseRunBody", () => { const s = await store.create({ agent: "grida" }); await persistIncomingTail(store, s.id, parsed.messages); const persisted = await store.listVisibleMessages(s.id); - expect(persisted[0].parts[1].type).toBe("data-user_file_attachments"); + expect(persisted[0].parts.map((part) => part.type)).toEqual([ + "text", + "file", + "data-user_file_attachments", + ]); const model = buildModelMessages(persisted); - const marker = model[0].parts[1] as { type: string; text: string }; + const marker = model[0].parts.find( + (part): part is { type: "text"; text: string } => + typeof part === "object" && + part !== null && + "type" in part && + part.type === "text" && + "text" in part && + typeof part.text === "string" && + part.text.includes("") + ); + expect(marker).toBeDefined(); + if (!marker) throw new Error("missing user_file_attachments marker"); expect(marker.text).toContain(""); - expect(marker.text).toContain('"path": "upload-1.pdf"'); - expect(marker.text.indexOf("Document.pdf")).toBeLessThan( + expect(marker.text).toContain('"path": "upload-1.gif"'); + expect(marker.text).toContain('"provider_file_index": 0'); + expect(marker.text.indexOf("Original.gif")).toBeLessThan( marker.text.indexOf("Notes.txt") ); }); + it.each([ + [[-1], "malformed"], + [[1], "out of range"], + [[0, 0], "duplicated"], + ])( + "rejects attachment provider-file correlation when it is %s", + async (indices) => { + const parsed = await parseRunBody( + { + messages: [ + { + role: "user", + parts: [ + { + type: "file", + mediaType: "image/png", + url: "data:image/png;base64,AAAA", + }, + { + type: "data-user_file_attachments", + data: { + location: "scratch", + files: indices.map((provider_file_index, index) => ({ + name: `image-${index}.png`, + mime: "image/png", + size: 3, + path: `upload-${index}.png`, + provider_file_index, + })), + }, + }, + ], + }, + ], + scratch_seed: indices.map((_, index) => ({ + path: `upload-${index}.png`, + base64: "AQID", + })), + }, + deps as never + ); + + expect(parsed).toBeInstanceOf(Response); + expect(parsed instanceof Response ? parsed.status : 200).toBe(400); + } + ); + it.each([ [undefined, "missing body"], [{ path: "upload.pdf", base64: "AQIDBA==" }, "wrong size"], diff --git a/packages/grida-ai-agent/src/runtime/run-input.ts b/packages/grida-ai-agent/src/runtime/run-input.ts index 3e6b823d8..15c8b092c 100644 --- a/packages/grida-ai-agent/src/runtime/run-input.ts +++ b/packages/grida-ai-agent/src/runtime/run-input.ts @@ -146,6 +146,13 @@ export async function parseRunBody( { status: 400 } ); } + // Approval answers are explicit request fields rather than assistant tool + // results. Like any assistant-tail continuation, they fire no new user + // message even when the client resends an older user row as the array tail. + // The runtime later proves every caller-owned row already belongs to the + // session before accepting the continuation. + const approvalAnswer = coerceApprovalAnswer(b.approval_answer); + const firedUser = approvalAnswer ? undefined : tailUserMessage(messages); const scratchSeed = parseScratchSeed(b.scratch_seed); if (scratchSeed.error) { return Response.json( @@ -153,14 +160,23 @@ export async function parseRunBody( { status: 400 } ); } - const attachmentError = validateAttachmentSeeds(messages, scratchSeed.value); + if (scratchSeed.value && !firedUser) { + return Response.json( + { + error: "scratch_seed requires a fired tail user message", + code: "invalid-scratch-seed", + }, + { status: 400 } + ); + } + const attachmentError = validateAttachmentSeeds(firedUser, scratchSeed.value); if (attachmentError) { return Response.json( { error: attachmentError, code: "invalid-file-attachments" }, { status: 400 } ); } - const directoryReferences = parseDirectoryReferences(messages); + const directoryReferences = parseDirectoryReferences(firedUser); if (directoryReferences.error) { return Response.json( { @@ -285,7 +301,7 @@ export async function parseRunBody( // 400. A well-formed answer is still matched against the persisted pending // approval; the runtime rejects the request when that authoritative match // returns false. - approval_answer: coerceApprovalAnswer(b.approval_answer), + approval_answer: approvalAnswer, scratch_seed: scratchSeed.value, directory_scopes: directoryReferences.value, session_id: @@ -296,17 +312,28 @@ export async function parseRunBody( } /** - * Validate and collect directory descriptors from the LAST user message only. + * The tail user candidate. Resent history can contain older user rows, while a + * question continuation ends in an assistant tool result. The caller also + * suppresses this candidate for a valid explicit approval continuation. + */ +function tailUserMessage( + messages: NormalizedMessage[] +): NormalizedMessage | undefined { + const tail = messages.at(-1); + return tail?.role === "user" ? tail : undefined; +} + +/** + * Validate and collect directory descriptors from the fired tail user only. * The client resends history; old/forked reference parts are durable facts but * must never be reinterpreted as fresh authority. Exact descriptor validation * also prevents a raw host path or write access from being smuggled into the * model-visible marker before the registry compares it to canonical facts. */ -function parseDirectoryReferences(messages: NormalizedMessage[]): { +function parseDirectoryReferences(user: NormalizedMessage | undefined): { value?: DirectoryScopeDescriptor[]; error?: string; } { - const user = messages.findLast((message) => message.role === "user"); if (!user) return {}; const directories: DirectoryScopeDescriptor[] = []; const ids = new Set(); @@ -437,15 +464,18 @@ function isSafeScratchPath(path: string): boolean { * remain durable while their session scratch was seeded on the original turn. */ function validateAttachmentSeeds( - messages: NormalizedMessage[], + user: NormalizedMessage | undefined, scratchSeed: ScratchSeedEntry[] | undefined ): string | null { - const user = messages.findLast((message) => message.role === "user"); if (!user) return null; + const providerFileCount = user.parts.filter( + (part) => part.type === "file" + ).length; const seedByPath = new Map( (scratchSeed ?? []).map((seed) => [seed.path, seed]) ); const described = new Set(); + const providerFileIndices = new Set(); for (const part of user.parts) { if (part.type !== USER_FILE_ATTACHMENTS) continue; const payload = part.data; @@ -457,6 +487,15 @@ function validateAttachmentSeeds( return `attachment path is described more than once: ${file.path}`; } described.add(file.path); + if (file.provider_file_index !== undefined) { + if (file.provider_file_index >= providerFileCount) { + return `attachment provider_file_index is out of range: ${file.provider_file_index}`; + } + if (providerFileIndices.has(file.provider_file_index)) { + return `attachment provider_file_index is described more than once: ${file.provider_file_index}`; + } + providerFileIndices.add(file.provider_file_index); + } const seed = seedByPath.get(file.path); if (!seed) return `attachment body is missing from scratch_seed: ${file.path}`; @@ -496,7 +535,11 @@ function isUserFileAttachmentsData( Number.isSafeInteger(f.size) && f.size >= 0 && typeof f.path === "string" && - isSafeScratchPath(f.path) + isSafeScratchPath(f.path) && + (f.provider_file_index === undefined || + (typeof f.provider_file_index === "number" && + Number.isSafeInteger(f.provider_file_index) && + f.provider_file_index >= 0)) ); }); } @@ -561,8 +604,7 @@ async function isModelAvailableFromProvider( export function extractTailUserMessageId( msgs: NormalizedMessage[] ): string | undefined { - const tail = msgs.at(-1); - return tail?.role === "user" ? tail.id : undefined; + return tailUserMessage(msgs)?.id; } /** @@ -624,40 +666,42 @@ export async function persistIncomingTail( incoming: NormalizedMessage[], options: { resolveAssistantToolResults?: boolean } = {} ): Promise { - // Ids already persisted for this session. The AI SDK client resends - // the full history with stable ids every turn, so most incoming ids - // are already here and skip below. Doubles as the intra-request dedup - // set — a client DB-hydration race can place the same user message in - // one outgoing array twice — so we record each id as we go. - const seen = new Set(await store.listMessageIds(sessionId)); - for (const m of incoming) { - if (m.role === "assistant") { - // The recorder owns assistant messages — it writes them from the model - // stream. The ONE thing the stream can't carry is a CLIENT-resolved tool - // result: a session with no server-side fs (the desktop file window's - // single-file sidebar) resolves fs tools in the renderer, and the result - // arrives only on the next request's assistant message. Fill just those - // into the existing (recorder-written) tool row so the server-authoritative - // model view (`buildModelMessages`) stops dropping the call as incomplete. - if (options.resolveAssistantToolResults !== false) { - await persistResolvedToolResults(store, sessionId, m); + await store.withTransaction(async () => { + // Ids already persisted for this session. The AI SDK client resends + // the full history with stable ids every turn, so most incoming ids + // are already here and skip below. Doubles as the intra-request dedup + // set — a client DB-hydration race can place the same user message in + // one outgoing array twice — so we record each id as we go. + const seen = new Set(await store.listMessageIds(sessionId)); + for (const m of incoming) { + if (m.role === "assistant") { + // The recorder owns assistant messages — it writes them from the model + // stream. The ONE thing the stream can't carry is a CLIENT-resolved tool + // result: a session with no server-side fs (the desktop file window's + // single-file sidebar) resolves fs tools in the renderer, and the result + // arrives only on the next request's assistant message. Fill just those + // into the existing (recorder-written) tool row so the server-authoritative + // model view (`buildModelMessages`) stops dropping the call as incomplete. + if (options.resolveAssistantToolResults !== false) { + await persistResolvedToolResults(store, sessionId, m); + } + continue; + } + if (seen.has(m.id)) continue; + seen.add(m.id); + // Idempotent insert: a concurrent run on the same session can land + // this id between the snapshot above and now (the client may re-POST + // /agent/run while one is still in flight). ON CONFLICT DO NOTHING + // turns that race into a no-op instead of a UNIQUE-constraint 500. + await store.appendMessageIfAbsent(sessionId, { id: m.id, role: m.role }); + // Parts are keyed by (messageId, index) — upsert is idempotent, so a + // re-send refreshes content without duplicating rows. + for (let i = 0; i < m.parts.length; i += 1) { + const part = m.parts[i]; + await store.upsertPart(m.id, { index: i, type: part.type, data: part }); } - continue; - } - if (seen.has(m.id)) continue; - seen.add(m.id); - // Idempotent insert: a concurrent run on the same session can land - // this id between the snapshot above and now (the client may re-POST - // /agent/run while one is still in flight). ON CONFLICT DO NOTHING - // turns that race into a no-op instead of a UNIQUE-constraint 500. - await store.appendMessageIfAbsent(sessionId, { id: m.id, role: m.role }); - // Parts are keyed by (messageId, index) — upsert is idempotent, so a - // re-send refreshes content without duplicating rows. - for (let i = 0; i < m.parts.length; i += 1) { - const part = m.parts[i]; - await store.upsertPart(m.id, { index: i, type: part.type, data: part }); } - } + }); } /** diff --git a/packages/grida-ai-agent/src/runtime/runtime.live.test.ts b/packages/grida-ai-agent/src/runtime/runtime.live.test.ts index 544791894..7710deec3 100644 --- a/packages/grida-ai-agent/src/runtime/runtime.live.test.ts +++ b/packages/grida-ai-agent/src/runtime/runtime.live.test.ts @@ -1,8 +1,8 @@ /** * LIVE end-to-end — real provider, real model. The durability bar for inline - * image input: prove the agent actually SEES a pasted/dropped image, that the - * image survives into later turns (DB rebuild), and that it survives a process - * restart (resume). + * image input: prove the agent actually SEES a pasted/dropped image, can + * operate on its byte-exact scratch twin by path, that the image survives into + * later turns (DB rebuild), and that it survives a process restart (resume). * * Gated + excluded from CI. Run explicitly with a real BYOK key: * @@ -12,18 +12,25 @@ * Env knobs: * GRIDA_LIVE_AGENT=1 — required, opts in. * OPENROUTER_API_KEY / GRIDA_BYOK_KEY — the BYOK key. + * GRIDA_LIVE_AUTH_DIR — alternatively, an existing agent auth-store + * directory (the key is read in place, not + * copied into the test fixture). * GRIDA_BYOK_PROVIDER — "openrouter" (default) | "vercel". * GRIDA_LIVE_MODEL — a multimodal catalog model id (default below). * - * Perception probe: a solid RED square (generated in-process, no fixture - * binary) + "name the dominant color". A model cannot answer "red" without - * seeing the pixels, so a correct answer is real multimodal delivery — and the - * multi-turn / resume variants prove the image came from the DB, not the - * request payload. + * Perception probe: solid-color squares (generated in-process, no fixture + * binary). A model cannot name the color without seeing the pixels. The + * operability probe sends contrasting images so selecting the BLUE + * provider-native image and copying its correlated scratch twin proves the two + * delivery routes still identify the same resource. */ import fs from "node:fs/promises"; -import { assistantTextFromSse, sessionIdFromSse } from "../testing/sse"; +import { + assistantTextFromSse, + chunksOf, + sessionIdFromSse, +} from "../testing/sse"; import os from "node:os"; import path from "node:path"; import { fileURLToPath } from "node:url"; @@ -32,10 +39,11 @@ import { afterEach, beforeEach, describe, expect, it } from "vitest"; import { Hono } from "hono"; import { AuthStore } from "@grida/daemon/server"; import { SecretsStore } from "@grida/daemon/server"; -import { WorkspaceRegistry } from "@grida/daemon/server"; +import { runUnsandboxedShell, WorkspaceRegistry } from "@grida/daemon/server"; import { openSessionsDb } from "../session/db"; import { SessionsStore } from "../session/store"; import { AGENT_SESSION_AGENT } from "../protocol/run"; +import { scratchRootFor } from "../session/scratch"; import { AgentRuntime } from "."; import { StreamRegistry } from "./stream-registry"; import { registerAgentRoutes } from "../http/routes/agent"; @@ -43,6 +51,7 @@ import { registerAgentRoutes } from "../http/routes/agent"; const LIVE = process.env.GRIDA_LIVE_AGENT === "1"; const PROVIDER_KEY = process.env.OPENROUTER_API_KEY ?? process.env.GRIDA_BYOK_KEY ?? ""; +const LIVE_AUTH_DIR = process.env.GRIDA_LIVE_AUTH_DIR?.trim() || undefined; const PROVIDER_ID = (process.env.GRIDA_BYOK_PROVIDER ?? "openrouter") as | "openrouter" | "vercel"; @@ -55,7 +64,8 @@ const BUNDLED_SKILLS_DIR = path.resolve( "../../../../skills" ); -const liveDescribe = LIVE && PROVIDER_KEY ? describe : describe.skip; +const liveDescribe = + LIVE && (PROVIDER_KEY || LIVE_AUTH_DIR) ? describe : describe.skip; // --- minimal solid-color PNG encoder (no deps; in-process fixture) --------- function pngChunk(type: string, data: Buffer): Buffer { @@ -106,7 +116,8 @@ function buildHost( baseDir: string, opts?: { bundled_dir?: string; registry?: WorkspaceRegistry } ): Host { - const auth = new AuthStore(baseDir); + const authDir = PROVIDER_KEY ? baseDir : (LIVE_AUTH_DIR ?? baseDir); + const auth = new AuthStore(authDir); const secrets = new SecretsStore(auth); const db = openSessionsDb({ user_data_path: baseDir }); const store = new SessionsStore(db); @@ -120,11 +131,12 @@ function buildHost( workspace_registry: registry, sessions_store: store, streams: new StreamRegistry(), + secrets_root: authDir, drain_cooldown_ms: 20, // Mirror the shipped desktop: per-session scratch + shell, so a workspace // turn has the full toolset (a deck run writes files + may run the shell). scratch_base: path.join(baseDir, "scratch"), - shell_execution_allowed: true, + shell_executor: runUnsandboxedShell, // Wire the host-bundled skills tree so built-ins are discovered + advertised. skill_discovery: opts?.bundled_dir ? { bundled_dir: opts.bundled_dir, include_user_scoped: false } @@ -140,6 +152,7 @@ async function setKey(baseDir: string): Promise { } const RED: [number, number, number] = [220, 30, 30]; +const BLUE: [number, number, number] = [30, 70, 220]; liveDescribe("LIVE — inline image perception + durability", () => { let baseDir: string; @@ -147,7 +160,7 @@ liveDescribe("LIVE — inline image perception + durability", () => { beforeEach(async () => { baseDir = await fs.mkdtemp(path.join(os.tmpdir(), "grida-agent-live-")); - await setKey(baseDir); + if (PROVIDER_KEY) await setKey(baseDir); host = buildHost(baseDir); }); @@ -194,7 +207,154 @@ liveDescribe("LIVE — inline image perception + durability", () => { ); it( - "(b) multi-turn durability: a later text-only turn still sees the image", + "(b) scratch operability: the model copies the staged image byte-for-byte", + async () => { + const workspaceDir = path.join(baseDir, "ingress-workspace"); + await fs.mkdir(workspaceDir); + const workspace = await host.registry.open(workspaceDir); + const redDataUrl = solidPngDataUrl(RED); + const blueDataUrl = solidPngDataUrl(BLUE); + const redBase64 = redDataUrl.slice(redDataUrl.indexOf(",") + 1); + const blueBase64 = blueDataUrl.slice(blueDataUrl.indexOf(",") + 1); + const blueOriginal = Buffer.from(blueBase64, "base64"); + const res = await host.app.request("/agent/run", { + method: "POST", + body: JSON.stringify({ + model_id: MODEL_ID, + workspace_id: workspace.id, + // S4: a copy wholly inside scratch is pre-authorized even in the + // supervised default posture; this turn must not pause for Allow. + mode: "accept-edits", + messages: [ + { + id: "u-operable-image", + role: "user", + parts: [ + { + type: "text", + text: "Look at the two attached provider images, identify the BLUE one, then use provider_file_index in the attachment metadata to copy that same resource's scratch file byte-for-byte to blue-copy.png in the same scratch directory. Do not reconstruct it from pixels and do not inspect the scratch images to choose. Reply ONLY BLUE after the copy exists.", + }, + { + type: "file", + mediaType: "image/png", + url: redDataUrl, + filename: "attachment-a.png", + }, + { + type: "file", + mediaType: "image/png", + url: blueDataUrl, + filename: "attachment-b.png", + }, + { + type: "data-user_file_attachments", + data: { + location: "scratch", + files: [ + { + name: "attachment-a.png", + mime: "image/png", + size: Buffer.from(redBase64, "base64").byteLength, + path: "attachment-a.png", + provider_file_index: 0, + }, + { + name: "attachment-b.png", + mime: "image/png", + size: blueOriginal.byteLength, + path: "attachment-b.png", + provider_file_index: 1, + }, + ], + }, + }, + ], + }, + ], + scratch_seed: [ + { path: "attachment-a.png", base64: redBase64 }, + { path: "attachment-b.png", base64: blueBase64 }, + ], + }), + }); + expect(res.status).toBe(200); + const sse = await res.text(); + const sessionId = sessionIdFromSse(sse); + expect(sessionId).toBeTruthy(); + const scratchDir = scratchRootFor( + path.join(baseDir, "scratch"), + sessionId + ); + const chunks = chunksOf(sse); + expect( + chunks.some( + (chunk) => + chunk.type === "tool-input-available" && + chunk.toolName === "view_image" + ) + ).toBe(false); + const commandInput = chunks.find( + (chunk) => + chunk.type === "tool-input-available" && + chunk.toolName === "run_command" && + typeof chunk.input === "object" && + chunk.input !== null && + (chunk.input as { command?: unknown }).command === "cp" + ); + const commandTrace = commandInput + ? chunks.filter((chunk) => chunk.toolCallId === commandInput.toolCallId) + : []; + expect(commandInput).toBeDefined(); + if ( + commandTrace.some((chunk) => chunk.type === "tool-approval-request") + ) { + throw new Error( + `scratch-local command unexpectedly requested approval; trace=${JSON.stringify(commandTrace)}` + ); + } + const command = commandInput?.input as + | { command?: unknown; args?: unknown; workdir?: unknown } + | undefined; + expect(command?.command).toBe("cp"); + const commandArgs = command?.args; + if ( + !Array.isArray(commandArgs) || + !commandArgs.every( + (operand): operand is string => typeof operand === "string" + ) + ) { + throw new Error(`run_command args were not strings`); + } + expect(commandArgs).toHaveLength(2); + const effectiveWorkdir = + typeof command?.workdir === "string" ? command.workdir : workspaceDir; + const resolvedCommandPaths = await Promise.all( + commandArgs.map((operand) => + fs.realpath(path.resolve(effectiveWorkdir, operand)) + ) + ); + const expectedCommandPaths = await Promise.all( + ["attachment-b.png", "blue-copy.png"].map((name) => + fs.realpath(path.join(scratchDir, name)) + ) + ); + expect(resolvedCommandPaths).toEqual(expectedCommandPaths); + expect(assistantTextFromSse(sse).toLowerCase()).toContain("blue"); + + const copiedPath = path.join(scratchDir, "blue-copy.png"); + const copied = await fs.readFile(copiedPath).catch((cause) => { + throw new Error( + `run_command did not create ${copiedPath}; trace=${JSON.stringify(commandTrace)}`, + { cause } + ); + }); + expect(copied).toEqual(blueOriginal); + }, + TIMEOUT_MS + ); + + it( + "(c) multi-turn durability: a later text-only turn still sees the image", async () => { const t1 = await host.app.request("/agent/run", { method: "POST", @@ -254,7 +414,7 @@ liveDescribe("LIVE — inline image perception + durability", () => { ); it( - "(c) resume durability: the image survives a process restart", + "(d) resume durability: the image survives a process restart", async () => { const t1 = await host.app.request("/agent/run", { method: "POST", @@ -603,8 +763,4 @@ describe("image input — still-deferred capabilities (not yet implemented)", () // The agent DISCOVERS images on its own: list_files surfaces binary files so // the agent can pick one to view without the user naming the path. it.todo("discovers and views a workspace image without being given the path"); - - // A pasted/dropped image becomes OPERABLE ("convert this to .gif"): staged to - // scratch → file-ref so the agent has a path + shell, not just pixels. - it.todo("operates on a pasted image as a file (scratch-staging → file-ref)"); }); diff --git a/packages/grida-ai-agent/src/runtime/runtime.test.ts b/packages/grida-ai-agent/src/runtime/runtime.test.ts index 8ca2dc701..60092369b 100644 --- a/packages/grida-ai-agent/src/runtime/runtime.test.ts +++ b/packages/grida-ai-agent/src/runtime/runtime.test.ts @@ -15,6 +15,7 @@ import { WorkspaceAgentFsBackend, } from "./workspace-agent-bindings"; import { ProviderHttp } from "../providers/http"; +import { runUnsandboxedShell } from "@grida/daemon/server"; const symlinkIt = process.platform === "win32" ? it.skip : it; @@ -90,7 +91,7 @@ describe("agent workspace bindings", () => { it("GRIDA-SEC-004: withholds the command capability unless shell execution is allowed (fail-closed)", async () => { await fixture.write_workspace_file("canvas.svg", ""); - // Default (no shell_execution_allowed) — fs + todos, but NO command. + // Default (no shell executor) — fs + todos, but NO command. const denied = await createWorkspaceAgentBindings( { workspace_root: fixture.workspace_root }, { workspace_registry: fixture.registry } @@ -102,7 +103,10 @@ describe("agent workspace bindings", () => { // Explicit opt-in — command capability is wired. const allowed = await createWorkspaceAgentBindings( { workspace_root: fixture.workspace_root }, - { workspace_registry: fixture.registry, shell_execution_allowed: true } + { + workspace_registry: fixture.registry, + shell_executor: runUnsandboxedShell, + } ); expect(allowed!.command).toBeDefined(); expect(allowed!.command!.default_workdir).toBe(fixture.workspace_root); diff --git a/packages/grida-ai-agent/src/runtime/sandbox-repro.live.test.ts b/packages/grida-ai-agent/src/runtime/sandbox-repro.live.test.ts index bb9399e27..ba9bc491d 100644 --- a/packages/grida-ai-agent/src/runtime/sandbox-repro.live.test.ts +++ b/packages/grida-ai-agent/src/runtime/sandbox-repro.live.test.ts @@ -4,17 +4,19 @@ * too strict" reproduction (the hardcoded allowlist blocked `python3`/`node`); * now it pins the fix. * - * Mirrors the shipped macOS desktop: `shell_execution_allowed` is TRUE and - * `run_command` IS in the tool registry. There is no command allowlist anymore — - * a per-session MODE gates the shell: + * Injects the explicit raw shell executor, so `run_command` is in the tool + * registry without making a Desktop confinement claim. A per-session MODE + * gates the shell: * * - `auto`: every command runs. The agent writes a script and runs it * (`python3`/`node`), producing `chart.svg`. (Headline: the fix works.) - * - `accept-edits`: only read-only commands auto-run; a mutating/executing - * command **pauses for a supervised Allow/Deny** (a `tool-approval-request` - * chunk; the SDK's native `needsApproval`). This harness never approves, so - * the interpreter never runs here. The answer/resume boundary is unit-pinned - * in `store.test.ts` (`answerApproval`) + `workspace-agent-bindings.test.ts`. + * - `accept-edits`: read-only commands and the narrow scratch-local `cp`/`mv` + * exception auto-run; other mutating/executing commands **pause for a + * supervised Allow/Deny** (a `tool-approval-request` chunk; the SDK's native + * `needsApproval`). This harness never approves, so the interpreter never + * runs here. The answer/resume boundary and scratch exception are + * unit-pinned in `store.test.ts` (`answerApproval`) + + * `workspace-agent-bindings.test.ts`. * * NOT exercised here: the srt Seatbelt OUTER wrap (this test runs unsandboxed — * it pins the in-process mode gate + shell runner, the agent-visible behavior). @@ -34,7 +36,7 @@ import { afterEach, beforeEach, describe, expect, it } from "vitest"; import { Hono } from "hono"; import { AuthStore } from "@grida/daemon/server"; import { SecretsStore } from "@grida/daemon/server"; -import { WorkspaceRegistry } from "@grida/daemon/server"; +import { runUnsandboxedShell, WorkspaceRegistry } from "@grida/daemon/server"; import { openSessionsDb } from "../session/db"; import { SessionsStore } from "../session/store"; import { AgentRuntime } from "."; @@ -235,10 +237,10 @@ function buildShellHost(baseDir: string, registry: WorkspaceRegistry): Host { sessions_store: store, streams: new StreamRegistry(), secrets_root: baseDir, - // The shipped macOS desktop sets this true (srt confines the tree). With it - // true, run_command IS registered — so we exercise the real allowlist gate, - // not the fail-closed "no shell at all" gate. - shell_execution_allowed: true, + // Desktop registers run_command by injecting an OS-confined shell_executor. + // This live harness injects the raw runner so it exercises the same + // permission-mode gate without claiming Desktop confinement. + shell_executor: runUnsandboxedShell, drain_cooldown_ms: 20, }); registerAgentRoutes(app, runtime); diff --git a/packages/grida-ai-agent/src/runtime/scratch.live.test.ts b/packages/grida-ai-agent/src/runtime/scratch.live.test.ts index 487ab2b4c..2ac150b84 100644 --- a/packages/grida-ai-agent/src/runtime/scratch.live.test.ts +++ b/packages/grida-ai-agent/src/runtime/scratch.live.test.ts @@ -6,9 +6,10 @@ * extract an archive into scratch (not the project), inspect it, then PROMOTE * the wanted file out into the workspace. * - * Mirrors the shipped macOS desktop: `shell_execution_allowed` is TRUE, mode is - * `auto` (commands run without a supervised pause), and `scratch_base` is wired - * (the runtime derives + creates the per-session dir and tells the agent). + * Injects the explicit raw shell executor, uses `auto` mode (commands run + * without a supervised pause), and wires `scratch_base` (the runtime derives + + * creates the per-session dir and tells the agent). Desktop confinement is not + * exercised here. * * Gated + excluded from CI. Run with a real BYOK key (source the gitignored env * file so process.env carries the key — vitest does NOT auto-load it): @@ -26,7 +27,7 @@ import { afterEach, beforeEach, describe, expect, it } from "vitest"; import { Hono } from "hono"; import { AuthStore } from "@grida/daemon/server"; import { SecretsStore } from "@grida/daemon/server"; -import { WorkspaceRegistry } from "@grida/daemon/server"; +import { runUnsandboxedShell, WorkspaceRegistry } from "@grida/daemon/server"; import { openSessionsDb } from "../session/db"; import { SessionsStore } from "../session/store"; import { AgentRuntime } from "."; @@ -132,7 +133,7 @@ function buildHost( streams: new StreamRegistry(), secrets_root: baseDir, scratch_base: scratchBase, - shell_execution_allowed: true, + shell_executor: runUnsandboxedShell, drain_cooldown_ms: 20, }); registerAgentRoutes(app, runtime); diff --git a/packages/grida-ai-agent/src/runtime/workspace-agent-bindings.test.ts b/packages/grida-ai-agent/src/runtime/workspace-agent-bindings.test.ts index 245072143..edc20f6b7 100644 --- a/packages/grida-ai-agent/src/runtime/workspace-agent-bindings.test.ts +++ b/packages/grida-ai-agent/src/runtime/workspace-agent-bindings.test.ts @@ -13,7 +13,11 @@ import { afterEach, beforeEach, describe, expect, it, vi } from "vitest"; import fs from "node:fs/promises"; import os from "node:os"; import path from "node:path"; -import { workspaceFs, WorkspaceRegistry } from "@grida/daemon/server"; +import { + runUnsandboxedShell, + workspaceFs, + WorkspaceRegistry, +} from "@grida/daemon/server"; import type { SecretsStore } from "@grida/daemon/server"; import { AgentFs } from "../fs"; import { AgentVision } from "../vision"; @@ -354,37 +358,175 @@ describe("createWorkspaceAgentBindings — supervised approval wiring", () => { it("accept-edits: pauses a mutating command, auto-runs a read-only one", async () => { const bindings = await createWorkspaceAgentBindings( { workspace_root: workspaceRoot, mode: "accept-edits" }, - { workspace_registry: registry, shell_execution_allowed: true } + { workspace_registry: registry, shell_executor: runUnsandboxedShell } ); const needsApproval = bindings?.command?.needs_approval; expect(needsApproval).toBeDefined(); // A mutating/executing command requires approval... - expect(needsApproval!({ command: "python3", args: ["x.py"] })).toBe(true); + expect( + needsApproval!({ + command: "python3", + args: ["x.py"], + workdir: workspaceRoot, + }) + ).toBe(true); // ...a read-only inspection command does not. - expect(needsApproval!({ command: "ls", args: ["-la"] })).toBe(false); + expect( + needsApproval!({ + command: "ls", + args: ["-la"], + workdir: workspaceRoot, + }) + ).toBe(false); + }); + + it("accept-edits: scratch-local copy/move is pre-authorized, promotion is not", async () => { + const scratchDir = path.join(baseDir, "scratch"); + await fs.mkdir(scratchDir); + const scratchRoot = await fs.realpath(scratchDir); + const bindings = await createWorkspaceAgentBindings( + { workspace_root: workspaceRoot, mode: "accept-edits" }, + { + workspace_registry: registry, + shell_executor: runUnsandboxedShell, + scratch_dir: scratchRoot, + } + ); + const needsApproval = bindings!.command!.needs_approval!; + await fs.writeFile(path.join(scratchRoot, "source.png"), "source"); + expect( + needsApproval({ + command: "cp", + args: ["source.png", "copy.png"], + workdir: scratchRoot, + }) + ).toBe(false); + await fs.writeFile(path.join(scratchRoot, "copy.png"), "copy"); + expect( + needsApproval({ + command: "mv", + args: [ + path.join(scratchRoot, "copy.png"), + path.join(scratchRoot, "renamed.png"), + ], + workdir: scratchRoot, + }) + ).toBe(false); + await fs.rm(path.join(scratchRoot, "copy.png")); + expect( + needsApproval({ + command: "cp", + args: [ + path.join(scratchRoot, "source.png"), + path.join(scratchRoot, "copy.png"), + ], + // Absolute scratch operands stay scratch-local even when the omitted + // tool workdir resolves to its workspace default. + workdir: workspaceRoot, + }) + ).toBe(false); + expect( + needsApproval({ + command: "cp", + args: ["source.png", path.join(workspaceRoot, "kept.png")], + workdir: scratchRoot, + }) + ).toBe(true); + expect( + needsApproval({ + command: "cp", + args: ["source.png", "copy.png"], + workdir: workspaceRoot, + }) + ).toBe(true); + await fs.writeFile(path.join(scratchRoot, "existing.png"), "existing"); + expect( + needsApproval({ + command: "cp", + args: ["source.png", "existing.png"], + workdir: scratchRoot, + }) + ).toBe(true); + expect( + needsApproval({ + command: "cp", + args: ["--recursive", "source", "copy"], + workdir: scratchRoot, + }) + ).toBe(true); + }); + + it("accept-edits: symlinked scratch operands cannot bypass workspace approval", async () => { + if (process.platform === "win32") return; + const scratchDir = path.join(baseDir, "scratch"); + await fs.mkdir(scratchDir); + const scratchRoot = await fs.realpath(scratchDir); + await fs.writeFile(path.join(scratchRoot, "source.png"), "source"); + await fs.writeFile(path.join(workspaceRoot, "workspace.png"), "workspace"); + const bindings = await createWorkspaceAgentBindings( + { workspace_root: workspaceRoot, mode: "accept-edits" }, + { + workspace_registry: registry, + shell_executor: runUnsandboxedShell, + scratch_dir: scratchRoot, + } + ); + // Plant these after fs hydration; they exercise only the synchronous + // approval classifier and should not produce unrelated hydrate warnings. + await fs.symlink(workspaceRoot, path.join(scratchRoot, "workspace-link")); + await fs.symlink( + path.join(workspaceRoot, "workspace.png"), + path.join(scratchRoot, "file-link.png") + ); + const needsApproval = bindings!.command!.needs_approval!; + expect( + needsApproval({ + command: "cp", + args: ["source.png", "workspace-link/copied.png"], + workdir: scratchRoot, + }) + ).toBe(true); + expect( + needsApproval({ + command: "cp", + args: ["file-link.png", "copied.png"], + workdir: scratchRoot, + }) + ).toBe(true); }); it("auto: supplies no approval predicate (every command auto-runs)", async () => { const bindings = await createWorkspaceAgentBindings( { workspace_root: workspaceRoot, mode: "auto" }, - { workspace_registry: registry, shell_execution_allowed: true } + { workspace_registry: registry, shell_executor: runUnsandboxedShell } ); expect(bindings?.command).toBeDefined(); expect(bindings?.command?.needs_approval).toBeUndefined(); }); - it("no shell containment: no command capability at all", async () => { + it("no shell containment: scratch remains a structured-fs capability", async () => { + const scratchDir = path.join(baseDir, "scratch"); + await fs.mkdir(scratchDir); + const scratchRoot = await fs.realpath(scratchDir); + await fs.writeFile(path.join(scratchRoot, "seeded.txt"), "from scratch"); const bindings = await createWorkspaceAgentBindings( { workspace_root: workspaceRoot, mode: "accept-edits" }, - { workspace_registry: registry, shell_execution_allowed: false } + { + workspace_registry: registry, + scratch_dir: scratchRoot, + } ); expect(bindings?.command).toBeUndefined(); + expect(bindings?.scratch_dir).toBe(scratchRoot); + expect( + await bindings?.fs.readBytes(path.join(scratchRoot, "seeded.txt")) + ).toEqual(new TextEncoder().encode("from scratch")); }); it("serializes a concurrent write before a later command reads it", async () => { const bindings = await createWorkspaceAgentBindings( { workspace_root: workspaceRoot, mode: "auto" }, - { workspace_registry: registry, shell_execution_allowed: true } + { workspace_registry: registry, shell_executor: runUnsandboxedShell } ); expect(bindings?.command).toBeDefined(); @@ -450,7 +592,7 @@ describe("createWorkspaceAgentBindings — scratch reach", () => { { workspace_root: workspaceRoot, mode: "auto" }, { workspace_registry: registry, - shell_execution_allowed: true, + shell_executor: runUnsandboxedShell, secrets_root: secretsRoot, scratch_dir: scratchRoot, } @@ -480,7 +622,7 @@ describe("createWorkspaceAgentBindings — scratch reach", () => { { workspace_root: workspaceRoot, mode: "auto" }, { workspace_registry: registry, - shell_execution_allowed: true, + shell_executor: runUnsandboxedShell, secrets_root: secretsRoot, scratch_dir: linkScratch, // raw, symlinked — realpath ≠ this } @@ -503,7 +645,7 @@ describe("createWorkspaceAgentBindings — scratch reach", () => { { workspace_root: workspaceRoot, mode: "auto" }, { workspace_registry: registry, - shell_execution_allowed: true, + shell_executor: runUnsandboxedShell, secrets_root: secretsRoot, scratch_dir: scratchRoot, } @@ -546,7 +688,7 @@ describe("createWorkspaceAgentBindings — scratch reach", () => { { workspace_root: workspaceRoot, mode: "auto" }, { workspace_registry: registry, - shell_execution_allowed: true, + shell_executor: runUnsandboxedShell, secrets_root: secretsRoot, // no scratch_dir } @@ -609,7 +751,7 @@ describe("createWorkspaceAgentBindings — image_gen gating", () => { { workspace_root: workspaceRoot, mode: "auto" }, { workspace_registry: registry, - shell_execution_allowed: true, + shell_executor: runUnsandboxedShell, ...deps, } ); @@ -776,7 +918,7 @@ describe("createWorkspaceAgentBindings — read-only directory references", () = ], mode: "auto", }, - { workspace_registry: registry, shell_execution_allowed: true } + { workspace_registry: registry, shell_executor: runUnsandboxedShell } ); } diff --git a/packages/grida-ai-agent/src/runtime/workspace-agent-bindings.ts b/packages/grida-ai-agent/src/runtime/workspace-agent-bindings.ts index 0fbf00e0b..3d4f83e7d 100644 --- a/packages/grida-ai-agent/src/runtime/workspace-agent-bindings.ts +++ b/packages/grida-ai-agent/src/runtime/workspace-agent-bindings.ts @@ -9,6 +9,7 @@ import path from "node:path"; import { createHash } from "node:crypto"; +import { lstatSync, realpathSync } from "node:fs"; import { realpath } from "node:fs/promises"; import { generateImage } from "ai"; import { AgentFs } from "../fs"; @@ -37,8 +38,13 @@ import { SCAN_MAX_DEPTH, SCAN_MAX_FILES, } from "@grida/daemon/server"; -import type { Workspace, WorkspaceRegistry } from "@grida/daemon/server"; +import type { + ShellExecutor, + Workspace, + WorkspaceRegistry, +} from "@grida/daemon/server"; import type { ProviderHttp } from "../providers/http"; +import type { RunCommandApprovalInput } from "../tools"; export type WorkspaceAgentBindingRequest = { workspace_root?: string; @@ -112,21 +118,24 @@ export async function createWorkspaceAgentBindings( */ secrets_root?: string; /** - * GRIDA-SEC-004 — fail-closed shell gate. When falsy (the default), the - * `command` capability is NOT returned, so `run_command` never enters the - * tool registry and the model cannot run a shell. The host sets this true - * only when an OS sandbox confines the process tree (srt) or it has - * explicitly opted into an unsandboxed shell. "No containment ⇒ no shell." + * GRIDA-SEC-004 — host-owned finite-command capability. When absent, + * `run_command` is withheld. A Desktop host supplies an OS-confined + * executor; an explicitly unsandboxed host may supply the raw runner. */ - shell_execution_allowed?: boolean; + shell_executor?: ShellExecutor; + /** Register each live command with the owning turn's settlement barrier. */ + track_command_execution?: (task: Promise) => void; /** * The session's scratch dir (WG `scratch.md`): a per-session ephemeral * working area the shell may `cd`/write into though it is NOT a workspace - * (S5). Threaded onto the command backend as an additional allowed cwd root - * and surfaced on the returned `command` binding so the agent can be told - * its path. Absent ⇒ no scratch reach (the command stays workspace-only). + * (S5). Threaded into the exact command scope and surfaced on the returned + * `command` binding so the agent can be told its path. Absent ⇒ no scratch + * reach (the command stays workspace-only). */ scratch_dir?: string; + /** Shared scratch base threaded into the executor scope so the host can + * deny every sibling session root while allowing this session's scratch. */ + scratch_base?: string; /** * The host's `SecretsStore` (BYOK keys). Needed to build the image * generator (`generate_image`); credentials never leave the package. Absent @@ -161,8 +170,12 @@ export async function createWorkspaceAgentBindings( /** Real path of the session scratch dir, when wired — the agent reaches it * via the shell and is told it through the scratch capability hint. */ scratch_dir?: string; - needs_approval?: (input: { command: string; args: string[] }) => boolean; + needs_approval?: (input: RunCommandApprovalInput) => boolean; }; + /** Real, host-provisioned session scratch root already included in `fs`. + * Independent of command exposure so a fail-closed shell posture can still + * advertise the structured filesystem's scratch reach. */ + scratch_dir?: string; /** Image generator backing `generate_image`, when a provider key + scratch * sink are available (S3: produced files land in scratch). */ image_gen?: AgentGen.ImageGenerator; @@ -191,6 +204,10 @@ export async function createWorkspaceAgentBindings( workspace && deps.scratch_dir ? await realpath(deps.scratch_dir).catch(() => deps.scratch_dir!) : undefined; + const scratchBase = + workspace && deps.scratch_base + ? await realpath(deps.scratch_base).catch(() => deps.scratch_base!) + : undefined; // GRIDA-SEC-004 — the workspace-bound agent fs refuses no-clobber writes // (`.git`, lockfiles, rc files, …). The standalone/client-resolved fs gets no // guard, so its behavior is unchanged. @@ -218,37 +235,40 @@ export async function createWorkspaceAgentBindings( ); await fs.hydrate(); const todos = new AgentTodos(); - // GRIDA-SEC-004 fail-closed: only wire shell execution when the host - // affirmed containment (or an explicit unsandboxed opt-in). Otherwise the - // workspace still gets fs + todos, but no `run_command`. + // GRIDA-SEC-004 fail-closed: only wire shell execution when the host supplied + // a finite-command capability. Otherwise the workspace still gets fs + todos, + // but no `run_command`. const mode = req.mode ?? AGENT_DEFAULT_MODE; const command = - workspace && req.workspace_root && deps.shell_execution_allowed + workspace && req.workspace_root && deps.shell_executor ? { - backend: createAgentCommandBackend( - deps.workspace_registry, - deps.secrets_root ? [deps.secrets_root] : [], - // Scratch is a sanctioned cwd root though it is not a workspace (S5). - scratchDir ? [scratchDir] : [], + backend: createAgentCommandBackend({ + workspace_root: workspace.root, + scratch_root: scratchDir, + scratch_base: scratchBase, + protected_read_roots: deps.secrets_root ? [deps.secrets_root] : [], + executor: deps.shell_executor, // Flush the agent fs's pending writes before a command runs, so a // script the agent just wrote via write_file is on disk when the // shell reads it (closes the debounced-write vs immediate-read race). - () => fs.flush(), + before_run: () => fs.flush(), // Tool calls in one model step may execute concurrently. Share the // fs FIFO across the entire command so call order, persistence, and // subsequent operation-time reads agree. - (action) => fs.runExclusive(action) - ), - default_workdir: req.workspace_root, + run_exclusive: (action) => fs.runExclusive(action), + track_execution: deps.track_command_execution, + }), + default_workdir: workspace.root, scratch_dir: scratchDir, // Supervised gate (RFC `permission modes`, Phase 2). In `accept-edits` - // a non-read-only command pauses for Allow/Deny (the tool's - // `needsApproval`); a read-only inspection command still auto-runs. In - // `auto` the predicate is absent — every command runs without asking. + // a command pauses unless it is read-only or a narrowly constrained + // scratch-local copy/move. In `auto` the predicate is absent — every + // command runs without asking. needs_approval: mode === "accept-edits" - ? ({ command, args }: { command: string; args: string[] }) => - !isReadOnlyCommand(command, args) + ? (input: RunCommandApprovalInput) => + !isReadOnlyCommand(input.command, input.args) && + !isScratchLocalCopyOrMove(input, scratchDir) : undefined, } : undefined; @@ -286,7 +306,81 @@ export async function createWorkspaceAgentBindings( const skill_load_body = scratchDir ? createMaterializingSkillLoader(scratchDir) : undefined; - return { fs, todos, command, image_gen, skill_load_body }; + return { + fs, + todos, + command, + scratch_dir: scratchDir, + image_gen, + skill_load_body, + }; +} + +/** + * Scratch is a pre-authorized working area (WG scratch S4). In supervised + * mode, a plain two-path `cp` or `mv` that stays wholly inside that area must + * not pause for approval. Keep this deliberately narrow: flags, another cwd, + * promotion into the workspace, or any unknown command retain the ordinary + * mutating-command approval. + * + * This is an approval-UX classifier, not the containment boundary. The command + * backend and OS sandbox still validate and confine the actual process. + */ +function isScratchLocalCopyOrMove( + input: RunCommandApprovalInput, + scratchDir: string | undefined +): boolean { + if ( + !scratchDir || + (input.command !== "cp" && input.command !== "mv") || + input.args.length !== 2 || + input.args.some((arg) => arg.length === 0 || arg.startsWith("-")) + ) { + return false; + } + const workdir = path.resolve(input.workdir); + let resolvedWorkdir: string | undefined; + try { + resolvedWorkdir = realpathSync(workdir); + } catch { + // Absolute operands do not need cwd; relative operands fail closed below. + } + const operands = input.args.map((arg) => { + if (!path.isAbsolute(arg)) { + if (!resolvedWorkdir || !containsPath(scratchDir, resolvedWorkdir)) { + return null; + } + } + const resolved = path.resolve(workdir, arg); + return containsPath(scratchDir, resolved) ? resolved : null; + }); + if (operands.some((operand) => operand === null)) return false; + const [source, destination] = operands as [string, string]; + + try { + // Copy/move only an existing regular file whose canonical target remains + // in scratch. This rejects a scratch symlink that points into the workspace. + const sourceStat = lstatSync(source); + if (!sourceStat.isFile() || sourceStat.isSymbolicLink()) return false; + if (!containsPath(scratchDir, realpathSync(source))) return false; + + // Never auto-approve an overwrite. Besides preserving the scratch input, + // this avoids truncating a workspace file through a scratch hard link. + try { + lstatSync(destination); + return false; + } catch (err) { + if ((err as NodeJS.ErrnoException).code !== "ENOENT") return false; + } + + // The destination itself is new, so canonicalize its existing parent. + // A lexical `/link/file` whose `link` targets the workspace then + // fails containment instead of bypassing supervised approval. + const destinationParent = realpathSync(path.dirname(destination)); + return containsPath(scratchDir, destinationParent); + } catch { + return false; + } } /** File extension for a produced image's media type (best-effort). */ diff --git a/packages/grida-ai-agent/src/server.test.ts b/packages/grida-ai-agent/src/server.test.ts index 5e1f22057..0693c035a 100644 --- a/packages/grida-ai-agent/src/server.test.ts +++ b/packages/grida-ai-agent/src/server.test.ts @@ -23,7 +23,7 @@ describe("agentTenantOptionsFromDaemon — no host field is silently dropped", ( // Every host-supplied tenant field must ride from the composed daemon into // the tenant. `skills_root` shipped disabled precisely because it was omitted // here → the agent discovered ZERO built-in skills on the desktop. - it("forwards skills_root, gg_base_url, scratch_base, provider_http, and the capability flags", () => { + it("forwards host capabilities, including the finite shell executor", () => { const provider_http = { request: globalThis.fetch, download: globalThis.fetch, @@ -40,6 +40,9 @@ describe("agentTenantOptionsFromDaemon — no host field is silently dropped", ( originator: "grida-test", default_model_id: "openai/gpt-5.6-terra" as const, }; + const shell_executor = async () => { + throw new Error("not invoked by the option-mapping test"); + }; const out = agentTenantOptionsFromDaemon( { password: "p", @@ -49,6 +52,7 @@ describe("agentTenantOptionsFromDaemon — no host field is silently dropped", ( gg_base_url: "https://grida.co", scratch_base: "/tmp/scratch", image_model_id: "img/model", + shell_executor, sandbox_enforced: true, external_agent_execution: "disabled", allow_unsandboxed_shell: false, @@ -63,6 +67,7 @@ describe("agentTenantOptionsFromDaemon — no host field is silently dropped", ( expect(out.gg_base_url).toBe("https://grida.co"); expect(out.scratch_base).toBe("/tmp/scratch"); expect(out.image_model_id).toBe("img/model"); + expect(out.shell_executor).toBe(shell_executor); expect(out.sandbox_enforced).toBe(true); expect(out.external_agent_execution).toBe("disabled"); expect(out.interactive).toBe(true); diff --git a/packages/grida-ai-agent/src/server.ts b/packages/grida-ai-agent/src/server.ts index d4f73dbe8..40e15f30b 100644 --- a/packages/grida-ai-agent/src/server.ts +++ b/packages/grida-ai-agent/src/server.ts @@ -20,10 +20,12 @@ import os from "node:os"; import type { Hono } from "hono"; import { DaemonServer, + runUnsandboxedShell, type DaemonCapabilities, type DaemonHttpAccess, type DaemonServices, type DaemonTenant, + type ShellExecutor, } from "@grida/daemon/server"; import { buildDaemonSandboxPolicy } from "@grida/daemon/sandbox"; import { registerSecretsRoutes } from "./http/routes/secrets"; @@ -65,6 +67,7 @@ export type { ChatGptOAuthConfig, } from "./providers/chatgpt-credentials"; export type { ChatGptProviderConfig } from "./providers/chatgpt"; +export { defaultScratchBase, prepareScratchAuthority } from "./session/scratch"; // Re-exported for hosts that compose or probe the daemon through this // package (the CLI, tests). The daemon package is the owner. @@ -78,6 +81,9 @@ export { type DaemonHttpAccess, type DaemonServices, type DaemonTenant, + type ShellExecutionScope, + type ShellExecutor, + type ShellRunOptions, } from "@grida/daemon/server"; /** @@ -143,12 +149,22 @@ export type AgentTenantOptions = { * model is not an agent argument. Omit to use the catalog default. */ image_model_id?: string; + /** + * GRIDA-SEC-004 — host-owned finite-command capability. The executor receives + * the exact current workspace/scratch scope for every invocation and must + * enforce it at the process boundary. Omission withholds `run_command`. + * + * `sandbox_enforced` is not an executor: wrapping the multi-session daemon + * once cannot isolate sibling session scratch roots. Standalone hosts that + * deliberately accept ambient filesystem authority use + * {@link allow_unsandboxed_shell}, which injects the raw runner explicitly. + */ + shell_executor?: ShellExecutor; /** * GRIDA-SEC-004 — whether this host's process tree is confined by an OS - * sandbox (srt Seatbelt/bubblewrap). Default `false`: with no explicit - * shell-only opt-in the `run_command` tool is withheld, and an external ACP - * mode of `"sandboxed"` is unavailable. The desktop supervisor sets this - * true only when it actually wrapped the sidecar spawn. + * sandbox (srt Seatbelt/bubblewrap). This attests only the coarse outer + * process tree for external ACP's `"sandboxed"` disposition; it does not + * expose `run_command`. Finite commands require {@link shell_executor}. */ sandbox_enforced?: boolean; /** @@ -164,11 +180,12 @@ export type AgentTenantOptions = { */ external_agent_execution?: "enabled" | "sandboxed" | "disabled"; /** - * GRIDA-SEC-004 — deliberate escape hatch for hosts that run WITHOUT an OS - * sandbox (the `grida-agent` CLI, local dev). When true, `run_command` is - * exposed even though `sandbox_enforced` is false. Off by default; enabling - * it is an explicit, logged decision by the host author who accepts that the - * shell child has no kernel-level fs/network containment. + * GRIDA-SEC-004 — deliberate raw-execution escape hatch (the `grida-agent` + * CLI, local dev). When true and no {@link shell_executor} is supplied, the + * package injects {@link runUnsandboxedShell}. Off by default; enabling it is + * an + * explicit, logged decision by the host author who accepts that the shell + * child has no session-bound kernel-level filesystem containment. * * This does not affect external ACP agents; their independent disposition is * controlled by `external_agent_execution`. @@ -340,20 +357,22 @@ export function createAgentTenant(opts: AgentTenantOptions = {}): DaemonTenant { provider_http: providerHttp, }); } - // GRIDA-SEC-004 — the single fail-closed shell decision. Shell execution - // is off unless the host confirmed an OS sandbox confines the tree, or it - // explicitly opted into an unsandboxed shell. Computed here (one auditable - // place) and threaded to the runtime → bindings → tool registry. - const shellExecutionAllowed = - opts.sandbox_enforced === true || opts.allow_unsandboxed_shell === true; + // GRIDA-SEC-004 — a finite command is a host capability, not a boolean + // attestation about the daemon's broad process tree. Standalone/CLI hosts + // retain raw execution only through the explicit unsandboxed switch. + const shellExecutor = + opts.shell_executor ?? + (opts.allow_unsandboxed_shell === true + ? runUnsandboxedShell + : undefined); if ( opts.allow_unsandboxed_shell === true && - opts.sandbox_enforced !== true + opts.shell_executor === undefined ) { console.warn( - "[grida-agent] GRIDA-SEC-004: run_command exposed WITHOUT an OS sandbox " + - "(allow_unsandboxed_shell). The shell child has no kernel-level " + - "fs/network containment — only the in-process allowlist + arg checks." + "[grida-agent] GRIDA-SEC-004: run_command exposed with the raw executor " + + "(allow_unsandboxed_shell). The shell child has no session-bound " + + "kernel-level filesystem containment — only exact-cwd + arg checks." ); } // Per-session scratch base (WG `scratch.md`). Host-injected; default at @@ -365,7 +384,7 @@ export function createAgentTenant(opts: AgentTenantOptions = {}): DaemonTenant { // underneath a running command by a still-in-flight async sweep. const scratchBase = opts.scratch_base ?? defaultScratchBase(services.user_data_path); - sweepScratch(scratchBase); + sweepScratch(scratchBase, services.user_data_path); const runtime = new AgentRuntime({ secrets: services.secrets, endpoints: endpointsStore, @@ -380,12 +399,11 @@ export function createAgentTenant(opts: AgentTenantOptions = {}): DaemonTenant { directory_scopes: directoryScopes, streams, // GRIDA-SEC-004: the daemon's own secret dir (auth.json, sessions.db, - // workspaces.json, recent.json). Threaded to the shell runner so the - // agent's `run_command` cannot read it back into the transcript. NOT - // added to the srt deny_read policy — the daemon itself reads auth.json. + // workspaces.json, recent.json). Threaded into exact command scope so a + // confined executor can deny it while the daemon itself retains access. secrets_root: services.user_data_path, scratch_base: scratchBase, - shell_execution_allowed: shellExecutionAllowed, + shell_executor: shellExecutor, // GRIDA-SEC-004 — the sandboxed ACP disposition consumes this host // attestation; the explicit `enabled` disposition does not. sandbox_enforced: opts.sandbox_enforced === true, diff --git a/packages/grida-ai-agent/src/session/scratch.test.ts b/packages/grida-ai-agent/src/session/scratch.test.ts index e78e5f034..391ed59ce 100644 --- a/packages/grida-ai-agent/src/session/scratch.test.ts +++ b/packages/grida-ai-agent/src/session/scratch.test.ts @@ -5,7 +5,7 @@ * grep-able. Pure derivation is asserted without I/O; the thin I/O helpers run * against a real temp dir under `os.tmpdir()`. */ -import { afterEach, beforeEach, describe, expect, it } from "vitest"; +import { afterEach, beforeEach, describe, expect, it, vi } from "vitest"; import fs from "node:fs/promises"; import os from "node:os"; import path from "node:path"; @@ -13,6 +13,8 @@ import { assertOutsideSecretsRoot, defaultScratchBase, ensureScratch, + listScratchFilePaths, + prepareScratchAuthority, removeScratch, scratchRootFor, sweepScratch, @@ -84,15 +86,21 @@ describe("scratch I/O helpers", () => { await expect(ensureScratch(root)).resolves.toBeUndefined(); }); - it("ensureScratch creates the scratch dir owner-only (0700)", async () => { + it("ensureScratch creates every authority level owner-only (0700)", async () => { // Other local accounts must not read produced/extracted artifacts on a // shared machine (`` can resolve under a world-traversable // `/tmp`). Skip on Windows, which has no POSIX mode bits. if (process.platform === "win32") return; const root = scratchRootFor(base, "ses_mode"); await ensureScratch(root); - const stat = await fs.stat(root); - expect(stat.mode & 0o777).toBe(0o700); + for (const dir of [ + base, + path.join(base, "sessions"), + path.dirname(root), + root, + ]) { + expect((await fs.lstat(dir)).mode & 0o777).toBe(0o700); + } }); it("ensureScratch refuses a scratch dir nested in the secret root (S4)", async () => { @@ -103,22 +111,91 @@ describe("scratch I/O helpers", () => { await expect(ensureScratch(badDir, secrets)).rejects.toThrow(/secret root/); }); - it("ensureScratch tightens a pre-existing permissive scratch dir to 0700", async () => { + it("ensureScratch tightens every pre-existing permissive authority dir to 0700", async () => { // `mkdir` won't change an existing dir's mode, so an attacker-pre-created - // world-readable scratch dir would otherwise keep leaking. Skip on Windows. + // world-readable level would otherwise keep leaking. Skip on Windows. if (process.platform === "win32") return; const root = scratchRootFor(base, "ses_pre"); await fs.mkdir(root, { recursive: true }); // Force a permissive starting mode regardless of the runner's umask (which // could otherwise mask mkdir's mode down to 0700 and skip the tightening // path this test exists to exercise). - await fs.chmod(path.dirname(root), 0o755); - await fs.chmod(root, 0o755); - expect((await fs.stat(root)).mode & 0o777).toBe(0o755); + const dirs = [base, path.join(base, "sessions"), path.dirname(root), root]; + for (const dir of dirs) { + await fs.chmod(dir, 0o755); + } await ensureScratch(root); - expect((await fs.stat(root)).mode & 0o777).toBe(0o700); - // The session dir (parent) is tightened too. - expect((await fs.stat(path.dirname(root))).mode & 0o777).toBe(0o700); + for (const dir of dirs) { + expect((await fs.lstat(dir)).mode & 0o777).toBe(0o700); + } + }); + + it("ensureScratch rejects authority ancestry not owned by the current uid or root", async () => { + if (process.platform === "win32" || process.getuid === undefined) return; + const realUid = process.getuid(); + const uid = realUid === Number.MAX_SAFE_INTEGER ? realUid - 1 : realUid + 1; + const getuid = vi.spyOn(process, "getuid").mockReturnValue(uid); + try { + await expect( + ensureScratch(scratchRootFor(base, "ses_foreign")) + ).rejects.toThrow(/uid/); + } finally { + getuid.mockRestore(); + } + }); + + it("ensureScratch rejects a sticky parent owned by an unprivileged foreign uid", async () => { + if ( + process.platform === "win32" || + process.getuid === undefined || + process.getuid() === 0 + ) { + return; + } + const foreignParent = path.join(base, "foreign-sticky-parent"); + await fs.mkdir(foreignParent); + await fs.chmod(foreignParent, 0o1777); + const realUid = process.getuid(); + const getuid = vi + .spyOn(process, "getuid") + .mockReturnValue( + realUid === Number.MAX_SAFE_INTEGER ? realUid - 1 : realUid + 1 + ); + try { + await expect( + ensureScratch( + scratchRootFor(path.join(foreignParent, "authority"), "ses_x") + ) + ).rejects.toThrow(/nor privileged uid 0/); + } finally { + getuid.mockRestore(); + } + }); + + it("ensureScratch rejects a group/other-writable non-sticky base parent", async () => { + if (process.platform === "win32") return; + const unsafeParent = path.join(base, "unsafe-parent"); + await fs.mkdir(unsafeParent, { mode: 0o777 }); + await fs.chmod(unsafeParent, 0o777); + const unsafeBase = path.join(unsafeParent, "authority"); + + await expect( + ensureScratch(scratchRootFor(unsafeBase, "ses_x")) + ).rejects.toThrow(/group\/other-writable without the sticky bit/); + await expect(fs.lstat(unsafeBase)).rejects.toThrow(/ENOENT/); + }); + + it("ensureScratch accepts a sticky shared base parent", async () => { + if (process.platform === "win32") return; + const stickyParent = path.join(base, "sticky-parent"); + await fs.mkdir(stickyParent); + await fs.chmod(stickyParent, 0o1777); + const root = scratchRootFor(path.join(stickyParent, "authority"), "ses_x"); + + await expect(ensureScratch(root)).resolves.toBeUndefined(); + expect( + (await fs.lstat(path.dirname(path.dirname(root)))).mode & 0o777 + ).toBe(0o700); }); it("ensureScratch refuses a SYMLINKED base that resolves into the secret root (S4)", async () => { @@ -147,6 +224,34 @@ describe("scratch I/O helpers", () => { expect(new Uint8Array(await fs.readFile(out))).toEqual(bytes); }); + it("writeScratchFile preserves generated-artifact overwrite behavior", async () => { + const root = scratchRootFor(base, "ses_overwrite"); + await ensureScratch(root); + const target = await writeScratchFile( + root, + "image.png", + new Uint8Array([1, 2, 3]) + ); + await writeScratchFile(root, "image.png", new Uint8Array([4, 5])); + expect(new Uint8Array(await fs.readFile(target))).toEqual( + new Uint8Array([4, 5]) + ); + }); + + it("writeScratchFile can reject a seed collision without truncating the original", async () => { + const root = scratchRootFor(base, "ses_no_clobber"); + await ensureScratch(root); + const original = new Uint8Array([1, 2, 3]); + const target = await writeScratchFile(root, "input.bin", original); + + await expect( + writeScratchFile(root, "input.bin", new Uint8Array([9]), { + overwrite: false, + }) + ).rejects.toThrow(/EEXIST/); + expect(new Uint8Array(await fs.readFile(target))).toEqual(original); + }); + it("writeScratchFile writes the produced file owner-only (0600)", async () => { // Shared-machine reasoning, same as the dir mode. Skip on Windows (no POSIX // mode bits). @@ -189,6 +294,25 @@ describe("scratch I/O helpers", () => { await expect(fs.stat(outside)).rejects.toThrow(/ENOENT/); }); + it("listScratchFilePaths reports only live direct regular files and fails closed", async () => { + const root = scratchRootFor(base, "ses_list"); + await ensureScratch(root); + await fs.writeFile(path.join(root, "live.bin"), "bytes"); + await fs.mkdir(path.join(root, "nested")); + await fs.writeFile(path.join(root, "nested", "hidden.bin"), "bytes"); + if (process.platform !== "win32") { + await fs.symlink( + path.join(root, "live.bin"), + path.join(root, "linked.bin") + ); + } + + expect(await listScratchFilePaths(root)).toEqual(new Set(["live.bin"])); + expect(await listScratchFilePaths(path.join(root, "missing"))).toEqual( + new Set() + ); + }); + it("removeScratch is recursive and idempotent (S2)", async () => { const root = scratchRootFor(base, "ses_rm"); await ensureScratch(root); @@ -200,14 +324,192 @@ describe("scratch I/O helpers", () => { await expect(removeScratch(base, "ses_never")).resolves.toBeUndefined(); }); - it("sweepScratch reclaims every session dir; a missing base is a no-op (S2)", async () => { + it("removeScratch unlinks a session symlink without deleting its target", async () => { + if (process.platform === "win32") return; + prepareScratchAuthority(base); + const target = path.join(base, "outside-session"); + await fs.mkdir(target); + await fs.writeFile(path.join(target, "keep.txt"), "keep"); + const link = path.join(base, "sessions", "ses_link"); + await fs.symlink(target, link); + + await removeScratch(base, "ses_link"); + + await expect(fs.lstat(link)).rejects.toThrow(/ENOENT/); + await expect( + fs.readFile(path.join(target, "keep.txt"), "utf8") + ).resolves.toBe("keep"); + }); + + it("removeScratch rejects a symlinked sessions authority without touching its target", async () => { + if (process.platform === "win32") return; + const authority = path.join(base, "remove-authority"); + const target = path.join(base, "remove-target"); + await fs.mkdir(authority); + await fs.mkdir(path.join(target, "ses_victim"), { recursive: true }); + await fs.writeFile(path.join(target, "ses_victim", "keep.txt"), "keep"); + await fs.symlink(target, path.join(authority, "sessions")); + + await expect(removeScratch(authority, "ses_victim")).rejects.toThrow( + /non-symlink directory/ + ); + await expect( + fs.readFile(path.join(target, "ses_victim", "keep.txt"), "utf8") + ).resolves.toBe("keep"); + }); + + it("sweepScratch reclaims every session dir and establishes a fresh authority (S2)", async () => { await ensureScratch(scratchRootFor(base, "ses_a")); await ensureScratch(scratchRootFor(base, "ses_b")); // Synchronous — the host calls it before serving runs (no race). sweepScratch(base); expect(await fs.readdir(path.join(base, "sessions"))).toEqual([]); - // A base that was never used (fresh host) sweeps without error. - expect(() => sweepScratch(path.join(base, "does-not-exist"))).not.toThrow(); + // A fresh authority is securely established even when there is nothing to + // reclaim, ready for the first turn. + const fresh = path.join(base, "does-not-exist"); + expect(() => sweepScratch(fresh)).not.toThrow(); + expect(await fs.readdir(path.join(fresh, "sessions"))).toEqual([]); + }); + + it("sweepScratch rejects a symlinked base without deleting target contents", async () => { + if (process.platform === "win32") return; + const target = path.join(base, "base-target"); + await fs.mkdir(path.join(target, "sessions", "ses_victim"), { + recursive: true, + }); + const keep = path.join(target, "sessions", "ses_victim", "keep.txt"); + await fs.writeFile(keep, "keep"); + const link = path.join(base, "base-link"); + await fs.symlink(target, link); + + expect(() => sweepScratch(link)).toThrow(/non-symlink directory/); + await expect(fs.readFile(keep, "utf8")).resolves.toBe("keep"); + }); + + it("sweepScratch rejects physical secret-root overlap before touching a symlink target", async () => { + if (process.platform === "win32") return; + const secrets = path.join(base, "secret-root"); + const target = path.join(secrets, "scratch-authority"); + await fs.mkdir(path.join(target, "sessions", "ses_victim"), { + recursive: true, + }); + const keep = path.join(target, "sessions", "ses_victim", "keep.txt"); + await fs.writeFile(keep, "keep"); + const link = path.join(base, "secret-link"); + await fs.symlink(target, link); + + expect(() => sweepScratch(link, secrets)).toThrow( + /physically overlaps the secret root/ + ); + await expect(fs.readFile(keep, "utf8")).resolves.toBe("keep"); + }); + + it("sweepScratch rejects a broad base containing the secret root before chmod or deletion", async () => { + if (process.platform === "win32") return; + const broadBase = path.join(base, "broad-base"); + const secrets = path.join(broadBase, "user-data"); + const victim = path.join(broadBase, "sessions", "ses_victim"); + await fs.mkdir(secrets, { recursive: true }); + await fs.mkdir(victim, { recursive: true }); + const keep = path.join(victim, "keep.txt"); + await fs.writeFile(keep, "keep"); + await fs.chmod(broadBase, 0o755); + + expect(() => sweepScratch(broadBase, secrets)).toThrow( + /must not contain the secret root/ + ); + expect((await fs.lstat(broadBase)).mode & 0o777).toBe(0o755); + await expect(fs.readFile(keep, "utf8")).resolves.toBe("keep"); + }); + + it("sweepScratch rejects a symlinked sessions root without deleting target contents", async () => { + if (process.platform === "win32") return; + const authority = path.join(base, "sweep-authority"); + const target = path.join(base, "sessions-target"); + await fs.mkdir(authority); + await fs.mkdir(path.join(target, "ses_victim"), { recursive: true }); + const keep = path.join(target, "ses_victim", "keep.txt"); + await fs.writeFile(keep, "keep"); + await fs.symlink(target, path.join(authority, "sessions")); + + expect(() => sweepScratch(authority)).toThrow(/non-symlink directory/); + await expect(fs.readFile(keep, "utf8")).resolves.toBe("keep"); + }); + + it("sweepScratch unlinks child symlinks instead of recursing into them", async () => { + if (process.platform === "win32") return; + prepareScratchAuthority(base); + const target = path.join(base, "sweep-child-target"); + await fs.mkdir(target); + const keep = path.join(target, "keep.txt"); + await fs.writeFile(keep, "keep"); + const link = path.join(base, "sessions", "ses_link"); + await fs.symlink(target, link); + + sweepScratch(base); + + await expect(fs.lstat(link)).rejects.toThrow(/ENOENT/); + await expect(fs.readFile(keep, "utf8")).resolves.toBe("keep"); + }); + + it("sweepScratch logs an entry deletion failure and continues reclaiming siblings", async () => { + if ( + process.platform === "win32" || + process.getuid === undefined || + process.getuid() === 0 + ) { + return; + } + prepareScratchAuthority(base); + const sessionsDir = path.join(base, "sessions"); + for (const name of ["ses_one", "ses_two"]) { + const nested = path.join(sessionsDir, name, "nested"); + await fs.mkdir(nested, { recursive: true }); + await fs.writeFile(path.join(nested, "artifact.txt"), "bytes"); + } + const entries = await fs.readdir(sessionsDir); + expect(entries).toHaveLength(2); + const [blockedName, reclaimableName] = entries as [string, string]; + const blocked = path.join(sessionsDir, blockedName); + const reclaimable = path.join(sessionsDir, reclaimableName); + await fs.chmod(blocked, 0); + const warn = vi.spyOn(console, "warn").mockImplementation(() => undefined); + + try { + expect(() => sweepScratch(base)).not.toThrow(); + expect(warn).toHaveBeenCalledWith( + expect.stringContaining(JSON.stringify(blockedName)) + ); + await expect(fs.lstat(blocked)).resolves.toBeDefined(); + await expect(fs.lstat(reclaimable)).rejects.toThrow(/ENOENT/); + } finally { + warn.mockRestore(); + await fs.chmod(blocked, 0o700).catch(() => undefined); + } + }); + + it("prepareScratchAuthority secures base and sessions without sweeping", async () => { + if (process.platform === "win32") return; + const authority = path.join(base, "prepare-only"); + prepareScratchAuthority(authority); + const marker = path.join(authority, "sessions", "keep.txt"); + await fs.writeFile(marker, "keep"); + + prepareScratchAuthority(authority); + + await expect(fs.readFile(marker, "utf8")).resolves.toBe("keep"); + expect((await fs.lstat(authority)).mode & 0o777).toBe(0o700); + expect( + (await fs.lstat(path.join(authority, "sessions"))).mode & 0o777 + ).toBe(0o700); + }); + + it("prepareScratchAuthority never recursively creates a missing parent", async () => { + const missingParent = path.join(base, "missing-parent"); + expect(() => + prepareScratchAuthority(path.join(missingParent, "authority")) + ).toThrow(/cannot inspect parent/); + await expect(fs.lstat(missingParent)).rejects.toThrow(/ENOENT/); }); }); @@ -228,4 +530,11 @@ describe("defaultScratchBase", () => { expect(a).not.toBe(b); expect(defaultScratchBase("/home/u/.grida/agent")).toBe(a); }); + + it("lets a native host inject its own temp authority root", () => { + const nativeTemp = path.join(os.tmpdir(), "desktop-owned-temp"); + const b = defaultScratchBase("/home/u/.grida/agent", nativeTemp); + expect(path.dirname(b)).toBe(nativeTemp); + expect(path.basename(b)).toMatch(/^grida-agent-[0-9a-f]{16}$/); + }); }); diff --git a/packages/grida-ai-agent/src/session/scratch.ts b/packages/grida-ai-agent/src/session/scratch.ts index 770c2d882..6cf5d56f2 100644 --- a/packages/grida-ai-agent/src/session/scratch.ts +++ b/packages/grida-ai-agent/src/session/scratch.ts @@ -8,8 +8,9 @@ * path-out, no I/O, headlessly testable. These carry the package-owned * INVARIANTS the host cannot override: per-session isolation (S1) and the * refusal to sit inside the host's secret root (S4 containment). - * - THIN I/O (`ensureScratch`, `removeScratch`, `sweepScratch`) — mkdir/rm - * wrappers the runtime wires into the session lifecycle. + * - AUTHORITY I/O (`prepareScratchAuthority`, `ensureScratch`, + * `removeScratch`, `sweepScratch`) — fail-closed ownership/mode checks plus + * bounded creation and cleanup wired into the session lifecycle. * * WHERE scratch physically lives is host-owned I/O: the host injects a `base` * and the default (`defaultScratchBase`) is resolved at the host/CLI entrypoint @@ -22,8 +23,27 @@ * is both outside the secret root and naturally ephemeral. */ -import { chmod, mkdir, rm, realpath, open } from "node:fs/promises"; -import { readdirSync, rmSync, constants as fsConstants } from "node:fs"; +import { + chmod, + lstat, + mkdir, + open, + readdir, + realpath, + rm, + unlink, +} from "node:fs/promises"; +import { + chmodSync, + constants as fsConstants, + lstatSync, + mkdirSync, + readdirSync, + realpathSync, + rmSync, + unlinkSync, +} from "node:fs"; +import type { Stats } from "node:fs"; import crypto from "node:crypto"; import os from "node:os"; import path from "node:path"; @@ -38,8 +58,8 @@ const SCRATCH_NAMESPACE = "grida-agent"; * Owner-only (`rwx------`) mode for every scratch dir we create. On a shared * Unix machine the default base (`/grida-agent`) can resolve under * a world-traversable `/tmp`, so without this another local account could list - * the sessions tree and read produced/extracted artifacts. `mkdir` applies this - * to each level it creates (subject to umask, which only removes bits). + * the sessions tree and read produced/extracted artifacts. Creation requests + * this mode, then chmod + lstat verify it on every authority level. */ const SCRATCH_DIR_MODE = 0o700; @@ -47,12 +67,225 @@ const SCRATCH_DIR_MODE = 0o700; * shared-machine reasoning as {@link SCRATCH_DIR_MODE}. */ const SCRATCH_FILE_MODE = 0o600; +function isErrno(err: unknown, code: NodeJS.ErrnoException["code"]): boolean { + return (err as NodeJS.ErrnoException).code === code; +} + +function scratchAuthorityError(target: string, reason: string): Error { + return new Error(`unsafe scratch authority at ${target}: ${reason}`); +} + +function currentUid(): number | undefined { + return process.platform === "win32" || typeof process.getuid !== "function" + ? undefined + : process.getuid(); +} + +/** + * Resolve a host-provided authority path once and reject broad/unstable + * targets. Scratch locations are host-owned absolute paths; accepting `/` + * would let the mode-tightening below chmod the filesystem root. + */ +function resolveAuthorityBase(base: string): string { + if (!path.isAbsolute(base)) { + throw scratchAuthorityError(base, "the base must be an absolute path"); + } + const resolved = path.resolve(base); + if (path.dirname(resolved) === resolved) { + throw scratchAuthorityError(base, "the filesystem root cannot be a base"); + } + return resolved; +} + +/** + * A local account must not be able to rename or replace the authority entry + * through its parent. A group-/other-writable parent is safe only with the + * sticky bit (the normal `/tmp` contract). This is a POSIX ownership boundary; + * Windows does not expose equivalent uid/mode semantics through Node. + */ +function assertSafeParent(parent: string, stat: Stats): void { + if (stat.isSymbolicLink() || !stat.isDirectory()) { + throw scratchAuthorityError( + parent, + "the parent must be a non-symlink directory" + ); + } + const uid = currentUid(); + if (uid === undefined) return; + if (stat.uid !== uid && stat.uid !== 0) { + throw scratchAuthorityError( + parent, + `parent uid ${stat.uid} is neither process uid ${uid} nor privileged uid 0` + ); + } + const groupOrOtherWritable = (stat.mode & 0o022) !== 0; + const sticky = (stat.mode & 0o1000) !== 0; + if (groupOrOtherWritable && !sticky) { + throw scratchAuthorityError( + parent, + "the parent is group/other-writable without the sticky bit" + ); + } +} + +function assertOwnedDirectory(target: string, stat: Stats): void { + if (stat.isSymbolicLink() || !stat.isDirectory()) { + throw scratchAuthorityError(target, "expected a non-symlink directory"); + } + const uid = currentUid(); + if (uid !== undefined && stat.uid !== uid) { + throw scratchAuthorityError( + target, + `directory uid ${stat.uid} does not match process uid ${uid}` + ); + } +} + +function verifyPrivateMode(target: string, stat: Stats): void { + if (currentUid() !== undefined && (stat.mode & 0o777) !== SCRATCH_DIR_MODE) { + throw scratchAuthorityError( + target, + `directory mode is ${(stat.mode & 0o777).toString(8)}, expected 700` + ); + } +} + +/** + * Establish one authority directory without following a final symlink. + * + * The parent is inspected BEFORE mkdir. Creation is deliberately + * non-recursive: no unchecked intermediate can be created or followed on the + * way to a predictable temp path. On POSIX, an existing directory must belong + * to this uid; then chmod + a second lstat make 0700 a verified postcondition + * rather than a best-effort creation hint. + */ +function ensurePrivateDirectorySync(target: string): void { + const parent = path.dirname(target); + let parentStat: Stats; + try { + parentStat = lstatSync(parent); + } catch (err) { + throw scratchAuthorityError( + parent, + `cannot inspect parent (${(err as NodeJS.ErrnoException).code ?? "unknown error"})` + ); + } + assertSafeParent(parent, parentStat); + + try { + mkdirSync(target, { mode: SCRATCH_DIR_MODE, recursive: false }); + } catch (err) { + if (!isErrno(err, "EEXIST")) throw err; + } + + let stat = lstatSync(target); + assertOwnedDirectory(target, stat); + if (currentUid() !== undefined) { + chmodSync(target, SCRATCH_DIR_MODE); + stat = lstatSync(target); + assertOwnedDirectory(target, stat); + verifyPrivateMode(target, stat); + } +} + +async function ensurePrivateDirectory(target: string): Promise { + const parent = path.dirname(target); + let parentStat: Stats; + try { + parentStat = await lstat(parent); + } catch (err) { + throw scratchAuthorityError( + parent, + `cannot inspect parent (${(err as NodeJS.ErrnoException).code ?? "unknown error"})` + ); + } + assertSafeParent(parent, parentStat); + + try { + await mkdir(target, { mode: SCRATCH_DIR_MODE, recursive: false }); + } catch (err) { + if (!isErrno(err, "EEXIST")) throw err; + } + + let stat = await lstat(target); + assertOwnedDirectory(target, stat); + if (currentUid() !== undefined) { + await chmod(target, SCRATCH_DIR_MODE); + stat = await lstat(target); + assertOwnedDirectory(target, stat); + verifyPrivateMode(target, stat); + } +} + +function realpathNearestSync(p: string): string { + let current = path.resolve(p); + const tail: string[] = []; + for (;;) { + try { + const real = realpathSync(current); + return tail.length ? path.join(real, ...tail.reverse()) : real; + } catch (err) { + if (!isErrno(err, "ENOENT") && !isErrno(err, "ENOTDIR")) { + throw err; + } + const parent = path.dirname(current); + if (parent === current) return path.resolve(p); + tail.push(path.basename(current)); + current = parent; + } + } +} + +function assertAuthorityOutsideSecretsSync( + base: string, + secretsRoot: string | undefined +): void { + if (!secretsRoot) return; + assertOutsideSecretsRoot(base, secretsRoot); + if (containsPath(path.resolve(base), path.resolve(secretsRoot))) { + throw new Error( + `scratch base must not contain the secret root (GRIDA-SEC-004): ${base}` + ); + } + const realBase = realpathNearestSync(base); + const realSecrets = realpathNearestSync(secretsRoot); + if ( + containsPath(realSecrets, realBase) || + containsPath(realBase, realSecrets) + ) { + throw new Error( + `scratch base physically overlaps the secret root (GRIDA-SEC-004): ${base}` + ); + } +} + +/** + * Synchronously establish the shared scratch authority, without deleting any + * session data. Native hosts may call this before touching another private + * child such as `/commands`. + * + * The base and `sessions` are both non-symlink, current-uid-owned 0700 + * directories on POSIX. A predictable path pre-created by another local uid, + * a symlink aimed elsewhere, or physical overlap with `secretsRoot` fails + * closed before any mode or content mutation. + */ +export function prepareScratchAuthority( + base: string, + secretsRoot?: string +): void { + const resolvedBase = resolveAuthorityBase(base); + assertAuthorityOutsideSecretsSync(resolvedBase, secretsRoot); + ensurePrivateDirectorySync(resolvedBase); + ensurePrivateDirectorySync(path.join(resolvedBase, SESSIONS_DIRNAME)); +} + /** * Default base directory for session scratch areas when the host injects none. - * `/grida-agent-`, where the tag is a short hash of the - * host's `userData` dir. Resolved at the host/server boundary (the thin adapter - * shell), so the runtime core never names a temp path itself — a future host - * with a different filesystem reality (a cloud sandbox) injects its own. + * `/grida-agent-`, where `tempRoot` defaults to + * `os.tmpdir()` and the tag is a short hash of the host's `userData` dir. + * Resolved at the host/server boundary (the thin adapter shell), so the runtime + * core never names a temp path itself — a host with a narrower filesystem + * reality may inject its own temp authority root. * * Namespaced PER HOST so two default-configured hosts on the same machine (e.g. * a desktop sidecar and a `cli serve`) don't share a base — otherwise one host's @@ -60,13 +293,16 @@ const SCRATCH_FILE_MODE = 0o600; * The tag is stable across restarts of the same host (same `userData`), so the * sweep still reclaims that host's prior-run scratch. */ -export function defaultScratchBase(userDataPath: string): string { +export function defaultScratchBase( + userDataPath: string, + tempRoot: string = os.tmpdir() +): string { const tag = crypto .createHash("sha256") .update(path.resolve(userDataPath)) .digest("hex") .slice(0, 16); - return path.join(os.tmpdir(), `${SCRATCH_NAMESPACE}-${tag}`); + return path.join(tempRoot, `${SCRATCH_NAMESPACE}-${tag}`); } /** @@ -126,7 +362,10 @@ async function realpathNearest(p: string): Promise { try { const real = await realpath(current); return tail.length ? path.join(real, ...tail.reverse()) : real; - } catch { + } catch (err) { + if (!isErrno(err, "ENOENT") && !isErrno(err, "ENOTDIR")) { + throw err; + } const parent = path.dirname(current); if (parent === current) return path.resolve(p); tail.push(path.basename(current)); @@ -135,10 +374,44 @@ async function realpathNearest(p: string): Promise { } } +function parseScratchRoot(scratchDir: string): { + base: string; + sessionDir: string; + scratchDir: string; +} { + if (!path.isAbsolute(scratchDir)) { + throw scratchAuthorityError( + scratchDir, + "the scratch root must be an absolute path" + ); + } + const resolvedScratch = path.resolve(scratchDir); + if (path.basename(resolvedScratch) !== SCRATCH_DIRNAME) { + throw scratchAuthorityError( + scratchDir, + `expected the final path segment to be ${SCRATCH_DIRNAME}` + ); + } + const sessionDir = path.dirname(resolvedScratch); + assertSafeSessionId(path.basename(sessionDir)); + const sessionsDir = path.dirname(sessionDir); + if (path.basename(sessionsDir) !== SESSIONS_DIRNAME) { + throw scratchAuthorityError( + scratchDir, + `expected the session parent to be ${SESSIONS_DIRNAME}` + ); + } + return { + base: resolveAuthorityBase(path.dirname(sessionsDir)), + sessionDir, + scratchDir: resolvedScratch, + }; +} + /** - * Create a scratch dir on demand (`mkdir -p`, owner-only). Idempotent. Takes the - * already-derived dir (from {@link scratchRootFor}) so the path isn't computed - * twice — the caller holds it for the agent binding anyway. + * Create a scratch dir on demand, owner-only. Idempotent. Takes the + * already-derived absolute dir (from {@link scratchRootFor}) so the path isn't + * computed twice — the caller holds it for the agent binding anyway. * * Containment (GRIDA-SEC-004) is checked in TWO layers: a cheap lexical * pre-check ({@link assertOutsideSecretsRoot}), then an AUTHORITATIVE physical @@ -152,8 +425,9 @@ export async function ensureScratch( secretsRoot?: string ): Promise { assertOutsideSecretsRoot(scratchDir, secretsRoot); + const authority = parseScratchRoot(scratchDir); if (secretsRoot) { - const realScratch = await realpathNearest(scratchDir); + const realScratch = await realpathNearest(authority.scratchDir); const realSecrets = await realpathNearest(secretsRoot); if (containsPath(realSecrets, realScratch)) { throw new Error( @@ -161,14 +435,9 @@ export async function ensureScratch( ); } } - await mkdir(scratchDir, { recursive: true, mode: SCRATCH_DIR_MODE }); - // `mkdir`'s mode only applies to dirs it CREATES — a pre-existing (possibly - // world-readable, attacker-pre-created) scratch or session dir keeps its mode. - // Force owner-only on both, and FAIL CLOSED: a dir we can't restrict (e.g. one - // we don't own → EPERM) must throw rather than silently serve artifacts to - // other local accounts. The session dir is `path.dirname(scratchDir)`. - await chmod(scratchDir, SCRATCH_DIR_MODE); - await chmod(path.dirname(scratchDir), SCRATCH_DIR_MODE); + prepareScratchAuthority(authority.base, secretsRoot); + await ensurePrivateDirectory(authority.sessionDir); + await ensurePrivateDirectory(authority.scratchDir); } /** @@ -206,22 +475,30 @@ function assertSafeFilename(filename: string): void { * outside the session tree — a TOCTOU that the lexical checks above can't catch * (#920 review). `O_NOFOLLOW` is POSIX-only; on Windows it is absent (the `?? 0` * fallback), where scratch's owner-only model is already a no-op. + * + * Generated artifacts keep the default overwrite behavior. Caller-supplied + * turn seeds pass `{ overwrite: false }`, which adds `O_EXCL`: a replay or + * colliding upload then fails before it can truncate a path already correlated + * with durable history. */ export async function writeScratchFile( scratchDir: string, filename: string, - bytes: Uint8Array + bytes: Uint8Array, + opts: { overwrite?: boolean } = {} ): Promise { assertSafeFilename(filename); const full = path.join(scratchDir, filename); if (path.dirname(path.resolve(full)) !== path.resolve(scratchDir)) { throw new Error(`scratch filename escapes the scratch dir: ${filename}`); } + const collisionFlag = + opts.overwrite === false ? fsConstants.O_EXCL : fsConstants.O_TRUNC; const handle = await open( full, fsConstants.O_WRONLY | fsConstants.O_CREAT | - fsConstants.O_TRUNC | + collisionFlag | (fsConstants.O_NOFOLLOW ?? 0), SCRATCH_FILE_MODE ); @@ -233,6 +510,64 @@ export async function writeScratchFile( return full; } +/** + * Snapshot the currently live flat scratch paths for model-view liveness. + * Direct regular files only: directories and symlinks are not operable + * attachment bodies. Missing or unreadable scratch fails closed to an empty + * set, because persisted descriptors are facts about a prior turn, not proof + * that ephemeral bytes survived. + */ +export async function listScratchFilePaths( + scratchDir: string +): Promise> { + try { + const entries = await readdir(scratchDir, { withFileTypes: true }); + return new Set( + entries.filter((entry) => entry.isFile()).map((entry) => entry.name) + ); + } catch (err) { + const code = (err as NodeJS.ErrnoException).code; + if (code !== "ENOENT") { + console.warn( + `[agent] scratch listing failed (${code ?? "unknown filesystem error"})` + ); + } + return new Set(); + } +} + +async function removeAuthorityEntry(target: string): Promise { + let stat: Stats; + try { + stat = await lstat(target); + } catch (err) { + if (isErrno(err, "ENOENT")) return; + throw err; + } + if (stat.isSymbolicLink() || !stat.isDirectory()) { + await unlink(target); + return; + } + assertOwnedDirectory(target, stat); + await rm(target, { recursive: true, force: true }); +} + +function removeAuthorityEntrySync(target: string): void { + let stat: Stats; + try { + stat = lstatSync(target); + } catch (err) { + if (isErrno(err, "ENOENT")) return; + throw err; + } + if (stat.isSymbolicLink() || !stat.isDirectory()) { + unlinkSync(target); + return; + } + assertOwnedDirectory(target, stat); + rmSync(target, { recursive: true, force: true }); +} + /** * Remove a session's scratch subtree (the whole `/sessions/`, so no * empty session dir lingers). Recursive and idempotent — removing a session that @@ -241,13 +576,15 @@ export async function writeScratchFile( */ export async function removeScratch( base: string, - sessionId: string + sessionId: string, + secretsRoot?: string ): Promise { assertSafeSessionId(sessionId); - await rm(path.join(base, SESSIONS_DIRNAME, sessionId), { - recursive: true, - force: true, - }); + const resolvedBase = resolveAuthorityBase(base); + prepareScratchAuthority(resolvedBase, secretsRoot); + await removeAuthorityEntry( + path.join(resolvedBase, SESSIONS_DIRNAME, sessionId) + ); } /** @@ -256,23 +593,28 @@ export async function removeScratch( * runs, so a freshly resumed session's `ensureScratch` can't race a still-running * async sweep that would delete the dir underneath it. A single-instance daemon's * prior in-flight scratch is dead after a restart, so this bounds scratch's - * lifetime even across a crash (S2). Best-effort — a missing base is a no-op; an - * unreadable entry is skipped. + * lifetime even across a crash (S2). + * + * The authority is established BEFORE listing or deletion. A symlinked base or + * `sessions` root therefore fails closed without touching its target. Child + * symlink entries are unlinked directly, never passed to recursive removal. + * Once that shared authority is validated, one stale entry that cannot be + * removed is logged and left in place without preventing reclamation of the + * remaining independent session entries. */ -export function sweepScratch(base: string): void { - const sessionsDir = path.join(base, SESSIONS_DIRNAME); - let entries: string[]; - try { - entries = readdirSync(sessionsDir); - } catch { - // No base yet (fresh host) — nothing to reclaim. - return; - } +export function sweepScratch(base: string, secretsRoot?: string): void { + const resolvedBase = resolveAuthorityBase(base); + prepareScratchAuthority(resolvedBase, secretsRoot); + const sessionsDir = path.join(resolvedBase, SESSIONS_DIRNAME); + const entries = readdirSync(sessionsDir); for (const name of entries) { try { - rmSync(path.join(sessionsDir, name), { recursive: true, force: true }); - } catch { - // Skip an entry we can't remove; the next sweep retries. + removeAuthorityEntrySync(path.join(sessionsDir, name)); + } catch (err) { + const code = (err as NodeJS.ErrnoException).code; + console.warn( + `[agent] scratch sweep failed for ${JSON.stringify(name)} (${code ?? "unknown filesystem error"}); continuing` + ); } } } diff --git a/packages/grida-ai-agent/src/session/store.ts b/packages/grida-ai-agent/src/session/store.ts index 0dde70fe9..8c02c26e1 100644 --- a/packages/grida-ai-agent/src/session/store.ts +++ b/packages/grida-ai-agent/src/session/store.ts @@ -151,6 +151,15 @@ export class SessionsStore { this.db = opened.db; } + /** + * Run a short store mutation as one SQLite transaction. The opened DB's + * re-entrant owner gate lets ordinary store methods be called inside `fn` + * without interleaving another operation on the shared connection. + */ + async withTransaction(fn: () => Promise): Promise { + return await this.opened.withTx(fn); + } + // ──────────────────────────── sessions ──────────────────────────── async create(input: CreateSessionInput): Promise { diff --git a/packages/grida-ai-agent/src/tools/index.ts b/packages/grida-ai-agent/src/tools/index.ts index c38a06c71..d7d611901 100644 --- a/packages/grida-ai-agent/src/tools/index.ts +++ b/packages/grida-ai-agent/src/tools/index.ts @@ -56,6 +56,7 @@ import { } from "./names"; import { createRunCommandTool, + type RunCommandApprovalInput, type RunCommandBackend, type RunCommandOutcome, } from "./run-command"; @@ -71,7 +72,12 @@ export { SURFACE_LIST_OPEN_TOOL_NAME, SKILL_TOOL_NAME, }; -export type { AgentToolName, RunCommandBackend, RunCommandOutcome }; +export type { + AgentToolName, + RunCommandApprovalInput, + RunCommandBackend, + RunCommandOutcome, +}; export type ToolsetCapabilities = { /** Server-side AgentFs binding. When provided, fs tools get @@ -111,7 +117,7 @@ export type ToolsetCapabilities = { * straight to `createRunCommandTool` → the tool's `needsApproval`. The * host computes it from the session mode + `isReadOnlyCommand`; absent in * `auto` (every command auto-runs). */ - needs_approval?: (input: { command: string; args: string[] }) => boolean; + needs_approval?: (input: RunCommandApprovalInput) => boolean; }; /** Inject the discovered skill index. When provided, the locked `skill` * tool joins the registry, letting the model load any advertised skill diff --git a/packages/grida-ai-agent/src/tools/run-command.test.ts b/packages/grida-ai-agent/src/tools/run-command.test.ts new file mode 100644 index 000000000..3b5f1b7fe --- /dev/null +++ b/packages/grida-ai-agent/src/tools/run-command.test.ts @@ -0,0 +1,44 @@ +import { describe, expect, it, vi } from "vitest"; +import { createRunCommandTool, type RunCommandBackend } from "./run-command"; + +describe("createRunCommandTool", () => { + it("forwards the AI SDK turn abort signal to the command backend", async () => { + const backend = vi.fn(async () => ({ + stdout: "", + stderr: "", + exit_code: 0, + signal: null, + timed_out: false, + truncated: false, + })); + const command = createRunCommandTool({ + backend, + default_workdir: "/workspace", + }); + const controller = new AbortController(); + + await command.execute!( + { + command: "pwd", + args: [], + description: "print the working directory", + }, + { + toolCallId: "tool-call-1", + messages: [], + abortSignal: controller.signal, + } + ); + + expect(backend).toHaveBeenCalledWith( + { + command: "pwd", + args: [], + workdir: "/workspace", + timeout_ms: undefined, + description: "print the working directory", + }, + controller.signal + ); + }); +}); diff --git a/packages/grida-ai-agent/src/tools/run-command.ts b/packages/grida-ai-agent/src/tools/run-command.ts index 16b018966..f7f7ff22b 100644 --- a/packages/grida-ai-agent/src/tools/run-command.ts +++ b/packages/grida-ai-agent/src/tools/run-command.ts @@ -20,18 +20,18 @@ * allowlist; the OS sandbox is the structural boundary. * * GRIDA-SEC-004 — this tool owns the supervised-approval gate. The - * `needsApproval` predicate below is what PAUSES a mutating command for an - * Allow/Deny in `accept-edits` (and is absent in `auto`). The gate lives on + * `needsApproval` predicate below is what PAUSES a host-classified command for + * an Allow/Deny in `accept-edits` (and is absent in `auto`). The gate lives on * the tool, NOT the backend: by the time the backend's `execute` runs, the - * call is already cleared (auto, or user-approved), so the backend cannot - * re-gate on mode. The mode→predicate wiring is at - * `workspace-agent-bindings.ts`; the read-only classification is - * `permissions.ts` `isReadOnlyCommand`; the server-authoritative answer is - * atomically bound to its consuming run by + * call is already cleared (auto, host-pre-authorized, or user-approved), so + * the backend cannot re-gate on mode. The mode→predicate wiring is at + * `workspace-agent-bindings.ts`; it combines the read-only classification + * with the narrow scratch-local copy/move exception. The server-authoritative + * answer is atomically bound to its consuming run by * `store.commitApprovalContinuation`. See SECURITY.md. */ -import { tool } from "ai"; +import { tool, type ToolExecutionOptions } from "ai"; import { z } from "zod"; import { RUN_COMMAND_TOOL_NAME } from "./names"; @@ -41,7 +41,7 @@ export { RUN_COMMAND_TOOL_NAME }; export type RunCommandToolName = typeof RUN_COMMAND_TOOL_NAME; /** Shape the agent expects from the injected command backend. The - * caller's job is to apply its allowlist, resolve the workdir, spawn the + * caller's job is to validate and confine the workdir/scope, spawn the * process, and aggregate the result. The agent doesn't know or care * how that happens. */ export type RunCommandResult = { @@ -69,13 +69,25 @@ export type RunCommandFailure = { export type RunCommandOutcome = RunCommandResult | RunCommandFailure; -export type RunCommandBackend = (input: { +export type RunCommandApprovalInput = { command: string; args: string[]; + /** Effective workdir after applying the tool's default. Approval policy + * needs the resolved value to distinguish a scratch-local operation from a + * workspace mutation. */ workdir: string; - timeout_ms?: number; - description: string; -}) => Promise; +}; + +export type RunCommandBackend = ( + input: { + command: string; + args: string[]; + workdir: string; + timeout_ms?: number; + description: string; + }, + signal?: ToolExecutionOptions["abortSignal"] +) => Promise; /** * Build the command tool bound to a specific backend + default workdir. @@ -90,21 +102,21 @@ export function createRunCommandTool(opts: { * Supervised-approval gate (RFC `permission modes`, Phase 2). When this * returns true for a given call, the AI SDK emits a `tool-approval-request` * and PAUSES — `execute` does not run until the user approves (Allow). In - * `accept-edits` the host wires this to "true unless the command is - * read-only"; in `auto` it's absent (every command auto-runs). The decision - * lives here, NOT in the backend, because the backend's `execute` can't tell - * an approved call from an un-approved one — by the time `execute` runs, the - * SDK has already cleared the call (auto, or user-approved). + * `accept-edits` the host wires its read-only and narrowly pre-authorized + * scratch-operation policy here; in `auto` it's absent (every command + * auto-runs). The decision lives here, NOT in the backend, because the + * backend's `execute` can't tell an approved call from an un-approved one — + * by the time `execute` runs, the SDK has already cleared the call. */ - needs_approval?: (input: { command: string; args: string[] }) => boolean; + needs_approval?: (input: RunCommandApprovalInput) => boolean; }) { const policy = opts.policy_description ?? - "The host backend is responsible for command allowlisting, workdir " + - "validation, timeout caps, and process isolation."; + "The host backend is responsible for workdir validation, scope " + + "confinement, timeout caps, and process isolation."; return tool({ description: - "Run a host-approved command in the workspace. This directly " + + "Run a host-approved command in an authorized working directory. This directly " + "spawns an executable with argv arguments; it is not a shell, " + "so pipes, redirects, glob expansion, env assignment, and `&&` " + `are not interpreted. ${policy} ` + @@ -114,9 +126,7 @@ export function createRunCommandTool(opts: { command: z .string() .min(1) - .describe( - "Bare executable name. Must be accepted by the host backend." - ), + .describe("Bare executable name resolved by the host backend."), args: z .array(z.string()) .default([]) @@ -126,7 +136,7 @@ export function createRunCommandTool(opts: { .optional() .describe( "Optional absolute path. Defaults to the workspace root. " + - "Must resolve inside the workspace." + "Must resolve inside a host-authorized root." ), timeout_ms: z .number() @@ -162,22 +172,23 @@ export function createRunCommandTool(opts: { opts.needs_approval!({ command: input.command, args: input.args ?? [], + workdir: input.workdir ?? opts.default_workdir, }) : false, - execute: async ({ - command, - args, - workdir, - timeout_ms: timeoutMs, - description, - }) => { - return await opts.backend({ - command, - args: args ?? [], - workdir: workdir ?? opts.default_workdir, - timeout_ms: timeoutMs, - description, - }); + execute: async ( + { command, args, workdir, timeout_ms: timeoutMs, description }, + { abortSignal } + ) => { + return await opts.backend( + { + command, + args: args ?? [], + workdir: workdir ?? opts.default_workdir, + timeout_ms: timeoutMs, + description, + }, + abortSignal + ); }, }); } diff --git a/packages/grida-daemon/src/__public-api__.test.ts b/packages/grida-daemon/src/__public-api__.test.ts index 51de3bd66..f25fbd273 100644 --- a/packages/grida-daemon/src/__public-api__.test.ts +++ b/packages/grida-daemon/src/__public-api__.test.ts @@ -30,7 +30,9 @@ import { containsPath, Daemon, DaemonServer, + runUnsandboxedShell, SecretsStore, + validateShellRequest, WorkspaceRegistry, workspaceFs, type BuiltServer, @@ -39,6 +41,9 @@ import { type DaemonServices, type DaemonTenant, type DaemonTenantHandle, + type ShellExecutionScope, + type ShellExecutor, + type ShellRunOptions, } from "./server"; import { buildDaemonSandboxPolicy, @@ -147,6 +152,16 @@ describe("@grida/daemon public API", () => { expect(typeof workspaceFs.readDir).toBe("function"); expect(typeof workspaceFs.iterateDir).toBe("function"); expect(containsPath("/a", "/a/b")).toBe(true); + const executor: ShellExecutor = runUnsandboxedShell; + const scope: ShellExecutionScope = { + workspace_root: "/workspace", + protected_read_roots: [], + }; + const runOptions: ShellRunOptions = {}; + expect(typeof executor).toBe("function"); + expect(typeof validateShellRequest).toBe("function"); + expect(scope.workspace_root).toBe("/workspace"); + expect(runOptions).toEqual({}); }); }); diff --git a/packages/grida-daemon/src/server.ts b/packages/grida-daemon/src/server.ts index f17be2f55..c22938910 100644 --- a/packages/grida-daemon/src/server.ts +++ b/packages/grida-daemon/src/server.ts @@ -62,10 +62,14 @@ export { } from "./auth/file"; export { runShell, + runUnsandboxedShell, validateShellRequest, - type AdditionalAllowedRoots, + type AllowedCwdRoots, type ProtectedReadRoots, + type ShellExecutionScope, + type ShellExecutor, type ShellRunError, + type ShellRunOptions, type ShellRunRequest, type ShellRunResult, } from "./shell/runner"; diff --git a/packages/grida-daemon/src/shell/runner.test.ts b/packages/grida-daemon/src/shell/runner.test.ts index 8269ddcb5..6e3566c02 100644 --- a/packages/grida-daemon/src/shell/runner.test.ts +++ b/packages/grida-daemon/src/shell/runner.test.ts @@ -10,12 +10,10 @@ * * Two layers, increasing scope: * - * 1. `validateShellRequest` — touches real fs (`realpath`, `stat`) - * and a real `WorkspaceRegistry` pointed at a temp userData - * dir. Covers the structural gates the agent server route runs before - * spawning: cwd-resolve, cwd-is-directory, cwd-in-workspace, and the - * secret-arg containment check. (Command identity is gated upstream by - * mode — see `permissions.test.ts`.) + * 1. `validateShellRequest` — touches real fs (`realpath`, `stat`). Covers + * cwd resolution, exact granted-root containment, and the secret-arg + * check. (Command identity is gated upstream by mode — see + * `permissions.test.ts`.) * 2. `runShell` — actually spawns child processes via * `child_process.spawn`. Uses `echo`, `pwd`, `ls`, `sleep` from * the host PATH; the runner uses `shell: false` so these @@ -23,43 +21,40 @@ * is the same code path the agent host takes in production. * * All cases are deterministic: fixed inputs, fixed expected outputs. - * The temp dir lives under `os.tmpdir()` so the registry's git-root - * walk doesn't pick up the grida repo root by accident — `os.tmpdir()` - * resolves to `/private/var/folders/…` on macOS and `/tmp` on Linux, - * neither of which is inside a git tree. + * The temp dir lives under `os.tmpdir()` so tests do not touch the checkout. */ /* eslint-disable jest/no-conditional-expect */ -import { afterEach, beforeEach, describe, expect, it } from "vitest"; +import { afterEach, beforeEach, describe, expect, it, vi } from "vitest"; import fs from "node:fs/promises"; import os from "node:os"; import path from "node:path"; import { runShell, validateShellRequest } from "./runner"; -import { WorkspaceRegistry } from "../workspaces"; /* ────────────────────── validate + runShell scaffold ───────────── */ /** - * Per-test fixture: a temp dir with a real `WorkspaceRegistry` that - * has registered a child of the temp dir as its single workspace. - * `workspaceRoot` is already `realpath`'d so tests can compare - * against it without doing the macOS `/var` → `/private/var` dance. + * Per-test fixture with two sibling workspace-shaped roots. The validator is + * handed exactly one; the other proves global/open-workspace authority cannot + * bleed into the current command. Roots are already `realpath`'d so tests can + * compare without doing the macOS `/var` → `/private/var` dance. */ async function makeFixture(): Promise<{ workspace_root: string; - registry: WorkspaceRegistry; + other_workspace_root: string; cleanup: () => Promise; }> { const base = await fs.mkdtemp(path.join(os.tmpdir(), "grida-shell-test-")); const workspaceDir = path.join(base, "workspace"); + const otherWorkspaceDir = path.join(base, "other-workspace"); const userDataDir = path.join(base, "userdata"); await fs.mkdir(workspaceDir); + await fs.mkdir(otherWorkspaceDir); await fs.mkdir(userDataDir); const workspaceRoot = await fs.realpath(workspaceDir); - const registry = new WorkspaceRegistry(userDataDir); - await registry.open(workspaceRoot); + const otherWorkspaceRoot = await fs.realpath(otherWorkspaceDir); return { workspace_root: workspaceRoot, - registry, + other_workspace_root: otherWorkspaceRoot, cleanup: async () => { await fs.rm(base, { recursive: true, force: true }); }, @@ -77,10 +72,10 @@ describe("validateShellRequest", () => { await fixture.cleanup(); }); - it("accepts an allowlisted cmd with an in-workspace cwd", async () => { + it("accepts a request with a cwd inside the exact workspace root", async () => { const result = await validateShellRequest( { cmd: "echo", args: ["hi"], cwd: fixture.workspace_root }, - fixture.registry + [fixture.workspace_root] ); expect(result.ok).toBe(true); if (result.ok) { @@ -89,13 +84,23 @@ describe("validateShellRequest", () => { } }); - it("rejects cwd outside any registered workspace", async () => { - // The OS tmpdir itself is the *parent* of our registered - // workspace; `containsPath` does a prefix-with-separator check, - // so the parent is correctly rejected. + it("rejects cwd outside the exact granted root", async () => { + // The OS tmpdir itself is the *parent* of our workspace; `containsPath` + // does a prefix-with-separator check, so the parent is correctly rejected. const result = await validateShellRequest( { cmd: "echo", args: [], cwd: os.tmpdir() }, - fixture.registry + [fixture.workspace_root] + ); + expect(result.ok).toBe(false); + if (!result.ok) { + expect(result.error.code).toBe("cwd-not-in-workspace"); + } + }); + + it("rejects a sibling workspace when only the current workspace is granted", async () => { + const result = await validateShellRequest( + { cmd: "echo", args: [], cwd: fixture.other_workspace_root }, + [fixture.workspace_root] ); expect(result.ok).toBe(false); if (!result.ok) { @@ -110,7 +115,7 @@ describe("validateShellRequest", () => { args: [], cwd: path.join(fixture.workspace_root, "no-such-dir"), }, - fixture.registry + [fixture.workspace_root] ); expect(result.ok).toBe(false); if (!result.ok) { @@ -123,7 +128,7 @@ describe("validateShellRequest", () => { await fs.writeFile(filePath, "x"); const result = await validateShellRequest( { cmd: "echo", args: [], cwd: filePath }, - fixture.registry + [fixture.workspace_root] ); expect(result.ok).toBe(false); if (!result.ok) { @@ -162,7 +167,7 @@ describe("validateShellRequest — protected secret roots", () => { it("rejects an absolute arg inside the secrets root", async () => { const result = await validateShellRequest( { cmd: "cat", args: [authJsonAbs], cwd: fixture.workspace_root }, - fixture.registry, + [fixture.workspace_root], [secretsRoot] ); expect(result.ok).toBe(false); @@ -180,7 +185,7 @@ describe("validateShellRequest — protected secret roots", () => { args: ["../userdata/auth.json"], cwd: fixture.workspace_root, }, - fixture.registry, + [fixture.workspace_root], [secretsRoot] ); expect(result.ok).toBe(false); @@ -193,7 +198,7 @@ describe("validateShellRequest — protected secret roots", () => { await fs.writeFile(path.join(fixture.workspace_root, "file.txt"), "ok"); const result = await validateShellRequest( { cmd: "cat", args: ["./file.txt"], cwd: fixture.workspace_root }, - fixture.registry, + [fixture.workspace_root], [secretsRoot] ); expect(result.ok).toBe(true); @@ -202,23 +207,22 @@ describe("validateShellRequest — protected secret roots", () => { it("does not treat flags or plain text as paths", async () => { const result = await validateShellRequest( { cmd: "grep", args: ["-n", "needle"], cwd: fixture.workspace_root }, - fixture.registry, + [fixture.workspace_root], [secretsRoot] ); expect(result.ok).toBe(true); }); }); -/* ───────────── additional allowed roots (session scratch) ──────── */ +/* ───────────────── exact roots (session scratch) ───────────────── */ /** * WG `scratch.md` S4: the session scratch dir is a sanctioned cwd root even - * though it is NOT a registered workspace (S5). It is supplied via - * `additionalAllowedRoots`; without it, a cwd in scratch is rejected exactly - * like any other out-of-workspace path. The scratch dir lives in its own temp - * dir here (outside both the workspace and the secrets root). + * though it is NOT a workspace (S5). It is one of this session's exact roots; + * without it, a cwd in scratch is rejected like any other ungranted path. The + * scratch dir lives outside both the workspace and the secrets root. */ -describe("validateShellRequest — additional allowed roots (scratch)", () => { +describe("validateShellRequest — exact scratch root", () => { let fixture: Awaited>; let scratchRoot: string; beforeEach(async () => { @@ -235,9 +239,7 @@ describe("validateShellRequest — additional allowed roots (scratch)", () => { it("cwd inside scratch passes when scratch is an allowed root (S4)", async () => { const result = await validateShellRequest( { cmd: "ls", args: [], cwd: scratchRoot }, - fixture.registry, - [], - [scratchRoot] + [fixture.workspace_root, scratchRoot] ); expect(result.ok).toBe(true); if (result.ok) expect(result.request.cwd).toBe(scratchRoot); @@ -246,7 +248,7 @@ describe("validateShellRequest — additional allowed roots (scratch)", () => { it("cwd inside scratch fails when scratch is not an allowed root", async () => { const result = await validateShellRequest( { cmd: "ls", args: [], cwd: scratchRoot }, - fixture.registry + [fixture.workspace_root] ); expect(result.ok).toBe(false); if (!result.ok) expect(result.error.code).toBe("cwd-not-in-workspace"); @@ -263,9 +265,8 @@ describe("validateShellRequest — additional allowed roots (scratch)", () => { args: ["a.zip", "-d", path.join(scratchRoot, "out")], cwd: fixture.workspace_root, }, - fixture.registry, - [secretsRoot], - [scratchRoot] + [fixture.workspace_root, scratchRoot], + [secretsRoot] ); expect(result.ok).toBe(true); }); @@ -279,6 +280,7 @@ describe("runShell", () => { fixture = await makeFixture(); }); afterEach(async () => { + vi.restoreAllMocks(); await fixture.cleanup(); }); @@ -354,4 +356,93 @@ describe("runShell", () => { expect(result.signal).not.toBeNull(); expect(result.duration_ms).toBeLessThan(2000); }); + + it("kills the child when its host generation is aborted", async () => { + const controller = new AbortController(); + const pending = runShell( + { + cmd: "sleep", + args: ["5"], + cwd: fixture.workspace_root, + }, + { signal: controller.signal } + ); + controller.abort(); + + const result = await pending; + expect(result.timed_out).toBe(false); + expect(result.signal).not.toBeNull(); + expect(result.duration_ms).toBeLessThan(2000); + }); + + it("kills background descendants before releasing command authority", async () => { + if (process.platform === "win32") return; + const lateWrite = path.join(fixture.workspace_root, "late-write.txt"); + const result = await runShell({ + cmd: "/bin/sh", + args: ["-c", `(sleep 0.4; echo escaped > '${lateWrite}') &`], + cwd: fixture.workspace_root, + }); + + expect(result.exit_code).toBe(0); + await new Promise((resolve) => setTimeout(resolve, 650)); + await expect(fs.stat(lateWrite)).rejects.toThrow(/ENOENT/); + }); + + it("excludes process-group teardown polling from command duration", async () => { + if (process.platform === "win32") return; + const processKill = process.kill.bind(process); + let existenceProbes = 0; + vi.spyOn(process, "kill").mockImplementation((pid, signal) => { + if (pid >= 0) return processKill(pid, signal); + if (signal !== 0) return true; + existenceProbes += 1; + if (existenceProbes <= 20) return true; + throw Object.assign(new Error("no such process"), { code: "ESRCH" }); + }); + + const wallStartedAt = Date.now(); + const result = await runShell({ + cmd: "echo", + args: ["done"], + cwd: fixture.workspace_root, + }); + const wallDurationMs = Date.now() - wallStartedAt; + + expect(existenceProbes).toBe(21); + expect(wallDurationMs - result.duration_ms).toBeGreaterThanOrEqual(100); + }); + + it("escalates immediately when process-group exit cannot be observed", async () => { + if (process.platform === "win32") return; + const processKill = process.kill.bind(process); + const groupSignals: Array<{ + signal: Parameters[1]; + at: number; + }> = []; + vi.spyOn(process, "kill").mockImplementation((pid, signal) => { + if (pid >= 0) return processKill(pid, signal); + groupSignals.push({ signal, at: Date.now() }); + if (signal === 0) { + throw Object.assign(new Error("operation not permitted"), { + code: "EPERM", + }); + } + return true; + }); + + await runShell({ + cmd: "echo", + args: ["done"], + cwd: fixture.workspace_root, + }); + + expect(groupSignals.map(({ signal }) => signal)).toEqual([ + "SIGTERM", + 0, + "SIGKILL", + 0, + ]); + expect(groupSignals[2]!.at - groupSignals[0]!.at).toBeLessThan(100); + }); }); diff --git a/packages/grida-daemon/src/shell/runner.ts b/packages/grida-daemon/src/shell/runner.ts index 147d52307..2c69d92ff 100644 --- a/packages/grida-daemon/src/shell/runner.ts +++ b/packages/grida-daemon/src/shell/runner.ts @@ -15,28 +15,25 @@ * gone — the OS sandbox (srt) is the structural boundary, and the supervised * mode's read-only-vs-mutating gate lives upstream in the command backend * (`runtime/command-backend.ts`, via `permissions.ts`). What remains here are - * the two STRUCTURAL gates that hold in every mode: + * the request-validation gates that hold in every mode: * * 1. cwd must be `realpath`-resolvable AND contained by a - * currently-registered workspace OR by one of the caller-supplied - * `additionalAllowedRoots` (the session's scratch dir — a sanctioned - * ephemeral working area outside the workspace; see WG `scratch.md`). - * Without an opened workspace and no allowed root, the call fails. + * caller-supplied exact allowed root. The agent tenant supplies only the + * current session's workspace and scratch roots; a global workspace + * registry is deliberately not accepted here. * 2. No arg may resolve to a path inside a protected-secret root * (the agent host's own `userData`, where BYOK `auth.json` and * the sessions db live). The srt outer policy can't deny that * root — the host process itself reads it for provider auth — so * this in-process arg check keeps `cat ${userData}/auth.json` - * from leaking the key to the shell child. See `sandbox/policy.ts`. - * (A kernel-level per-call deny is the planned hardening; until then - * this is the load-bearing guard for the host's own key — though an - * interpreter in `auto` can read by a computed path, which is why the - * per-call sub-policy is the real fix.) + * from leaking the key through a direct arg. A confined + * {@link ShellExecutor} also receives the protected roots and must deny + * them at the finite child-process boundary. See `sandbox/policy.ts`. * - * GRIDA-SEC-004: gate 2 is NOT general arg containment (deferred to the - * srt per-cmd sub-policy). It denies exactly the secret root, nothing - * more — so an arg's "is this a path?" guess only ever costs the secret - * dir, never a false rejection of normal in-workspace work. + * GRIDA-SEC-004: gate 2 is defense-in-depth, NOT general arg containment. It + * denies exactly the secret root, nothing more — so an arg's "is this a path?" + * guess only ever costs the secret dir, never a false rejection of normal + * in-workspace work. Computed paths are contained by the host executor. * * Timeout: 30s hard cap. Long-running processes are killed (SIGKILL * after a SIGTERM grace). The route returns `exitCode: null, @@ -50,12 +47,12 @@ import { spawn } from "node:child_process"; import fs from "node:fs/promises"; import path from "node:path"; -import type { WorkspaceRegistry } from "../workspaces"; import { containsPath } from "../path-contains"; const DEFAULT_TIMEOUT_MS = 30_000; const MAX_BUFFER_BYTES = 1 * 1024 * 1024; // 1 MiB combined const SIGTERM_GRACE_MS = 250; +const PROCESS_GROUP_POLL_MS = 10; export type ShellRunRequest = { cmd: string; @@ -78,6 +75,37 @@ export type ShellRunResult = { truncated: boolean; }; +export type ShellRunOptions = Readonly<{ + /** Host-lifecycle cancellation (for example, a retired sidecar generation). */ + signal?: AbortSignal; +}>; + +/** + * Immutable authority the host must enforce around one finite command. + * + * The tenant derives this from server-owned session state. A confined host + * executor must independently bind and validate the roots before spawning; + * the in-process cwd/arg validation below is defense-in-depth, not the + * filesystem sandbox. + */ +export type ShellExecutionScope = Readonly<{ + workspace_root: string; + scratch_root?: string; + scratch_base?: string; + protected_read_roots: ProtectedReadRoots; +}>; + +/** + * Host-owned finite-command capability. Desktop implementations confine each + * invocation at the OS boundary; deliberately unsandboxed hosts may inject + * {@link runUnsandboxedShell} explicitly. + */ +export type ShellExecutor = ( + request: ShellRunRequest, + scope: ShellExecutionScope, + signal?: AbortSignal +) => Promise; + export type ShellRunError = | { code: "cwd-not-in-workspace"; cwd: string } | { code: "cwd-not-a-directory"; cwd: string } @@ -93,30 +121,26 @@ export type ShellRunError = export type ProtectedReadRoots = readonly string[]; /** - * Absolute roots — beyond the registered workspaces — a cwd is allowed to sit - * inside. The session's scratch dir (WG `scratch.md`): a sanctioned ephemeral - * working area the agent may `cd`/write into without it being a workspace. + * Exact absolute roots a cwd may sit inside. For a workspace-bound agent this + * is the current session's workspace plus, when present, its own scratch root. * Realpath'd here for symlink stability, same as the protected roots. */ -export type AdditionalAllowedRoots = readonly string[]; +export type AllowedCwdRoots = readonly string[]; /** - * Validates a shell-run request against the workspace registry and the - * protected-secret roots (the two structural gates; command identity is gated - * upstream by mode — see the module header). Returns either `{ok, request}` - * with the cwd-`realpath`'d request, or `{ok:false, error}` with a structured - * error the route handler can return as 400/403. + * Validates a shell-run request against exact allowed cwd roots and the + * protected-secret roots. Command identity is gated upstream by mode — see the + * module header. Returns either `{ok, request}` with the cwd-`realpath`'d + * request, or `{ok:false, error}` with a structured error. * * `protectedReadRoots` are absolute secret roots (the agent host's * `userData`) the shell child must not read through any arg — see the * module header's gate (2). Omit for the no-bindings path. - * `additionalAllowedRoots` are extra cwd roots (the session scratch dir). */ export async function validateShellRequest( req: ShellRunRequest, - registry: WorkspaceRegistry, - protectedReadRoots: ProtectedReadRoots = [], - additionalAllowedRoots: AdditionalAllowedRoots = [] + allowedCwdRoots: AllowedCwdRoots, + protectedReadRoots: ProtectedReadRoots = [] ): Promise< { ok: true; request: ShellRunRequest } | { ok: false; error: ShellRunError } > { @@ -149,22 +173,15 @@ export async function validateShellRequest( if (!stat.isDirectory()) { return { ok: false, error: { code: "cwd-not-a-directory", cwd: req.cwd } }; } - // Force the registry to be loaded — the route handler calls - // `registry.list()` upstream so this is usually warm, but the sync - // `containsPath` requires it. - await registry.list(); - // cwd is allowed inside a registered workspace OR a caller-supplied extra - // root (the session scratch dir). Scratch is NOT a workspace (WG `scratch.md` - // S5), so it goes through this separate allowance — keeping the workspace - // registry and its semantics untouched. - const inWorkspace = registry.containsPath(realCwd); + // Bind cwd to THIS command grant, never the process-global registry. A + // daemon can have many opened workspaces and many session scratch roots; the + // current command receives exactly one workspace and at most one scratch. const inAllowedRoot = - !inWorkspace && - additionalAllowedRoots.length > 0 && - (await realpathRoots(additionalAllowedRoots)).some((r) => - containsPath(r, realCwd) + allowedCwdRoots.length > 0 && + (await realpathRoots(allowedCwdRoots)).some((root) => + containsPath(root, realCwd) ); - if (!inWorkspace && !inAllowedRoot) { + if (!inAllowedRoot) { return { ok: false, error: { code: "cwd-not-in-workspace", cwd: realCwd }, @@ -255,17 +272,22 @@ async function realpathNearest(abs: string): Promise { } /** - * Runs the validated request. The caller is expected to have run - * `validateShellRequest` first; this function does NOT re-validate - * (passing an unvalidated request is a programming error). + * Low-level raw process runner. It does NOT validate or confine the request: + * callers either run `validateShellRequest` first under an explicit + * unsandboxed posture, or pass an already sandbox-wrapped command from a + * trusted host executor. * * Always resolves — never rejects on child failure. Non-zero exit, * signal kill, and timeout are all expressed in the returned * `ShellRunResult`. */ -export async function runShell(req: ShellRunRequest): Promise { +export async function runShell( + req: ShellRunRequest, + options: ShellRunOptions = {} +): Promise { const timeoutMs = Math.min(req.timeout_ms ?? DEFAULT_TIMEOUT_MS, 60_000); const startedAt = Date.now(); + const ownsProcessGroup = process.platform !== "win32"; return await new Promise((resolve) => { // `shell: false` is critical — args are passed straight to the @@ -275,6 +297,12 @@ export async function runShell(req: ShellRunRequest): Promise { const child = spawn(req.cmd, req.args, { cwd: req.cwd, shell: false, + // A finite command owns a fresh POSIX process group. Timeout, host abort, + // and even a normally exiting wrapper terminate the whole group before + // the caller releases command-scoped filesystem authority. Windows + // Desktop withholds run_command; the raw CLI fallback retains direct + // child termination there. + detached: ownsProcessGroup, // Fresh-ish env — keep PATH so the shell can find binaries, but // strip anything the agent host might have set that shouldn't leak // into a user-issued command. A fuller env scrub waits for srt. @@ -295,25 +323,79 @@ export async function runShell(req: ShellRunRequest): Promise { let truncated = false; let timedOut = false; let killTimer: NodeJS.Timeout | null = null; + let terminationStarted = false; + let settled = false; - const timeout = setTimeout(() => { - timedOut = true; + const terminate = (fromTimeout: boolean) => { + if (terminationStarted) return; + terminationStarted = true; + if (fromTimeout) timedOut = true; // SIGTERM first, then SIGKILL after a short grace if the child // hasn't exited. Matches Unix convention; gives well-behaved // children a chance to clean up. - try { - child.kill("SIGTERM"); - } catch { - // already dead - } + signalProcessTree(child.pid, child, ownsProcessGroup, "SIGTERM"); killTimer = setTimeout(() => { - try { - child.kill("SIGKILL"); - } catch { - // already dead - } + signalProcessTree(child.pid, child, ownsProcessGroup, "SIGKILL"); }, SIGTERM_GRACE_MS); - }, timeoutMs); + }; + const timeout = setTimeout(() => terminate(true), timeoutMs); + const abort = () => terminate(false); + + const cleanupListeners = () => { + clearTimeout(timeout); + options.signal?.removeEventListener("abort", abort); + }; + + const finish = async ( + exitCode: number | null, + exitSignal: NodeJS.Signals | null, + spawnError?: Error + ) => { + if (settled) return; + settled = true; + // Report command lifetime through the child exit/error event. Process- + // group revocation below is a host cleanup barrier, not command work, + // and can consume one or two grace windows independently. + const durationMs = Date.now() - startedAt; + cleanupListeners(); + + // A shell wrapper can exit successfully after launching a background + // descendant. Revoke that group's lifetime before the host removes its + // private temp or releases SRT's per-command state. + if (ownsProcessGroup && child.pid) { + if (!terminationStarted) { + terminationStarted = true; + signalProcessTree(child.pid, child, true, "SIGTERM"); + } + const afterTerm = await waitForProcessGroupExit( + child.pid, + SIGTERM_GRACE_MS + ); + if (afterTerm !== "exited") { + // An unknown status (for example EPERM from kill(-pgid, 0)) cannot + // prove revocation. Escalate immediately rather than spending a + // grace window repeatedly making an unverifiable probe. + signalProcessTree(child.pid, child, true, "SIGKILL"); + await waitForProcessGroupExit(child.pid, SIGTERM_GRACE_MS); + } + } + if (killTimer) clearTimeout(killTimer); + + resolve({ + cmd: req.cmd, + args: req.args, + cwd: req.cwd, + exit_code: spawnError ? -1 : exitCode, + signal: exitSignal ?? null, + stdout, + stderr: spawnError + ? stderr + `\n[spawn error] ${spawnError.message}` + : stderr, + duration_ms: durationMs, + timed_out: timedOut, + truncated, + }); + }; const append = (which: "stdout" | "stderr", chunk: Buffer) => { if (truncated) return; @@ -337,37 +419,79 @@ export async function runShell(req: ShellRunRequest): Promise { // Spawn-time errors (ENOENT etc.) surface here. Treat as // exit code -1 with the error message in stderr so the client // gets a meaningful response. - clearTimeout(timeout); - if (killTimer) clearTimeout(killTimer); - resolve({ - cmd: req.cmd, - args: req.args, - cwd: req.cwd, - exit_code: -1, - signal: null, - stdout, - stderr: stderr + `\n[spawn error] ${err.message}`, - duration_ms: Date.now() - startedAt, - timed_out: timedOut, - truncated, - }); + void finish(null, null, err); }); child.on("exit", (code, signal) => { - clearTimeout(timeout); - if (killTimer) clearTimeout(killTimer); - resolve({ - cmd: req.cmd, - args: req.args, - cwd: req.cwd, - exit_code: code, - signal: signal ?? null, - stdout, - stderr, - duration_ms: Date.now() - startedAt, - timed_out: timedOut, - truncated, - }); + void finish(code, signal); }); + + if (options.signal?.aborted) abort(); + else options.signal?.addEventListener("abort", abort, { once: true }); }); } + +function signalProcessTree( + pid: number | undefined, + child: ReturnType, + ownsProcessGroup: boolean, + signal: NodeJS.Signals +): void { + if (ownsProcessGroup && pid) { + try { + process.kill(-pid, signal); + return; + } catch (error) { + if (isNoSuchProcess(error)) return; + // Fall through to the direct child as defense in depth. + } + } + try { + child.kill(signal); + } catch { + // already dead + } +} + +async function waitForProcessGroupExit( + pid: number, + timeoutMs: number +): Promise { + const deadline = Date.now() + timeoutMs; + while (true) { + const status = processGroupStatus(pid); + if (status !== "alive") return status; + if (Date.now() >= deadline) return "alive"; + await new Promise((resolve) => setTimeout(resolve, PROCESS_GROUP_POLL_MS)); + } +} + +type ProcessGroupStatus = "alive" | "exited" | "unknown"; + +function processGroupStatus(pid: number): ProcessGroupStatus { + try { + process.kill(-pid, 0); + return "alive"; + } catch (error) { + return isNoSuchProcess(error) ? "exited" : "unknown"; + } +} + +function isNoSuchProcess(error: unknown): boolean { + return ( + error instanceof Error && + "code" in error && + (error as NodeJS.ErrnoException).code === "ESRCH" + ); +} + +/** + * Explicitly unsafe adapter for hosts that intentionally expose the raw + * process runner without an OS confinement layer. + * + * Keeping this separate from {@link runShell} prevents the low-level runner + * from being accidentally assigned as a {@link ShellExecutor}: the latter + * receives a security scope that a real host executor must enforce. + */ +export const runUnsandboxedShell: ShellExecutor = (request, _scope, signal) => + runShell(request, { signal }); diff --git a/packages/grida-desktop-bridge/src/index.ts b/packages/grida-desktop-bridge/src/index.ts index ceb766af7..ed7b84853 100644 --- a/packages/grida-desktop-bridge/src/index.ts +++ b/packages/grida-desktop-bridge/src/index.ts @@ -73,6 +73,13 @@ export type DesktopAgentCapabilities = { * as an old host that accepts text-only scratch seeds. */ scratch_seed_base64?: boolean; + /** + * The host exposes binary-operable tools for scratch-backed files (today, + * the confined `run_command` capability). Omission/false means a path-only + * PDF/archive/etc. would be inert, so renderers must not offer scratch-only + * arbitrary-byte attachments. + */ + scratch_binary_tools?: boolean; }; export type DesktopCapabilities = { diff --git a/test/desktop-agent-chat-busy-image-blocked.md b/test/desktop-agent-chat-busy-image-blocked.md index c23ba3072..2b1f369e6 100644 --- a/test/desktop-agent-chat-busy-image-blocked.md +++ b/test/desktop-agent-chat-busy-image-blocked.md @@ -1,25 +1,28 @@ --- id: TC-DESKTOP-AGENT-CHAT-005 -title: Image submit is blocked while a turn is streaming (queue is text-only) +title: Attachment submit is blocked while the session is busy (queue is text-only) module: desktop area: agent-chat -tags: [agent-chat, composer, image, queue, multimodal] +tags: [agent-chat, composer, attachment, image, queue, multimodal] status: untested severity: medium date: 2026-06-07 -updated: 2026-06-07 +updated: 2026-07-30 automatable: false -covered_by: [] +covered_by: + - editor/lib/agent-chat/turn-queue.test.ts + - editor/lib/agent-chat/use-turn-queue-controller.test.ts --- ## Behavior -Submitting while a turn is streaming **enqueues** the message (RFC `queue`). The -turn queue persists **text only**, so image attachments cannot ride a queued -send in v1. When the user tries to submit with an image attached while the -session is busy, the composer **blocks** the submit, shows a one-line notice, -and — crucially — does **not** clear the attachments, so the user keeps their -image and can send it once the turn finishes. +Submitting while the session is busy **enqueues** an ordinary text message (RFC +`queue`). The turn queue persists **text only**, so provider-native files, +scratch uploads, and other out-of-band attachment context cannot ride a queued +send in v1. When the user tries to submit any such attachment while the session +is busy, the composer **blocks** the submit, shows a one-line notice, and — +crucially — does **not** clear the draft. The user keeps the attachment and text +and can send them once the session becomes idle. Text-only submits while busy still enqueue normally (unchanged behavior). @@ -30,9 +33,9 @@ Text-only submits while busy still enqueue normally (unchanged behavior). streaming (the round button shows Stop). 3. While it streams, paste/drop an image (chip appears), type some text, and press Enter / click Send. - - Expected: a one-line notice like "Can't queue images — wait for the current - turn." The message is NOT sent, and the image chip + text REMAIN in the - composer. + - Expected: "Can't send attachments while the session is busy — wait until + it's idle." The message is NOT sent, and the image chip + text REMAIN in + the composer. 4. While it still streams, submit **text only** (remove the image first, or use a fresh empty composer). - Expected: the text message enqueues normally (appears in the queued tray). @@ -42,12 +45,14 @@ Text-only submits while busy still enqueue normally (unchanged behavior). ## Notes -- Block + no-clear logic: `agent-composer-input.tsx` `submit()` (guards on - `isStreaming && files.length > 0`); `composer.clear()` is skipped on the - blocked path so attachments survive. -- The queue carrying text only is intentional for v1 — queued image sends are a - deferred enhancement (`use-turn-queue-controller.ts` enqueue path). -- Capability note: every catalogued model is currently `multimodal: true`, so - the "non-vision model rejects images" path is covered by unit logic - (the `multimodal` gate in the composer) rather than a manual TC; add one if a - text-only model ships. +- Block + no-clear logic: `agent-composer-input.tsx` `submit()` guards on + `isBusy && hasOutOfBandResources`; `composer.clear()` is skipped on the + blocked path so the complete draft survives. +- The queue carrying text only is intentional for v1 — queued attachment sends + are a deferred enhancement (`use-turn-queue-controller.ts` enqueue path). +- The listed automated coverage verifies busy-state queue selection and the + text-only queue boundary. The composer notice, blocked submit, and preserved + draft remain manual. +- Provider perception is independent of this guard. A raster unsupported by the + selected provider may still be operable through scratch; either representation + remains out-of-band and therefore cannot enter the text queue. diff --git a/test/desktop-agent-chat-generic-file-drop.md b/test/desktop-agent-chat-generic-file-drop.md new file mode 100644 index 000000000..c00cb8ec9 --- /dev/null +++ b/test/desktop-agent-chat-generic-file-drop.md @@ -0,0 +1,46 @@ +--- +id: TC-DESKTOP-AGENT-CHAT-010 +title: Drop a generic file into the agent composer +module: desktop +area: agent-chat +tags: [agent-chat, composer, file, drag-drop, scratch] +status: verified +severity: high +date: 2026-07-30 +updated: 2026-07-30 +automatable: false +covered_by: + - editor/lib/agent-chat/input-resource-router.test.ts + - packages/grida-ai-agent/src/http/routes/agent.test.ts +--- + +## Behavior + +A dropped file that is not a supported raster, including plain text and SVG, +uses the generic operable-file route. It renders as a file attachment rather +than a raster preview, and a workspace-bound agent can read its staged bytes +from session scratch without receiving the source operating-system path. + +## Steps + +1. Open the desktop app, open a workspace, and focus the agent composer. +2. From Finder, drop a plain `.txt` file with distinctive known contents. + - Expected: a generic file chip appears, not a raster thumbnail. +3. Send a prompt asking the agent to read the attached file and quote its + contents. + - Expected: the agent reads the scratch-backed file and returns the exact + contents without asking for its source path. +4. Repeat with an `.svg` file. + - Expected: it follows the same generic file route and does not render as a + provider-native raster preview. + +## Notes + +- `InputResourceRouter` classifies supported raster types with + `isSupportedImageType`; `image/svg+xml` is deliberately excluded. +- File classification and scratch lowering are covered by + `input-resource-router.test.ts` and the agent route tests; the Finder gesture + and rendered chip remain manual. +- 2026-07-30: Manually verified in local Grida Desktop with plain text and SVG. + Both used the generic file route and were readable from scratch without a + source path. diff --git a/test/desktop-agent-chat-image-drop.md b/test/desktop-agent-chat-image-drop.md index 81ec58d69..551855c43 100644 --- a/test/desktop-agent-chat-image-drop.md +++ b/test/desktop-agent-chat-image-drop.md @@ -4,44 +4,56 @@ title: Drag-and-drop an image file into the agent composer module: desktop area: agent-chat tags: [agent-chat, composer, image, drag-drop, multimodal, vision] -status: untested +status: verified severity: high date: 2026-06-07 -updated: 2026-06-07 +updated: 2026-07-30 automatable: false -covered_by: [] +covered_by: + - editor/kits/composer/composer-transfer.test.ts + - editor/lib/agent-chat/image-attachment.test.ts + - editor/lib/agent-chat/input-resource-router.test.ts + - packages/grida-ai-agent/src/http/routes/agent.test.ts + - packages/grida-ai-agent/src/runtime/runtime.live.test.ts --- ## Behavior Dragging an image file from Finder (or another app) onto the agent composer -attaches it the same way a paste does: inline, perceive-only, downscaled + -base64-encoded into a `file` part. The drop is read from the drop event's bytes -in the renderer — the OS path is never resolved, so it works regardless of where -the file lives and never goes through the workspace-scoped agent fs. - -Dropping a NON-image (e.g. a `.txt`) or an `.svg` must NOT become an image -attachment (SVG is text); those fall through to the editor's normal handling. +attaches it the same way a paste does: a bounded provider-native image for +immediate perception plus a byte-exact original staged into session scratch for +file operations. The drop is read from the drop event's bytes in the renderer — +the source OS path is never resolved or exposed, so it works regardless of +where the file lives. ## Steps 1. Open the desktop app, open a workspace, focus the agent composer. -2. From Finder, drag a `.png`/`.jpg` onto the composer and drop. +2. In Terminal, run `shasum -a 256 ""` and + `wc -c < ""`, then record the source image's SHA-256 + digest, byte count, and filename extension. +3. From Finder, drag that supported raster image (such as a `.png` or `.jpg`) + onto the composer and drop. - Expected: a thumbnail chip appears; the editor text is unchanged. -3. Send "what is this?" → the model describes the dropped image's content. -4. Drag a large, high-resolution image (e.g. >5 MB or >2000px) and drop, then - send. - - Expected: it still sends and the model sees it (client downscale kept it - under the provider limit — no provider error). -5. Drag a plain `.txt` file and drop. - - Expected: it is NOT added as an image chip (non-image rejected at ingest). +4. Send "what is this?" → the model describes the dropped image's content. +5. Send "Use the attachment's scratch path to make a byte-for-byte copy named + `dropped-copy` with the same filename extension. Compute and report the + SHA-256 digest and byte count of both the attachment and the copy." + - Expected: the agent operates on the scratch copy without asking for a path + or reattachment. Both reported digests and byte counts are identical to + each other and to the values recorded in step 2, and the copy preserves + the original extension. ## Notes -- `image/*` files are forwarded by the kit; the desktop handler filters with - `isSupportedImageType` (raster only — `image/svg+xml` excluded) inside - `encodeImageFile`. +- `ComposerContent.onTransfer` and `ComposerTransfer` preserve drop provenance; + `InputResourceRouter` classifies supported raster types with + `isSupportedImageType` (`image/svg+xml` excluded) and prepares the selected + provider preview and scratch representation. - Multiple files dropped at once each become their own chip (see TC-DESKTOP-AGENT-CHAT-004). -- Downscale/cap policy + the model round-trip are covered by - `image-attachment.test.ts` and the gated live test. +- The model round-trip is covered by the gated live test; the Finder gesture + and rendered chip remain manual. +- 2026-07-30: Manually verified in local Grida Desktop. The dropped raster was + perceived, its scratch-backed original remained operable on later turns, and + the agent created the requested byte-for-byte copy without reattachment. diff --git a/test/desktop-agent-chat-image-paste.md b/test/desktop-agent-chat-image-paste.md index 1bad8befa..e5f0a89de 100644 --- a/test/desktop-agent-chat-image-paste.md +++ b/test/desktop-agent-chat-image-paste.md @@ -1,28 +1,35 @@ --- id: TC-DESKTOP-AGENT-CHAT-001 -title: Paste a clipboard image into the agent composer (perceive-only) +title: Paste a clipboard image into the agent composer module: desktop area: agent-chat tags: [agent-chat, composer, image, paste, multimodal, vision] status: untested severity: high date: 2026-06-07 -updated: 2026-06-07 +updated: 2026-07-30 automatable: false -covered_by: [] +covered_by: + - editor/kits/composer/composer-transfer.test.ts + - editor/lib/agent-chat/image-attachment.test.ts + - editor/lib/agent-chat/input-resource-router.test.ts + - packages/grida-ai-agent/src/http/routes/agent.test.ts + - packages/grida-ai-agent/src/runtime/runtime.live.test.ts --- ## Behavior -Pasting a copied image into the desktop agent composer attaches it as an -**inline, perceive-only** image: it is downscaled/encoded client-side to a -base64 data-URL `file` part and sent to the model so the model literally sees -the pixels. No filesystem path is surfaced to the agent (Claude-Code-style) — -the agent can describe the image but cannot operate on it as a file. +Pasting a copied image into the desktop agent composer gives the agent two +representations of one attachment: -The image rides the message as an AI-SDK `file` part. The pipeline already -forwards + persists `file` parts, so a later turn (even text-only) still has the -image in context — the model view is rebuilt from the DB each turn. +- a bounded, possibly resized/transcoded base64 `file` part, so a vision model + sees the pixels immediately; and +- the byte-exact original staged into session scratch, plus a descriptor naming + its scratch-relative path, so filesystem and shell tools can operate on it. + +Immediate perception and tool addressability are complementary rather than +mutually exclusive. If scratch is unavailable or its bounded seed budget is +full, provider-native perception remains the fallback. A pasted image must NOT be inserted as text (no base64 blob in the editor): the composer intercepts image clipboard data and turns it into an attachment chip @@ -39,7 +46,12 @@ instead. 4. Type "what's in this image?" and send. - Expected: the model's reply describes the actual image content (the specific shape/word), proving it saw the pixels — not a generic guess. -5. Without attaching anything, send a follow-up: "describe it again in one line." +5. Send: "Use the attachment's scratch path to make a byte-for-byte copy named + `pasted-copy.png`, then report its byte count." + - Expected: the agent uses the scratch path from the attachment descriptor; + the copy exists and it does not ask the user to save or attach the image + again. +6. Without attaching anything, send a follow-up: "describe it again in one line." - Expected: the model still references the same image (durability — it was persisted and re-sent from the DB on this turn). @@ -47,12 +59,16 @@ instead. - Encoding/policy: `editor/lib/agent-chat/image-attachment.ts` (`encodeImageFile`, downscale to ~1568px / ~5 MB, PNG→JPEG ladder). -- Paste/drop hook is a generic passthrough on the composer kit - (`editor/kits/composer/composer-react.tsx` → `onImageFiles`); the desktop - wiring + chip render is in +- Paste/drop enters through `ComposerContent.onTransfer`; `ComposerTransfer` + preserves the gesture provenance and original browser files. The desktop + wiring and chip render are in `editor/scaffolds/desktop/shared/agent-composer-input.tsx`. - The model→image round-trip (incl. multi-turn + resume) is automated against a real model in `packages/grida-ai-agent/src/runtime/runtime.live.test.ts` (gated `GRIDA_LIVE_AGENT=1`); this TC covers the UI gesture that test can't. +- Dual routing, raw-byte preservation, scratch budgeting, and provider-only + fallback are automated in `input-resource-policy.test.ts` and + `input-resource-router.test.ts`; the clipboard gesture and rendered chip + remain manual. - A server-side size guard rejects inline images >~8 MB before persistence (`run-input.ts` `normalizeWireParts`). diff --git a/test/desktop-agent-chat-large-raster-drop.md b/test/desktop-agent-chat-large-raster-drop.md new file mode 100644 index 000000000..72f97582f --- /dev/null +++ b/test/desktop-agent-chat-large-raster-drop.md @@ -0,0 +1,48 @@ +--- +id: TC-DESKTOP-AGENT-CHAT-009 +title: Drop a large raster without exceeding provider limits +module: desktop +area: agent-chat +tags: [agent-chat, composer, image, drag-drop, multimodal, vision, limits] +status: verified +severity: high +date: 2026-07-30 +updated: 2026-07-30 +automatable: false +covered_by: + - editor/lib/agent-chat/image-attachment.test.ts + - editor/lib/agent-chat/input-resource-router.test.ts + - packages/grida-ai-agent/src/runtime/runtime.live.test.ts +--- + +## Behavior + +A large raster dropped into a workspace-bound agent composer remains +perceivable without sending provider-hostile source bytes. The provider-native +representation is bounded independently from the byte-exact scratch twin, so +provider limits do not silently make the attachment unusable. + +## Steps + +1. Open the desktop app, open a workspace, and focus the agent composer. +2. From Finder, drop the checked-in asset `editor/public/west/poster.png` + (2,910 × 4,338 px; 303,282 bytes; SHA-256 + `4439944baae26bd9895274f50445d9f16036457626714bb3297fee189e7b2c99`). + Its 4,338 px longest edge deterministically exceeds the 1,568 px provider + representation ceiling while its original bytes fit the scratch budget. + - Expected: a thumbnail chip appears without an attachment error. +3. Send "What is shown in this black-and-white image?" + - Expected: the model identifies multiple cowboys riding horses (and may + mention birds or smoke) without a provider size error. + +## Notes + +- Downscaling affects only the provider-native perception representation. When + the scratch budget permits, file operations use the original bytes. +- The byte caps and provider representation are covered by + `image-attachment.test.ts` and `input-resource-router.test.ts`; the operating + system drop gesture remains manual. +- 2026-07-30: Manually verified in local Grida Desktop with an oversized + raster; it produced a thumbnail and remained perceivable without a provider + size error. The checked-in high-resolution asset above makes future runs + deterministic without modifying the engine-owned image fixture snapshot. diff --git a/test/desktop-agent-chat-multi-image.md b/test/desktop-agent-chat-multi-image.md index 4fe8550dc..ef919f679 100644 --- a/test/desktop-agent-chat-multi-image.md +++ b/test/desktop-agent-chat-multi-image.md @@ -7,24 +7,27 @@ tags: [agent-chat, composer, image, multimodal, vision] status: untested severity: medium date: 2026-06-07 -updated: 2026-06-07 +updated: 2026-07-30 automatable: false -covered_by: [] +covered_by: + - editor/kits/composer/composer-transfer.test.ts + - editor/lib/agent-chat/input-resource-router.test.ts --- ## Behavior Several images can be attached to a single message — by dropping multiple files at once, or by pasting/dropping repeatedly before sending. Each becomes its own -thumbnail chip, each is sent as its own `file` part, and the model can compare -them in one turn. Removing one chip (hover ✕) leaves the others intact. +thumbnail chip and provider-native `file` part, so the model can compare them in +one turn. Removing one chip (hover ✕) leaves the others intact. ## Steps 1. Open the desktop app, open a workspace, focus the agent composer. -2. Select two different images in Finder and drag both onto the composer. +2. Select two different supported raster images, such as PNG or JPEG files, in + Finder and drag both onto the composer. - Expected: two thumbnail chips appear. -3. Paste a third image (**⌘V**) before sending. +3. Paste a third supported raster image (**⌘V**) before sending. - Expected: a third chip appears. 4. Remove one chip via its hover ✕. - Expected: that chip disappears; the other two remain. @@ -34,9 +37,18 @@ them in one turn. Removing one chip (hover ✕) leaves the others intact. ## Notes -- Each image is independently downscaled/encoded by `encodeImageFile`; all - qualifying `file-attachment` parts are mapped to `file` parts by - `toFileUiParts` on submit. +- `InputResourceRouter.prepareBatch` prepares each provider rendition and raw + scratch body in attachment order under a bounded draft-retention budget. + `InputResourceRouter.lower` emits the provider `file` parts plus the admitted + scratch seed and ordered descriptors. If a chip is removed, retained twins + are re-evaluated against the current submit-time scratch capacity. +- When the scratch budget permits, each original is staged as its own operable + copy. A provider-capable image remains available for perception if its + optional scratch copy cannot be admitted. +- Automated routing tests cover aggregate-budget arbitration: scratch-only + resources reserve capacity before optional raster twins, and remaining + provider-capable rasters fall back in attachment order. This manual case + verifies the multi-gesture, card-removal, and model-comparison behavior. - Chips + remove come from the composer kit's `ComposerAttachmentCards`. - Watch the context meter: several inlined images add up — large sets can approach the context window (and are re-sent each turn until the deferred diff --git a/test/desktop-agent-chat-screenshot-drop.md b/test/desktop-agent-chat-screenshot-drop.md index 4b833e128..b1be4036e 100644 --- a/test/desktop-agent-chat-screenshot-drop.md +++ b/test/desktop-agent-chat-screenshot-drop.md @@ -7,9 +7,14 @@ tags: [agent-chat, composer, image, drag-drop, macos, screenshot, sandbox] status: untested severity: high date: 2026-06-07 -updated: 2026-06-07 +updated: 2026-07-30 automatable: false -covered_by: [] +covered_by: + - editor/kits/composer/composer-transfer.test.ts + - editor/lib/agent-chat/image-attachment.test.ts + - editor/lib/agent-chat/input-resource-router.test.ts + - packages/grida-ai-agent/src/http/routes/agent.test.ts + - packages/grida-ai-agent/src/runtime/runtime.live.test.ts --- ## Behavior @@ -20,8 +25,9 @@ saves. Its backing file lives under a system temp path (`/var/folders/.../T/TemporaryItems/...`), OUTSIDE any workspace. Dragging it into the agent composer must attach it like any other image: the -renderer reads the **bytes** off the drop event and inlines them — it must NOT -try to resolve/read the `/var/folders/...` path through the workspace agent fs +renderer reads the **bytes** off the drop event, creates the provider-native +preview, and stages a byte-exact session-scratch copy. It must NOT try to +resolve/read the `/var/folders/...` path through the workspace agent fs (which would reject it as outside the workspace, GRIDA-SEC-004). **This is the one genuinely-uncertain case** and the reason it gets its own TC: @@ -46,9 +52,12 @@ is the failure mode to capture here. **Run this case early.** - Mechanically identical to TC-DESKTOP-AGENT-CHAT-002; only the _source_ (a temp-backed drag promise) differs, which is what makes the sandbox behavior - worth verifying explicitly. -- The path is never handed to the agent — inline/perceive-only by design, so the - `/var/folders` location is irrelevant to the agent fs boundary. + worth verifying explicitly. The listed automated coverage exercises the + ordinary transfer, routing, and delivery paths; it does not exercise this + macOS drag-promise gesture. +- The source `/var/folders` path is never handed to the agent. The operable path + is a host-chosen scratch-relative name, so the temp location remains outside + the agent fs boundary. - If this fails, the fallback is the clipboard path: **⌘⇧Ctrl+4** copies the screenshot to the clipboard, then paste (TC-DESKTOP-AGENT-CHAT-001), which never touches a temp file.