Sandbox denials are invisible to the model when the confined program rewrites the kernel error

[tommy0103]

What happened

In a standard workspace-write session, the agent needed to write to ~/.obelisk (outside the workspace root). The Seatbelt profile correctly denied the write — but the agent had no way to know the sandbox was involved:

1. A SQLite command failed with a translated error ("unable to open database file"-class — SQLite maps the kernel's EPERM onto its own error codes). No denial marker appeared in the tool result.
2. The model's first hypothesis was disk full. It spent several turns probing that theory (e.g. testing whether a full disk blocks all SQLite file operations) before a manual touch ~/.obelisk happened to print Operation not permitted and revealed the sandbox.
3. Only then did it request escalation to danger-full-access — with a correct justification, through the proper approval flow.

To be fair to the escalation design: from workspace-write, danger-full-access is the only wider mode (WIDER_MODES), and writing to ~/.obelisk genuinely requires it. The escalation mechanism worked as intended. What failed is the diagnostic path: the model reached the right conclusion by accident, after a wrong-hypothesis detour, instead of from a harness-supplied fact. With a slightly different error translation it could just as easily have concluded "this command cannot work at all" and given up — or asked for full access with a bogus justification like "the disk is broken". The user shouldn't have to wait through (and approve around) a misdiagnosis the harness could have prevented with one line of output.

Root cause: denial detection is a stderr-substring heuristic

ctx.sandbox.confine() returns per-backend denialSignatures (operation not permitted for Seatbelt, permission denied for Landlock, etc.), and dsh-bash-sandbox classifies a failed run with matchesSignature(exitCode, stderr, signatures) — a plain case-insensitive substring match on stderr (packages/shell/bash-sandbox/src/helpers.ts).

That only works when the confined program prints the kernel's errno verbatim. Real programs translate EPERM into their own vocabulary:

• SQLite: unable to open database file, disk I/O error
• Node libraries: wrapped EACCES/EPERM messages that may or may not survive
• countless tools: cannot open, failed to create, or silence

When the translation happens, no signature matches → sandbox.denied: false → the [sandbox: file access denied under workspace-write mode] marker (packages/sandbox/sandbox/src/escalation.ts) is never emitted → the model reasons about the error at face value. The misdiagnosis is not a model weakness; the harness withheld the one fact (confinement is active, and this write fell outside the policy) that would have made the diagnosis trivial.

Notably, the fs-capability fence does not have this problem: it refuses mutations itself and reports the exact denial marker. The asymmetry is confined to the subprocess path, where the kernel — not the harness — delivers the refusal.

Directions worth discussing

1. Always-on confinement context. Include the active mode and writable roots in every confined tool result (or at least in every non-zero-exit result), not just on heuristic hits — so the model can attribute ambiguous I/O errors itself. Cost: a few tokens per result.
2. Path-based attribution as a second classifier. On non-zero exit under a confined mode, extract absolute paths from stderr/stdout and flag ones outside the policy's writable roots with a softer marker ([sandbox: this failure touched paths outside writable roots — possibly sandbox-related]). Heuristic too, but catches the translated-error case that signatures miss.
3. Tool-description teaching. State in the bash tool description that under workspace-write, writes outside the workspace fail and programs may report this as generic I/O errors — cheap, prompt-level, covers today's gap with no mechanism change.
4. Distinguish "proven denial" from "proven not-denied". Today denied: false conflates "command genuinely failed" with "we couldn't tell". Surfacing the uncertainty itself would already change model behavior.

Happy to prototype whichever direction the maintainers prefer — (1) or (2) look like the best signal-to-complexity ratio to me.