docs(openspec): propose approval-policy-v2#940
Draft
Aaronontheweb wants to merge 27 commits intonetclaw-dev:devfrom
Draft
docs(openspec): propose approval-policy-v2#940Aaronontheweb wants to merge 27 commits intonetclaw-dev:devfrom
Aaronontheweb wants to merge 27 commits intonetclaw-dev:devfrom
Conversation
2 tasks
Aaronontheweb
force-pushed
the
openspec/approval-policy-v2
branch
2 times, most recently
from
81a306f to
574774a
Compare
Collaborator
Author
Eval Suite Results — Approval Policy v2Targeted N=5 runs against the local inference endpoint ( Baseline → Variant B (prompt rewrites only)
Infrastructure observationThe local inference endpoint is intermittently producing stalled streams — This affects eval reliability but is outside this PR's scope — it points at adding a streaming idle timeout on the daemon's OpenAI-compatible HTTP client so stalled streams surface as real errors instead of producing partial responses. Will file separately. Eval-infra fixes shipped in this PRBoth were latent bugs, surfaced when investigating why the v2 cases scored 0% initially:
What we know works
Acceptance gate statusPer |
Breaking redesign of the persistent approval store and prompt UX: - typed (verb, directory) ApprovalEntry replaces the v1 flat string list; v1 file quarantines to .v1.bak on first read (no migration) - safe-verb ∩ safe-space short-circuit (per-OS verb list, audience-aware roots from ToolAudienceProfileResolver) auto-runs read-only inspection inside session_dir / project_dir - ShellTool cwd defaults to project_dir → session_dir (today inherits daemon-process cwd) - ShellTokenizer refuses pattern extraction on bash control-flow / unbalanced input so junk fragments never persist - 5-button prompt (Once / This chat / Always here / Always anywhere / Deny) with danger styling on the destructive options; one-line resolution message - netclaw approvals trust-verb <verb> CLI for unattended/scheduled grants - AGENTS.md + tool description + failure-path hint coordinate to push the agent toward set_working_directory; eval cases (positive/negative/recovery/ schedule pre-approval) lock the behavior in
Foundation for the approval-policy-v2 storage refactor. Adds: - ApprovalEntry record (Verb required, Directory nullable for global wildcard) - ToolApprovalEntryComparer.Equals(ApprovalEntry, ApprovalEntry) overload that delegates to the existing platform-correct string comparison No behavior change: ToolApprovalStore still operates on the v1 string-based API and the existing test suite (274 tests) passes unchanged. The actual storage cutover, matcher refactor, and caller updates land in subsequent commits per openspec/changes/approval-policy-v2/tasks.md sections 1-6.
Section 1 of the approval-policy-v2 OpenSpec change. Refactors
ToolApprovalStore to a typed (verb, directory) ApprovalEntry model with
a versioned on-disk schema, replacing the v1 flat string list.
What changed:
- ToolApprovalStore now serializes/deserializes ToolApprovalData with
"version": 2 and List<ApprovalEntry> per (audience, tool).
- Two-step Load(): peek schema version via JsonDocument; quarantine
legacy v1 files to tool-approvals.json.v1.bak; quarantine unparseable
files to .invalid; in either case, return an empty store.
- AddApproval/RemoveApproval/RemoveAllForTool/Snapshot operate on
ApprovalEntry. New GetApprovedEntries replaces GetApprovedPatterns.
- AddApproval normalizes the directory portion (trims trailing
separators while preserving "/" and "C:\") so the on-disk file does
not accumulate trailing-slash variants of the same logical entry.
- ToolApprovalEntryComparer gains NormalizeDirectory + Normalize(entry)
helpers; Equals(ApprovalEntry, ApprovalEntry) normalizes both sides.
Caller updates required to compile:
- ToolApprovalActor: persistent writes wrap incoming verb strings as
ApprovalEntry { Verb=pattern, Directory=null } (interim semantic
preserved until section 2 lands the directory-aware matcher).
- ApprovalsListView/ApprovalsCommand: list output renders entries as
"<verb> in <dir>" or "<verb> anywhere"; --json emits the typed
ApprovalEntry shape; --json uses IndentedOmitNull so the CLI shape
matches the file shape (nulls omitted).
- ApprovalsCommand.WarnIfQuarantined surfaces both .v1.bak and
.invalid quarantine paths with distinct remediation guidance.
- ApprovalsManagerViewModel/Page: rendering uses entry.DisplayText.
- ToolAudienceProfilesDoctorCheck: drops the v1 stale-path-aware
pattern detection (irrelevant under v2; v1 contents quarantine on
first read).
Tests:
- ToolApprovalStoreTests rewritten for the v2 API and gain coverage
for v1 quarantine, malformed quarantine, fresh-write-after-quarantine,
trailing-slash normalization, and idempotent add.
- ApprovalsCommand/ApprovalsManagerPage tests rewritten to use
ApprovalEntry and the new "<verb> in <dir>" / "<verb> anywhere"
rendering.
- Stale-pattern doctor test removed.
All 3348 tests pass; dotnet slopwatch analyze reports no new
violations; file-header verification passes.
Section 2 of the approval-policy-v2 OpenSpec change. Refactors the approval matcher and gate to consume v2 typed ApprovalEntry records, plumbs the candidate cwd through the execution context, and deletes the v1 string-shape inspection logic. Matcher contract changes: - IToolApprovalMatcher.ExtractDirectoryRoots is removed; the v2 matcher has no concept of "directory roots extracted from arguments." The directory half of every (verb, directory) pair is the candidate's cwd from ToolExecutionContext. - ExtractApprovalEntries renamed to ExtractCandidateVerbs and now returns pure verb chains. The v1 fallback to normalized commands or bare directory roots is gone. - IsApproved signature: now takes (toolName, args, IReadOnlyList<ApprovalEntry>, cwd) and dispatches to ApprovalPatternMatching.MatchesShellApproval which enforces verb equality + (directory null || cwd under directory) + no-symlink-segment. Cwd plumbing: - ToolExecutionContext gains a Cwd property the session pipeline sets from candidate args / WorkingContext.ProjectDirectory / session_dir (sections 4 + 5 cover the resolution side). - IToolApprovalService.GetUnapprovedPatternsAsync and RecordApprovalAsync take a cwd parameter; AkkaToolApprovalService threads it through GetUnapprovedPatterns and RecordToolApproval actor messages. - ToolApprovalContext: ApprovalEntries field renamed to CandidateVerbs; DirectoryRoots stays but is always populated empty by the gate (section 7's prompt redesign removes the field). SessionOutput, SessionOutputDto, ParentSessionApprovalBridge, PendingToolInteraction, and the protocol mapper rename consistently. Shared symlink-segment guard: - PathUtility.ContainsSymlinkSegment hoisted from ScopedFileAccessPolicy so the matcher and the file-access policy share one implementation. Tests: - Configuration.Tests, Cli.Tests, Daemon.Tests, MemoryRetrievalPoC.Tests, Search.Tests, Security.Tests (397 incl new matcher cases), and Actors.Tests (1483) all pass. - ShellApprovalMatcherTests rewritten to assert the v2 (verb, cwd, entries) semantics: global-wildcard matches anywhere, folder-scoped matches when cwd under directory, requires concrete cwd, recurses into bash -c. - ToolApprovalGateTests' v1 directory-roots assertions replaced with v2 candidate-verb assertions; DirectoryRoots is asserted empty. - ToolApprovalActor's session HashSet now uses ToolApprovalEntryComparer.Comparer so session approvals follow the same platform-correct case rules as the persistent store. - Test plumbing across the codebase passes cwd: null where the invocation isn't directory-anchored. Slopwatch clean; file headers verified.
Section 3 of the approval-policy-v2 OpenSpec change. Adds a cheap
structural scan to ShellTokenizer that detects bash control-flow
keywords and unbalanced quotes/brackets, refuses verb-chain
extraction in those cases, and plumbs an IsMessy flag through the
gate and protocol so the section 7 prompt builder can show "complex
command" hints and omit persistent-grant buttons.
Detection (ShellTokenizer.IsMessyCompoundCommand):
- Single-pass scan that tracks quote state and (), [], {} balance.
- Flags any unquoted standalone token equal to one of:
for, while, do, done, then, fi, case, esac.
- Flags unbalanced quotes (open without close) and unbalanced brackets
(close without open OR open without close).
- Cheap structural only — no semantic bash parsing. Heredocs, command
substitution, and process substitution are not analyzed beyond
bracket balance.
SplitCompoundCommand:
- Returns an empty list when IsMessyCompoundCommand returns true. The
matcher's ExtractCandidateVerbs and ExtractPatterns therefore both
return empty for messy commands, and ShellApprovalMatcher.IsApproved
short-circuits to false (cannot auto-approve what we cannot extract).
Gate / protocol plumbing:
- IToolApprovalMatcher gains IsMessy(toolName, args). Default-false
for DefaultApprovalMatcher and FilePathApprovalMatcher; ShellApprovalMatcher
delegates to ShellTokenizer.IsMessyCompoundCommand.
- ToolApprovalContext gains an IsMessy bool field.
- ToolInteractionRequest, SessionOutputDto (InteractionIsMessy),
PendingToolInteraction, IParentApprovalBridge.RequestApprovalAsync,
and ParentSessionApprovalBridge all carry the flag through.
- DispatchingToolExecutor short-circuits messy invocations to
RequiresApproval regardless of empty CandidateVerbs, so the user
always sees the prompt for messy input.
Trade-off accepted: a bare standalone `done`/`fi`/`esac` token at the
end of a command (e.g. `git fetch && echo done`) is a false positive
for the cheap heuristic — the user gets the "complex command" prompt
(Once/Deny only) instead of the full 4-button row. The mitigation if
this bites real usage is a smarter detector that requires the keyword
to appear in a syntactically meaningful position; for now the trade
favors a clean approval store over coverage of edge bash idioms. One
existing test (SplitCompound_preserves_quoted_operators) updated
accordingly to use a different sentinel word.
Tests:
- ShellTokenizerTests: positive cases (for/while/if/case/unbalanced
quote/unbalanced bracket), negative cases (well-formed compounds,
command substitution, brace expansion, trailing commands), and
guards against keyword-substring false positives ("format",
"fido"). SplitCompoundCommand returns empty for messy input;
still splits well-formed compounds.
- ShellApprovalMatcherTests: IsMessy true for control-flow,
IsMessy false for well-formed; IsApproved returns false for
messy commands even when every conceivable verb is approved.
- All 3367 tests pass; slopwatch clean; file headers verified.
Section 4 of the approval-policy-v2 OpenSpec change. Establishes a deterministic cwd resolution chain for shell invocations so the approval policy can reason about safe-space membership and the spawned process never inherits the daemon's cwd. Resolution order (ToolExecutionContext.ResolveShellCwd): 1. Explicit args.WorkingDirectory when the agent provided one. 2. WorkingContext.ProjectDirectory when set via set_working_directory. 3. SessionDirectory (the per-session ~/.netclaw/sessions/<id>/ scratch). 4. null only when none is available. Plumbing: - ToolExecutionContext gains ProjectDirectory and ResolveShellCwd. The session pipeline populates ProjectDirectory at context-build time from _state.WorkingContext.ProjectDirectory. - SessionToolExecutionPipeline.ExecuteToolsAsync / ExecuteSingleToolAsync / BuildToolExecutionContext gain a projectDirectory parameter; LlmSessionActor passes _state.WorkingContext.ProjectDirectory at every dispatch. - ShellTool.ExecuteAsync uses context.ResolveShellCwd(args.WorkingDirectory) to set psi.WorkingDirectory; never falls through to ProcessStartInfo's default-of-inheriting-the-daemon's-cwd, which is a footgun the approval policy cannot reason about. - DispatchingToolExecutor.AuthorizeCoreAsync calls the same resolver and writes context.Cwd before GetUnapprovedPatternsAsync, so the approval gate evaluates folder-scoped ApprovalEntry records against the same cwd the spawned process will run in. Tests: - Cwd_falls_back_to_project_directory_when_no_explicit_arg - Cwd_falls_back_to_session_directory_when_project_directory_null - Cwd_explicit_arg_overrides_project_and_session_directories - Cwd_does_not_inherit_daemon_process_directory (asserts the spawned pwd output is the resolved session_dir, not Environment.CurrentDirectory) All 3371 tests pass; slopwatch clean; file headers verified.
Section 5 of the approval-policy-v2 OpenSpec change. Adds the
load-bearing friction-reduction layer: read-only verbs invoked inside
declared safe spaces auto-allow without prompting, while every other
combination still routes through the interactive approval gate.
Three-position policy:
layer 1 ToolPathPolicy hard-deny (unchanged)
layer 1.5 NEW: safe-verb ∩ safe-space short-circuit (this commit)
layer 2 interactive approval gate (unchanged)
A candidate (verb, cwd) short-circuits to Allow when ALL hold:
- verb is on the curated SafeVerbList for the current OS
- cwd resolves under one of the audience-aware safe-space roots
(Personal/Team: session_dir + project_dir; Public: session_dir)
- no segment of the cwd path is a filesystem symlink (reparse point)
Bundled lists (Netclaw.Configuration/SafeVerbs/safe-verbs.*.json
embedded as resources, additive user override at
~/.netclaw/config/safe-verbs.<os>.json):
Linux/macOS: ls, find, grep, egrep, fgrep, rg, cat, head, tail, wc,
sort, uniq, cut, tr, awk, sed -n, file, pwd, which, stat, tree,
du, df, git status, git log, git diff, git show, git branch,
git remote, git rev-parse, git ls-files, git blame.
Windows: dir, type, more, where, findstr, Get-ChildItem,
Get-Content, Select-String, Get-Item, Test-Path, Get-Location,
Resolve-Path, plus the same git read subcommands.
Mutating verbs (git push, sed -i, awk -i inplace, rm, mv, etc.) are
intentionally absent from both lists. sed is pinned to "sed -n" so
the matcher refuses to short-circuit "sed -i". The verb-chain
matcher means "awk" auto-allows but "awk -i inplace" hits the gate
because ExtractVerbChain stops at the first flag.
Plumbing:
- New SafeVerbList (Configuration) with platform-correct comparer.
- New SafeVerbLoader that reads the bundled JSON resource and merges
the user override file additively. Malformed override → silently
fall back to bundled defaults (the doctor will surface the problem
out of band; we do not refuse to start the daemon).
- New ScopedShellSafeVerbPolicy (Netclaw.Actors.Tools) mirroring
ScopedFileAccessPolicy: takes (verb, cwd, context), returns a
short-circuit decision; reuses PathUtility.ContainsSymlinkSegment
and the audience model.
- ToolAccessPolicy gains a SafeVerbList ctor parameter and runs the
safe-verb check inline in CheckApprovalGate after the messy/Auto
filters but before producing the approval-prompt context. The cwd
it evaluates is resolved by ToolExecutionContext.ResolveShellCwd
and written back to context.Cwd so the downstream gate and the
spawned process agree on "where this runs."
- DispatchingToolExecutor's duplicate cwd resolution removed —
CheckApprovalGate now owns the write to context.Cwd.
- Program.cs constructs a SafeVerbList at startup and registers it
alongside ToolAccessPolicy.
- NetclawPaths.SafeVerbsOverridePath returns the per-OS user file.
Tests (3388 → 3398 across the suite):
- SafeVerbLoaderTests: bundled defaults present per OS, user override
extends additively, malformed override falls back, missing override
ignored, platform-correct case rules.
- ScopedShellSafeVerbPolicyTests: all seven scenarios from the spec —
safe verb + project_dir → allow; safe verb + session_dir → allow;
safe verb + outside → prompt; mutating verb in safe space →
prompt; Public audience cannot use project_dir as safe space;
symlink segment in cwd breaks short-circuit; AllShortCircuit
fails-loud when any candidate is unsafe.
Slopwatch clean; file headers verified.
Section 6 of the approval-policy-v2 OpenSpec change. Replaces the
section 1 interim revoke parser with a strict parser for the user-
visible scope labels emitted by 'list', and adds the 'trust-verb'
subcommand for pre-approving global wildcards from the CLI.
Revoke parser:
- Accepts only the two forms 'list' emits:
'<verb> in <directory>' -> (verb, directory) entry
'<verb> anywhere' -> (verb, null) global wildcard
- Anything else exits 1 with a clear message — bare verb input
no longer silently treated as a global wildcard, so an operator
typo cannot widen the intended scope. The TryParseRevokePattern
helper is internal so tests can exercise the parser surface
directly without the CLI shell.
trust-verb subcommand:
- 'netclaw approvals trust-verb <verb> [--audience <a>] [--tool <t>]'
- Default audience = personal, default tool = shell_execute.
- Writes a (verb, null) entry to tool-approvals.json — the
global-wildcard form. Idempotent: existing entry exits zero with
a "No changes" message; otherwise prints "Trusted '<verb> anywhere'
for <audience> / <tool>".
- This is the deliberate scriptable path the spec calls out for
unattended/scheduled task pre-approval. Combined with section 5's
safe-verb short-circuit it covers two distinct user goals:
short-circuit (read-only verbs in safe spaces, no persistence)
versus trust-verb (any verb, anywhere, persisted).
Help text updated to document both new forms; quarantine note from
section 1 already covers the .v1.bak case.
Tests (Cli.Tests 620 -> 629):
- Revoke folder-scoped form removes entry with matching directory;
folder-scoped form does not match a global-wildcard entry;
unrecognized pattern exits 1 with clear message.
- trust-verb adds global wildcard with default audience/tool;
idempotent on repeated invocation; honors --audience/--tool;
missing verb argument exits 1 with usage; unknown audience flag
exits 1.
- Help output mentions trust-verb subcommand.
TUI display already shows verb + directory via DisplayText (landed
in section 1). The trust-verb-from-TUI affordance is deferred — the
agent path is CLI-only and the CLI works for human operators too;
revisit if friction surfaces.
All 3397 tests pass; slopwatch clean; file headers verified.
Section 7 of the approval-policy-v2 OpenSpec change. Replaces the
v1 Slack approval prompt (4 buttons + Patterns/Directory Roots
sections) with the v2 design: 5 buttons, danger styling on the
elevated decisions, cwd in the header, verbs as bullets, and a
single-line resolution message.
Five-button row (ApprovalOptionKeys):
Once (primary) - no persist
This chat (default) - session-scoped only
Always here (default) - persist (verb, cwd)
Always anywhere (danger) - persist (verb, null)
Deny (danger) - refuse this call
ApprovalOptionKeys gains ApproveEverywhere/ApproveEverywhereLabel
("Always anywhere") and renames the existing labels to the spec
spelling: "Once" / "This chat" / "Always here" / "Deny". The wire
keys are unchanged so persisted resolutions still decode.
ApprovalDecision and ParentApprovalDecision gain ApprovedEverywhere
so the runtime can distinguish folder-scoped persistence from global
wildcard. LlmSessionActor maps the new button key, picks
cwd-or-null based on which decision was chosen, and threads through
RecordApprovalAsync. ToolApprovalActor's persistent-write path now
uses msg.Cwd directly (replacing the section 1 interim that always
wrote null), so:
Always here -> AddApproval(audience, tool, (verb, msg.Cwd))
Always anywhere -> AddApproval(audience, tool, (verb, null))
Button-row gating by IsMessy / cwd-shallow:
IsMessy -> only Once + Deny (no persistence possible)
cwd shallow -> Always here omitted (This chat / Always
anywhere still available; matches the
tool-approval-gates "Shallow directory prevents
Always here" scenario)
otherwise -> all five buttons
Cwd-shallow check in ToolAccessPolicy: a path with fewer than two
non-empty path segments under its root (e.g. /, /etc/, C:\) cannot
host a folder-scoped grant; fail-closed on Always here so an
operator cannot accidentally persist a too-shallow root.
Slack prompt body changes:
Header (single verb): "Approve git status in /home/user/repos/foo?"
Header (multi-verb): "Approve in /home/user/repos/foo?"
+ "• git fetch / • git rebase / • git status"
Messy: "_complex command — only one-shot approval
available_"
The Patterns and Directory Roots sections are gone; verb display
flows from CandidateVerbs (the v2 matcher's pure verb-chain
extraction) with a Patterns fallback for legacy callers.
Resolution message single-line format:
Always here -> "Saved: <verbs> in <cwd>"
Always anywhere -> "Saved: <verbs> anywhere"
This chat -> "Saved for this chat: <verbs> in <cwd>"
Once -> "Approved (no save)"
Deny -> "Denied"
Tests (Actors.Tests 1497 -> 1507):
- New SlackApprovalBlockBuilderTests covers all the spec scenarios:
single-verb header, multi-verb bulleted header, messy hint,
five-button row with danger styling on Always anywhere + Deny,
legacy Directory Roots / Patterns sections gone, and all five
resolution-message branches (Always here / Always anywhere /
This chat / Once / Deny).
- Existing DiscordApprovalPromptBuilderTests label expectations
bumped to the new spelling ("Once" / "Always here").
All 3407 tests pass; slopwatch clean; file headers verified.
Discord rendering still on v1 — section 8 mirrors this design over.
Section 8 of the approval-policy-v2 OpenSpec change. Brings the
Discord approval prompt to parity with the Slack v2 layout from
section 7: same 5-button row, same danger styling rules, same
header format, same single-line resolution message.
DiscordApprovalPromptBuilder changes:
- BuildButtonPrompt now renders the v2 header
("Approve git status in /home/user/repos/foo?" for single-verb,
"Approve in /home/user/repos/foo?" + bulleted verbs for multi-verb)
and surfaces the "complex command — only one-shot approval
available" hint when IsMessy is true.
- BuildResolvedPromptText emits the single-line resolution form
identical to Slack:
Always here -> "Saved: <verbs> in <cwd>"
Always anywhere -> "Saved: <verbs> anywhere"
This chat -> "Saved for this chat: <verbs> in <cwd>"
Once -> "Approved (no save)"
Deny -> "Denied"
- GetButtonStyle applies DiscordButtonStyle.Danger to both
ApproveEverywhere and Deny, mirroring Slack's danger pair.
- Verb display sources from CandidateVerbs (v2) with a Patterns
fallback for legacy callers.
- GetDecisionLabel handles ApproveEverywhere alongside the existing
keys.
No Discord-side response-handler changes required: the transport
decodes button values and forwards selectedKey to the session actor,
and LlmSessionActor's switch (updated in section 7) already routes
ApproveEverywhere for both channels.
Tests (Actors.Tests 1507 -> 1514):
- Existing two BuildResolvedPromptText cases bumped to assert the
v2 single-line form ("Approved (no save)" / "Denied") instead of
the v1 "Decision: <label>" string.
- Seven new V2_ tests parallel to SlackApprovalBlockBuilderTests:
single-verb header collapse, multi-verb generic header with
bullets, messy-command hint with two-button row, five-button row
with danger styling on Always anywhere and Deny, and the three
persistent-resolution branches (Always here / Always anywhere /
This chat).
All 3414 tests pass; slopwatch clean; file headers verified.
Both Slack and Discord approval flows now end-to-end on v2.
…hint
Section 9 of the approval-policy-v2 OpenSpec change. Steers the
agent toward declaring its project root early and gives it a
self-correction path when a shell call is denied for cwd-outside-
safe-spaces.
netclaw-operations SKILL.md (bumped to v2.0.0):
- Rewrote Approval Prompts around the v2 (verb, directory) model:
three-layer gate (hard-deny / safe-verb short-circuit / interactive
prompt), the five-button row and its scope semantics, when fewer
buttons appear (messy / shallow cwd), and how set_working_directory
affects prompt cadence.
- Added "Pre-approving for unattended tasks (load-bearing)" section
documenting the schedule-creation pre-approval flow. Replaces the
v1 "run interactively first" pattern with the new
'netclaw approvals trust-verb <verb>' path; agent dialogue example
shows how to ask the user before pre-approving.
- Updated the Approval Requirements for Reminders/Webhooks section
to point at trust-verb instead of interactive-first.
- Updated the inspecting/revoking section: list emits typed entries
('<verb> in <dir>' / '<verb> anywhere'); revoke accepts those forms
verbatim; trust-verb is the deliberate scriptable path.
- Last-resort recovery now mentions both .v1.bak and .invalid
quarantine paths.
Resources/AGENTS.md (Personal+Team identity file):
- New top-level "Declare Your Project Root Early (load-bearing)"
section. Tells the agent its FIRST shell-related action MUST be
set_working_directory when the task is project-scoped, with the
consequence framing ("burns the user's attention and your token
budget" if skipped). Includes a recovery rubric: when shell denial
surfaces a set_working_directory hint, read it and self-correct
rather than re-prompting the user.
- AGENTS.public.md unchanged because set_working_directory is
profile-managed away from Public.
set_working_directory tool description:
- Reframed from "set the project directory for this session" to
"Declare your project root and expand your trusted scope." Spells
out the safe-verb short-circuit consequence so the model sees
*why* this tool matters for friction reduction. Removed the cd-
style framing.
- Added public ToolName constant so the failure-path hint logic can
reference it without string duplication.
Failure-path hint (SessionToolExecutionPipeline.BuildSetWorkingDirectoryHint):
- Emits a one-line hint pointing at 'set_working_directory <cwd>' when:
* tool is shell_execute
* decision is Denied (not TimedOut, not hard-deny)
* cwd is non-null
* cwd is NOT inside SessionDirectory or ProjectDirectory
* set_working_directory is exposed to the current audience
- LlmSessionActor pre-computes setWorkingDirectoryAvailable from the
ToolAccessPolicy's IsToolExposed check and threads the bool into
ExecuteToolsAsync; the pipeline appends the hint to the deny-result
text the model sees on its next turn.
- Suppresses for non-shell tools, timeouts, hard-deny refusals, cwd
already inside a safe space, and audiences without the tool — so
Public sessions don't see misleading "use set_working_directory"
guidance.
Tests (Actors.Tests 1514 -> 1521):
- Seven hint-helper unit tests cover all the spec scenarios:
emitted on cwd-outside denial; suppressed when tool unavailable;
suppressed for TimedOut; suppressed for non-shell tools; suppressed
when cwd is inside session_dir; suppressed when cwd is inside
project_dir; suppressed when cwd is null.
All 3421 tests pass; slopwatch clean; file headers verified.
Cleanup pass on the approval-policy-v2 PR. Two related dead-code
removals that were marked "to delete in section 7" but never trimmed.
Dead v1 directory-extraction helpers:
- IShellApprovalSemantics.ExtractDirectoryRoots (interface + impl).
- ShellApprovalSemanticsBase.TryCreateDirectoryApprovalRoot,
ExtractDisplayDirectory, NormalizeDisplayDirectory,
IsRelativeDisplayPath, EnsureTrailingSeparator, CountPathSegments,
GetLastShellSeparatorIndex.
- PosixShellApprovalSemantics.ExtractDisplayDirectory and
EnsureTrailingSeparator overrides.
- ShellTokenizer.ExtractDirectoryRoots and MinDirectoryScopeDepth.
- DirectoryApprovalRoot record (file deleted).
- ShellTokenizerTests.ExtractDirectoryRoots_* test methods plus the
AbsoluteRootCases / RelativeRootCases / WindowsAbsoluteDirectoryRootCases
TheoryData properties that fed them.
These were the v1 "extract directory roots from path arguments" path.
v2 derives directory exclusively from ToolExecutionContext.Cwd so
nothing in production calls these anymore.
DirectoryRoots field plumbing:
- ToolApprovalContext, ToolInteractionRequest (SessionOutput),
SessionOutputDto.InteractionDirectoryRoots, the mapper round-trip,
PendingToolInteraction, IParentApprovalBridge.RequestApprovalAsync,
ParentSessionApprovalBridge, SubAgentActor caller, the pipeline emit
site, and the TUI rendering in ChatViewModel/ChatPage.
- All carriers always passed [], per the spec's "REMOVED Requirement:
Directory root extraction via IToolApprovalMatcher" and the section 7
prompt redesign which moved cwd into the prompt header.
Tests updated:
- DaemonClientMappingTests no longer round-trips DirectoryRoots.
- ParentSessionApprovalBridgeTests passes a real verb chain instead of
the synthetic "/tmp/work/logs/" placeholder it was carrying.
- ToolApprovalGateTests drops Assert.Empty(DirectoryRoots) calls that
only existed to document the empty-after-cutover state.
- ChatPage approval prompt rendering updated to the v2 button labels
("Once / This chat / Always here / Always anywhere / Deny").
3411 tests pass (10 fewer than before because the
ExtractDirectoryRoots_* test methods were removed; nothing else
changed). Slopwatch clean; file headers verified.
Followups from the simplification review pass. ApprovalEntry now owns its display + parse round-trip: - Format: ApprovalEntry.FormatScope() emits "<verb> in <dir>" or "<verb> anywhere". - Parse: ApprovalEntry.TryParseScope(input, out entry, out error) is the inverse, accepting only the two user-visible forms. - Both helpers replace duplicated implementations in ApprovalsCommand.FormatEntryForList, ApprovalsCommand.TryParseRevokePattern, and ApprovalDisplayItem.DisplayText. One round-trip source of truth. Hot-path: the actor's per-message file read. - ToolApprovalActor.GetUnapprovedPatterns now snapshots the persisted approvals once per message rather than re-reading + re-parsing tool-approvals.json per candidate verb. For a compound shell with N verbs that's N file reads → 1. Hot-path: per-verb cwd / safe-roots / symlink work. - ScopedShellSafeVerbPolicy.AllShortCircuit hoists Path.GetFullPath, ResolveSafeSpaceRoots, and ContainsSymlinkSegment out of the per-verb loop. The cwd doesn't change between verbs in the same invocation, so a 4-verb compound now does 1 path-normalize + 1 symlink scan instead of 4. ShortCircuitsApproval becomes a thin wrapper that forwards to AllShortCircuit. Wire ApprovalOptionKeys.IsDangerStyled in both channel builders instead of inlining the same `Deny or ApproveEverywhere` switch arm in two files. Consolidate WorkingDirectory/Command extraction in ShellApprovalMatcher to call ToolArgumentHelper.GetString — the helper already handles the PascalCase ↔ camelCase round-trip via key normalization, so the inline two-key TryGetValue duplication was needless and slightly inconsistent with the rest of the codebase. 3411 tests pass; slopwatch clean; file headers verified.
…approval
Section 10 of the approval-policy-v2 OpenSpec change. Adds a new
"Approval Policy v2" eval category covering the four behavioral
guardrails introduced in sections 5 + 9:
- approval_set_working_directory_positive
Project-scoped prompt mentions a repo path. Asserts the agent
calls set_working_directory before any shell tool call into that
tree (order check: SWD line < first shell_execute line).
- approval_set_working_directory_negative
Unrelated prompts ("what's 2+2?", "explain a hash table"). Asserts
the agent does NOT preemptively call set_working_directory just
because AGENTS.md mentions it.
- approval_recovery_hint (multi-turn)
T1 plants the cwd-outside-safe-spaces denial hint into the
conversation; T2 asserts the agent self-corrects by calling
set_working_directory rather than re-prompting the user. Scripting
an actual denial inside the eval container would require a
preconfigured project_dir mismatch we don't have plumbing for; the
hint-shape feed exercises the same self-correction code path.
- approval_schedule_pre_approval
User asks to schedule a daily reminder using the freshdesk verb.
Asserts the agent calls `netclaw approvals trust-verb freshdesk`
via shell_execute as part of schedule setup.
Task 10.1 cross-checked: "Pre-approving for unattended tasks
(load-bearing)" section in netclaw-operations SKILL.md (added in
section 9) covers the agent-driven trust-verb flow with example
dialogue. No additional skill text needed.
Task 10.6 (run the suite, document baseline pass rate) is deferred
to local execution — the suite needs NETCLAW_EVAL_PROVIDER_* env +
Docker daemon container which only Aaron has set up. Listed in
acceptance gates.
`bash -n evals/run-evals.sh` parses cleanly.
Folds the change's delta specs into main specs and archives the change to openspec/changes/archive/2026-05-08-approval-policy-v2/. - tool-approval-gates: rewrites shell pattern matching, persistent approval storage, and directory-root approvals around the v2 ApprovalEntry model; adds requirements for safe-verb short-circuit, five-button prompt, single-line resolution, and bash control-flow refusal. - session-cwd: adds shell tool cwd defaults, failure-path hint, and the safe-space expansion contract that set_working_directory now carries; modifies set_working_directory tool to reflect the new framing. Also fixes a pre-existing structural defect (spec was authored with the delta '## ADDED Requirements' heading instead of '## Purpose' + '## Requirements'). - netclaw-cli: replaces the Operator CLI for persistent tool approvals requirement with the v2 version (scope-labeled list, strict revoke parser, trust-verb subcommand, .v1.bak quarantine note).
Two bugs together meant evals never tested in-repo skill changes:
1. The skill scanner expects '<skills>/.system/<skill-name>/SKILL.md'
but the eval script copied to '.system/files/<skill-name>/SKILL.md'
(matching the repo's feeds/ layout, not the runtime layout). The
local copies were silently invisible.
2. The daemon then synced from the live R2 feed, which ships the last
released set of skills. So evals always exercised whatever was
published, not the source tree.
Result: a v2 'netclaw-operations' SKILL.md bumped in this PR was a
no-op for evals — the model in the container saw the older 1.x copy
from R2 and missed the new approval/trust-verb guidance entirely.
Fix:
- Copy '.../files/<skill>/' → '$EVAL_HOME/skills/.system/<skill>/'.
- Set 'NETCLAW_SkillSync__DisableSystemSkillSync=true' in the eval
container so the daemon doesn't fetch+overwrite from the live feed.
Confirmed via re-run: skill_load("netclaw-operations") now succeeds
inside the eval container (previously: "Skill not found"). The new
v2 approval cases ('approval_set_working_directory_positive',
'approval_schedule_pre_approval') visibly improve once the model can
see the bumped skill content.
Two cases had genuine eval-design problems independent of the v2
implementation, surfaced once N=5 baselines stabilized.
approval_set_working_directory_positive
Old prompt: 'I'm working on the Netclaw repository at /tmp. List the
files in that directory and tell me what's there.' This is ambiguous
between sustained project work (which the v2 spec says SHOULD pre-
declare) and a one-shot directory listing (which the spec explicitly
says should NOT pre-declare). The model going straight to shell was
arguably a correct read of the prompt, not a guidance failure.
New prompt makes the sustained-work signal explicit ('debugging
session... multiple shell commands across the tree').
approval_recovery_hint
Old structure was multi-turn: T1 fed a denial message and instructed
'do not call any tools yet', T2 said 'now call the tool.' Several
failure runs showed the model getting stuck in T1's no-tools
conditioning and refusing T2 ('I will not call any tools.'). That
tests prompt-flip resilience, not recovery-hint comprehension.
Rewrote as a single conversational prompt that delivers the denial
hint and asks 'how should I unblock this?' which is what a real
recovery turn looks like.
Side note for the PR: full N=5 baselines on local provider show the
inference endpoint is intermittently flaky (Dutchman-style stream
stalls — 'out=3 tok_s=8' instead of normal 'out=80 tok_s=27'),
which produces eval variance unrelated to either the v2
implementation or these prompts. Aaron will validate via binary-
swap before merging.
Aaronontheweb
force-pushed
the
openspec/approval-policy-v2
branch
from
c38eb08 to
1c96848
Compare
…ually scopes
ToolApprovalContext was missing a Cwd field, so SessionToolExecutionPipeline
emitted ToolInteractionRequest with Cwd=null even though ToolExecutionContext
already had it resolved. PendingToolInteraction.Cwd was therefore always null
on the session-actor side, and the persistence path
var persistCwd = decision == ApprovedEverywhere ? null : pending.Cwd;
silently turned every "Always here" click (ApprovedAlways) into a global
wildcard ("Always anywhere"). Confirmed in a live session: tool-approvals.json
contained nine entries, all with directory=null, despite most coming from
"Always here" button clicks.
The fix is small but the bug was load-bearing — folder-scoped trust is the
whole point of the v2 button row. Without cwd flowing through, the "here"
button was UX theater.
- Add 'Cwd' to ToolApprovalContext (string?, defaults null).
- Resolve cwd up-front in ToolAccessPolicy.CheckApprovalGate so it's
populated for every shell approval path (not only the safe-verb
short-circuit branch).
- Pass ctx.Cwd into ToolInteractionRequest.
- Regression test (SessionToolExecutionPipelineTests.Approval_request_
propagates_cwd_from_approval_context) asserts the emitted request
carries the cwd through.
Three bug-fix pillars on top of v2 (which has not deployed beyond a single dogfood operator): verb extraction emits the command head only, the first path-like argument becomes the candidate's effective directory, and pure side-effect clauses (echo / printf / true / false) authorize once but do not pollute persistence. The dogfood evidence (D0AC6CKBK5K/1778303523.861279) showed nine 'tool-approvals.json' entries that never matched future calls because each call's path got baked into the persisted verb. The fix reframes the (verb, directory) pair so verbs are reusable and paths declare scope implicitly. Persistence shape is unchanged. Includes proposal.md, design.md, and a delta spec for the 'tool-approval-gates' capability with all five MODIFIED/ADDED requirements covering verb classification, effective-directory matching, file-parent inference, multi-path tiebreak, and the side-effect skip list. Tasks are next.
Aaronontheweb
added
shell
…nontheweb/netclaw into openspec/approval-policy-v2
Seven sections covering path classification, matcher updates, persistence, side-effect skip list, agent guidance, tests, and the sync/archive flow. Acceptance gates include a manual binary-swap check that explicitly validates the dogfood failure mode (find /repo → Always here → find /repo/sub auto-runs).
Implements sections 1-4 of approval-policy-path-extraction. The verb
half of (verb, directory) is now the command head only; the directory
half comes from the first path-like argument when present, falling
back to cwd. Folder-scoped trust now compounds across deeper paths.
Section 1 — Path classification + verb-only extraction
- Add IsPathToken predicate (conservative: /, ~/, ./, ../ prefixes
or exact ~, ., ..). Internal-slash regexes and URLs are NOT paths.
- Strip the path-aware-append from ExtractVerbChain. For path-aware
verbs (cat, grep, find, ls, ...) we now stop at the first verb so
call-specific paths don't bake into the persisted verb chain.
- New ExtractFirstPathArgument applies the file-parent rule via
Path.HasExtension so cat ~/.bashrc resolves to "~", not "~/.bashrc".
- New ApprovalCandidate(Verb, Directory?) record + ExtractCandidates
on IToolApprovalMatcher.
Section 2 — Matcher uses effective directory
- New MatchesShellApproval overload taking
(candidateVerb, candidateDirectory, cwd, approvedEntries).
- Effective directory = candidateDirectory ?? cwd; relative paths
resolve against cwd via PathUtility.ExpandAndNormalize.
- Backwards-compat overload retained for v2.0 callers.
- Symlink-segment guard runs against the resolved effective directory.
Section 3 — Persistence on Always here uses effective directory
- New PersistApprovalCandidatesAsync in LlmSessionActor groups
candidates by effective directory and writes one
RecordApprovalAsync call per bucket. (find /repo + ls /repo) →
one (verb=find, dir=/repo) + one (verb=ls, dir=/repo) entry.
- ApprovedEverywhere collapses all candidates to (verb, null).
- ApprovedSession uses the same per-bucket grouping with persistent=false.
Section 4 — Side-effect skip list
- IsPureSideEffect(ApprovalCandidate) — true when verb is one of
{echo, printf, :, true, false} AND Directory is null. Redirect
detection is implicit: a redirect target shows up as the
candidate's directory via ExtractFirstPathArgument, so
echo X > /tmp/log persists as (echo, /tmp) instead of being skipped.
- LlmSessionActor's persistence loop drops side-effect candidates
entirely. They're authorized for the current call by the decision
but don't pollute the store.
Threading + protocol
- ApprovalCandidate flows ToolApprovalContext → ToolInteractionRequest
→ PendingToolInteraction → persistence. Existing CandidateVerbs
(verb-only) is now derived from Candidates for the renderers that
bullet-list verbs in the prompt body.
Tests
- 12 new ShellApprovalMatcherPathExtractionTests covering verb-only
extraction, file-parent rule, no-path-arg, compound clauses,
effective-directory matching, side-effect skip.
- ShellTokenizer tests updated for verb-only ExtractVerbChain.
- ToolApprovalGateTests updated for new candidate shape.
- Full suite: 3,463 tests passing across 7 projects, 0 failures.
Deferred (binary-swap validation):
- 3.2 per-candidate shallow-path skip + resolution-line note
- 3.3, 3.4 LlmSessionActor end-to-end persistence tests
- 4.4 resolution-line "Saved" vs "Authorized for this call"
- 4.5 same — matcher-level coverage exists, end-to-end deferred
Section 5 of approval-policy-path-extraction. The runtime now treats
path arguments as implicit scope declarations, and the agent-facing
guidance has to match or the model will keep over-using
set_working_directory and under-using folder-scoped trust.
- netclaw-operations SKILL.md (bumped 2.0.0 → 2.1.0):
- Rewrote `verb` / `directory` definitions to reflect the v2.1
extractor: verb is the command head only; directory comes from
the first path argument (with file-parent rule) when present,
else cwd.
- Added "Folder-scoped trust compounds" explanation so the model
knows (find, /repo) covers find /repo/sub.
- Added the side-effect-only clauses note (echo / printf / : / true
/ false get authorized but not persisted).
- Reframed the troubleshooting suggestions: read-only re-prompts in
a repo usually mean the commands aren't carrying a path arg;
set_working_directory is the fix for that case specifically.
- Resources/AGENTS.md:
- Renamed "Declare Your Project Root Early" to "Declaring Project
Scope" because the imperative no longer applies universally.
- Path arguments now declare scope implicitly. set_working_directory
is repositioned as the fallback for sessions running multiple
commands without explicit paths (REPL work, git status loops,
make -C / git -C flag-hidden paths).
- Kept the consequence framing ("burns the user's attention and
your token budget") for the case where the agent skips scope
declaration entirely.
- SetWorkingDirectoryTool description:
- Added a one-sentence note that shell commands with path arguments
declare scope automatically, framing the tool as most useful for
multi-command sessions without explicit paths.
Two corrections after applying sections 1-5 of approval-policy-path- extraction: - Revert the netclaw-operations SKILL.md version bump back to 2.0.0. v2.0.0 hasn't shipped to the live skills feed yet, so there's no installed-base to differentiate from. Bumping is only meaningful once the previous version has shipped. - Mark sections 6 (evals) and 7 (sync/archive) tasks as deferred- with-rationale rather than open. Eval cases for click-driven approval persistence can't be scripted from the eval framework (which only checks model output text), and the existing flaky inference provider blocks 6.1 re-runs anyway. Section 7 needs manual binary-swap validation before sync/archive.
Aaron's first dogfood click on Always anywhere crashed the turn
(corr id b48cc740). Trace:
ToolApprovalRequiredException: Tool 'shell_execute' requires approval
at DispatchingToolExecutor.AuthorizeCoreAsync
at DispatchingToolExecutor.ExecuteAsync
at SessionToolExecutionPipeline.ExecuteToolAttemptAsync
at SessionToolExecutionPipeline.ExecuteSingleToolAsync
at SessionToolExecutionPipeline.ExecuteToolsAsync
Sequence:
1. Compound command "cd ~/repo && git status && echo \"---\" &&
git remote -v && echo \"...\" && git branch ... && echo \"...\" &&
git log ..." prompts.
2. User clicks Always anywhere. Persistence runs:
- 5 entries written: cd, git status, git remote, git branch, git log
- echo skipped per side-effect rule (correct).
3. Pipeline retries the original call.
4. AuthorizeCoreAsync re-checks ALL 6 candidate verbs (incl. echo)
against the store. echo isn't there → unapproved.Count > 0 →
RequiresApproval → throw.
5. The throw escapes the catch at SessionToolExecutionPipeline.cs:269
because we're already inside that catch handling the ORIGINAL
exception. Sibling catches (line 381 / 393 / 404) don't apply
once execution is inside another catch. ExecuteToolsAsync's
outer Exception handler tells the actor to FailCurrentTurn.
Root cause: the side-effect skip handled persistence but not match-
time. Side-effect verbs need to be treated as authorized regardless
of whether they're in the store, so the matcher and persistence
agree on what's reachable.
Fixes:
- DispatchingToolExecutor.AuthorizeCoreAsync filters
IsPureSideEffect candidates from the unapproved-check verb list,
using approvalContext.Candidates when available. If every
candidate is side-effect-only, the decision is Allow up-front.
- ShellApprovalMatcher.IsApproved skips side-effect candidates in
the per-candidate match loop, mirroring the same intent at the
matcher boundary used by GetUnapprovedPatternsAsync.
- ShellTokenizer.SingleTokenSideEffectVerbs added; ExtractVerbChain
now caps at one token for these (in addition to PathAwareVerbs)
so "echo hello" resolves to verb "echo" instead of a 2-token
chain that the side-effect skip wouldn't match. Aaron's exact
dogfood case ("echo \"---X---\"") was already capped because
--- starts with - and breaks the loop on its own; the new cap
protects against shapes like build-script "echo done"-equivalents.
Tests:
- New ShellApprovalMatcher regression: IsApproved_treats_side_
effect_candidates_as_authorized covers the retry-after-Always-
anywhere case end-to-end at the matcher level.
- New ExtractCandidates_caps_echo_at_one_token ensures the
tokenizer cap holds.
- DispatchingToolExecutorTests.{One_time_approval_*, Session_
approval_*} updated to use git status (non-side-effect) so they
still exercise the approval flow rather than auto-allowing.
Full suite: 3,465 tests passing across 7 projects.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
OpenSpec change proposing a breaking redesign of the persistent tool-approval store and the Slack/Discord approval prompt UX. No code changes in this PR — the proposal, design, delta specs, and tasks list.
/opsx-applywork follows in subsequent PRs.What's in scope
(verb, directory)ApprovalEntryreplaces the v1 flat string list. v1 file quarantines to.v1.bakon first read; no automatic migration (we have no production users).safe-verbs.linux.json,safe-verbs.windows.json) plus the agent's existingsession_dirand optionalproject_dirform a three-position policy: auto-run when both axes match; prompt otherwise; hard-deny list unchanged. ReusesToolAudienceProfileResolverand the symlink-segment guard fromScopedFileAccessPolicy.project_dirif set, elsesession_dir. Today it inherits the daemon-process cwd, which is a footgun.ShellTokenizerreturns no verb chains forfor/while/do/done/etc. or unbalanced quotes/brackets. Approval gate offers onlyOnceandDenyfor messy input. Stops the on-disk store from accumulating fragments likedone,for pid,awk {print $2}).Once/This chat/Always here/Always anywhere/Deny, with danger styling onAlways anywhereandDeny. Replaces the v1Patterns+Directory Rootsbody sections with a singleApprove in <cwd>?header + verb bullets. Resolution message collapses to one line.netclaw approvals trust-verb <verb>writes a global wildcard(verb, null)entry;listandrevokelabel entries by scope (<verb> in <dir>/<verb> anywhere).set_working_directoryearly, tool-description rewrite, shell failure-path hint pointing atset_working_directory. Three new eval cases (positive, negative, recovery) lock adoption in. Schedule-creation flow proactively suggests pre-approval for unattended verbs.Phasing
The
tasks.mdplans two implementation PRs under this single change:Validation
Test plan
D0AC6CKBK5K/1778238489.065719?trust-verbis the only path to(verb, null)and that we're not regressing anything inlist/revoke.