feat: staged-context verification — a subagent declares what its workspace needs, the worker checks it before the agent runs - #152
Merged
Conversation
Scope INVERTED from the first draft on consumer evidence, and the argument is structural rather than preferential: a declaration nobody checks is exactly the failure this area keeps producing. contextShape is a sentence with five declaration sites and no reader; the consumer's own expectedExecutions recorded one capability while binding two; a stable agent name pointed at three content hashes. Shipping supply-declaration without verification would add a sixth unread declaration to a system whose problem is unread declarations. Their line decides it: "I'll keep hand-writing briefs; I can't hand-write detection." So v1 is contextRequires + worker verification. Declaration and prompt-rendering are v2 and ship only behind v1, because rendering an unverified declaration trades N unchecked prose claims for one unchecked JSON claim. THE PROBLEM IS DRIFT, NOT IGNORANCE. The agent is not under-informed — briefs are more explicit than any generated manifest. What is missing is detection when a brief goes stale: bind a new toolchain bundle and 22 true statements become false with nothing noticing. FAIL-FAST RATIONALE CORRECTED. The first draft justified failing before the agent as preventing false greens. That is not what happened: the consumer's verifier answered a NARROWER question truthfully, because they wrote criteria to what it could check. The real failure is misattribution — a missing PATH yields "command not found" which reads as a plan problem, and they lost a cycle to that class with a cause four layers from the report. Justified by false-greens the feature aims at verifiers; justified by misattribution it aims at every dispatch. They had reached the same conclusion independently before seeing the proposal. NON-GOAL STATED PLAINLY, because 17's framing invited the conflation: this does NOT give a verifier the base tree and does NOT verify "patch applied". 17 bundled two problems and this solves only the toolchain half. "Patch applied" is excluded because checking it means re-doing the apply, not observing a property. PLACEMENT is option A of the packs decision spec — contextRequires on the subagent def, which the worker already reads. Explicitly NOT resting on both-paths coverage: the first consumer is orchestrator-only and asked not to be counted as evidence for that. The argument is only that the checker must be able to read the requirement. Two v2 constraints recorded now so v1 does not foreclose them, both consumer-found: declaration must be optional with absence meaning "declares nothing" rather than "stages nothing" (a generated 200-file snapshot bundle has nothing sane to declare); and array-union is the wrong merge rule — pangolin-setup.sh is last-write-wins with a silent drop, so union declarations on top would assert both scripts while one ran. Conflicting supply for the same path must refuse, not merge. Every code citation self-checked against the tree. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
… only Implements v1 of the 2026-08-03 staged-context spec. Verification only: no declaration manifest, no prompt rendering. Those are v2 and ship behind v1, because rendering an unverified declaration trades N unchecked prose claims for one unchecked JSON claim in a system whose recurring defect is unread declarations. SHAPE. Two roots collapse to one — the core contract fans out to the client (persists) and the worker (evaluates), which rejoin at the lifecycle wiring, then docs. Real parallelism at the middle layer; the chain is four deep, not linear. PRE-AUTHORING GREP, per the anti-cascade rule. The subagent def is CONTENT-HASHED and six client test files assert those hashes (cancel, describe, dispatch, dispatch-fire, integration, retention). Adding a field naively would move every one of them. The `verify` field at subagent-register.ts:79 is the exact hash-stable precedent — written only when set — and task-client-accept-requires is scoped to mirror it, with an acceptance criterion demanding a concrete hash equality against a value captured before the edit rather than a "did not throw". TWO ORDERING CONSTRAINTS ARE MADE EXECUTABLE RATHER THAN TRUSTED. The check must run AFTER step 9, because pangolin-setup.sh is what installs a toolchain and an `exec` requirement is unanswerable before it. It must run BEFORE captureBaseline, because that helper runs `git init` (patch-capture.ts:19), after which every workspace has a .git and `needs: 'history'` becomes trivially satisfiable. The second is pinned by an acceptance criterion that would PASS INCORRECTLY if the order were wrong — the executable form of the constraint, not a comment asserting it. A subtlety worth the reviewer's attention: `exec` resolves against the MERGED runtime env's PATH, not the worker's process.env. Checking the worker's PATH would answer a different question than "can the agent run this", since filterRuntimeEnv may legitimately drop entries the worker holds. Pinned by a criterion that clears process.env.PATH for the call. Every absence criterion carries a positive control in the same task — zero adapter invocations on the failing run paired with exactly one on the passing run; a binary found when PATH includes it paired with the same binary missed when it does not. Validation: 5 unique ids, no cycles, no undefined deps, no file-scope conflicts, 6 mermaid edges matching 6 depends_on edges, nodes matching ids, H1/H2/H3/H4/H5/H6/H7 all clean (one wiring task exempt from H7 by declaration). Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…ocking 7/7 lenses ran, reconciled by one opus pass, no downgrades — all ten lens-proposed blockings upheld and two DEFERRED promoted after a second lens falsified the detection story each rested on. The through-line is that four of my own verification claims do not hold. The check fails OPEN because an elided sibling left 'paths' and 'git' with no push, so an unchecked requirement reads as satisfied. The ordering pin does not discriminate because captureBaseline never commits, making the criterion identical before and after. The named hash fence was fabricated — five of six cited files never register a subagent, and the real fence was omitted. And AC1 asserts on a detail field the lifecycle event deliberately does not carry. Plus a runtime split no test would catch: no glob engine is declared anywhere, and the obvious choice typechecks on CI's Node 22 then throws on the worker's Node 20. 8 findings, 5 edit sites, 6 of them in two tasks. Findings are the author's to fix; no task was mutated. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
All eight blocking findings addressed as five joint edits, plus every interaction guard the reconciler flagged. R-A — the evaluator is now EXHAUSTIVE BY CONSTRUCTION. The elided sibling that caused B1 is gone: paths and git are written out, the switch has a `never` default that pushes met:false, and an arity invariant (results.length === reqs.length) is an acceptance criterion at both levels. A branch pushing nothing previously read as SATISFIED downstream, which is the exact silent pass this module exists to remove. worktree is now defined in the core doc comment rather than left to two implementers. R-B — the ordering pin is re-based from history to WORKTREE, the only predicate that flips across captureBaseline's git init. history could not serve: captureBaseline never commits, so it was met:false under both orderings and the test passed either way. The insertion point is corrected from "extending the cast at :534" (20 lines AFTER captureBaseline, unreachable from before it) to a local cast immediately before :514. R-C — the glob engine is decided, not left open: export the existing matchesGlob from overlay-engine.ts and pair it with readdir(recursive). The task now explicitly warns off fs.promises.glob, which typechecks against @types/node 25 and passes on CI's Node 22, then throws in the Node-20 worker image. overlay-engine.ts joins the files list. R-D — the hash fence is pinned to a literal I re-derived rather than to a self-comparison: sha256:5001767f... pre-edit, and sha256:1ba525f0... if `?? []` leaks in. The six-file claim is replaced with subagent-register.test.ts, the file that actually registers; five of the six named before never call registerSubagent at all. R-E — the wiring task names integration.test.ts as its harness, splits the two observation surfaces (reason on the lifecycle event, detail on stdout NDJSON, since detail is deliberately withheld from the event), and adds the capability-bundle and inputs/<key> criteria that are the only ones positively binding workspaceDir to the real staged directory. Interaction guards, all five: the ordering pin and the .git-bundle case are required to be two separate dispatches over two workspaces (a shared workspace would make worktree met:true and silently destroy the pin — the fix-A-becomes-finding-B cycle); the git-absent fail-closed divergence from captureBaseline's best-effort posture is stated in Context so an implementer copying the neighbour is warned; the glob decision lands with R-A rather than after; minCount short-circuits the walk so node_modules/** does not cost the cycle it saves; and the jsonBytes idiom keeps pangolin-client out of the worker, which also removes the spurious client -> entrypoint edge. Net: +0 tasks, +1 file in scope, -1 edge, ~+120 lines. Re-validated: 5 unique ids, no cycles, no undefined deps, no file-scope conflicts, 5 mermaid edges matching 5 depends_on, H1-H7 clean. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Defines the observable-only requirement vocabulary shared by client (persists) and worker (evaluates): paths/exec/git kinds. Excludes intent claims like 'patch applied' by design. Documents the history-vs-worktree git.needs distinction that a downstream task's ordering pin depends on.
…ntext-requirement - Doc-comment glob root-relativity, minCount default (1, never 0), and exec.bin PATH-resolution semantics, matching the existing git.needs register. - Test now imports from the package barrel (src/index.js) instead of the sub-file, so the barrel export is actually exercised. - Rename rejected -> _rejected and drop the vacuous toBeDefined() assertion; note the @ts-expect-error guard is type-level only. - Reword captureBaseline comment: worktree/history diverge only on a workspace that is not already a git repo, not on every call.
… + worker in parallel)
Persists contextRequires onto the content-hashed subagent def, written only when set so existing subagent identities do not move.
…orkspace Adds checkContextRequirements, returning one result per requirement (exec/paths/git) so a missing branch can never read as satisfied downstream. Exports overlay-engine's matchesGlob for reuse instead of adding a glob dependency.
…isterSubagent Test 1 hand-constructed a def object literal and hashed it directly, independent of subagent-register.ts, so it stayed green even when the guarded contextRequires assignment was patched to '?? []' (only test 3 caught that regression). Rewrite test 1 to call registerSubagent with no capabilities/verify/contextRequires and assert the returned handle's real contentHash against the same golden value. Verified the new test goes red when the guard regresses to '?? []', then restored the guard (subagent-register.ts content is unchanged, byte-identical to HEAD).
…it, hardened git env, revert stray reformat - Add nested-path recursion test (paths kind) pinning **/separator behavior; replace the materialize-then-break paths walk with a real early-exit directory-at-a-time walker that never reads past minCount and avoids Dirent.parentPath (Node 20/22 hazard). - Spawn the git kind against buildGitEnv() + inherited PATH instead of the full merged runtime env, restoring the documented git-env invariant: GIT_DIR/GIT_WORK_TREE/GIT_COMMON_DIR from the merged env can no longer override -C workspaceDir and false-satisfy a requirement, and no agent credential rides along into a child process run against an agent-controlled .git tree. Test pins the GIT_DIR-override case. - Mirror patch-capture.ts's killed-by-signal formatting in runGit's observed message. - Revert the unrequested whole-file prettier reformat of overlay-engine.ts, keeping only the export keyword on matchesGlob and its doc-comment addition.
…e captureBaseline Evaluates the subagent def's contextRequires after the setup script (step 9) and before captureBaseline, failing the dispatch with reason: worker-failed when any requirement is unmet. reason lives on the dispatch.failed lifecycle event; detail (naming the unmet requirement) stays on stdout only, per the existing redaction boundary.
…lity passes Twelve suggestions across six files. All four tasks were APPROVED with these outstanding; none blocked merge. CORE CONTRACT. The glob DIALECT is now pinned, not just its base: `**` spans segments, `*` stops at `/`, no braces, no character classes, no dotfile special case — read off matchesGlob's actual implementation rather than assumed. Also states outright that DIRECTORIES COUNT AS MATCHES, since `logs/**` being satisfied by an empty `logs/sub/` is the kind of thing a gate author discovers the hard way. minCount now says an out-of-contract value (< 1) is MALFORMED and must be rejected rather than clamped — clamping turns a typo into a gate that can never fail. The `exec` JSDoc moved inside the union member; before it sat ahead of the `|` and documented nothing, so hovering `bin` showed no text. The core test's "unenforced at runtime" was a category slip — a @ts-expect-error has no runtime dimension at all. Reworded to name the real gap (no tsconfig.test.json) and the real gate (any src/ construction site, which pangolin-worker now provides). EXEC SEMANTICS — and the suggestion as given was INCOMPLETE. It proposed X_OK to stop a non-executable file satisfying an `exec` requirement. Measured in the Linux worker image before applying: access(0644 file, F_OK) -> pass (why the default was wrong) access(0644 file, X_OK) -> fail (X_OK fixes that half) access(directory, X_OK) -> PASS (X_OK does NOT fix the directory case) On a directory the execute bit means "traversable", so the reviewer's own directory example still passed under X_OK alone. Added an isFile() stat as well, and a regression test for the directory case specifically — verified it bites by removing the guard and confirming exactly one failure. Local runs cannot demonstrate the X_OK half at all, since Windows has no execute bit and degrades X_OK to F_OK; the comment records that so nobody reads a green Windows run as proof. GIT ARM. Carries patch-capture.ts's three hardening flags (fsmonitor/pager/hooksPath). The reviewer verified these are not live for `rev-parse`, and that holds — they are added so the asymmetry with the file this arm otherwise mirrors is not a trap for whoever adds a `status`/`log`/`diff` arm later, where they WOULD be live. runGit now resolves on 'close' rather than 'exit': 'exit' can fire before stderr has flushed, and here stderr IS the user-facing diagnostic rather than incidental error text. WALK COST, stated rather than optimised. The short-circuit only helps the MET direction; an unmet `paths` requirement still walks the whole tree uncapped on the pre-agent path. That is bounded by one failing dispatch, so it is accepted — but the header now says so plainly, with the pruning fix named, so the short-circuit is not misread as making the miss case cheap. TESTS. The fail-closed test promised "naming the cause" and asserted only observed.length > 0, which any string satisfies including a generic `exited 128`; now matches the specific unavailable-branch prefix. The GIT_DIR test's name claimed GIT_WORK_TREE too and only set GIT_DIR; both are set now. ENTRYPOINT + DOCS. The header's failure-mode table gains the new cause — mildly ironic to have omitted it from a feature whose purpose is findable causes. The client-api reference said `paths` matches "files"; corrected to "entries" with the directory case spelled out, matching the core contract. Full repo tests green, `pnpm -r lint` and `pnpm -r typecheck` clean, docs build passing with all internal links valid. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…r the narrowed PATH The `installs pnpm into $HOME` test is itPosix, so it never executes on Windows and its first-ever run was on CI, where it failed at the very first assertion (`expected 1 to be +0`). Root cause, reproduced on Linux with a control: the test's env bundle REPLACES PATH with `$HOME/bin` — a directory whose only occupant is the stub the setup script is trying to create. Bare `mkdir` and `chmod` are therefore unresolvable, the script dies with 127 having created nothing, and the dispatch fails at step 9 (setup script) without ever reaching the context check the test exists to exercise. Verified: exit 127 and no directory under the narrow PATH; exit 0 and an executable stub with `/bin`+`/usr/bin` added back. Fixed by making the script's own commands absolute rather than by widening the bundle's PATH. Widening would also go green, but at the cost of the test's teeth: a real pnpm on a system directory would satisfy the `exec` requirement and the assertion would pass without the setup script having done anything. Not a defect in the context check — X_OK is not implicated, since the candidate file never comes into existence. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
BrettNye
added a commit
that referenced
this pull request
Aug 3, 2026
…ce about a new posix-gated test (#153) Harvested from #152, where a newly authored `itPosix` test failed on its first-ever execution. Because Windows skips it, there was no prior green to regress from — the local suite reported `6 passed | 1 skipped` and the 1 skipped was the only test covering the new path. Records the cheap counter-move (reproduce the mechanism under `docker run node:20`) and the near-miss: widening the test's PATH also goes green while hollowing the test out. Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
This was referenced Aug 4, 2026
BrettNye
added a commit
that referenced
this pull request
Aug 4, 2026
…the rule (#159) A `status:` line is written once, at authoring time, and nothing updates it when the work lands. Measured across docs/superpowers on 2026-08-03: 12 of 47 specs claimed draft / "design (plan pending)" / "DESIGNED — ready for a plan" for work already on main — including staged-context verification, which said "ready for a plan" the day after it merged in #152. 2 plans still carried `status: pending` on every task for code shipped in May, under a header instructing agents to execute them. Following that header would have meant re-implementing the live DispatchExecutor and orchestrator. Every correction cites the marker verified on main (`context-check.ts`, `executors/dispatch.ts`, `secret-store/`, `audit/audit-log.ts`, `mailbox/local-dir.ts`, `buildGitEnv`, `terminationSignal`, and so on), and keeps the original text under **Originally:**. Only specs with positive code evidence were touched: cron-trigger, the three callback specs, mneme integration and worker-env-block-exposure were left alone, because their statuses are accurate — that work genuinely is not built. The two shipped plans get a HISTORICAL header rather than flipped task markers. Flipping them would fake a per-task verification nobody performed; the markers describe the plan at authoring time, and that is worth preserving. Added docs/superpowers/README.md stating the rule — update the status in the PR that lands the work — and explaining why there is deliberately no status index: it would need the same maintenance these lines already fail to get, and would become a second stale source disagreeing with the first. Same class as the KNOWN-ISSUES backfill (#156): a marker that is not maintained is worse than none, because it is believed. Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Implements v1 of the staged-context design:
contextRequiresonsubagent.register,verified by the worker after
pangolin-setup.shand before baseline capture, failing thedispatch when unmet.
Spec, DAG plan, gate-2 audit record, and implementation are all on this branch. Companion
decision spec in #151 (independent — this plan restates its argument inline).
The problem is drift, not ignorance
Agents are not under-informed; authored briefs are more explicit than any generated
manifest. What was missing is detection when a brief goes stale — bind a new toolchain
bundle and a brief's true statements silently become false. As the consumer put it: "I'll
keep hand-writing briefs; I can't hand-write detection."
Scope was inverted during design on their evidence. The first draft shipped a
declaration manifest with verification optional.
contextShapealready has fivedeclaration sites and zero readers — adding a sixth unread declaration to a system whose
problem is unread declarations would be net-negative. v1 is verification only.
Failing before the agent is about misattribution, not false greens
I originally justified this as preventing a verifier from reporting unearned success. The
consumer corrected it: their verifier answered a narrower question truthfully — they
lowered the question rather than getting a wrong answer. The real failure is
misattribution: an unmet requirement surfaces as
command not foundat agent time andreads as a plan problem, four layers from the cause. That correction changed what the
feature aims at.
Design
ContextRequirementis observable-only by construction —paths(glob),exec(binary on
PATH),git(worktree/history). It deliberately cannot express"patch applied", because verifying that means re-doing the apply rather than observing a
property. That is a stated non-goal, not a gap: it leaves half of KNOWN-ISSUES 17 open and
the spec says so plainly, because 17's framing is what invited the conflation.
It lives on the subagent def, which the worker already reads — not on
SubagentShape,which per ADR-0018 D11 the worker is ignorant of. Written only when set, so existing
subagent content hashes do not move.
Gate-2 audit: NOT READY, 8 blocking — all fixed before execution
7 lenses, no downgrades. The worst finding: the check failed open — an elided sibling
left
paths/gitwith nopush, so an unchecked requirement read as satisfied. Also: theordering pin did not discriminate (
captureBaselinenever commits, sohistorywasmet:falseunder both orderings — re-based ontoworktree, the only predicate thatflips); a fabricated test fence (five of six named files never call
registerSubagent);and
fs.promises.glob, which typechecks on CI's Node 22 and throws on the worker's Node20.
Execution: 5/5 done, three needed a fix round
Reviewers running code found 7 more the prose audit could not:
gitwith the full merged agent env,contradicting the invariant at
runtime-env-filter.ts:24-26.GIT_DIR/GIT_WORK_TREEthere override
-C workspaceDir— the check could reportmet:truefor a workspacewith no git at all. Now
buildGitEnv()+PATH, with a regression test that proves thebug first.
.kind — .observedsnippet would never contain the binary name, failing its own acceptance criterion; and
the prescribed hash test stayed green when the guard was deliberately broken.
readdir(recursive:true)completes the wholewalk before the loop.
Verification worth naming
Reviewers ran mutations rather than reading assertions: deleting
{recursive:true},moving the check after
captureBaseline, reverting the git env — each confirmed exactlythe intended test went red. The ordering constraint is enforced, not asserted.
One suggestion I applied turned out incomplete, caught by measuring in the Linux image:
X_OKrejects a non-executable file but a directory passes X_OK (the bit meanstraversable). Added an
isFile()stat and a regression test. Local Windows runs cannotdemonstrate either half — X_OK degrades to F_OK there — and the code says so.
Full repo tests green,
pnpm -r lintandpnpm -r typecheckclean, docs build with allinternal links valid.
🤖 Generated with Claude Code