Skip to content

feat: staged-context verification — a subagent declares what its workspace needs, the worker checks it before the agent runs - #152

Merged
BrettNye merged 19 commits into
mainfrom
spec/staged-context-verification
Aug 3, 2026
Merged

feat: staged-context verification — a subagent declares what its workspace needs, the worker checks it before the agent runs#152
BrettNye merged 19 commits into
mainfrom
spec/staged-context-verification

Conversation

@BrettNye

@BrettNye BrettNye commented Aug 3, 2026

Copy link
Copy Markdown
Contributor

Implements v1 of the staged-context design: contextRequires on subagent.register,
verified by the worker after pangolin-setup.sh and before baseline capture, failing the
dispatch when unmet.

Spec, DAG plan, gate-2 audit record, and implementation are all on this branch. Companion
decision spec in #151 (independent — this plan restates its argument inline).

The problem is drift, not ignorance

Agents are not under-informed; authored briefs are more explicit than any generated
manifest. What was missing is detection when a brief goes stale — bind a new toolchain
bundle and a brief's true statements silently become false. As the consumer put it: "I'll
keep hand-writing briefs; I can't hand-write detection."

Scope was inverted during design on their evidence. The first draft shipped a
declaration manifest with verification optional. contextShape already has five
declaration sites and zero readers — adding a sixth unread declaration to a system whose
problem is unread declarations would be net-negative. v1 is verification only.

Failing before the agent is about misattribution, not false greens

I originally justified this as preventing a verifier from reporting unearned success. The
consumer corrected it: their verifier answered a narrower question truthfully — they
lowered the question rather than getting a wrong answer. The real failure is
misattribution: an unmet requirement surfaces as command not found at agent time and
reads as a plan problem, four layers from the cause. That correction changed what the
feature aims at.

Design

ContextRequirement is observable-only by construction — paths (glob), exec
(binary on PATH), git (worktree / history). It deliberately cannot express
"patch applied"
, because verifying that means re-doing the apply rather than observing a
property. That is a stated non-goal, not a gap: it leaves half of KNOWN-ISSUES 17 open and
the spec says so plainly, because 17's framing is what invited the conflation.

It lives on the subagent def, which the worker already reads — not on SubagentShape,
which per ADR-0018 D11 the worker is ignorant of. Written only when set, so existing
subagent content hashes do not move.

Gate-2 audit: NOT READY, 8 blocking — all fixed before execution

7 lenses, no downgrades. The worst finding: the check failed open — an elided sibling
left paths/git with no push, so an unchecked requirement read as satisfied. Also: the
ordering pin did not discriminate (captureBaseline never commits, so history was
met:false under both orderings — re-based onto worktree, the only predicate that
flips); a fabricated test fence (five of six named files never call registerSubagent);
and fs.promises.glob, which typechecks on CI's Node 22 and throws on the worker's Node
20
.

Execution: 5/5 done, three needed a fix round

Reviewers running code found 7 more the prose audit could not:

  • A security finding. The worker spawned git with the full merged agent env,
    contradicting the invariant at runtime-env-filter.ts:24-26. GIT_DIR/GIT_WORK_TREE
    there override -C workspaceDir — the check could report met:true for a workspace
    with no git at all. Now buildGitEnv() + PATH, with a regression test that proves the
    bug first.
  • Two of my own plan instructions were wrong — the illustrative .kind — .observed
    snippet would never contain the binary name, failing its own acceptance criterion; and
    the prescribed hash test stayed green when the guard was deliberately broken.
  • A comment that lied about performancereaddir(recursive:true) completes the whole
    walk before the loop.

Verification worth naming

Reviewers ran mutations rather than reading assertions: deleting {recursive:true},
moving the check after captureBaseline, reverting the git env — each confirmed exactly
the intended test went red. The ordering constraint is enforced, not asserted.

One suggestion I applied turned out incomplete, caught by measuring in the Linux image:
X_OK rejects a non-executable file but a directory passes X_OK (the bit means
traversable). Added an isFile() stat and a regression test. Local Windows runs cannot
demonstrate either half — X_OK degrades to F_OK there — and the code says so.

Full repo tests green, pnpm -r lint and pnpm -r typecheck clean, docs build with all
internal links valid.

🤖 Generated with Claude Code

BrettNye and others added 18 commits August 3, 2026 10:19
Scope INVERTED from the first draft on consumer evidence, and the argument is
structural rather than preferential: a declaration nobody checks is exactly the
failure this area keeps producing. contextShape is a sentence with five declaration
sites and no reader; the consumer's own expectedExecutions recorded one capability
while binding two; a stable agent name pointed at three content hashes. Shipping
supply-declaration without verification would add a sixth unread declaration to a
system whose problem is unread declarations. Their line decides it: "I'll keep
hand-writing briefs; I can't hand-write detection."

So v1 is contextRequires + worker verification. Declaration and prompt-rendering are
v2 and ship only behind v1, because rendering an unverified declaration trades N
unchecked prose claims for one unchecked JSON claim.

THE PROBLEM IS DRIFT, NOT IGNORANCE. The agent is not under-informed — briefs are
more explicit than any generated manifest. What is missing is detection when a brief
goes stale: bind a new toolchain bundle and 22 true statements become false with
nothing noticing.

FAIL-FAST RATIONALE CORRECTED. The first draft justified failing before the agent as
preventing false greens. That is not what happened: the consumer's verifier answered
a NARROWER question truthfully, because they wrote criteria to what it could check.
The real failure is misattribution — a missing PATH yields "command not found" which
reads as a plan problem, and they lost a cycle to that class with a cause four layers
from the report. Justified by false-greens the feature aims at verifiers; justified
by misattribution it aims at every dispatch. They had reached the same conclusion
independently before seeing the proposal.

NON-GOAL STATED PLAINLY, because 17's framing invited the conflation: this does NOT
give a verifier the base tree and does NOT verify "patch applied". 17 bundled two
problems and this solves only the toolchain half. "Patch applied" is excluded because
checking it means re-doing the apply, not observing a property.

PLACEMENT is option A of the packs decision spec — contextRequires on the subagent
def, which the worker already reads. Explicitly NOT resting on both-paths coverage:
the first consumer is orchestrator-only and asked not to be counted as evidence for
that. The argument is only that the checker must be able to read the requirement.

Two v2 constraints recorded now so v1 does not foreclose them, both consumer-found:
declaration must be optional with absence meaning "declares nothing" rather than
"stages nothing" (a generated 200-file snapshot bundle has nothing sane to declare);
and array-union is the wrong merge rule — pangolin-setup.sh is last-write-wins with a
silent drop, so union declarations on top would assert both scripts while one ran.
Conflicting supply for the same path must refuse, not merge.

Every code citation self-checked against the tree.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
… only

Implements v1 of the 2026-08-03 staged-context spec. Verification only: no
declaration manifest, no prompt rendering. Those are v2 and ship behind v1, because
rendering an unverified declaration trades N unchecked prose claims for one unchecked
JSON claim in a system whose recurring defect is unread declarations.

SHAPE. Two roots collapse to one — the core contract fans out to the client
(persists) and the worker (evaluates), which rejoin at the lifecycle wiring, then
docs. Real parallelism at the middle layer; the chain is four deep, not linear.

PRE-AUTHORING GREP, per the anti-cascade rule. The subagent def is CONTENT-HASHED and
six client test files assert those hashes (cancel, describe, dispatch, dispatch-fire,
integration, retention). Adding a field naively would move every one of them. The
`verify` field at subagent-register.ts:79 is the exact hash-stable precedent — written
only when set — and task-client-accept-requires is scoped to mirror it, with an
acceptance criterion demanding a concrete hash equality against a value captured
before the edit rather than a "did not throw".

TWO ORDERING CONSTRAINTS ARE MADE EXECUTABLE RATHER THAN TRUSTED. The check must run
AFTER step 9, because pangolin-setup.sh is what installs a toolchain and an `exec`
requirement is unanswerable before it. It must run BEFORE captureBaseline, because
that helper runs `git init` (patch-capture.ts:19), after which every workspace has a
.git and `needs: 'history'` becomes trivially satisfiable. The second is pinned by an
acceptance criterion that would PASS INCORRECTLY if the order were wrong — the
executable form of the constraint, not a comment asserting it.

A subtlety worth the reviewer's attention: `exec` resolves against the MERGED runtime
env's PATH, not the worker's process.env. Checking the worker's PATH would answer a
different question than "can the agent run this", since filterRuntimeEnv may
legitimately drop entries the worker holds. Pinned by a criterion that clears
process.env.PATH for the call.

Every absence criterion carries a positive control in the same task — zero adapter
invocations on the failing run paired with exactly one on the passing run; a binary
found when PATH includes it paired with the same binary missed when it does not.

Validation: 5 unique ids, no cycles, no undefined deps, no file-scope conflicts,
6 mermaid edges matching 6 depends_on edges, nodes matching ids, H1/H2/H3/H4/H5/H6/H7
all clean (one wiring task exempt from H7 by declaration).

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…ocking

7/7 lenses ran, reconciled by one opus pass, no downgrades — all ten lens-proposed
blockings upheld and two DEFERRED promoted after a second lens falsified the
detection story each rested on.

The through-line is that four of my own verification claims do not hold. The check
fails OPEN because an elided sibling left 'paths' and 'git' with no push, so an
unchecked requirement reads as satisfied. The ordering pin does not discriminate
because captureBaseline never commits, making the criterion identical before and
after. The named hash fence was fabricated — five of six cited files never register
a subagent, and the real fence was omitted. And AC1 asserts on a detail field the
lifecycle event deliberately does not carry.

Plus a runtime split no test would catch: no glob engine is declared anywhere, and
the obvious choice typechecks on CI's Node 22 then throws on the worker's Node 20.

8 findings, 5 edit sites, 6 of them in two tasks. Findings are the author's to fix;
no task was mutated.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
All eight blocking findings addressed as five joint edits, plus every interaction
guard the reconciler flagged.

R-A — the evaluator is now EXHAUSTIVE BY CONSTRUCTION. The elided sibling that
caused B1 is gone: paths and git are written out, the switch has a `never` default
that pushes met:false, and an arity invariant (results.length === reqs.length) is an
acceptance criterion at both levels. A branch pushing nothing previously read as
SATISFIED downstream, which is the exact silent pass this module exists to remove.
worktree is now defined in the core doc comment rather than left to two implementers.

R-B — the ordering pin is re-based from history to WORKTREE, the only predicate that
flips across captureBaseline's git init. history could not serve: captureBaseline
never commits, so it was met:false under both orderings and the test passed either
way. The insertion point is corrected from "extending the cast at :534" (20 lines
AFTER captureBaseline, unreachable from before it) to a local cast immediately before
:514.

R-C — the glob engine is decided, not left open: export the existing matchesGlob from
overlay-engine.ts and pair it with readdir(recursive). The task now explicitly warns
off fs.promises.glob, which typechecks against @types/node 25 and passes on CI's Node
22, then throws in the Node-20 worker image. overlay-engine.ts joins the files list.

R-D — the hash fence is pinned to a literal I re-derived rather than to a
self-comparison: sha256:5001767f... pre-edit, and sha256:1ba525f0... if `?? []` leaks
in. The six-file claim is replaced with subagent-register.test.ts, the file that
actually registers; five of the six named before never call registerSubagent at all.

R-E — the wiring task names integration.test.ts as its harness, splits the two
observation surfaces (reason on the lifecycle event, detail on stdout NDJSON, since
detail is deliberately withheld from the event), and adds the capability-bundle and
inputs/<key> criteria that are the only ones positively binding workspaceDir to the
real staged directory.

Interaction guards, all five: the ordering pin and the .git-bundle case are required
to be two separate dispatches over two workspaces (a shared workspace would make
worktree met:true and silently destroy the pin — the fix-A-becomes-finding-B cycle);
the git-absent fail-closed divergence from captureBaseline's best-effort posture is
stated in Context so an implementer copying the neighbour is warned; the glob decision
lands with R-A rather than after; minCount short-circuits the walk so node_modules/**
does not cost the cycle it saves; and the jsonBytes idiom keeps pangolin-client out of
the worker, which also removes the spurious client -> entrypoint edge.

Net: +0 tasks, +1 file in scope, -1 edge, ~+120 lines. Re-validated: 5 unique ids, no
cycles, no undefined deps, no file-scope conflicts, 5 mermaid edges matching 5
depends_on, H1-H7 clean.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Defines the observable-only requirement vocabulary shared by client
(persists) and worker (evaluates): paths/exec/git kinds. Excludes
intent claims like 'patch applied' by design. Documents the
history-vs-worktree git.needs distinction that a downstream task's
ordering pin depends on.
…ntext-requirement

- Doc-comment glob root-relativity, minCount default (1, never 0), and
  exec.bin PATH-resolution semantics, matching the existing git.needs
  register.
- Test now imports from the package barrel (src/index.js) instead of
  the sub-file, so the barrel export is actually exercised.
- Rename rejected -> _rejected and drop the vacuous toBeDefined()
  assertion; note the @ts-expect-error guard is type-level only.
- Reword captureBaseline comment: worktree/history diverge only on a
  workspace that is not already a git repo, not on every call.
Persists contextRequires onto the content-hashed subagent def, written
only when set so existing subagent identities do not move.
…orkspace

Adds checkContextRequirements, returning one result per requirement
(exec/paths/git) so a missing branch can never read as satisfied
downstream. Exports overlay-engine's matchesGlob for reuse instead of
adding a glob dependency.
…isterSubagent

Test 1 hand-constructed a def object literal and hashed it directly,
independent of subagent-register.ts, so it stayed green even when the
guarded contextRequires assignment was patched to '?? []' (only test 3
caught that regression). Rewrite test 1 to call registerSubagent with
no capabilities/verify/contextRequires and assert the returned
handle's real contentHash against the same golden value. Verified the
new test goes red when the guard regresses to '?? []', then restored
the guard (subagent-register.ts content is unchanged, byte-identical
to HEAD).
…it, hardened git env, revert stray reformat

- Add nested-path recursion test (paths kind) pinning **/separator
  behavior; replace the materialize-then-break paths walk with a real
  early-exit directory-at-a-time walker that never reads past minCount
  and avoids Dirent.parentPath (Node 20/22 hazard).
- Spawn the git kind against buildGitEnv() + inherited PATH instead of
  the full merged runtime env, restoring the documented git-env
  invariant: GIT_DIR/GIT_WORK_TREE/GIT_COMMON_DIR from the merged env
  can no longer override -C workspaceDir and false-satisfy a
  requirement, and no agent credential rides along into a child
  process run against an agent-controlled .git tree. Test pins the
  GIT_DIR-override case.
- Mirror patch-capture.ts's killed-by-signal formatting in runGit's
  observed message.
- Revert the unrequested whole-file prettier reformat of
  overlay-engine.ts, keeping only the export keyword on matchesGlob
  and its doc-comment addition.
…e captureBaseline

Evaluates the subagent def's contextRequires after the setup script (step 9)
and before captureBaseline, failing the dispatch with reason: worker-failed
when any requirement is unmet. reason lives on the dispatch.failed lifecycle
event; detail (naming the unmet requirement) stays on stdout only, per the
existing redaction boundary.
…lity passes

Twelve suggestions across six files. All four tasks were APPROVED with these
outstanding; none blocked merge.

CORE CONTRACT. The glob DIALECT is now pinned, not just its base: `**` spans
segments, `*` stops at `/`, no braces, no character classes, no dotfile special
case — read off matchesGlob's actual implementation rather than assumed. Also
states outright that DIRECTORIES COUNT AS MATCHES, since `logs/**` being satisfied
by an empty `logs/sub/` is the kind of thing a gate author discovers the hard way.
minCount now says an out-of-contract value (< 1) is MALFORMED and must be rejected
rather than clamped — clamping turns a typo into a gate that can never fail. The
`exec` JSDoc moved inside the union member; before it sat ahead of the `|` and
documented nothing, so hovering `bin` showed no text.

The core test's "unenforced at runtime" was a category slip — a @ts-expect-error has
no runtime dimension at all. Reworded to name the real gap (no tsconfig.test.json)
and the real gate (any src/ construction site, which pangolin-worker now provides).

EXEC SEMANTICS — and the suggestion as given was INCOMPLETE. It proposed X_OK to
stop a non-executable file satisfying an `exec` requirement. Measured in the Linux
worker image before applying:

  access(0644 file, F_OK) -> pass    (why the default was wrong)
  access(0644 file, X_OK) -> fail    (X_OK fixes that half)
  access(directory,  X_OK) -> PASS   (X_OK does NOT fix the directory case)

On a directory the execute bit means "traversable", so the reviewer's own directory
example still passed under X_OK alone. Added an isFile() stat as well, and a
regression test for the directory case specifically — verified it bites by removing
the guard and confirming exactly one failure. Local runs cannot demonstrate the X_OK
half at all, since Windows has no execute bit and degrades X_OK to F_OK; the
comment records that so nobody reads a green Windows run as proof.

GIT ARM. Carries patch-capture.ts's three hardening flags (fsmonitor/pager/hooksPath).
The reviewer verified these are not live for `rev-parse`, and that holds — they are
added so the asymmetry with the file this arm otherwise mirrors is not a trap for
whoever adds a `status`/`log`/`diff` arm later, where they WOULD be live. runGit now
resolves on 'close' rather than 'exit': 'exit' can fire before stderr has flushed,
and here stderr IS the user-facing diagnostic rather than incidental error text.

WALK COST, stated rather than optimised. The short-circuit only helps the MET
direction; an unmet `paths` requirement still walks the whole tree uncapped on the
pre-agent path. That is bounded by one failing dispatch, so it is accepted — but the
header now says so plainly, with the pruning fix named, so the short-circuit is not
misread as making the miss case cheap.

TESTS. The fail-closed test promised "naming the cause" and asserted only
observed.length > 0, which any string satisfies including a generic `exited 128`;
now matches the specific unavailable-branch prefix. The GIT_DIR test's name claimed
GIT_WORK_TREE too and only set GIT_DIR; both are set now.

ENTRYPOINT + DOCS. The header's failure-mode table gains the new cause — mildly
ironic to have omitted it from a feature whose purpose is findable causes. The
client-api reference said `paths` matches "files"; corrected to "entries" with the
directory case spelled out, matching the core contract.

Full repo tests green, `pnpm -r lint` and `pnpm -r typecheck` clean, docs build
passing with all internal links valid.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…r the narrowed PATH

The `installs pnpm into $HOME` test is itPosix, so it never executes on
Windows and its first-ever run was on CI, where it failed at the very
first assertion (`expected 1 to be +0`).

Root cause, reproduced on Linux with a control: the test's env bundle
REPLACES PATH with `$HOME/bin` — a directory whose only occupant is the
stub the setup script is trying to create. Bare `mkdir` and `chmod` are
therefore unresolvable, the script dies with 127 having created nothing,
and the dispatch fails at step 9 (setup script) without ever reaching
the context check the test exists to exercise. Verified: exit 127 and no
directory under the narrow PATH; exit 0 and an executable stub with
`/bin`+`/usr/bin` added back.

Fixed by making the script's own commands absolute rather than by
widening the bundle's PATH. Widening would also go green, but at the
cost of the test's teeth: a real pnpm on a system directory would
satisfy the `exec` requirement and the assertion would pass without the
setup script having done anything.

Not a defect in the context check — X_OK is not implicated, since the
candidate file never comes into existence.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@BrettNye
BrettNye merged commit 29e5e30 into main Aug 3, 2026
5 checks passed
@BrettNye
BrettNye deleted the spec/staged-context-verification branch August 3, 2026 21:52
BrettNye added a commit that referenced this pull request Aug 3, 2026
…ce about a new posix-gated test (#153)

Harvested from #152, where a newly authored `itPosix` test failed on its
first-ever execution. Because Windows skips it, there was no prior green
to regress from — the local suite reported `6 passed | 1 skipped` and the
1 skipped was the only test covering the new path.

Records the cheap counter-move (reproduce the mechanism under
`docker run node:20`) and the near-miss: widening the test's PATH also
goes green while hollowing the test out.

Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
BrettNye added a commit that referenced this pull request Aug 4, 2026
…the rule (#159)

A `status:` line is written once, at authoring time, and nothing updates it when
the work lands. Measured across docs/superpowers on 2026-08-03:

  12 of 47 specs claimed draft / "design (plan pending)" / "DESIGNED — ready for
  a plan" for work already on main — including staged-context verification,
  which said "ready for a plan" the day after it merged in #152.

  2 plans still carried `status: pending` on every task for code shipped in May,
  under a header instructing agents to execute them. Following that header would
  have meant re-implementing the live DispatchExecutor and orchestrator.

Every correction cites the marker verified on main (`context-check.ts`,
`executors/dispatch.ts`, `secret-store/`, `audit/audit-log.ts`,
`mailbox/local-dir.ts`, `buildGitEnv`, `terminationSignal`, and so on), and keeps
the original text under **Originally:**. Only specs with positive code evidence
were touched: cron-trigger, the three callback specs, mneme integration and
worker-env-block-exposure were left alone, because their statuses are accurate —
that work genuinely is not built.

The two shipped plans get a HISTORICAL header rather than flipped task markers.
Flipping them would fake a per-task verification nobody performed; the markers
describe the plan at authoring time, and that is worth preserving.

Added docs/superpowers/README.md stating the rule — update the status in the PR
that lands the work — and explaining why there is deliberately no status index:
it would need the same maintenance these lines already fail to get, and would
become a second stale source disagreeing with the first.

Same class as the KNOWN-ISSUES backfill (#156): a marker that is not maintained
is worse than none, because it is believed.

Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant