feat(loop): separate builder and verifier roles with bounded repair convergence

[kunaldhongade]

Parent

Part of #675.

Problem

The existing codedecay loop can drive a user-owned agent and revalidate its edits. That proves orchestration, but one agent still receives the repair bundle and can effectively mark its own homework. Stopping is based on aggregate report progress rather than a requirement-and-hypothesis state machine, and the loop has limited budget, scope, and independence controls.

A 10/10 AI-native safety system needs an adversarial verifier that attempts to disprove the builder's result with trusted evidence.

User outcome

A developer supplies a task and chooses user-owned builder and verifier agents. The builder implements or repairs the change. The verifier independently challenges unmet requirements and consequence hypotheses, requests allowed experiments, and cannot declare success. CodeDecay stops only when the current tree has the required trusted proof or clearly explains why a human decision is required.

Acceptance criteria

• Add explicit builder and verifier roles with separate configured commands/providers, bundles, outputs, and identities.
• The verifier receives requirements, current-tree context, evidence, limitations, and observable behavior, but not the builder's hidden reasoning or an expected answer.
• Model the loop as a versioned state machine over requirement status, hypothesis status, experiment/check status, changed-tree fingerprint, and unresolved human decisions.
• Required transitions include plan, build/edit, analyze, challenge, verify, repair, current-tree reverify, and terminal verdict.
• The verifier can propose hypotheses and proof tasks but cannot edit code or mark a criterion verified.
• Only trusted deterministic, OSS-tool, or runtime evidence can move a criterion to verified or a hypothesis to confirmed/refuted.
• Enforce configurable maximum rounds, wall time, command time, model calls, token/cost budget when available, changed-file count, and allowed path scope.
• Detect oscillation, repeated patches, no evidence gain, widening scope, protected-path edits, and worsening verification.
• Terminal states distinguish at least verified, shallow-proof, unverified, stuck, budget-exhausted, unsafe-change, needs-human, builder-error, and verifier-error.
• Every successful edit is followed by current-tree analysis and required checks, including the final allowed round.
• Persist an inspectable per-round audit with agent identity, evidence IDs, commands, changed paths, decisions, budgets, and stop reason.
• Resume after interruption without rerunning successful destructive-looking setup or trusting stale evidence.
• Never auto-commit, push, merge, deploy, migrate production data, or silently revert user changes.

UAT contract

• UAT-LOOP-1: The builder makes an implementation that passes its unit test but violates a planted API requirement; the independent verifier finds it through evidence and the builder repairs it.
• UAT-LOOP-2: The final current tree passes the required experiment and requirement trace before verified.
• UAT-LOOP-3: A clean implementation finishes without unnecessary edits.
• UAT-LOOP-4: Oscillation, no progress, budget exhaustion, protected-path edits, and blocked commands stop with the correct status.
• UAT-LOOP-5: A verifier outage does not discard deterministic evidence or let the builder self-verify.
• UAT-LOOP-6: The packed npm CLI completes the workflow in an isolated child repository.

Agentic QA

CI uses deterministic fake builder and verifier processes with planted defects and clean decoys. Add an opt-in real-agent evaluation with at least one user-owned provider, recording model/configuration, budget, timing, edits, proof, and nondeterminism. The expected defect must not appear in the verifier prompt.

Dependencies

• Builds on feat(loop): add autonomous CodeDecay closed-loop orchestrator #610, feat(loop): make merge-safety verdict evidence-aware #614, fix(loop): revalidate final agent edits before reporting terminal state #665, feat(traceability): map requirements to implementation and verification evidence #669, feat(agent): add continuous before-during-after guidance sessions #677, feat(reasoning): turn agent investigation into falsifiable consequence hypotheses #678, and feat(verification): generate safe base-head behavioral experiments from agent hypotheses #679.

Validation

• transition, resume, budget, scope, oscillation, final freshness, and safety tests
• built CLI child-repository closed-loop UAT
• optional explicit real-agent run
• pnpm run lint
• pnpm typecheck
• pnpm test
• pnpm eval:pr-safety -- --run-id independent-loop
• pnpm build
• pnpm --filter @submuxhq/codedecay pack --dry-run