[EPIC] Continuous verification and earned autonomy

[Joncallim]

Issue Type

Product / platform Epic

Context

Forge now has structured planning, durable work packages, bounded execution, approval gates, execution artifacts, GitHub-native issue/PR workflows, and current MCP admission work. The next trust layer should answer two questions continuously:

1. What still works?
2. Which narrowly scoped capabilities have earned more autonomy through verified performance?

This Epic introduces a deterministic operation catalog, continuous proof checks, independent verification, a capability reliability ledger, and evidence-based autonomy promotion/revocation. It extends the existing Architect → Work Package → Worker → Review flow rather than creating a second orchestration system.

The design is inspired by Scout → Manager → Worker → Inspector loops, but maps them into existing Forge concepts:

• Project Sentinel detects actionable changes or regressions.
• Architect converts findings into bounded work packages and typed operation requests.
• Operation Catalog / Worker runtime validates and executes approved deterministic operations.
• Verification Workforce independently checks the result.

Desired Outcome

Forge can select and execute typed deterministic operations, continuously verify project goals, record structured execution and verification evidence, calculate reliability for specific capabilities and scopes, grant only bounded earned autonomy, and automatically revoke that autonomy when reliability or safety evidence deteriorates.

Autonomy must be:

• capability- and operation-specific, not agent-global;
• project/scope-specific;
• policy-versioned;
• evidence-backed;
• time-bounded or periodically requalified;
• immediately revocable after critical failures or policy violations;
• visible and overridable by the human operator.

Design Principles

1. Models select; Forge executes. Agents request approved typed operations rather than inventing arbitrary command strings.
2. Transport success is not task success. A successful API/ACP response can still represent refusal, blocking, invalid output, or incomplete work.
3. Verification is independent. The worker must not be the sole grader of its own output.
4. Deterministic signals first. Scheduled checks inspect CI, tests, task state, goals, and repository evidence before invoking an LLM.
5. Evidence before autonomy. Promotions use comparable historical attempts, objective verification, recency, and safety history.
6. Fail closed. Missing or stale definitions, policy, scope, or evidence must never silently increase permission.
7. No global trust score. Reliability belongs to a capability + operation + scope + runtime/model/policy context.
8. Human authority remains final. Operators can cap, revoke, or override autonomy.

Dependencies And Relationships

• Builds on the executable Workforce and work-package data model ([EPIC] Executable Workforce Beta: capability-brokered sequential specialist execution #119 and related implementation).
• Should consume the unified MCP admission decisions from [EPIC] MCP Execution Readiness and Bounded Context Grants #172 rather than bypass them.
• Uses [FEATURE] Add deterministic operation catalog and typed execution harness #201 as the deterministic execution surface beneath earned autonomy.
• Complements cross-agent checkpoints ([FEATURE] Add cross-agent continuation and checkpointing #32), the AI-native backlog (Build an AI-native issue tracker with GitHub sync and automatic Epic/Milestone grouping #42), adversarial mode ([FEATURE] Add adversarial agent mode with default rules and project override editor #40), and security/verification workforce architecture (Add security review workforce architecture and subagent taxonomy #60).
• Reuses tasks, task_attempts, work_packages, agent_runs, artifacts, approval gates, and repository evidence rather than creating parallel execution truth.
• Does not replace the closed GitHub-native workflow Epic [EPIC] GitHub-native agent workflow #141; GitHub remains the durable implementation record.

Tasks

• [FEATURE] Add deterministic operation catalog and typed execution harness #201
• [FEATURE] Normalize execution outcomes and stop reasons #185
• [FEATURE] Add capability reliability ledger #186
• [FEATURE] Add project verification goals and scheduled proof runs #187
• [FEATURE] Add independent Verification Workforce execution #188
• [FEATURE] Add evidence-based earned autonomy policy engine #189
• [FEATURE] Add Project Sentinel detection and escalation flow #190
• [FEATURE] Add reliability, autonomy, and regression reporting #191

Child Issues

• [FEATURE] Add deterministic operation catalog and typed execution harness #201 — Add deterministic operation catalog and typed execution harness
• [FEATURE] Normalize execution outcomes and stop reasons #185 — Normalize execution outcomes and stop reasons
• [FEATURE] Add capability reliability ledger #186 — Add capability reliability ledger
• [FEATURE] Add project verification goals and scheduled proof runs #187 — Add project verification goals and scheduled proof runs
• [FEATURE] Add independent Verification Workforce execution #188 — Add independent Verification Workforce execution
• [FEATURE] Add evidence-based earned autonomy policy engine #189 — Add evidence-based earned autonomy policy engine
• [FEATURE] Add Project Sentinel detection and escalation flow #190 — Add Project Sentinel detection and escalation flow
• [FEATURE] Add reliability, autonomy, and regression reporting #191 — Add reliability, autonomy, and regression reporting

Acceptance Criteria

• Models request versioned typed operations rather than directly supplying arbitrary executable command strings.
• Forge validates operation inputs, scope, capability, policy, and verification before and after execution.
• Forge records a canonical structured outcome for execution, verification, refusal, blocking, and failure paths.
• Comparable capability attempts produce a durable evidence-backed reliability history.
• Autonomy policies can promote, hold, demote, expire, and revoke scoped capabilities and operations.
• Projects can define durable “what still works” verification goals and run them on demand or schedule.
• Project Sentinel detects deterministic regressions and escalates them without attempting speculative repairs first.
• Verification can run independently from the implementation worker and persist findings/evidence.
• Operators can inspect reliability, autonomy state, regressions, and the evidence behind every policy decision.
• Critical failures, policy violations, or missing verification can revoke autonomy immediately.
• Initial delivery does not auto-merge changes or grant broad host/system permissions.

Out of Scope

• Arbitrary model-authored shell execution as an autonomy mechanism.
• Global autonomy for an agent or model.
• Fully autonomous repair of every detected regression.
• Auto-merge based only on a success percentage.
• Worker self-grading as the sole source of reliability.
• Broad host-level cron scripts with unrestricted filesystem or credential access.
• Replacing GitHub, existing work packages, approval gates, MCP admission, or the Forge task state machine.
• A polished dashboard before the underlying contracts and evidence paths are reliable.

Technical Notes

Suggested core entities:

• versioned operation definitions/requests/runs from [FEATURE] Add deterministic operation catalog and typed execution harness #201;
• execution_outcomes or an equivalent canonical outcome contract linked to task/work-package/agent run;
• capability_attempts for comparable execution and verification history;
• autonomy_policies and autonomy_decisions scoped by project, capability, operation, resource scope, runtime/model, and policy version;
• verification_goals and verification_goal_runs for scheduled/on-demand proof checks;
• sentinel_findings for deterministic detection and escalation evidence;
• verification artifacts/findings linked through existing agent_runs and artifacts where possible.

Suggested autonomy ladder:

• L0 — plan only;
• L1 — execute a typed operation in sandbox after approval;
• L2 — execute bounded low-risk operations, human reviews result;
• L3 — create branch and draft PR after verification;
• L4 — open ready-for-review PR after independent verification;
• L5 — reserved for narrowly defined reversible operations; not general auto-merge.

Recommended implementation order:

0. [FEATURE] Add deterministic operation catalog and typed execution harness #201 — Add deterministic operation catalog and typed execution harness.
1. [FEATURE] Normalize execution outcomes and stop reasons #185 — Normalize execution outcomes and stop reasons.
2. [FEATURE] Add capability reliability ledger #186 — Add capability reliability ledger.
3. [FEATURE] Add project verification goals and scheduled proof runs #187 — Add project verification goals and proof runs.
4. [FEATURE] Add independent Verification Workforce execution #188 — Add independent Verification Workforce execution.
5. [FEATURE] Add evidence-based earned autonomy policy engine #189 — Add earned-autonomy policy and enforcement.
6. [FEATURE] Add Project Sentinel detection and escalation flow #190 — Add Project Sentinel and escalation.
7. [FEATURE] Add reliability, autonomy, and regression reporting #191 — Add evidence-first reporting.

Implementation Scope

Large — cross-cutting platform Epic. Start with typed operations, contracts, persistence, and deterministic tests before automation or UI.

[Joncallim]

Roadmap extension opened in PR #192.

Repository path: docs/continuous-verification-and-earned-autonomy-roadmap.md

The document positions this Epic as a P2.5 trust layer between bounded Workforce/MCP execution and broader Forge Workspace expansion, and links implementation phases #185–#191.