Skip to content

[FEATURE] Add independent Verification Workforce execution #188

Description

@Joncallim

Problem Statement

Forge currently has manual QA/Reviewer/Security gates and package validation, but reliability cannot be established when the implementation worker is also the sole evaluator of its own output.

Parent Epic: #184
Depends on: #185
Related: #60

Desired Outcome

Forge can run an independent Verification Workforce against completed implementation output or project verification goals, using a fresh bounded context and persisting structured findings and evidence separately from the worker’s execution record.

User Story

As a Forge operator,
I want implementation work to be checked by an independent verifier,
So that reliability and autonomy decisions do not depend on worker self-assessment.

Requirements

  • Define a verification-run contract linked to:
  • Verification must run in a fresh session/context. Do not provide the worker’s private reasoning transcript.
  • Provide the verifier only the bounded inputs needed to judge the output:
    • objective and acceptance criteria;
    • changed files/diff or sandbox artifacts;
    • validation commands and results;
    • relevant repository rules;
    • known risk classification;
    • prior review findings when explicitly part of rework.
  • Run deterministic checks before model-based review.
  • Support at least these verification results:
    • passed;
    • failed;
    • inconclusive;
    • blocked;
    • needs_human_review.
  • Persist structured findings with:
    • finding type/severity;
    • affected surface/file/criterion;
    • evidence refs;
    • verification method;
    • required fix or follow-up;
    • confidence only as non-authoritative context;
    • verifier runtime/model snapshot.
  • Keep automated verification distinct from existing manual approval decisions.
  • Allow higher-risk packages to require a different runtime/model or explicit Security verifier policy.
  • Feed independently verified outcomes into [FEATURE] Add capability reliability ledger #186.
  • Fail closed when required verification evidence is missing, stale, or malformed.
  • Add re-verification support after rework without overwriting the original finding history.

Acceptance Criteria

  • The implementation worker cannot mark its own attempt as independently verified.
  • Verification runs have separate agent-run/artifact identity and fresh context.
  • Every acceptance criterion is reported as passed, failed, inconclusive, or not tested.
  • Deterministic test/build/lint evidence is preserved separately from model review findings.
  • Missing required evidence cannot produce a verified pass.
  • Re-verification appends history and links to the prior failed/inconclusive run.
  • High-risk work can require a Security verifier or human gate.
  • Verification results emit canonical outcomes compatible with [FEATURE] Normalize execution outcomes and stop reasons #185 and ledger entries compatible with [FEATURE] Add capability reliability ledger #186.
  • Tests cover independence, bounded context, evidence requirements, re-verification, and fail-closed behaviour.

Out of Scope

  • Replacing human final approval.
  • Making all current manual QA/Reviewer/Security gates autonomous immediately.
  • Giving the verifier repository-write or merge authority.
  • Providing the worker’s full model transcript or hidden reasoning to the verifier.
  • Autonomy promotion/demotion; that belongs to [FEATURE] Add evidence-based earned autonomy policy engine #189.

Implementation Scope

Large — execution path, contracts, artifacts, gates, evidence, and tests.

Technical Notes

Reuse existing agent_runs, artifacts, work-package acceptance criteria, repository evidence, review/rework state, and gate records where possible. Add new persistence only where existing records cannot represent verification attempts and structured findings cleanly.

The verification architecture should align with #60, but this issue owns the concrete independent execution/evidence path required by Epic #184.

Metadata

Metadata

Assignees

No one assigned

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions