Skip to content

[FEATURE] Add Project Sentinel detection and escalation flow #190

Description

@Joncallim

Problem Statement

Forge has task execution and project evidence, but it does not yet continuously inspect projects for regressions, stuck workflows, stale review state, or failed proof goals. A naive scheduled LLM scan would be costly, noisy, and prone to speculative findings.

Parent Epic: #184
Depends on: #185, #187
Related: #42

Desired Outcome

Forge provides a lightweight Project Sentinel that runs deterministic project checks first, creates deduplicated findings with evidence, and escalates actionable findings to the operator, GitHub backlog, or Architect without attempting an unreviewed repair.

User Story

As a Forge operator,
I want projects to be checked periodically for clear regressions and workflow problems,
So that important failures are surfaced early without handing an always-on agent broad authority.

Requirements

  • Define a versioned Sentinel finding contract including:
    • project id;
    • finding type and severity;
    • source signal;
    • first/last observed timestamps;
    • repository ref/environment fingerprint;
    • evidence references;
    • deduplication fingerprint;
    • current state (open, acknowledged, resolved, suppressed);
    • recommended escalation/action;
    • linked issue/task/work package/goal where applicable.
  • Implement deterministic detectors before any model call, including at least:
    • failed or regressed verification goals from [FEATURE] Add project verification goals and scheduled proof runs #187;
    • CI/check failures where GitHub is configured;
    • work packages stuck, blocked, repeatedly retried, or dead-lettered;
    • stale required review/approval state;
    • open needs-clarification / agent-blocked implementation work;
    • unresolved critical verification/security findings;
    • provider/MCP readiness regressions where relevant.
  • Deduplicate repeated observations of the same underlying problem.
  • Resolve or age findings when the underlying signal recovers.
  • Add controlled schedule/on-demand execution through the Forge queue or GitHub integration, not an unrestricted host cron script.
  • Add cost and frequency guards.
  • Invoke an LLM only when deterministic evidence needs classification, summarization, or bounded work-order drafting.
  • Never give Sentinel direct code-edit, merge, credential, or broad filesystem authority.
  • Support escalation paths:
  • Preserve the evidence before any repair request is created.
  • Add audit logs and tests for detection, deduplication, recovery, suppression, and escalation.

Acceptance Criteria

  • Sentinel can detect a failed verification goal without using an LLM.
  • Repeated scans update one finding rather than creating duplicates.
  • Recovery of the underlying signal can resolve the finding while preserving history.
  • High/critical findings can trigger autonomy review/revocation before any repair attempt.
  • Sentinel cannot modify project code or merge changes directly.
  • Optional LLM use receives bounded deterministic evidence, not an unbounded repository dump.
  • Schedule frequency and model cost are configurable and bounded.
  • Findings can link to GitHub issues/tasks without replacing GitHub as implementation truth.
  • Tests cover each initial detector, deduplication, suppression, recovery, and fail-closed escalation.

Out of Scope

Implementation Scope

Large — detector framework, persistence, scheduling, escalation, integrations, and tests.

Technical Notes

Suggested initial finding types:

  • verification_goal_regression;
  • ci_failure;
  • stuck_work_package;
  • retry_exhausted;
  • review_stale;
  • agent_request_blocked;
  • critical_verification_finding;
  • provider_or_mcp_readiness_regression.

Suggested persistence: sentinel_findings plus append-only observations/events. Deterministic source evidence should remain queryable even if a later model summary changes.

Metadata

Metadata

Assignees

No one assigned

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions