Skip to content

epic(gate): four verified paths to a wrong auto-verdict (2 false-merge, 1 false-close, 1 lost enforcement) #9461

Description

@JSONbored

Summary

An adversarial audit of the review engine (2026-07-27, seven parallel sweeps against origin/main @ 776c4147a, plus live inspection of edge-nl-01) found four independently-verified paths by which the gate reaches a wrong terminal verdict — two that let a PR auto-merge when it should have been blocked, one that auto-closes a legitimate PR, and one that silently drops one-shot close enforcement.

Every finding below was re-verified by reading the cited lines at HEAD. This is the tracking epic; each sub-issue is one PR-sized fix.

Why these are grouped

All four share one root pattern: a subsystem failure is interpreted as "no problem found" rather than "cannot determine". The pre-merge-check machinery already models this correctly — an unresolvable enforced check yields PRE_MERGE_CHECK_UNRESOLVED_CODE → NEUTRAL → held, explicitly documented as "never silently skipping a hard requirement (auto-merge bypass)" (packages/loopover-engine/src/review/pre-merge-checks.ts:9-13,38-49). The four paths below each violate that principle in a different subsystem.

The four paths

# Path Wrong outcome Status
1 AI blocker text trips the public-safe word filter → collapses to "no defect" False merge Verified in code; live config affected
2 Screenshot-gate capture-retry marker never cleared → gate skipped forever for that head False merge Verified in code; gate live on all 3 repos
3 Browserless render failure swallowed → treated as "no visual evidence" False close Verified in code
4 reopen-reclose job dies on lock contention → disallowed reopen stands Enforcement lost Verified in code and in production (PR #9450)

Deployment context (verified live on edge-nl-01, 2026-07-27)

  • AI_PROVIDER=claude-code,ollamaresolveAiReviewerPlan returns one reviewer slot with combine: "single" (src/selfhost/ai.ts:1596-1613). Path 1 lives in the single branch, so the live configuration is the affected one.
  • screenshotTableGate.action: close is set in the private server-side config for all three repos (/opt/loopover/loopover-config/jsonbored__{loopover,metagraphed,awesome-claude}/.loopover.yml), and the gate was re-enabled two commits ago in 73d4e94 (chore(manifest): re-enable the screenshot-table gate scoped to genuinely visual paths #9455). Paths 2 and 3 are live, not theoretical.
  • Deployed image is ghcr.io/jsonbored/loopover-selfhost:orb-v3.5.0-beta.10.

Expected outcome

When this epic closes, no subsystem failure in the review pipeline can produce a merge or a close by default. Every "cannot determine" state resolves to a hold with a named reason, matching the pre_merge_check_unresolved precedent.

Acceptance criteria for the epic

  • All four sub-issues closed with regression tests that fail against current main.
  • A shared invariant test asserts that for each gate with an action: close mode, an induced infrastructure failure produces a hold (not a merge and not a close).
  • No behaviour change for the happy path (verified by existing gate tests staying green).

Non-goals

  • Re-tuning gate precision or thresholds (tracked separately for secret_leak).
  • Any change to the public repo .loopover.yml; review policy lives in the private server-side config.

Metadata

Metadata

Assignees

Labels

maintainer-onlyOwner-only work — yields no Gittensor points.orbGittensory Orb related - maintainer self-hosting analytics.

Projects

No projects

Relationships

None yet

Development

No branches or pull requests

Issue actions