Skip to content

gate(screenshot-table): surface the finding when capture-unobtainable degrades enforcement #10060

Description

@JSONbored

⚠️ Definition of Done: this issue must be completed in full, in a single PR. Do not split this
work across multiple PRs, and do not defer any Deliverable below to a follow-up issue. A PR that
satisfies only some of the Deliverables, stubs a required test, or leaves a checkbox
partially-done does NOT resolve this issue and will be closed.

Context

#9881 added an enforcement degrade to the screenshot-table gate: when the bot has proved it can never
produce a capture for this repo, a close must not fire. Both the engine and the call site say, explicitly,
that the finding survives the degrade even though the enforcement does not.

packages/loopover-engine/src/review/screenshot-table-gate.ts:343

  /** #9881: set when the violation stands but ENFORCEMENT must not. The bot proved this repo produces no
   *  preview deployment at all, so `review.visual.enabled` can never satisfy the gate here and a CLOSE would
   *  destroy PRs over evidence no contributor action could supply. The finding is still raised -- the PR
   *  really does lack visual evidence -- but the caller degrades `action: "close"` to advisory and says this.

src/queue/processors.ts:3572

  // #9881: the bot proved this repo produces no preview deployment at all, so `review.visual.enabled` can
  // never satisfy this gate here. The violation stands and still surfaces as a finding, but a CLOSE would
  // destroy a contributor's PR over a maintainer-side pipeline gap no contributor action can close, and
  // which nothing had ever told the maintainer about. Degrade to advisory and say so.

And the reason string itself claims to be contributor-visible —
packages/loopover-engine/src/review/screenshot-table-gate.ts:404:

/** #9881: the reason attached when enforcement is degraded. Stated in the contributor-visible comment, so
 *  the PR author learns the gate could not be satisfied here rather than being left to guess. */
export const CAPTURE_UNOBTAINABLE_REASON =

None of that is true. There is exactly one code path in this repo that turns a screenshot-table violation
into a finding, and it refuses to run in precisely the degraded case:

src/queue/processors.ts:8761

  if (!args.screenshotTableGateConfig.enabled || args.screenshotTableGateConfig.action !== "advisory") return;

The degrade only ever applies when the configured action is close or block — with action: "advisory"
there is no enforcement to degrade. So maybeAddScreenshotTableAdvisoryFinding always returns early on the
degraded path and never appends screenshot_table_missing. Meanwhile in the planner input:

  • screenshotTableMatch is undefined (src/queue/processors.ts:3579, gated on
    !screenshotTableEnforcementDegraded),
  • screenshotEvidenceHold is undefined (src/queue/processors.ts:3600, same gate),
  • screenshotTableEvidenceUnresolved is false (src/queue/processors.ts:3619, same gate),

so planAgentMaintenanceActions (src/settings/agent-actions.ts:10291044) falls straight through to
ordinary merit/CI evaluation. CAPTURE_UNOBTAINABLE_REASON reaches exactly one place — an internal audit
event's detail at src/queue/processors.ts:3613 — and nothing else. It is grep-confirmed unused outside
packages/loopover-engine/src/review/screenshot-table-gate.ts:420 and that audit call.

What breaks in practice: on a repo with requireScreenshotTable.action: "close" (the default action) whose
preview pipeline is proved unobtainable, a PR with no visual evidence is silently merged as if the gate had
never been configured. The maintainer is never told their gate is unsatisfiable, the contributor is never
told anything, and the only trace is an audit row nobody reads. The exact failure #9881's own migration
comment names — "nothing had ever told the maintainer about" the gap — is reproduced by the fix for it.

test/unit/screenshot-table-gate.test.ts:909 covers the pure evaluator only (that
enforcementDegradedReason is set and violated stays true); nothing tests the wiring.

Requirements

  • When evaluateScreenshotTableGate returns an enforcementDegradedReason, a warning-severity advisory
    finding MUST be appended to the PR's advisory findings regardless of the configured action, carrying both
    the gate's own reason and CAPTURE_UNOBTAINABLE_REASON so the maintainer reads the remedy.
  • maybeAddScreenshotTableAdvisoryFinding (src/queue/processors.ts:8748) is the function that must be
    extended — it must accept the captureUnobtainable fact and pass it into evaluateScreenshotTableGate,
    and its action !== "advisory" early return must become "return only when the action is not advisory
    AND enforcement was not degraded". Do NOT add a second, parallel finding-emitting function.
  • The call site at src/queue/processors.ts:12119 must pass the same
    Boolean(pr.headSha) && pr.visualCaptureUnobtainableSha === pr.headSha value the enforcement call site at
    src/queue/processors.ts:3535 computes, so the two evaluations of the same pure check cannot disagree.
  • Behaviour that must NOT change: action: "advisory" with no degrade must keep emitting exactly the finding
    it emits today, with the same code, severity, title, action and publicText; a degraded gate must
    still produce NO close, NO hold, and NO blocker; VISUAL/gate blocker classification must not change
    (screenshot_table_missing must remain unrecognized by isConfiguredGateBlocker); and a non-degraded
    close/block violation must keep producing no advisory finding (its close/hold comment already says it).
  • The finding emitted on the degraded path must be distinguishable from the ordinary advisory one — it must
    state that enforcement was degraded and why — so a maintainer can tell "you configured advisory" apart from
    "your gate cannot be satisfied here".

⚠️ Required pattern: mirror maybeAddLockfileTamperFinding (src/queue/processors.ts, immediately above
maybeAddScreenshotTableAdvisoryFinding) — one function, one advisory.findings.push, all evaluation
errors swallowed so the gate can never be destabilized. What does NOT satisfy this issue: (a) rewriting
screenshotTableGateConfig.action to "advisory" somewhere upstream so the existing early return happens
to pass — that silently changes what GET/settings surfaces report the repo's action to be; (b) emitting
the notice as a GitHub comment or check-run annotation instead of an advisory finding — the advisory
pipeline is the surface every sibling gate uses; (c) making the degraded case produce a hold or a close,
which is the exact behaviour #9881 removed; (d) a test-only PR asserting today's silence.

Deliverables

  • maybeAddScreenshotTableAdvisoryFinding (src/queue/processors.ts:8748) takes a
    captureUnobtainable: boolean argument, threads it into its evaluateScreenshotTableGate call, and no
    longer early-returns when result.enforcementDegradedReason !== undefined.
  • The degraded path pushes a finding whose detail/publicText contains CAPTURE_UNOBTAINABLE_REASON
    (imported from packages/loopover-engine/src/review/screenshot-table-gate.ts) in addition to the gate's
    own reason.
  • The call site at src/queue/processors.ts:12119 passes
    Boolean(pr.headSha) && pr.visualCaptureUnobtainableSha === pr.headSha.
  • Test in test/unit/screenshot-table-gate.test.ts: action: "close", gate violated,
    captureUnobtainable: truemaybeAddScreenshotTableAdvisoryFinding appends exactly one finding whose
    detail includes the CAPTURE_UNOBTAINABLE_REASON text, and the plan built from the same inputs
    contains no close and no manual-review hold.
  • Test in test/unit/screenshot-table-gate.test.ts: action: "block", gate violated,
    captureUnobtainable: true ⇒ the same finding is appended and no hold is planned.
  • Test in test/unit/screenshot-table-gate.test.ts: action: "close", gate violated,
    captureUnobtainable: false ⇒ NO advisory finding is appended (today's behaviour, pinned).
  • Test in test/unit/screenshot-table-gate.test.ts: action: "advisory", gate violated,
    captureUnobtainable: false ⇒ byte-identical finding to today.
  • Regression test in test/unit/screenshot-table-gate.test.ts named for this bug: a degraded
    action: "close" evaluation never produces a completely silent pass (assert advisory.findings is
    non-empty).

All Deliverables above are required in a single PR. A PR that satisfies only some of them — for example one
that only removes the action !== "advisory" early return without threading captureUnobtainable from the
call site (so the degrade is never detected there at all), or one that adds the finding but not the
captureUnobtainable: false pinning tests — does not resolve this issue.

Test Coverage Requirements

This repo enforces 99%+ Codecov patch coverage, branch-counted. vitest.config.ts's coverage.include
covers src/**/*.ts and packages/loopover-engine/src/**/*.ts — the touched file
(src/queue/processors.ts) is under src/**, so it is measured and gated. Every branch needs both arms:
the rewritten early return (!enabled, action !== "advisory" with degrade, action !== "advisory" without
degrade, action === "advisory"); the degraded vs non-degraded finding-shape branch; and the
Boolean(pr.headSha) && pr.visualCaptureUnobtainableSha === pr.headSha conjunction at the call site (headSha
absent, headSha present and matching, headSha present and not matching). If the PR also touches
packages/loopover-engine/src/review/screenshot-table-gate.ts, engine lines are credited by two uploads
whose hits are unioned — add the test to packages/loopover-engine/test/** as well as any root test/**
coverage, or the patch gate can still fail.

Expected Outcome

A repo whose screenshot-table gate cannot be satisfied stops failing silently. The contributor's PR is still
never closed for evidence the pipeline cannot produce, but the PR now carries a visible, non-blocking finding
naming the exact maintainer-side remedy — which is what both #9881's code comments and its migration already
claim happens.

Links & Resources

Metadata

Metadata

Assignees

No one assigned

    Labels

    gittensor:bugGittensor-scored bug fix — scores a 0.05x multiplier.help wantedExtra attention is needed

    Projects

    No projects

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions