Skip to content

feat(app): mark evidence check resources out of scope#3438

Merged
tofikwest merged 3 commits into
mainfrom
tofik/cs-719-bug-no-way-to-mark-resources-out-of-scope-aws-access
Jul 16, 2026
Merged

feat(app): mark evidence check resources out of scope#3438
tofikwest merged 3 commits into
mainfrom
tofik/cs-719-bug-no-way-to-mark-resources-out-of-scope-aws-access

Conversation

@tofikwest

@tofikwest tofikwest commented Jul 16, 2026

Copy link
Copy Markdown
Contributor

Closes CS-719

Problem

A customer failing an evidence item because of an intentional finding — e.g. an S3 bucket that must stay public because it only hosts a static website redirect — had no way to accept it. The "Access Restrictions" evidence item stayed failed with no route to resolve it, blocking their audit.

What already existed

The exception mechanism was already built end to end on the backend: FindingException rows keyed (connectionId, checkId, resourceId) are honored by both task run paths (manual run-check and the scheduled Trigger task, via the shared ActiveExceptionSet + countEffectiveFailures) and by the task runs display (excepted flag, adjusted failed count/status). The mark/revoke endpoints (POST /v1/cloud-security/findings/:id/exception, DELETE /v1/cloud-security/exceptions/:id, both integration:update) also already resolve evidence-task check result rows correctly — resolveCheckKey falls back to the run's manifest check id, which is exactly the namespace the task paths match on.

What was missing was purely the surface: no way to create or revoke an exception from the evidence item view, and the runs response didn't carry the exception's id/reason.

Changes

API

  • ActiveExceptionSet optionally carries per-exception metadata (id, reason); loadActiveExceptionSet populates it. Constructor stays backward-compatible for evaluation-only callers.
  • getTaskCheckRuns now returns exceptionId + exceptionReason on excepted result rows so the UI can show the documented reason and offer revoke.

App

  • Failing results on a task's check runs now offer Mark out of scope (latest run only). It opens the shared reason modal (min-20-char reason, optional reviewer + auto-review date) and creates the exception through the existing endpoint.
  • Excepted results show an Out of scope badge with the documented reason and offer Move back in scope (revoke, with confirm dialog).
  • After a scope change the affected check re-runs automatically, so the task status flips through the existing exception-aware run path without waiting for the next scheduled run. If a run is already in flight it is not duplicated — exceptions are loaded at aggregation time, so the in-flight run already honors the change.
  • Actions are gated client-side by integration:update (matching the endpoints); read-only users still see the out-of-scope state and reason.
  • MarkExceptionModal moved from cloud-tests/components to components/integrations with copy-override props — Cloud Tests keeps its exact previous copy via the defaults, the tasks surface brands the same flow "out of scope".

No schema changes. No new endpoints. Exceptions created here are the same rows Cloud Tests uses, recorded in the audit trail (exception_marked / exception_revoked).

Tests

  • API: finding-exceptions.spec 7/7 (new infoFor coverage incl. bare-key sets); task-integrations.controller.spec exception tests extended to assert exceptionId/exceptionReason.
  • App: new check-run-history.test.tsx (5 tests: mark payload, revoke payload incl. exception id, hidden without permission, hidden on non-latest runs, hidden when no actions provided); MarkExceptionModal.test.tsx extended for copy overrides (10/10 across both files).
  • Typecheck: api and app both show zero new errors vs origin/main (23 and 15 pre-existing spec-drift errors respectively, verified by baseline comparison with the same command).
  • Full-suite failures in src/integration-platform (4 suites/25 tests) and the app tasks+cloud-tests suites (5 suites/23 tests) are identical on unmodified origin/main — pre-existing order-dependent/env issues, not introduced here.

Summary by cubic

Adds out‑of‑scope controls to evidence task check results so customers can accept intentional resources and unblock audits. Addresses CS‑719 with mark/revoke actions in task run history, expanded lists for actions, and immediate status updates.

  • New Features

    • Failing results on the latest run show “Mark out of scope,” which opens a reason modal and creates an exception.
    • Excepted results show an “Out of scope” badge with the reason and “Move back in scope” to revoke; actions require integration:update.
    • Findings and out‑of‑scope lists expand beyond the first 3 sampled rows so every sampled resource is actionable.
    • After mark/revoke, the affected check re‑runs so the task status updates immediately; skipped only if the same check is already running.
    • Reuses a shared reason modal moved to components/integrations with copy overrides; the reason label uses out‑of‑scope wording.
  • API

    • ActiveExceptionSet optionally carries per‑exception metadata (id, reason); infoFor exposes it and loadActiveExceptionSet populates it.
    • getTaskCheckRuns returns exceptionId and exceptionReason on excepted results for display and revoke.
    • No schema changes; uses POST /v1/cloud-security/findings/:id/exception and DELETE /v1/cloud-security/exceptions/:id.

Written for commit 5113276. Summary will update on new commits.

Review in cubic

Customers failing an evidence item because of an intentional finding (e.g.
an S3 bucket that must stay public for a static website redirect) had no
way to accept it — the task stayed failed with no route to resolve it.

The exception mechanism already existed end to end: FindingException rows
keyed (connection, check, resource) are honored by both task run paths and
the task run display. What was missing was a way to create/revoke them from
the evidence item view.

- app: failing check results on a task now offer "Mark out of scope"
  (captures a documented reason via the shared MarkExceptionModal, moved to
  components/integrations with copy overrides); excepted results show the
  reason and offer "Move back in scope" (revoke)
- app: after a scope change the check re-runs automatically so the task
  status updates immediately
- api: task check runs response now carries exceptionId + exceptionReason
  on excepted results (ActiveExceptionSet optionally carries metadata)
- actions gated by integration:update, matching the endpoints
@linear

linear Bot commented Jul 16, 2026

Copy link
Copy Markdown

CS-719

@vercel

vercel Bot commented Jul 16, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
app Ready Ready Preview, Comment Jul 16, 2026 8:45pm
comp-framework-editor Ready Ready Preview, Comment Jul 16, 2026 8:45pm
1 Skipped Deployment
Project Deployment Actions Updated (UTC)
portal Skipped Skipped Jul 16, 2026 8:45pm

Request Review

@cubic-dev-ai cubic-dev-ai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

cubic analysis

All reported issues were addressed across 11 files

Confidence score: 5/5

  • Safe to merge after the addressed issues were fixed.

Linked issue analysis

Linked issue: CS-719: [BUG] No way to mark resources out of scope - AWS access restrictions task

Status Acceptance criteria Notes
Provide a “Mark out of scope” action on failing results in the evidence/task run history (latest run only). UI adds a `Mark out of scope` button on failing rows for the latest run and wires an action target with run/check coordinates.
Create an exception (finding exception) with reason/reviewer/expiry via the existing endpoint when the user marks a resource out of scope. MarkExceptionModal posts to the existing exception endpoint, captures reason/reviewer/expiry, and emits success handling for the parent to refresh.
Show excepted resources as “Out of scope” with the documented reason and offer “Move back in scope” (revoke) using the exception id. Task run result rows include exception metadata so the UI can display the reason and render a revoke action that passes the exception id to the revoke handler.
API surfaces exception id and reason on task check run results so the UI can display reason + revoke without extra queries. Controller populates exception metadata from ActiveExceptionSet and returns exceptionId/exceptionReason on excepted result rows; tests assert these fields.
After marking or revoking an exception, re-run the affected check so the task status updates immediately (without duplicating runs if one is already in flight). UI triggers a re-run of the affected check after mark/revoke and avoids duplicate runs when a run is already in flight; parent refresh/mutateRuns is also called.
Actions are gated to users with integration:update while read-only users still see out-of-scope state and reason. Permission check is read client-side and the exceptionActions.canManage flag controls whether action buttons render; tests cover hidden actions for non-manage users.

Reply with feedback, questions, or to request a fix.

Fix all with cubic | Re-trigger cubic

Comment thread apps/app/src/components/integrations/MarkExceptionModal.tsx
- re-run guard now only skips when the SAME check is in flight — a run of a
  different check doesn't recompute this one, so it must not suppress the
  re-run after a scope change
- findings and out-of-scope lists are expandable beyond the first three
  sampled rows, so every sampled resource is markable/revocable
- the reason field label follows the surface copy (out-of-scope wording on
  evidence items) via a reasonLabel override

@cubic-dev-ai cubic-dev-ai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

cubic analysis

No issues found across 11 files

Confidence score: 5/5

  • Automated review surfaced no issues in the provided summaries.
  • No files require special attention.

Linked issue analysis

Linked issue: CS-719: [BUG] No way to mark resources out of scope - AWS access restrictions task

Status Acceptance criteria Notes
Provide a UI action on the evidence item to mark a failing resource 'out of scope' (creates an exception) — available on the latest run only. The run history rows render a 'Mark out of scope' button for failing results on the latest run, opening the shared reason modal and creating the exception.
Show excepted results with an 'Out of scope' badge, display the documented reason, and offer 'Move back in scope' (revoke) with the exception id. Excepted rows display an out-of-scope badge and the exception reason; a 'Move back in scope' action is offered and includes the exception id so revoke can be performed without extra queries.
Backend surfaces exception id and reason on task-run result rows so the UI can display reason and revoke without extra queries. ActiveExceptionSet now carries per-exception metadata and loadActiveExceptionSet populates it; task runs controller attaches exceptionId and exceptionReason on excepted result rows.
After marking or revoking an exception, the affected check re-runs (unless an in-flight run of the same check exists) so task status updates immediately. UI logic triggers mutate + a targeted runCheck rerunAfterScopeChange, skipping if the same check is already running, to refresh the evidence and recompute status immediately.
Actions are permission-gated (integration:update) while read-only users still see out-of-scope state and reason; actions are hidden on non-latest runs. Permission check (usePermissions) gates rendering of action buttons; tests verify actions hidden without permission and hidden on stale runs while the out-of-scope state remains visible.

Re-trigger cubic

@tofikwest
tofikwest merged commit b840693 into main Jul 16, 2026
11 checks passed
@tofikwest
tofikwest deleted the tofik/cs-719-bug-no-way-to-mark-resources-out-of-scope-aws-access branch July 16, 2026 20:51
@claudfuen

Copy link
Copy Markdown
Contributor

🎉 This PR is included in version 3.104.0 🎉

The release is available on GitHub release

Your semantic-release bot 📦🚀

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants