Skip to content

Create REQUEST_CHANGES review for create_pull_request threat-warning mode#34133

Merged
pelikhan merged 5 commits into
mainfrom
copilot/update-safe-output-pr-behavior
May 22, 2026
Merged

Create REQUEST_CHANGES review for create_pull_request threat-warning mode#34133
pelikhan merged 5 commits into
mainfrom
copilot/update-safe-output-pr-behavior

Conversation

Copy link
Copy Markdown
Contributor

Copilot AI commented May 22, 2026

When threat detection returns warning, create_pull_request already adds a caution message but did not force a formal review gate. This change aligns threat-warning handling with protected-files behavior by creating a blocking-style review signal that requires human scrutiny.

  • Threat-warning review gating

    • Extend create_pull_request to submit a PR review with event: "REQUEST_CHANGES" when detection caution is active.
    • Review body explicitly calls out that changes must be scrutinized before merge, includes detection reason, and links to workflow run logs.
  • Policy composition with protected-files flow

    • Reuse the existing protected-files request-review path instead of introducing a parallel mechanism.
    • If both protected-files and threat-warning signals are present, compose a single review body with both sections.
  • Behavioral coverage updates

    • Add/adjust create_pull_request unit assertions to verify:
      • warning mode emits a REQUEST_CHANGES review with scrutiny language,
      • non-warning mode does not emit that review.
if (detectionCaution) {
  requestChangesSections.push(`Threat detection produced a warning for this pull request output.

These changes need to be scrutinized before merge and only merged after a careful manual review.

- Detection reason: \`${detectionReason}\`
- Review workflow run logs: ${runUrl}`);
}

Copilot AI and others added 2 commits May 22, 2026 22:34
Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
Copilot AI changed the title Add REQUEST_CHANGES review for create_pull_request threat warnings Create REQUEST_CHANGES review for create_pull_request threat-warning mode May 22, 2026
Copilot AI requested a review from pelikhan May 22, 2026 22:35
@pelikhan pelikhan marked this pull request as ready for review May 22, 2026 22:40
Copilot AI review requested due to automatic review settings May 22, 2026 22:40
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds a formal, blocking-style PR review gate when threat detection returns a warning, aligning threat-warning handling with the existing protected-files “request review” flow and composing both signals into a single review when applicable.

Changes:

  • Extend create_pull_request to submit a pulls.createReview with event: "REQUEST_CHANGES" when threat-warning caution is active (and compose with protected-files request-review content).
  • Refactor request-changes review body construction to support multiple sections separated by a Markdown divider.
  • Update unit tests to mock createReview in additional scenarios and assert threat-warning review behavior.
Show a summary per file
File Description
actions/setup/js/create_pull_request.cjs Compose protected-files + threat-warning “request changes” sections and submit a blocking PR review when either signal is present.
actions/setup/js/create_pull_request.test.cjs Add assertions/mocks to verify threat-warning triggers a REQUEST_CHANGES review and non-warning does not.

Copilot's findings

Tip

Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comments suppressed due to low confidence (1)

actions/setup/js/create_pull_request.cjs:2080

  • This code now creates a REQUEST_CHANGES review for both protected-files and threat-warning cases, but the log message below still says it’s "due to protected files". Update the message to reflect the actual reasons (protected files and/or threat-detection warning) to avoid misleading operational logs.
      if (requestChangesSections.length > 0) {
        const requestChangesBody = requestChangesSections.join("\n\n---\n\n");
        /** @type {{ owner: string, repo: string, pull_number: number, event: "REQUEST_CHANGES" | "COMMENT", body: string, commit_id?: string }} */
        const requestChangesParams = {
          owner: repoParts.owner,
          repo: repoParts.repo,
  • Files reviewed: 2/2 changed files
  • Comments generated: 3

Comment on lines +2056 to +2062
requestChangesSections.push(
`Protected files were modified in this pull request and require manual scrutiny before merge.

Please verify that each protected-file change is intentional, policy-compliant, and safe:

${protectedFilesList}`
);
Comment on lines +2067 to +2072
`Threat detection produced a warning for this pull request output.

These changes need to be scrutinized before merge and only merged after a careful manual review.

- Detection reason: \`${detectionReason}\`
- Review workflow run logs: ${runUrl}`
expect(createReviewCall.body).toContain("Threat detection produced a warning");
expect(createReviewCall.body).toContain("need to be scrutinized");
});

@pelikhan
Copy link
Copy Markdown
Collaborator

@copilot move templates to actions/setup/md and use rendertemplatefile also use renderFilesList helper

Copilot AI and others added 2 commits May 22, 2026 23:02
Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
Copy link
Copy Markdown
Contributor Author

Copilot AI commented May 22, 2026

@copilot move templates to actions/setup/md and use rendertemplatefile also use renderFilesList helper

Implemented in 6709de0 and 743ab75: moved REQUEST_CHANGES review templates to actions/setup/md, switched create_pull_request to renderTemplateFromFile, and wired protected-file rendering through renderFilesList.

@github-actions
Copy link
Copy Markdown
Contributor

@copilot review all comments
Please address the unresolved review feedback and post an update when it is ready for another look.

Generated by 👨‍🍳 PR Sous Chef · ● 3.2M ·

@pelikhan pelikhan merged commit c933f0c into main May 22, 2026
12 checks passed
@pelikhan pelikhan deleted the copilot/update-safe-output-pr-behavior branch May 22, 2026 23:26
github-actions Bot added a commit that referenced this pull request May 23, 2026
…-warning REQUEST_CHANGES review

The `protected-files` policy on `create-pull-request` and
`push-to-pull-request-branch` now defaults to `request_review`
(creates the PR and submits a REQUEST_CHANGES review) and accepts a
fourth value of that same name. The reference tables previously listed
only `blocked`/`allowed`/`fallback-to-issue` and marked `blocked` as the
default, which no longer matches runtime behavior.

PR #34133 additionally extended `create_pull_request` to submit a
REQUEST_CHANGES review when threat detection returns a warning (with the
detection reason and run-log link), composing with the protected-files
review when both fire. Added a short "When Detection Returns a Warning"
note under Error Handling.

Refs: #34133

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants