Add pull_request.opened event support to ai-moderator#16230
Conversation
Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
|
@copilot run on forks too |
Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
|
Just as a heads up, I was blocked by some firewall rules while working on your feedback. Expand below for details. Warning Firewall rules blocked me from connecting to one or more addresses (expand for details)I tried to connect to the following addresses, but was blocked by firewall rules:
If you need me to access, download, or install something from one of these locations, you can either:
|
Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
Added Warning Firewall rules blocked me from connecting to one or more addresses (expand for details)I tried to connect to the following addresses, but was blocked by firewall rules:
If you need me to access, download, or install something from one of these locations, you can either:
|
There was a problem hiding this comment.
Pull request overview
Updates the ai-moderator agentic workflow to run on newly created pull requests and guide the agent to inspect PR diffs for spam/link-spam signals, in addition to issues and issue comments.
Changes:
- Added
pull_request: types: [opened]trigger withforks: "*"and explicitconcurrency.cancel-in-progress: false. - Updated the moderation prompt with PR-specific instructions to fetch and analyze the PR diff and apply moderation labels.
- Regenerated the compiled workflow lock file to reflect the new trigger, concurrency group, and rate-limit events.
Reviewed changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated 4 comments.
| File | Description |
|---|---|
| .github/workflows/ai-moderator.md | Adds PR-open trigger + forks allowance, concurrency config, and PR diff analysis instructions in the prompt. |
| .github/workflows/ai-moderator.lock.yml | Compiled output reflecting the new PR trigger, updated concurrency group, and rate-limit events including pull_request. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| lock-for-agent: true | ||
| pull_request: | ||
| types: [opened] | ||
| forks: "*" |
There was a problem hiding this comment.
Enabling the workflow on pull_request from forks (forks: "*") means this workflow will run against untrusted PR content. The compiled workflow checks out repo content and executes local actions/scripts (e.g., uses: ./actions/setup), which a fork PR can modify to get arbitrary code execution in the runner. To mitigate, ensure checkouts that provide executable code are pinned to the default branch (or a trusted ref) for PR events, or avoid using local actions when running on fork PRs (alternatively switch to pull_request_target with a safe checkout strategy).
| forks: "*" |
| lock-for-agent: true | ||
| pull_request: | ||
| types: [opened] | ||
| forks: "*" |
There was a problem hiding this comment.
With pull_request enabled for forks, safe outputs that write (e.g., adding labels) will typically not work on fork PRs because secrets are not provided and GITHUB_TOKEN is read-only for fork-triggered pull_request workflows. If the intent is to label/moderate fork PRs, consider using pull_request_target (with strict safeguards) or add logic to skip/soft-fail write operations when the PR comes from a fork.
| forks: "*" |
| - Fetch the PR diff using `pull_request_read` with method `get_diff` | ||
| - Analyze the diff for spam patterns: |
There was a problem hiding this comment.
Instruction to fetch the full PR diff via pull_request_read --method get_diff is likely to exceed the MCP 25k token response limit on larger PRs (this repo documents pull_request_read large diffs exceeding limits). Consider adjusting the prompt to start with get_files (paginated) and only fetch diffs selectively / for suspicious files, or explicitly handle/avoid oversized diff responses.
| - Fetch the PR diff using `pull_request_read` with method `get_diff` | |
| - Analyze the diff for spam patterns: | |
| - First, list changed files using `pull_request_read` with method `get_files` (use pagination if needed) | |
| - Identify potentially suspicious files (for example, files with large additions, many links, unusual paths, or newly added binaries/scripts) | |
| - For only those suspicious or relevant files, fetch diffs using `pull_request_read` with method `get_diff` (or an equivalent per-file diff), taking care to avoid oversized responses that might exceed token limits | |
| - Analyze the selected file diffs for spam patterns: |
| 3. **For Pull Requests** (when pull request number is present): | ||
| - Fetch the PR diff using `pull_request_read` with method `get_diff` |
There was a problem hiding this comment.
This workflow prompt still contains guidance about handling workflow_dispatch, but the workflow’s on: configuration does not include a workflow_dispatch trigger. Consider removing that instruction (or adding the trigger if it’s intended) so the agent doesn’t try to follow a non-existent execution path.
Update ai-moderator to handle PR created events
Summary of Changes
Workflow Event Trigger:
pull_request: types: [opened]to the workflow'sonconfigurationforks: "*"to allow the workflow to run on PRs from forked repositoriesConcurrency Configuration:
cancel-in-progress: falseto allow concurrent moderation runs to complete without cancellationFork Support:
forks: "*"in the pull_request configurationPrompt Instructions:
pull_request_readwith methodget_diffto fetch the PR diff and analyze the changes for spam patterns"spam,link-spam,ai-generated) if issues detectedai-inspectedlabel if PR appears legitimateLock File Changes:
pull_request: types: [opened]event trigger${{ github.event.issue.number || github.event.pull_request.number }}cancel-in-progress: false(per feedback)💡 You can make Copilot smarter by setting up custom instructions, customizing its development environment and configuring Model Context Protocol (MCP) servers. Learn more Copilot coding agent tips in the docs.