Bypass Copilot review for manual reviewer handoffs - #240
Merged
trask merged 7 commits intoAug 12, 2026
Conversation
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Contributor
There was a problem hiding this comment.
Pull request overview
Updates reviewer-routing overrides to bypass Copilot review while retaining required-check gating.
Changes:
- Adds and tests Copilot bypass persistence and push reset behavior.
- Updates command replies and documentation.
- Records bypass state in dashboard facts.
Reviewed changes
Copilot reviewed 6 out of 6 changed files in this pull request and generated 1 comment.
Show a summary per file
| File | Description |
|---|---|
pull-request-dashboard/README.md |
Documents bypass behavior. |
.github/scripts/pull-request-dashboard/test_dashboard.py |
Tests routing and bypass lifecycle. |
.github/scripts/pull-request-dashboard/test_dashboard_override.py |
Updates command-reply expectations. |
.github/scripts/pull-request-dashboard/RATIONALE.md |
Explains the bypass rationale. |
.github/scripts/pull-request-dashboard/dashboard.py |
Implements Copilot gate bypass. |
.github/scripts/pull-request-dashboard/dashboard_override.py |
Refines command replies. |
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
Copilot comment: A push can immediately re-enable this bypass. `clear_overridden_actions()` recomputes `dashboard_override_cleared_count`/`dashboard_override_cleared_ci` from the durable command watermark on every refresh, so an old discussion or check that remains present keeps `override_cleared_actions` true even when `head_sha` changed. The new push test misses this production state because it omits both cleared fields. Treat cleared actions as a new handoff only when the command watermark is new; if the previous result already recorded a bypass for the same command and the head changed, keep the bypass off until a later command changes the watermark. Add the cleared field to the push regression test as well. Analysis: Cleared-action facts are recomputed from the durable command timestamp, so they only represent a new manual handoff when that timestamp changes. An existing bypass now carries forward only while both the command timestamp and head SHA remain unchanged. Upsides: A push restores the required Copilot gate even when old cleared items remain, while a later override command can establish a new manual handoff. Downsides: No material downside identified. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Contributor
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 6 out of 6 changed files in this pull request and generated no new comments.
Suppressed comments (2)
.github/scripts/pull-request-dashboard/dashboard_override.py:261
- The comma is not grammatical for the successful
routedbranch: it now renders@author, routed this pull request to reviewers., where the mention no longer serves as the subject and the remainder is a sentence fragment. Keep direct-address punctuation for the other branches, but render this case as something like@author, this pull request was routed to reviewers.
mention = f"@{user}, " if user else ""
.github/scripts/pull-request-dashboard/dashboard.py:1436
- Commands are identified only by their second-granularity timestamp, so two authorized overrides posted in the same second are treated as the same command. If the first bypass was ended by a push, an effective second command can clear old actions but
not same_override_commandremains false, so it cannot establish a new bypass. Include the pending command ID when detecting a newly effective override, and cover two commands sharingcreated_at.
same_override_command = (
bool(facts.get("dashboard_override_since"))
and facts.get("dashboard_override_since")
== previous_facts.get("dashboard_override_since")
Copilot comment:
The comma is not grammatical for the successful `routed` branch: it now renders `@author, routed this pull request to reviewers.`, where the mention no longer serves as the subject and the remainder is a sentence fragment. Keep direct-address punctuation for the other branches, but render this case as something like `@author, this pull request was routed to reviewers.`
```
mention = f"@{user}, " if user else ""
```
Analysis: The mention is direct address, so the successful routed branch needs a complete sentence after its comma. The acknowledgement now uses a passive sentence consistent with the other reply branches.
Upsides: Successful command acknowledgements are grammatical without changing punctuation or wording for the other outcomes.
Downsides: No material downside identified.
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
…ands
Copilot comment:
Commands are identified only by their second-granularity timestamp, so two authorized overrides posted in the same second are treated as the same command. If the first bypass was ended by a push, an effective second command can clear old actions but `not same_override_command` remains false, so it cannot establish a new bypass. Include the pending command ID when detecting a newly effective override, and cover two commands sharing `created_at`.
```
same_override_command = (
bool(facts.get("dashboard_override_since"))
and facts.get("dashboard_override_since")
== previous_facts.get("dashboard_override_since")
```
Analysis: The durable timestamp remains the cleared-action watermark, but a nonzero pending command ID identifies a newly effective command when timestamps collide. A pending ID is treated as the same command only when it matches the previous result; its later transition to zero remains the acknowledgement of that command.
Upsides: Distinct override commands posted in the same second can establish distinct reviewer handoffs without re-enabling an old command after a push.
Downsides: No material downside identified.
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Contributor
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 6 out of 6 changed files in this pull request and generated no new comments.
Suppressed comments (1)
.github/scripts/pull-request-dashboard/dashboard.py:1442
- An old acknowledged override can regain the bypass when this PR has no cached
previous_result(for example, after the PR was closed, evicted byremove_cached_dashboard_prs(), and reopened). The durable command watermark still makesclear_overridden_actions()report old cleared items, sosame_override_commandis false and this branch treats the old command as a new handoff—even if the current head was pushed after it. That violates the documented rule that a later push restores the gate. Only a currently pending command ID should establish a fresh bypass; acknowledged commands should only carry one forward from the same cached command/head.
manual_reviewer_handoff = (
override_cleared_actions and not same_override_command
) or bool(
…ypass
Copilot comment:
An old acknowledged override can regain the bypass when this PR has no cached `previous_result` (for example, after the PR was closed, evicted by `remove_cached_dashboard_prs()`, and reopened). The durable command watermark still makes `clear_overridden_actions()` report old cleared items, so `same_override_command` is false and this branch treats the old command as a new handoff—even if the current head was pushed after it. That violates the documented rule that a later push restores the gate. Only a currently pending command ID should establish a fresh bypass; acknowledged commands should only carry one forward from the same cached command/head.
```
manual_reviewer_handoff = (
override_cleared_actions and not same_override_command
) or bool(
```
Analysis: Recomputed cleared-action facts do not prove that an acknowledged command is newly effective when cached state is absent. A fresh bypass now requires a nonzero pending command ID, while an acknowledged command can only preserve a bypass already recorded for the same command watermark and head.
Upsides: Cache eviction and PR reopen flows cannot revive an old manual-review bypass after a push.
Downsides: No material downside identified.
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Contributor
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 6 out of 6 changed files in this pull request and generated no new comments.
Suppressed comments (1)
.github/scripts/pull-request-dashboard/dashboard.py:1447
- The bypass is durable only through an immediately successful previous result. A transient discussion-classification failure returns and caches at lines 1565–1582 before this field is set, replacing the last result with facts that omit
copilot_review_bypassed_by_override. The next successful refresh therefore restores the Copilot gate even when the command and head are unchanged; if the failure happened on the command's first refresh,same_override_commandcan also prevent that still-unacknowledged command from ever establishing the bypass. Preserve the prior bypass decision/identity in failed results (or retain the last successful routing state separately), and cover both failure-before-first-routing and failure-after-bypass refresh sequences.
) or bool(
previous_facts.get("copilot_review_bypassed_by_override")
and same_overridden_head
)
Copilot comment:
The bypass is durable only through an immediately successful previous result. A transient discussion-classification failure returns and caches at lines 1565–1582 before this field is set, replacing the last result with facts that omit `copilot_review_bypassed_by_override`. The next successful refresh therefore restores the Copilot gate even when the command and head are unchanged; if the failure happened on the command's first refresh, `same_override_command` can also prevent that still-unacknowledged command from ever establishing the bypass. Preserve the prior bypass decision/identity in failed results (or retain the last successful routing state separately), and cover both failure-before-first-routing and failure-after-bypass refresh sequences.
```
) or bool(
previous_facts.get("copilot_review_bypassed_by_override")
and same_overridden_head
)
```
Analysis: Classification failures are cached as the next result, so they now retain the prior command identity and bypass decision. The bypass is retained only when the command watermark and head are unchanged, while retaining the prior pending ID lets a command whose first routing attempt failed remain newly detectable.
Upsides: Transient classification failures no longer lose or prematurely consume a manual handoff, and pushes still terminate an existing bypass.
Downsides: Failed results intentionally expose the prior pending command ID as routing state until a successful refresh recomputes it.
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Pull request dashboard statusMerged · refreshed 2026-08-12 19:50 UTC Status above doesn't look right?
|
maryliag
approved these changes
Aug 12, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Treat an effective
/dashboard route:reviewerscommand as an explicit manual handoff, so the dashboard neither requests nor waits for another Copilot review of the current head.Required checks continue to gate the handoff, and a later push restores the normal Copilot review requirement. The bypass is carried across dashboard refreshes for the same command and head.