fix(ci): make agent gates deterministic per head - #311
Conversation
|
Warning Review limit reached
Next review available in: 41 minutes You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository. How can I continue?After more reviews become available, a review can be triggered using the To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews. How do review limits work?CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability. For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window. Please refer docs for additional details. Review details⚙️ Run configurationConfiguration used: defaults Review profile: CHILL Plan: Pro Plus Run ID: 📒 Files selected for processing (5)
📝 WalkthroughWalkthroughThis change makes Agent Gates deterministic per pull request head. It removes review-state triggers and approval-fetching logic, adds cancellation for superseded runs, retains protected-branch approval enforcement, and updates regression tests and operations documentation. ChangesDeterministic Agent Gates
Estimated code review effort: 3 (Moderate) | ~25 minutes Possibly related PRs
🚥 Pre-merge checks | ✅ 3 | ❌ 2❌ Failed checks (2 warnings)
✅ Passed checks (3 passed)
✨ Finishing Touches 💡 1📝 Generate docstrings 💡
🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 2
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In @.github/workflows/agent-gates.yml:
- Around line 5-9: Update the actions/checkout step in
.github/workflows/agent-gates.yml (lines 19-22) to set ref to ${{
github.event.pull_request.head.sha }}. Add a regression assertion in
scripts/test_lightweight_agent_gates.py (lines 124-134) verifying the workflow
contains this explicit PR-head checkout ref.
In `@scripts/test_lightweight_agent_gates.py`:
- Around line 124-126: Update the permission assertions in the relevant test to
reject any pull-requests permission, not only the read variant. Assert that the
broader “pull-requests:” key is absent from agent_gates while preserving the
existing checks for pull_request_review and --require-pr-approval.
🪄 Autofix
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: defaults
Review profile: CHILL
Plan: Pro Plus
Run ID: 7a28dac5-2563-4abb-8842-b2c531f77493
📒 Files selected for processing (8)
.agent-loop/CURRENT_STATE.md.agent-loop/initiatives/WS-CI-002-deterministic-agent-gates/STATUS.md.agent-loop/initiatives/WS-CI-002-deterministic-agent-gates/chunks/WS-CI-002-01-single-head-gate.md.github/workflows/agent-gates.ymlbackend/scripts/check_guide_extractor_dependencies.pybackend/tests/test_guide_extractor_dependencies.pydocs/operations_backend_testing.mdscripts/test_lightweight_agent_gates.py
| self.assertNotIn("pull_request_review:", agent_gates) | ||
| self.assertNotIn("--require-pr-approval", agent_gates) | ||
| self.assertNotIn("pull-requests: read", agent_gates) |
There was a problem hiding this comment.
🔒 Security & Privacy | 🟡 Minor | ⚡ Quick win
Reject all pull-requests permissions.
The test rejects only pull-requests: read. A change to pull-requests: write would pass this test and restore PR access. Assert that pull-requests: is absent.
Proposed fix
- self.assertNotIn("pull-requests: read", agent_gates)
+ self.assertNotIn("pull-requests:", agent_gates)📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| self.assertNotIn("pull_request_review:", agent_gates) | |
| self.assertNotIn("--require-pr-approval", agent_gates) | |
| self.assertNotIn("pull-requests: read", agent_gates) | |
| self.assertNotIn("pull_request_review:", agent_gates) | |
| self.assertNotIn("--require-pr-approval", agent_gates) | |
| self.assertNotIn("pull-requests:", agent_gates) |
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@scripts/test_lightweight_agent_gates.py` around lines 124 - 126, Update the
permission assertions in the relevant test to reject any pull-requests
permission, not only the read variant. Assert that the broader “pull-requests:”
key is absent from agent_gates while preserving the existing checks for
pull_request_review and --require-pr-approval.
d1cb4ad to
bd22524
Compare
2 participants
Workstream PR Trust Bundle
Chunk
WS-CI-002-01— Single-Head Agent GateRoot cause
Agent Gates used one required check name for both PR-head validation and live
approval polling. Dependency-manifest changes therefore produced failures
before approval and new runs after review submission or dismissal. GitHub could
retain contradictory same-head contexts, leaving an approved, fully tested PR
blocked until every historical failure was manually rerun.
What changed
dependency command, and rejection of the retired CLI argument.
CI integrity
No dependency hashes, platform constraints, parser import boundaries, tests,
coverage floors, Backend lanes, or branch-protection requirements were weakened.
There is no fallback success or
continue-on-error.Evidence
Human review focus
Confirm that repository validation is deterministic per PR head and that
independent approval remains enforced by protected-branch review rules.
Human merge ownership
Summary by CodeRabbit