feat(opencode): add empirical review-quality gate - #829
Conversation
|
Warning Review limit reached
Next review available in: 26 minutes Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available. How can I continue?After more reviews become available, a review can be triggered using the To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews. How do review limits work?CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability. For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window. Please refer docs for additional details. Review details⚙️ Run configurationConfiguration used: Organization UI Review profile: CHILL Plan: Pro Plus Run ID: 📒 Files selected for processing (9)
Comment |
|
@opencode-agent address Exact-head RED→GREEN repair for current Draft head OpenCode Review Quality CI run
Also preserve and make GREEN the existing contracts for case-fold duplicate reviewer names, required presence of both candidate/reference reviewers on each exact-head case, and Run the two focused quality modules first, then exact production statement+branch coverage 100%, public docstring coverage 100%, compile/diff hygiene, full central tests, and normal Strix/security/supply-chain checks. Update doctoring/changelog only if the governing schema contract materially changes. Commit normally to the existing branch only if the pre-write head/blobs remain unchanged. Keep Draft; do not modify central dispatch/reviewer credentials/model policy, approve, merge, or introduce temporary/write-capable workflows. |
|
Exact-current-head review request for The complete exact-head quality/security surface is terminal-success, including OpenCode Review Quality CI and Strix Changed Path Quality CI. This PR is now Ready and auto-merge is armed only behind repository policy. Please perform read-only current-head review; do not modify the branch, synthesize approval, weaken tests, or merge. @coderabbitai review |
|
|
|
@opencode-agent @cwl-noema-review Review-only request for exact current head Inspect the complete nine-file diff, scorer semantics, strict evidence schema, exact-head binding, true-positive contract, CI workflow, APA 7 doctoring, and claim boundaries. Distinguish substantive findings from lifecycle-pilot limitations and from infrastructure-only states. Submit any formal review only on this exact head. Do not modify the branch, update the base, approve on behalf of an author, merge, release, weaken tests, alter reviewer credentials/model policy, or reuse predecessor-head evidence. |
![opencode-agent[bot]](https://avatars.githubusercontent.com/in/1549082?s=60&v=4){kind=small}
There was a problem hiding this comment.
Pull request overview
OpenCode could not approve from deterministic current-head evidence because GitHub Checks have failed.
Findings
1. HIGH Current-head GitHub Checks - Fix failed required checks before approval
- Problem: Failed same-head checks remain for
523e883b9088d1ac7edad3f3f60eb6ebdb0948a7. - Root cause: The model-unavailable evidence fallback is allowed only when peer GitHub Checks are complete and clean.
- Fix: Read and fix the failed check logs below, then rerun the current-head checks.
- Regression test: Keep the model-unavailable fallback gated on an empty failed-check rollup.
Failed checks:
- Strix Security Scan/strix: FAILURE (https://github.com/ContextualWisdomLab/.github/actions/runs/31254306146/job/93095273845)
- Strix Security Scan/strix: failure (https://github.com/ContextualWisdomLab/.github/actions/runs/31254306146/job/93095273845)
Changed-File Evidence Map
flowchart LR
PR["PR changed files"] --> Evidence["OpenCode bounded evidence"]
Evidence --> S1["Workflow: opencode-review-quality-ci.yml"]
S1 --> I1["GitHub Actions review job"]
I1 --> R1["Review risk: Workflow: opencode-review-quality-ci.yml"]
R1 --> V1["actionlint plus required checks"]
Evidence --> S2["Changed file (2 files)"]
S2 --> I2["repository behavior"]
I2 --> R2["Review risk: Changed file (2 files)"]
R2 --> V2["required checks"]
Evidence --> S3["Docs (3 files)"]
S3 --> I3["operator or user guidance"]
I3 --> R3["Review risk: Docs (3 files)"]
R3 --> V3["docs review"]
Evidence --> S4["CI script: opencode_review_quality_score.py"]
S4 --> I4["review and security gate shell path"]
I4 --> R4["Review risk: CI script: opencode_review_quality_score.py"]
R4 --> V4["bash -n plus Strix self-test"]
Evidence --> S5["Test (2 files)"]
S5 --> I5["regression suite"]
I5 --> R5["Review risk: Test (2 files)"]
R5 --> V5["targeted test run"]
OpenCode Review Overview
Pull request overviewOpenCode could not approve from deterministic current-head evidence because GitHub Checks have failed. Findings1. HIGH Current-head GitHub Checks - Fix failed required checks before approval
Failed checks:
Changed-File Evidence Mapflowchart LR
PR["PR changed files"] --> Evidence["OpenCode bounded evidence"]
Evidence --> S1["Workflow: opencode-review-quality-ci.yml"]
S1 --> I1["GitHub Actions review job"]
I1 --> R1["Review risk: Workflow: opencode-review-quality-ci.yml"]
R1 --> V1["actionlint plus required checks"]
Evidence --> S2["Changed file (2 files)"]
S2 --> I2["repository behavior"]
I2 --> R2["Review risk: Changed file (2 files)"]
R2 --> V2["required checks"]
Evidence --> S3["Docs (3 files)"]
S3 --> I3["operator or user guidance"]
I3 --> R3["Review risk: Docs (3 files)"]
R3 --> V3["docs review"]
Evidence --> S4["CI script: opencode_review_quality_score.py"]
S4 --> I4["review and security gate shell path"]
I4 --> R4["Review risk: CI script: opencode_review_quality_score.py"]
R4 --> V4["bash -n plus Strix self-test"]
Evidence --> S5["Test (2 files)"]
S5 --> I5["regression suite"]
I5 --> R5["Review risk: Test (2 files)"]
R5 --> V5["targeted test run"]
|
1 participant
Purpose
Establish an evidence-based quality program for the organization-wide OpenCode reviewer and prevent unearned claims that it has reached CodeRabbit parity.
Exact integration identity
6eb06cdd08c79a06f7b390069d4ffa49e2eb7dba(main);feat/opencode-review-quality-gate;523e883b9088d1ac7edad3f3f60eb6ebdb0948a7;Empirical pilot and limitation
The committed pilot records directly observed lifecycle evidence from three organization pull requests where both reviewers left observable GitHub records:
ContextualWisdomLab/disksage#140;ContextualWisdomLab/EgressWeave#62;ContextualWisdomLab/inkspan#65.This is a deliberately selected operational pilot, not a random sample and not a head-matched expert-gold benchmark. The benchmark therefore remains
INSUFFICIENT_EVIDENCEuntil at least 50 head-matched cases and 50 expert-gold findings exist.Observed lifecycle totals are encoded rather than asserted in prose: OpenCode produced eight completed, infrastructure-only reviews with zero source findings and five duplicate reviews; CodeRabbit produced three completed reviews with eight actionable source comments, with one directly observed rate-limit event. These values measure lifecycle yield and observed availability only. They do not establish precision, recall, or CodeRabbit superiority on identical heads.
Implemented quality contract
Exact-current-head verification
The authoritative OpenCode Review Quality CI run
31252748759checked out exact head523e883b9088d1ac7edad3f3f60eb6ebdb0948a7and completed successfully:54 passed;303/303;118/118;100%;INSUFFICIENT_EVIDENCEas required.The exact-head Strix Changed Path Quality CI run
31252748779succeeded. The latest exact-head CodeQL, Python Security, SAST Semgrep, Security Scan, Secret Scan, OSV, Scorecard, and SBOM workflows also completed successfully.CodeRabbit commit status is successful, but the latest full-review attempt was rate-limited and is not treated as a completed formal review or independent approval. Formal review submissions and inline review threads are currently absent.
Production architecture consequence
The current central dispatch can transform coverage-evidence failure into a synthetic source-level
REQUEST_CHANGESfinding before semantic review publication. This PR intentionally does not edit that actively contested workflow. The required follow-up architecture remains:review_verdictindependent frommerge_readiness;Safety and merge gate
COPILOT_GITHUB_TOKENuse;Merge only after a fresh exact-current-head automated review completes, a qualifying independent non-author formal approval exists, every required check and branch-protection rule passes, and no valid unresolved finding remains. Rate-limited, status-only, author-only, absent, queued, cancelled, stale-head, or predecessor evidence is not acceptance.