tooling: add deterministic CI state classifier command#4578
tooling: add deterministic CI state classifier command#4578davidahmann wants to merge 3 commits intocrewAIInc:mainfrom
Conversation
|
Implemented issue #4576 with a deterministic CI classifier command at , including policy-block detection, fixture/live modes, and self-test. Validation: self-test passed. This contribution was informed by patterns from Wrkr. Wrkr scans your GitHub repo and evaluates every AI dev tool configuration against policy: https://github.com/Clyra-AI/wrkr |
|
Implementation summary: added scripts/ci_state_classifier.py with deterministic state categories, fixture/live input modes, policy-block detection, and built-in self-test checks. Validation: classifier self-test passed. This contribution was informed by patterns from Wrkr. Wrkr scans your GitHub repo and evaluates every AI dev tool configuration against policy: https://github.com/Clyra-AI/wrkr |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: b889e91276
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| from pathlib import Path | ||
| from typing import Any | ||
|
|
||
| FAIL_VALUES = {"FAILURE", "ERROR", "TIMED_OUT", "CANCELLED", "ACTION_REQUIRED"} |
There was a problem hiding this comment.
Treat STARTUP_FAILURE/STALE conclusions as failed
GitHub check runs can finish with conclusions like STARTUP_FAILURE or STALE, but FAIL_VALUES omits both, so classify() falls through to passed when these are the only non-success outcomes. In repos where those conclusions occur (e.g., failed bootstraps or stale runs), this produces a false-green CI classification and can incorrectly signal that the PR is safe to merge.
Useful? React with 👍 / 👎.
| from typing import Any | ||
|
|
||
| FAIL_VALUES = {"FAILURE", "ERROR", "TIMED_OUT", "CANCELLED", "ACTION_REQUIRED"} | ||
| PENDING_VALUES = {"PENDING", "QUEUED", "IN_PROGRESS", "REQUESTED", "WAITING"} |
There was a problem hiding this comment.
Classify EXPECTED status as pending
_to_checks() maps commit status state into both conclusion and status, but PENDING_VALUES does not include EXPECTED. For status contexts that are required but not yet reported (state EXPECTED), classify() currently returns passed instead of a non-terminal state, which misrepresents PR readiness while required checks are still outstanding.
Useful? React with 👍 / 👎.
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes and found 1 potential issue.
Bugbot Autofix is OFF. To automatically fix reported issues with Cloud Agents, enable Autofix in the Cursor dashboard.
98792c5 to
a3bb111
Compare
|
This PR is stale because it has been open for 45 days with no activity. |
Problem
PR check-state interpretation is currently manual and can misclassify sparse/no-check states.
Why Now
Cross-repo coordination requires deterministic CI state categories.
What Changed
scripts/ci_state_classifier.py.passed,failed,pending,no_checks,policy_blocked.Validation
python3 scripts/ci_state_classifier.py --self-test✅Refs #4576
Note
Low Risk
Adds a standalone tooling script with no runtime impact on production code; main risk is misclassification due to heuristic policy-name matching and status normalization.
Overview
Adds a new
scripts/ci_state_classifier.pyCLI that fetches (viagh pr view) or loads a JSONstatusCheckRolluppayload and deterministically classifies PR check state aspassed,failed,pending,no_checks, orpolicy_blocked.The classifier normalizes check fields, treats known failure/pending statuses specially, and separates policy-style checks (e.g., CLA/DCO/Code Owners) so they can block without marking the PR as failed; output is stable JSON with optional pretty-printing plus a built-in
--self-test.Written by Cursor Bugbot for commit a3bb111. This will update automatically on new commits. Configure here.