Add babysit-review skill - #125
Merged
Merged
Conversation
Sibling to babysit-prs, inverted for external contributions: check CI, re-run failures classified as flaky, and report the real ones. --all sweeps every open PR I've approved. The only GitHub write is re-running a flaky workflow; it never edits, pushes, merges, or comments. A small per-commit ledger (~/.local/state/babysit-review/reruns.json, keyed by run ID) bounds re-runs to once per commit, so a deterministic failure the classifier mislabels as flaky surfaces as a real failure after one re-run instead of looping under /loop.
There was a problem hiding this comment.
Pull request overview
Adds a new babysit-review skill intended to “babysit” PRs you’re reviewing by sweeping CI status, re-running failures classified as flaky (at most once per commit via a local ledger), and reporting real failures for follow-up.
Changes:
- Introduces
ai/skills/babysit-review/SKILL.mddefining the skill’s scope, argument contract, and step-by-step procedure. - Specifies a local rerun ledger at
~/.local/state/babysit-review/reruns.jsonto prevent repeated reruns of the same failing run. - Documents direct usage of
ci-monitorscripts for CI rollup, log fetching, and failure classification.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adds
ai/skills/babysit-review-- a skill for babysitting PRs I'm reviewing (external contributions), as the read-only mirror ofbabysit-prs.For each PR, it checks CI, re-runs failures classified as flaky, and reports the real ones so I can decide whether to follow up with the contributor. It drives the
ci-monitorscripts directly rather than invoking the skill, so it can sweep many PRs without the interactive fix/poll cycle. The only write action to GitHub isgh run rerun --failedon an existing run.A small ledger at
~/.local/state/babysit-review/reruns.jsontracks which run IDs have already been re-run, keyed by run ID (which is stable across re-runs but changes on a new commit). This bounds each flaky-looking failure to one re-run per commit: if it fails again after the re-run, the next sweep reports it as real instead of re-running it indefinitely.Test plan
/babysit-review <pr-url>against a PR with a green CI run: expect the "green" row in the summary table.gh run rerunfired, run ID recorded in the ledger./babysit-review --all --dry-run: verify nogh run reruncalls and the ledger file is not written or modified.