feat(bin): fm-coderabbit-poll.sh — hourly host-cron CodeRabbit review babysitter#281
Open
depohmel wants to merge 6 commits into
Open
feat(bin): fm-coderabbit-poll.sh — hourly host-cron CodeRabbit review babysitter#281depohmel wants to merge 6 commits into
depohmel wants to merge 6 commits into
Conversation
…abbit reviews Periodic host-cron poll that watches a captain-maintained queue of GitHub PRs, detects new CodeRabbit reviews via the GitHub reviews API, and dispatches a scoped one-shot crewmate to address each new actionable review. Designed for CodeRabbit free-tier's org-wide 1-review-per-hour rate limit: run hourly via host cron, at most one crewmate per new review. - bin/fm-coderabbit-poll.sh — the poll script. flock-guarded, watermarked per PR, spawns via existing fm-spawn.sh on opencode+litellm/coder. - docs/coderabbit-poll.md — setup, cadence, file layout, watermark semantics, crontab example. - docs/examples/coderabbit-queue.txt — commented example queue. Fixes are constrained by an explicit crewmate brief: docs and minor fixes only, declines destructive/scope-creep findings via PR comment rather than waking firstmate, never merges or force-pushes. Escalation reserved for hard blockers (auth, missing tools), not for judgment calls. Smoke-tested against empty/absent/malformed queue files locally.
…ing fixes If the crewmate actually pushed commits, it also posts a second PR comment "@coderabbitai review" to trigger a fresh review of the fixed diff. Skipped when the crewmate declined every finding — a re-review with no code change wastes the org-wide 1-per-hour quota. Also documents this in docs/coderabbit-poll.md.
…in the crewmate flow
…tier names Addresses the pipeline review finding that `bin/fm-coderabbit-poll.sh` baked in a specific LiteLLM tier alias, which is operator-specific infrastructure that should not appear in shared/public firstmate code. - FM_CODERABBIT_POLL_HARNESS - defaults to `opencode` (public, always available on a stock firstmate install) - FM_CODERABBIT_POLL_MODEL - when unset we do NOT pass --model at all; fm-spawn then resolves the model through config/crew-dispatch.json and the harness default, keeping any private tier names out of this script Docs updated to reflect the env-var-driven, no-hardcoded-default behavior.
…, log every decision Before this commit, the poll only ACTED on CodeRabbit reviews that had already been posted. If CodeRabbit had never been triggered on a queued PR, the poll would fire hourly and correctly report "nothing new" - but the actual work of kicking off the first review still had to be done by hand. Also: PRs that had been merged or closed stayed in the queue and their reviews kept being consumed against the org quota on every fire. Changes: - process_pr now checks pr.state first. Non-open PRs (merged, closed) are skipped with a log line, so the queue can safely keep them without wasting the CodeRabbit quota. - When there is NO existing CodeRabbit review on an open PR, the poll posts a single "@coderabbitai review" comment - at most one per fire cycle (REVIEW_REQUESTED_THIS_CYCLE), so the free-tier one-per-hour quota is respected. Remaining unreviewed PRs will get their request on subsequent hourly fires. - Every decision now logs a one-line reason (state=closed skip, no review yet, watermark match, N actionable dispatched, spawn failed), so the cron log becomes an audit trail: wake up, read one screen, understand what happened. Docs updated to reflect the new per-PR decision flow.
4 tasks
…uring cooldown Two fixes so the poll serves both a captain's own repos (paid coderabbitai[bot]) and OSS repos they contribute to (coderabbit-oss[bot]): - Match either bot login in the review query and everywhere else via a shared BOT_JQ_FILTER. Detect which variant is active on a given PR from the newest bot activity (review or issue comment) and mirror the @mention accordingly (@coderabbitai vs @coderabbit-oss). Fall back to @coderabbitai on PRs with no prior bot activity. - Before posting @... review, scan the newest bot issue comment for a "Review limit reached" cooldown notice. If present, log it (with any parseable countdown) and skip - otherwise every hourly fire burned a queued-request slot while CodeRabbit was in cooldown. The existing REVIEW_REQUESTED_THIS_CYCLE gate is preserved.
4 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adds a host-cron poller that watches a captain-maintained queue of GitHub PRs, detects new CodeRabbit reviews, and dispatches a scoped one-shot crewmate per new actionable review.
Motivation
CodeRabbit free-tier's rate limit is 1 review per hour for the whole org. Existing supervision has no hook for "new external bot review posted on my PR"; a captain either polls manually or leaves reviews unaddressed. Session-based
bin/fm-spawn.shbabysitters (tried on this fleet earlier this week) proved brittle for long-idle polling — the model wandered, hallucinated findings, or falsely declared clean without waiting for CodeRabbit to actually review.Host-cron sidesteps both problems: strict hourly cadence, no live firstmate needed, and each fire spawns a one-shot crewmate with a scoped brief containing the exact review body.
What's here
bin/fm-coderabbit-poll.sh— the poll.flock-guarded, per-PR watermarks instate/coderabbit-watermarks.tsv, spawns via existingbin/fm-spawn.sh.docs/coderabbit-poll.md— setup, cadence, files, watermark semantics, crontab example.docs/examples/coderabbit-queue.txt— commented example queue.Design decisions worth review
CronCreate: doesn't depend on firstmate being live. Simpler failure mode (cron log tells you).submitted_attimestamp, not id: naturally matches CodeRabbit's re-review flow; a fresh review with a newer timestamp triggers dispatch.Actionable comments posted: Nmarker; N=0 advances watermark without spawn. If CodeRabbit changes this format, the poll would need an update — this is the load-bearing external contract.Tests
Smoke-tested locally against:
poll cycle complete, exit 0.poll cycle complete, exit 0.poll cycle complete, exit 0.Not exercised end-to-end against a live CodeRabbit review here (would double-dispatch on the fleet's own in-flight PRs).
Not in this PR
data/coderabbit-queue.txt,state/coderabbit-watermarks.tsv,state/coderabbit-poll.lock) to section 2's layout list.tests/dir. The script is mostly shell glue overgh api+fm-spawn.sh; a full test harness would need to mock the GitHub API.CronCreatevariant. Deferred until there's a concrete need for firstmate-driven scheduling.🤖 Written by Claude Code
Pipeline
Updates from git push no-mistakes
Note: the no-mistakes pipeline was driven manually on this PR —
axi runcompleted intent/rebase/review/test/document/lint successfully; the auto-stamp step failed only because the scratch worktree had HTTPS origin without push credentials. The review pass emitted one actionable finding (the hardcoded LiteLLM tier name inbin/fm-coderabbit-poll.sh), which is fixed by commit2bb32a9on this branch (env-var-driven, no hardcoded operator-private tier names).