feat(#291): CodeRabbit rate limit tracker and quota management#303
Conversation
|
Important Review skippedAuto incremental reviews are disabled on this repository. Please check the settings in the CodeRabbit UI or the ⚙️ Run configurationConfiguration used: Path: .coderabbit.yaml Review profile: ASSERTIVE Plan: Pro Plus Run ID: You can disable this status message by setting the Use the checkbox below for a quick retry:
WalkthroughAdds a CodeRabbit review quota tracking feature: a new ChangesCodeRabbit Quota Feature
Estimated code review effort: 4 (Complex) | ~60 minutes Possibly related issues
🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
xsovad06
left a comment
xsovad06
left a comment
There was a problem hiding this comment.
PR Review: CodeRabbit Rate Limit Tracker (#291)
Summary: Well-structured feature adding CodeRabbit quota tracking across config, service, DB model, dashboard API, and UI widget. 1229 lines total, 682 of which are tests (55%). All CI checks pass, SonarCloud reports 100% coverage on new code.
Findings
[HIGH] Plan defaults may not match CodeRabbit actual limits
sova/config/models.py:368 -- The plan defaults {"free": 4, "pro": 5, "pro_plus": 10} are unverified for paid tiers. CodeRabbit Pro uses adaptive rate limiting, not a fixed per-hour cap. Consider defaulting paid plans to 0 (unlimited) since rate limiting is primarily a free-tier concern:
_plan_defaults = {"free": 4, "pro": 0, "pro_plus": 0}[HIGH] No Alembic migration for CodeRabbitEvent table
sova/db/models.py:347-363 -- New coderabbit_events table has no migration. Acknowledged in the PR body as "future PR", and mitigated by enabled=False default. Existing databases will crash with "no such table" if the feature is enabled before migration lands. Ensure the migration PR is tracked.
[MEDIUM] 20 sequential API calls on sync
sova/supervisor/coderabbit_quota.py:249-258 -- sync_from_github fetches reviews for up to 20 PRs individually (semaphore=5). Consider filtering by updated_at within the rolling window, or batching via GraphQL to reduce API calls.
[MEDIUM] Two coderabbit-prefixed config sections
sova/config/models.py:351 vs :286 -- [external_reviews.coderabbit] and [coderabbit_quota] coexist with similar env prefixes (SOVA_CODERABBIT_ vs SOVA_CODERABBIT_QUOTA_). The separation is justified (supervisor vs review-tool concerns) but may confuse users. Consider a note in settings descriptions referencing the relationship.
Verdict: Comment (no blocking issues)
The feature defaults to disabled, limiting blast radius. The two HIGH findings are worth addressing but neither requires blocking the PR.
What is done well
- Triple-registration discipline -- config model, loader, and settings metadata all present and correct per architecture rules.
- Thorough test coverage -- 28 test cases covering config defaults, service logic edge cases, API failures, bad JSON, deduplication, project isolation, and all router endpoints.
- Clean module placement -- new
sova/supervisor/subsystem with proper__init__.pydocstring sets up the namespace cleanly for future additions.
- Default paid plan quotas to 0 (unlimited) since CodeRabbit Pro/Pro+ use adaptive rate limiting, not fixed per-hour caps - Filter PR list by rolling window via `since` parameter to reduce GitHub API calls during sync - Add TODO for Alembic migration requirement on CodeRabbitEvent model - Add clarifying note in settings metadata distinguishing coderabbit_quota from external_reviews.coderabbit
Addressed review findingsAll four findings from PR review have been addressed in 30109a1: [HIGH] Plan defaults -- Changed paid plan defaults to [HIGH] No Alembic migration -- Added [MEDIUM] 20 sequential API calls -- Added [MEDIUM] Two coderabbit config sections -- Added a clarifying note to the All 3011 tests pass, lint clean. |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 3
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@sova/db/models.py`:
- Around line 347-368: Add the missing Alembic migration to create the
coderabbit_events table used by CodeRabbitEvent, including its columns, unique
constraint uq_coderabbit_event_review, and the recorded_at/project_slug indexes.
Make sure the migration is compatible with the existing CodeRabbitEvent model in
sova/db/models.py and supports the quota flow in
sova/supervisor/coderabbit_quota.py so reads and writes won’t hit a missing
table.
In `@sova/supervisor/coderabbit_quota.py`:
- Around line 217-244: The pulls listing in the quota check is still sending a
`since` parameter to `gh api repos/{repo}/pulls`, but that endpoint ignores it.
Remove the `since` field from the `run(...)` call in the quota logic and, if you
need time-based pruning, apply it after fetching by filtering on the returned
PRs’ updated timestamp or switch to the search-based flow.
- Around line 73-89: The next_available_minutes calculation in the quota logic
is using the oldest event, which under-reports when the window has more than the
allowed number of reviews. Update the quota calculation in coderabbit_quota.py
so QuotaStatus.next_available_minutes is based on the (reviews_in_window -
reviews_per_hour + 1)-th oldest event instead of min(e.recorded_at), and keep
the timezone handling and remaining-time math in the same quota/status flow.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: ASSERTIVE
Plan: Pro Plus
Run ID: 46fb8335-da31-42a6-9000-33676cb24d98
📒 Files selected for processing (10)
sova/config/loader.pysova/config/models.pysova/dashboard/app.pysova/dashboard/routers/quota.pysova/dashboard/settings_meta.pysova/dashboard/templates/agents.htmlsova/db/models.pysova/supervisor/__init__.pysova/supervisor/coderabbit_quota.pytests/test_coderabbit_quota.py
SonarCloud flagged json.JSONDecodeError as redundant when caught alongside ValueError (its parent). Remove the redundant class. Add 16 tests covering all previously uncovered code paths in coderabbit_quota.py: unlimited quota, empty sync, default timestamps, GitHub API fetch (success/failure/bad JSON/exceptions), and PR review parsing (non-CR users, missing fields, PENDING state, bad dates, null login). Coverage: 56% -> 100%.
Cover all paths in sova/dashboard/routers/quota.py: - GET /api/quota/coderabbit with enabled config (success + error) - POST /api/quota/coderabbit/sync (success, no-repo, error) - Fix line-length violations in test file Raises new-code coverage from 61.4% to meet SonarCloud 80% gate.
- Default paid plan quotas to 0 (unlimited) since CodeRabbit Pro/Pro+ use adaptive rate limiting, not fixed per-hour caps - Filter PR list by rolling window via `since` parameter to reduce GitHub API calls during sync - Add TODO for Alembic migration requirement on CodeRabbitEvent model - Add clarifying note in settings metadata distinguishing coderabbit_quota from external_reviews.coderabbit
xsovad06
dismissed
coderabbitai[bot]’s stale review
Findings addressed in latest push.
Closes #291
|
- Default paid plan quotas to 0 (unlimited) since CodeRabbit Pro/Pro+ use adaptive rate limiting, not fixed per-hour caps - Filter PR list by rolling window via `since` parameter to reduce GitHub API calls during sync - Add TODO for Alembic migration requirement on CodeRabbitEvent model - Add clarifying note in settings metadata distinguishing coderabbit_quota from external_reviews.coderabbit
1 participant
Summary
Changes
Config System
[coderabbit]section withenabled,plan,reviews_per_hour,min_pr_spacing_minutesfieldsCore Service (
sova/supervisor/coderabbit_quota.py)get_review_history()- queries GitHub API for coderabbitai[bot] PR reviews with DB cachingcan_create_pr()- checks quota availability against rolling 1-hour windownext_available_slot()- calculates when next review slot opensrequest_review()- posts@coderabbitai reviewcomment on PRsget_review_status()- distinguishes reviewed/rate_limited/pending/summary_only statesDatabase
CodeRabbitEventmodel with repo/pr_number/event_type/recorded_at fieldsDashboard
/api/quota/coderabbitendpoint returning quota statuscoderabbit.enabledconfigReview guidance
Key design decisions:
CodeRabbitQuotaConfigfrom existingCodeRabbitConfig(external_reviews) - quota tracking is supervisor-level concern, not review-tool integration/pulls/{pr}/reviewsnot/issues/{pr}/commentsrecorded_at > now - timedelta(hours=1))Verification focus:
repofieldTest plan
Closes #291