Skip to content

feat(renovate): bound fleet maintenance intake - #2893

Merged
stranske merged 2 commits into
mainfrom
codex/issue-2877-renovate-maintenance-windows
Aug 1, 2026
Merged

feat(renovate): bound fleet maintenance intake#2893
stranske merged 2 commits into
mainfrom
codex/issue-2877-renovate-maintenance-windows

Conversation

@stranske

@stranske stranske commented Aug 1, 2026

Copy link
Copy Markdown
Owner

Source: Issue #2877

Closes #2877

Automated Status Summary

Scope

renovate-presets/fleet.json:4-40 groups minor/patch updates but leaves digest updates outside the GitHub Actions group and does not define a schedule, commit rate, branch limit, PR limit, release age, PR-creation gate, or major-update approval. In the 2026-06-14..2026-08-01 audit window, 460 of 974 retrievable Renovate PRs were digest updates; grouped non-major work merged 174/180 while frequent digest families produced much more discarded work. This is a current intake-control gap, not evidence that routine Renovate grouping should be removed.

Tasks

  • Update renovate-presets/fleet.json with a documented weekly 4-hour schedule, commitHourlyLimit, prConcurrentLimit, and branchConcurrentLimit set to conservative fleet values.
  • Extend the GitHub Actions package rule at renovate-presets/fleet.json:29-35 to group digest, pin, minor, and patch updates for trusted actions.
  • Add routine-update minimumReleaseAge and prCreation policy without applying those delays to vulnerability alerts.
  • Add a major-update rule using dependencyDashboardApproval while preserving visible, actively handled majors.
  • Define grouped lock-file maintenance cadence explicitly instead of inheriting an always-available default.
  • Add tests/workflows/test_renovate_fleet_policy.py to parse the real preset and assert the schedule, limits, security bypass, digest grouping, dev-tool exclusions, and major lane.
  • Update docs/ci/TOOL_VERSION_MANAGEMENT.md, docs/WORKFLOW_GUIDE.md, and renovate-presets/README.md with the intake budget and emergency/security path.
  • Add the Renovate config validator command to scripts/dev_check.sh or the existing dependency-policy validation job.

Acceptance criteria

  • python -m pytest tests/workflows/test_renovate_fleet_policy.py tests/workflows/test_dependency_bot_conditions.py -q passes.
  • Renovate config validation accepts renovate.json, renovate-presets/fleet.json, and any inherited generated preset.
  • A representative Renovate dry-run for Workflows and one consumer reports at most three concurrent routine branches and groups trusted GitHub Actions digest updates into one branch.
  • The dry-run evidence shows a routine non-security release is held by the maintenance/release-age policy while a vulnerability-alert update is not held.
  • Deliberate-break gate: temporarily remove digest from the GitHub Actions group; tests/workflows/test_renovate_fleet_policy.py::test_trusted_action_digests_are_grouped must fail. Restore the rule before review.

@stranske stranske added agent:codex Agent-created issues from Codex agents:keepalive Use to initiate keepalive functionality with agents autofix Opt-in automated formatting & lint remediation labels Aug 1, 2026
Copilot AI review requested due to automatic review settings August 1, 2026 20:14
@stranske stranske added agent:retry Add to trigger agent retry after rate limit or pause agent:codex Agent-created issues from Codex agents:keepalive Use to initiate keepalive functionality with agents autofix Opt-in automated formatting & lint remediation labels Aug 1, 2026
@cursor

cursor Bot commented Aug 1, 2026

Copy link
Copy Markdown

Bugbot is not enabled for your account, so this pull request was not reviewed.

Enable Bugbot in the Cursor dashboard to get automatic reviews on future PRs.

@coderabbitai

coderabbitai Bot commented Aug 1, 2026

Copy link
Copy Markdown

Warning

Review limit reached

You’ve reached a temporary PR review limit under our Fair Usage Limits Policy.

Your recent review volume is higher than typical usage, so adaptive limits are currently applied.

Next review available in: 9 minutes

Your organization has reached its usage spending cap. Adjust your spending cap in the billing tab.

How can I continue?

After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews.

How do review limits work?

CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability.

For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window.

Please refer docs for additional details.

Review details
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Pro

Run ID: 9c5be482-11a4-47d1-a20b-342914f6721c

📥 Commits

Reviewing files that changed from the base of the PR and between d151eec and d2f540c.

📒 Files selected for processing (7)
  • docs/WORKFLOW_GUIDE.md
  • docs/ci/TOOL_VERSION_MANAGEMENT.md
  • docs/ops/CONSUMER_REPO_MAINTENANCE.md
  • renovate-presets/README.md
  • renovate-presets/fleet.json
  • scripts/dev_check.sh
  • tests/workflows/test_renovate_fleet_policy.py

Comment @coderabbitai help to get the list of available commands.

@github-actions

github-actions Bot commented Aug 1, 2026

Copy link
Copy Markdown
Contributor

Runner dispatch state for codex on PR #2893. Do not edit.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 18645eaa29

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread renovate-presets/fleet.json
Comment thread scripts/dev_check.sh

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Implements the bounded Renovate fleet intake policy described in Issue #2877 by tightening the shared renovate-presets/fleet.json preset (maintenance window, concurrency/commit budgets, routine gating, vulnerability bypass, major-update lane) and wiring in local validation plus documentation/tests so the contract is enforced across Workflows + consumer entrypoints.

Changes:

  • Add a weekly 4-hour maintenance window, conservative concurrency limits, routine PR gating, vulnerability-alert overrides, major-update dashboard approval, and explicit lock-file cadence to the fleet preset.
  • Add a dedicated pytest suite to assert key Renovate fleet policy invariants and that both Workflows + consumer templates extend the shared preset.
  • Add Renovate config validation to scripts/dev_check.sh and document the fleet intake budget/escape hatch in relevant docs.

Reviewed changes

Copilot reviewed 7 out of 7 changed files in this pull request and generated 1 comment.

Show a summary per file
File Description
tests/workflows/test_renovate_fleet_policy.py Adds regression tests for the Renovate fleet intake policy and shared entrypoint preset extension.
scripts/dev_check.sh Adds an npx renovate-config-validator quick-check step for Renovate configs/preset.
renovate-presets/README.md Documents the fleet preset’s intake budget, lanes, and the validation command.
renovate-presets/fleet.json Defines the bounded intake window/limits, vulnerability bypass, major dashboard approval, digest grouping, and lock-file cadence.
docs/WORKFLOW_GUIDE.md Documents the new “Fleet Renovate intake” maintenance surface and its policy.
docs/ops/CONSUMER_REPO_MAINTENANCE.md Documents that consumers inherit the same Renovate intake policy via the fleet preset.
docs/ci/TOOL_VERSION_MANAGEMENT.md Adds a Renovate intake validation section and the validator command for local checks.

Comment thread tests/workflows/test_renovate_fleet_policy.py
@agents-workflows-bot

Copy link
Copy Markdown
Contributor

🤖 Bot Comment Handler

  • Agent: codex
  • Bot comments to address: 3

The agent has been assigned to this PR to address the bot review comments.

Instructions for agent

  1. Implement suggested fixes that improve the code
  2. Skip suggestions that don't apply (note why in your response)

The bot comment handler workflow has prepared context in the artifacts.

@stranske
stranske force-pushed the codex/issue-2877-renovate-maintenance-windows branch from 18645ea to d2f540c Compare August 1, 2026 20:29
@stranske

stranske commented Aug 1, 2026

Copy link
Copy Markdown
Owner Author

Closer review recovery: resolved all three current findings in d2f540c. Documentation now accurately says prCreation=not-pending waits for update-branch checks (not base-branch checks); dev_check.sh --changed no longer exits before Renovate validation when only Renovate JSON changes; and the policy test now pins the full Workflows-owned dev-tool exclusion set. Validation: bash -n scripts/dev_check.sh; python -m pytest -q tests/workflows/test_renovate_fleet_policy.py tests/workflows/test_dependency_bot_conditions.py (10 passed); Renovate 43.285.3 validator for all three entrypoints; check_gate_diff_quality.py --base origin/main --head HEAD; and git diff --check.

@stranske
stranske merged commit 18f25a9 into main Aug 1, 2026
46 checks passed
@stranske
stranske deleted the codex/issue-2877-renovate-maintenance-windows branch August 1, 2026 20:38
@stranske stranske added the verify:compare Compare multiple LLM evaluations label Aug 1, 2026
@github-actions

github-actions Bot commented Aug 1, 2026

Copy link
Copy Markdown
Contributor

Provider Comparison Report

Provider Summary

Provider Model Verdict Confidence Summary
openai gpt-5.6-terra CONCERNS 82% The implementation appears well scoped and maintainable: it documents a weekly intake budget, adds conservative commit/PR/branch limits, groups trusted GitHub Actions digest/pin/minor/patch updates...
anthropic claude-sonnet-5 PASS 62% The PR adds a new test file (tests/workflows/test_renovate_fleet_policy.py), extends renovate-presets/fleet.json, updates three documentation files plus a new renovate-presets/README.md, and adds a...
📋 Full Provider Details (click to expand)

openai

  • Model: gpt-5.6-terra
  • Verdict: CONCERNS
  • Confidence: 82%
  • Scores:
    • Correctness: 8.0/10
    • Completeness: 7.0/10
    • Quality: 9.0/10
    • Testing: 8.0/10
    • Risks: 8.0/10
  • Summary: The implementation appears well scoped and maintainable: it documents a weekly intake budget, adds conservative commit/PR/branch limits, groups trusted GitHub Actions digest/pin/minor/patch updates, defines routine release-age and PR-creation policy, adds a major approval lane and explicit lock-file cadence, and adds focused policy tests plus a validator invocation. The primary remaining gap is acceptance evidence for Renovate's resolved runtime behavior across both this repository and a consumer. Static assertions are useful and include the requested digest grouping regression gate, but they do not replace the required dry-run demonstrations of branch limiting and security-update bypass behavior.
  • Concerns:
    • The change adds structural policy tests and config validation, but no representative Renovate dry-run output or committed evidence demonstrates the behavioral acceptance criteria: no more than three routine branches, digest updates being coalesced into one trusted-GitHub-Actions branch, and the different handling of routine versus vulnerability-alert updates.
    • The added tests primarily inspect JSON fields in the preset. They cannot establish that the effective inherited Renovate configuration, including Renovate's special vulnerability-alert processing and schedule/limit interactions, behaves as intended for Workflows and a consumer repository.
    • Although the validator command was added to dev_check.sh, the acceptance criterion calls for validation of renovate.json, the fleet preset, and inherited/generated presets. The available change summary does not establish that generated/inherited consumer configurations are actually materialized and validated rather than only validating the two static source files.

anthropic

  • Model: claude-sonnet-5
  • Verdict: PASS
  • Confidence: 62%
  • Scores:
    • Correctness: 7.0/10
    • Completeness: 6.0/10
    • Quality: 7.0/10
    • Testing: 7.0/10
    • Risks: 6.0/10
  • Summary: The PR adds a new test file (tests/workflows/test_renovate_fleet_policy.py), extends renovate-presets/fleet.json, updates three documentation files plus a new renovate-presets/README.md, and adds a validator step to scripts/dev_check.sh — matching the general shape of the required tasks. CI shows the Python test suite (which should include the new test file) passed in selftest-ci.yml, giving reasonable confidence the pytest acceptance criterion is satisfied. However, the fleet.json diff is quite small for the breadth of policy changes required, and I could not fully verify from the truncated diff that every specific field (minimumReleaseAge, prCreation, dependencyDashboardApproval major rule, explicit lock-file maintenance schedule) was implemented as specified. Additionally, several acceptance criteria are evidence/process-based (dry-run reports, deliberate-break gate) and are inherently unverifiable from a git diff; these depend on reviewer attestation rather than code inspection. Given passing CI and file coverage aligned with the task list, but with some completeness/verification gaps, this is a PASS with moderate confidence rather than high confidence.
  • Concerns:
    • The fleet.json diff is very small (+26/-2 lines) to plausibly cover all required policy additions (schedule, commitHourlyLimit, prConcurrentLimit, branchConcurrentLimit, extended digest/pin/minor/patch grouping, minimumReleaseAge, prCreation, major-update dependencyDashboardApproval rule, explicit lock-file maintenance cadence). Full diff content for fleet.json was not available to verify each item was actually implemented rather than partially covered.
    • Two acceptance criteria (representative Renovate dry-run evidence for Workflows + a consumer repo, and evidence that a routine release is held while a vulnerability alert is not) are process/evidence-based checks that cannot be verified from the code diff alone; no artifacts or evidence files were included in the changed file list.
    • The deliberate-break gate acceptance criterion is a manual verification step (temporarily break the rule, confirm test fails, then restore) — this is not something reflected in the merged diff and must be taken on faith that it was performed correctly during review.
    • Full content of the new test file (99 lines) was not fully visible in the truncated diff, so exact coverage of schedule/limits/security-bypass/digest-grouping/dev-tool-exclusions/major-lane assertions cannot be independently confirmed beyond file size and CI success signal.

Agreement

  • Correctness: scores within 1 point (avg 7.5/10, range 7.0-8.0)
  • Completeness: scores within 1 point (avg 6.5/10, range 6.0-7.0)
  • Testing: scores within 1 point (avg 7.5/10, range 7.0-8.0)

Disagreement

Dimension openai anthropic
Verdict CONCERNS PASS
Quality 9.0/10 7.0/10
Risks 8.0/10 6.0/10

Unique Insights

  • openai: The change adds structural policy tests and config validation, but no representative Renovate dry-run output or committed evidence demonstrates the behavioral acceptance criteria: no more than three routine branches, digest updates being coalesced into one trusted-GitHub-Actions branch, and the different handling of routine versus vulnerability-alert updates.; The added tests primarily inspect JSON fields in the preset. They cannot establish that the effective inherited Renovate configuration, including Renovate's special vulnerability-alert processing and schedule/limit interactions, behaves as intended for Workflows and a consumer repository.; Although the validator command was added to dev_check.sh, the acceptance criterion calls for validation of renovate.json, the fleet preset, and inherited/generated presets. The available change summary does not establish that generated/inherited consumer configurations are actually materialized and validated rather than only validating the two static source files.
  • anthropic: The fleet.json diff is very small (+26/-2 lines) to plausibly cover all required policy additions (schedule, commitHourlyLimit, prConcurrentLimit, branchConcurrentLimit, extended digest/pin/minor/patch grouping, minimumReleaseAge, prCreation, major-update dependencyDashboardApproval rule, explicit lock-file maintenance cadence). Full diff content for fleet.json was not available to verify each item was actually implemented rather than partially covered.; Two acceptance criteria (representative Renovate dry-run evidence for Workflows + a consumer repo, and evidence that a routine release is held while a vulnerability alert is not) are process/evidence-based checks that cannot be verified from the code diff alone; no artifacts or evidence files were included in the changed file list.; The deliberate-break gate acceptance criterion is a manual verification step (temporarily break the rule, confirm test fails, then restore) — this is not something reflected in the merged diff and must be taken on faith that it was performed correctly during review.; Full content of the new test file (99 lines) was not fully visible in the truncated diff, so exact coverage of schedule/limits/security-bypass/digest-grouping/dev-tool-exclusions/major-lane assertions cannot be independently confirmed beyond file size and CI success signal.

🔍 LangSmith Traces

@stranske

stranske commented Aug 1, 2026

Copy link
Copy Markdown
Owner Author

Closer verifier audit (split openai CONCERNS / anthropic PASS on comment 5153358352):

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

agent:codex Agent-created issues from Codex agent:retry Add to trigger agent retry after rate limit or pause agents:keepalive Use to initiate keepalive functionality with agents autofix Opt-in automated formatting & lint remediation verify:compare Compare multiple LLM evaluations

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Dependency/Sync] Add Renovate maintenance windows, limits, and digest grouping

3 participants