Skip to content

[change] Changelog bot: mitigate quota exhaustion and improved consistency#693

Merged
nemesifier merged 2 commits into
masterfrom
changelog-bot-model
Jun 3, 2026
Merged

[change] Changelog bot: mitigate quota exhaustion and improved consistency#693
nemesifier merged 2 commits into
masterfrom
changelog-bot-model

Conversation

@nemesifier
Copy link
Copy Markdown
Member

@nemesifier nemesifier commented Jun 3, 2026

Checklist

  • I have read the OpenWISP Contributing Guidelines.
  • I have manually tested the changes proposed in this pull request.
  • I have written new test cases for new code and/or updated existing tests for changes to existing code.
  • I have updated the documentation.

Reference to Existing Issue

No existing issue, it's a relatively small and trivial change.

Description of Changes

This PR hardens the changelog bot against Gemini API quota limits, adds model configurability, and ensures the GitHub Action workflow exits gracefully instead of failing when quotas are exhausted.

Context

The changelog bot was frequently failing workflows due to hitting the Gemini free-tier quota (RequestsPerDayPerProjectPerModel, 20 requests/day). This was compounded by two issues:

  1. The bot lacked a way to switch models.
  2. It could consume up to 3 requests per individual pull request.

Unlike the CI failure bot (which swallows these errors and stays green), the changelog bot would cause the entire workflow to turn red.

Changes

  • Configurable Models: Forwarded the GEMINI_MODEL variable through the action and reusable workflow, allowing a single variable to control both the changelog and CI failure bots.
  • Fallback Handling: Hardened model resolution to automatically fall back to the default model if the provided value is empty.
  • Reduced API Overhead: Lowered the generation retry limit from 3 to 2 to conserve daily quota.
  • Graceful Degradation: Modified the bot to exit cleanly on quota errors so that API exhaustion no longer triggers a workflow failure.
  • Deterministic Output: Normalized the changelog bot's output to ensure consistency across runs.

The changelog bot kept failing with Gemini free-tier quota errors
(RequestsPerDayPerProjectPerModel, 20/day) and marked the workflow as
failed, while the CI failure bot stayed green because it swallows the
same error. The changelog bot also had no way to switch models and
could spend up to three requests per pull request.

Forward GEMINI_MODEL through the action and reusable workflow so a
single variable controls both bots, harden model resolution so an
empty value falls back to the default, lower the generation retries
from three to two, and exit cleanly on quota errors so the workflow
no longer turns red on exhaustion.

Also document the GEMINI_MODEL variable for both bots.
@nemesifier nemesifier self-assigned this Jun 3, 2026
@nemesifier nemesifier added enhancement github_actions Pull requests that update GitHub Actions code helper-bots Helper bots, release management automation labels Jun 3, 2026
@coderabbitai
Copy link
Copy Markdown

coderabbitai Bot commented Jun 3, 2026

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: ASSERTIVE

Plan: Pro

Run ID: 6e274f32-c417-4f4b-9562-c974dbe5d521

📥 Commits

Reviewing files that changed from the base of the PR and between b57b938 and ada9cbc.

📒 Files selected for processing (5)
  • .github/actions/bot-changelog-generator/action.yml
  • .github/actions/bot-changelog-generator/generate_changelog.py
  • .github/actions/bot-changelog-generator/test_generate_changelog.py
  • .github/workflows/reusable-bot-changelog.yml
  • docs/developer/reusable-github-utils.rst
📜 Recent review details
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (14)
  • GitHub Check: Python==3.10 | django~=5.1.0
  • GitHub Check: Python==3.12 | django~=5.2.0
  • GitHub Check: Python==3.11 | django~=4.2.0
  • GitHub Check: Python==3.13 | django~=5.1.0
  • GitHub Check: Python==3.11 | django~=5.0.0
  • GitHub Check: Python==3.12 | django~=5.1.0
  • GitHub Check: Python==3.10 | django~=5.0.0
  • GitHub Check: Python==3.12 | django~=5.0.0
  • GitHub Check: Python==3.10 | django~=5.2.0
  • GitHub Check: Python==3.13 | django~=5.2.0
  • GitHub Check: Python==3.11 | django~=5.1.0
  • GitHub Check: Python==3.10 | django~=4.2.0
  • GitHub Check: Python==3.11 | django~=5.2.0
  • GitHub Check: Python==3.12 | django~=4.2.0
🧰 Additional context used
📓 Path-based instructions (1)
**/*.{md,rst,txt}

📄 CodeRabbit inference engine (AGENTS.md)

Update documentation when behavior, settings, public APIs, setup steps, QA rules, or supported versions change

Files:

  • docs/developer/reusable-github-utils.rst
🧠 Learnings (3)
📚 Learning: 2026-06-03T00:15:34.762Z
Learnt from: CR
Repo: openwisp/openwisp-utils PR: 0
File: AGENTS.md:0-0
Timestamp: 2026-06-03T00:15:34.762Z
Learning: Applies to **/*.{md,rst,txt} : Update documentation when behavior, settings, public APIs, setup steps, QA rules, or supported versions change

Applied to files:

  • docs/developer/reusable-github-utils.rst
📚 Learning: 2026-03-05T09:59:15.097Z
Learnt from: pushpitkamboj
Repo: openwisp/openwisp-utils PR: 584
File: .github/actions/bot-changelog-generator/generate_changelog.py:356-364
Timestamp: 2026-03-05T09:59:15.097Z
Learning: In generate_changelog.py, reviews should confirm that validate_changelog_output acts as an output safety filter: it should require the produced changelog text to begin with a valid tag ([feature], [fix], or [change]) and to include a correctly formed PR reference pattern. It should NOT require that the referenced PR number or URL exactly match the current PR. For review, check: (1) the first non-space token matches one of the allowed tags, (2) the PR reference pattern is present and well-formed (e.g., contains a recognizable PR identifier or URL), and (3) there are no additional hard-coded cross-checks that would make it overly strict. This pattern should apply to this file specifically and guide future changes to similar output-filter functions without assuming cross-repo constraints.

Applied to files:

  • .github/actions/bot-changelog-generator/generate_changelog.py
📚 Learning: 2026-06-03T00:15:34.762Z
Learnt from: CR
Repo: openwisp/openwisp-utils PR: 0
File: AGENTS.md:0-0
Timestamp: 2026-06-03T00:15:34.762Z
Learning: Applies to **/*.py : Add or update tests for every behavior change

Applied to files:

  • .github/actions/bot-changelog-generator/test_generate_changelog.py
🔇 Additional comments (23)
.github/actions/bot-changelog-generator/action.yml (2)

18-21: LGTM!


45-45: LGTM!

.github/workflows/reusable-bot-changelog.yml (1)

59-59: LGTM!

.github/actions/bot-changelog-generator/generate_changelog.py (11)

24-25: LGTM!


33-33: LGTM!


45-48: LGTM!


49-58: LGTM!


77-86: LGTM!


88-98: LGTM!


100-149: LGTM!


316-325: LGTM!


559-561: LGTM!


720-720: LGTM!


738-741: LGTM!

.github/actions/bot-changelog-generator/test_generate_changelog.py (7)

32-34: LGTM!


60-84: LGTM!


341-354: LGTM!


652-688: LGTM!


754-766: LGTM!


888-920: LGTM!


921-943: LGTM!

docs/developer/reusable-github-utils.rst (2)

427-434: LGTM!


562-572: LGTM!


📝 Walkthrough

Walkthrough

This PR enhances the changelog generator GitHub Action with configurable Gemini model selection and local output normalization. It adds a new optional gemini-model input to the action, introduces a resolve_model() helper that falls back to gemini-2.5-flash-lite when unset, and implements normalize_changelog_output() to reflow long body lines locally while preserving issue footers and blank lines. Gemini API quota and rate-limit errors are now detected and handled gracefully, emitting a warning and exiting successfully instead of failing the workflow. The generation flow applies normalization before validation to avoid unnecessary Gemini retries for deterministic formatting issues. MAX_GENERATION_ATTEMPTS is reduced from 3 to 2. The reusable workflow is updated to wire the input, and documentation is added covering model configuration and output normalization behavior.

Sequence Diagram

sequenceDiagram
  participant Workflow
  participant Action
  participant Script
  participant Gemini
  Workflow->>Action: inputs.gemini-model
  Action->>Script: GEMINI_MODEL env var
  Script->>Script: resolve_model()
  Script->>Gemini: call_gemini() with selected model
  Gemini->>Script: generated changelog or error
  alt quota/rate-limit error
    Script->>Script: emit warning
    Script->>Workflow: exit(0)
  else success
    Script->>Script: normalize_changelog_output()
    Script->>Script: validate
    alt valid after normalization
      Script->>Workflow: success
    else still invalid
      Script->>Gemini: retry (up to MAX=2)
    end
  end
Loading

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~20 minutes

Possibly related PRs

  • openwisp/openwisp-utils#682: Addresses long body lines in Gemini-produced messages—this PR adds local normalization wrapping, while the related PR tightens prompt rules and extends validation rejection.
  • openwisp/openwisp-utils#584: Original implementation of the Gemini changelog-generator action; this PR extends it with configurable model selection, output normalization, and updated error handling in the same codepath.
🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Title check ✅ Passed The title clearly and accurately summarizes the main changes: making the changelog bot more resilient to quota exhaustion and improving output consistency.
Description check ✅ Passed The description is comprehensive with all required sections completed: checklist items checked, issue reference provided (no existing issue noted), detailed change explanation with context and rationale, though no screenshot is included (not applicable here).
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Bug Fixes ✅ Passed Fixes quota exhaustion bug that caused workflow failure. Includes regression test test_exits_zero_on_quota_error() that verifies graceful exit; test fails without the fix.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch changelog-bot-model

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@github-project-automation github-project-automation Bot moved this from In progress to Reviewer approved in OpenWISP Priorities for next releases Jun 3, 2026
@nemesifier nemesifier moved this from Reviewer approved to Ready for review/testing in OpenWISP Priorities for next releases Jun 3, 2026
@nemesifier nemesifier merged commit 66f9a6f into master Jun 3, 2026
37 checks passed
@nemesifier nemesifier deleted the changelog-bot-model branch June 3, 2026 18:59
@github-project-automation github-project-automation Bot moved this from Ready for review/testing to Done in OpenWISP Priorities for next releases Jun 3, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement github_actions Pull requests that update GitHub Actions code helper-bots Helper bots, release management automation

Development

Successfully merging this pull request may close these issues.

1 participant