[fix] Fixed context scope to prevent hallucinations#607
Conversation
Previous method of fetching logs using gh view run was prone to weird context, whereas now it limits context and ensures it is correct.
|
Note Reviews pausedIt looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the Use the following commands to manage reviews:
Use the checkboxes below for quick actions:
📝 WalkthroughWalkthroughAdds an Sequence Diagram(s)sequenceDiagram
participant GitHubActor as GitHub Actor
participant ReusableWF as Reusable Workflow
participant TrustedRepo as trusted_scripts
participant LogFetcher as Log Collector
participant AIJob as analyze_failure.py
participant GitHubAPI as GitHub API
GitHubActor->>ReusableWF: trigger workflow (input: actor)
ReusableWF->>TrustedRepo: checkout ref=fix/ci-failure-bot
ReusableWF->>LogFetcher: enumerate failed jobs & steps
LogFetcher->>ReusableWF: failed_logs.txt
ReusableWF->>AIJob: run analysis (env: ACTOR, GEMINI_MODEL from vars)
AIJob->>GitHubAPI: post report/comment (uses greeting and tag_id)
Estimated code review effort🎯 3 (Moderate) | ⏱️ ~25 minutes Possibly related PRs
Suggested reviewers
🚥 Pre-merge checks | ✅ 2✅ Passed checks (2 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
|
Previously fetched logs were: - this included each and every log line whether it failed or not and without it step headline. the updated method now obtains final logs in this form and only for failed steps from failed jobs:- |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 3
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Inline comments:
In @.github/actions/bot-ci-failure/analyze_failure.py:
- Around line 100-107: The actor fallback currently uses actor =
os.environ.get("ACTOR", pr_author) which doesn't handle ACTOR set to an empty
string; change the logic that sets actor so an empty or whitespace-only ACTOR
falls back to pr_author (e.g., use actor_val = os.environ.get("ACTOR"); actor =
actor_val.strip() if actor_val and actor_val.strip() else pr_author) and keep
the existing greeting construction that compares pr_author.lower() to
actor.lower() to choose the appropriate greeting string.
In @.github/workflows/reusable-bot-ci-failure.yml:
- Line 54: The workflow currently pins the actions checkout ref to the feature
branch value 'fix/ci-failure-bot' (the ref key in the
reusable-bot-ci-failure.yml), which will break after the branch is deleted;
update the ref value to the repository's stable branch (e.g., 'master' or your
main branch name) or to a durable reference (like the default branch variable)
so the workflow continues to run after this PR is merged.
- Around line 108-109: The commands that set START_CLEAN and END_CLEAN use echo
$START and echo $END without quoting, which can cause word-splitting or
globbing; update the two assignments (symbols START_CLEAN and END_CLEAN) to
quote the variables when passing them to echo (i.e., use echo "$START" and echo
"$END") so the full values are preserved and safe from shell expansion.
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: ASSERTIVE
Plan: Pro
Run ID: b49b63ca-6a27-4033-ab58-38c999384662
📒 Files selected for processing (3)
.github/actions/bot-ci-failure/analyze_failure.py.github/workflows/bot-ci-failure.yml.github/workflows/reusable-bot-ci-failure.yml
📜 Review details
🧰 Additional context used
🧠 Learnings (2)
📓 Common learnings
Learnt from: pushpitkamboj
Repo: openwisp/openwisp-utils PR: 584
File: .github/workflows/reusable-bot-changelog.yml:49-49
Timestamp: 2026-03-05T09:38:03.232Z
Learning: In openwisp-utils, PR title prefixes are strictly limited to `[feature]`, `[fix]`, and `[change]` (exact bracketed tags, no scoping/sub-types). The regex `^\[(feature|fix|change)\]` in `.github/workflows/reusable-bot-changelog.yml` is intentional and correct — scoped variants like `[feature/bots]` are not valid and should not be matched.
Learnt from: nemesifier
Repo: openwisp/openwisp-utils PR: 584
File: .github/workflows/reusable-bot-changelog.yml:22-24
Timestamp: 2026-03-05T14:23:49.735Z
Learning: In `.github/workflows/reusable-bot-changelog.yml`, the maintainer (nemesifier) has explicitly decided that `github.event.review.author_association == 'COLLABORATOR'` should be allowed (alongside `OWNER` and `MEMBER`) to trigger the changelog bot workflow. The rationale is that the workflow is non-destructive and only posts a PR comment — it cannot make code changes. Do not flag `COLLABORATOR` as a security issue for this workflow.
📚 Learning: 2026-03-05T14:23:49.735Z
Learnt from: nemesifier
Repo: openwisp/openwisp-utils PR: 584
File: .github/workflows/reusable-bot-changelog.yml:22-24
Timestamp: 2026-03-05T14:23:49.735Z
Learning: In `.github/workflows/reusable-bot-changelog.yml`, the maintainer (nemesifier) has explicitly decided that `github.event.review.author_association == 'COLLABORATOR'` should be allowed (alongside `OWNER` and `MEMBER`) to trigger the changelog bot workflow. The rationale is that the workflow is non-destructive and only posts a PR comment — it cannot make code changes. Do not flag `COLLABORATOR` as a security issue for this workflow.
Applied to files:
.github/workflows/bot-ci-failure.yml.github/workflows/reusable-bot-ci-failure.yml
🔇 Additional comments (5)
.github/actions/bot-ci-failure/analyze_failure.py (1)
146-147: LGTM!The
{greeting}f-string substitution correctly integrates the conditional greeting logic into the system instruction, maintaining the expected response format with the dynamic header and commit SHA reference..github/workflows/reusable-bot-ci-failure.yml (3)
24-26: LGTM!The new
actorinput is properly defined as required and correctly wired to theACTORenvironment variable at line 130. This aligns with theanalyze_failure.pyscript which readsACTORfrom the environment (defaulting toPR_AUTHORif not provided).
132-132: Ensurevars.GEMINI_MODELis configured.
GEMINI_MODELis now sourced from repository/organization variables instead of workflow inputs. Ifvars.GEMINI_MODELis not defined, this will be empty, which may cause the Python script to use a default model or fail.Verify that
vars.GEMINI_MODELis configured in the repository or organization settings where this workflow will be used. If the variable might not exist, consider providing a fallback:GEMINI_MODEL: ${{ vars.GEMINI_MODEL || 'gemini-pro' }}
110-117: The awk filter assumption is correct. GitHub Actions logs are formatted with ISO8601 timestamps as the first field on each line (e.g.,2024-01-08T21:41:07.4620669Z message), exactly matching the script's expectation. The filter correctly extracts the timestamp, removes fractional seconds viasub(/\..*/, "", ts), and performs string-based comparison, which works correctly for ISO8601 format timestamps..github/workflows/bot-ci-failure.yml (1)
66-66: LGTM!The new
actorinput correctly passesgithub.actorto the reusable workflow, satisfying the newly required input parameter. This enables the downstreamanalyze_failure.pyscript to differentiate between the PR author and the workflow trigger actor for appropriate greeting logic.
Fixed actor empty case and greetings
nemesifier
left a comment
nemesifier
left a comment
There was a problem hiding this comment.
Can you please also increase security as suggested by coderabbit here?
#584 (comment)
Improved security by moving tag upwards so attacker cannot guess.
There was a problem hiding this comment.
Actionable comments posted: 1
♻️ Duplicate comments (1)
.github/workflows/reusable-bot-ci-failure.yml (1)
54-54:⚠️ Potential issue | 🔴 CriticalUse a durable checkout ref before merge.
Line [54] still pins
trusted_scriptsto a feature branch. After merge/branch cleanup, this breaks the workflow; usemaster(or another durable ref).🐛 Proposed fix
- ref: fix/ci-failure-bot + ref: master🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In @.github/workflows/reusable-bot-ci-failure.yml at line 54, The workflow currently pins the checkout ref for trusted_scripts to a short-lived feature branch ('fix/ci-failure-bot'), which will break after the branch is deleted; update the ref value used for trusted_scripts in .github/workflows/reusable-bot-ci-failure.yml (the checkout/uses entry that references trusted_scripts) to a durable branch or tag such as 'master' (or main/another long-lived ref) so the workflow continues working after merges/cleanup.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Inline comments:
In @.github/workflows/reusable-bot-ci-failure.yml:
- Around line 83-84: The JOB_IDS assignment only fetches the default first page
of jobs; change the gh command used to fetch jobs so it paginates and returns
all failed job ids. Replace the current JOB_IDS command invocation with the
paginated API call (use gh api --paginate repos/$REPO/actions/runs/$RUN_ID/jobs
with the same -q filter that selects .conclusion=="failure" and .id) so the
JOB_IDS variable includes failures from all pages; ensure the command still
assigns into JOB_IDS and preserves the existing -q filter expression.
---
Duplicate comments:
In @.github/workflows/reusable-bot-ci-failure.yml:
- Line 54: The workflow currently pins the checkout ref for trusted_scripts to a
short-lived feature branch ('fix/ci-failure-bot'), which will break after the
branch is deleted; update the ref value used for trusted_scripts in
.github/workflows/reusable-bot-ci-failure.yml (the checkout/uses entry that
references trusted_scripts) to a durable branch or tag such as 'master' (or
main/another long-lived ref) so the workflow continues working after
merges/cleanup.
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: ASSERTIVE
Plan: Pro
Run ID: 46e95730-becd-460a-a756-8a8922cdf1da
📒 Files selected for processing (2)
.github/actions/bot-ci-failure/analyze_failure.py.github/workflows/reusable-bot-ci-failure.yml
📜 Review details
🧰 Additional context used
🧠 Learnings (4)
📓 Common learnings
Learnt from: pushpitkamboj
Repo: openwisp/openwisp-utils PR: 584
File: .github/workflows/reusable-bot-changelog.yml:49-49
Timestamp: 2026-03-05T09:38:03.232Z
Learning: In openwisp-utils, PR title prefixes are strictly limited to `[feature]`, `[fix]`, and `[change]` (exact bracketed tags, no scoping/sub-types). The regex `^\[(feature|fix|change)\]` in `.github/workflows/reusable-bot-changelog.yml` is intentional and correct — scoped variants like `[feature/bots]` are not valid and should not be matched.
Learnt from: nemesifier
Repo: openwisp/openwisp-utils PR: 584
File: .github/workflows/reusable-bot-changelog.yml:22-24
Timestamp: 2026-03-05T14:23:49.735Z
Learning: In `.github/workflows/reusable-bot-changelog.yml`, the maintainer (nemesifier) has explicitly decided that `github.event.review.author_association == 'COLLABORATOR'` should be allowed (alongside `OWNER` and `MEMBER`) to trigger the changelog bot workflow. The rationale is that the workflow is non-destructive and only posts a PR comment — it cannot make code changes. Do not flag `COLLABORATOR` as a security issue for this workflow.
📚 Learning: 2026-03-05T09:38:03.232Z
Learnt from: pushpitkamboj
Repo: openwisp/openwisp-utils PR: 584
File: .github/workflows/reusable-bot-changelog.yml:49-49
Timestamp: 2026-03-05T09:38:03.232Z
Learning: In openwisp-utils, PR title prefixes are strictly limited to `[feature]`, `[fix]`, and `[change]` (exact bracketed tags, no scoping/sub-types). The regex `^\[(feature|fix|change)\]` in `.github/workflows/reusable-bot-changelog.yml` is intentional and correct — scoped variants like `[feature/bots]` are not valid and should not be matched.
Applied to files:
.github/workflows/reusable-bot-ci-failure.yml
📚 Learning: 2026-03-05T14:23:49.735Z
Learnt from: nemesifier
Repo: openwisp/openwisp-utils PR: 584
File: .github/workflows/reusable-bot-changelog.yml:22-24
Timestamp: 2026-03-05T14:23:49.735Z
Learning: In `.github/workflows/reusable-bot-changelog.yml`, the maintainer (nemesifier) has explicitly decided that `github.event.review.author_association == 'COLLABORATOR'` should be allowed (alongside `OWNER` and `MEMBER`) to trigger the changelog bot workflow. The rationale is that the workflow is non-destructive and only posts a PR comment — it cannot make code changes. Do not flag `COLLABORATOR` as a security issue for this workflow.
Applied to files:
.github/workflows/reusable-bot-ci-failure.yml
📚 Learning: 2026-03-05T09:59:15.097Z
Learnt from: pushpitkamboj
Repo: openwisp/openwisp-utils PR: 584
File: .github/actions/bot-changelog-generator/generate_changelog.py:356-364
Timestamp: 2026-03-05T09:59:15.097Z
Learning: In `.github/actions/bot-changelog-generator/generate_changelog.py`, the `validate_changelog_output` function's purpose is to act as an output safety filter — ensuring no sensitive information or arbitrary LLM-generated text gets posted as a PR comment. It checks that the output starts with a valid tag ([feature]/[fix]/[change]) and contains a correctly structured PR reference pattern. It is NOT intended to strictly validate that the referenced PR number/URL matches the current PR.
Applied to files:
.github/workflows/reusable-bot-ci-failure.yml.github/actions/bot-ci-failure/analyze_failure.py
🔇 Additional comments (4)
.github/actions/bot-ci-failure/analyze_failure.py (2)
100-107: Good defensive actor handling and greeting branching.The fallback on Line [100] correctly handles empty/whitespace
ACTOR, and the case-insensitive branch on Lines [104]-[107] avoids duplicate mentions cleanly.
109-120: Prompt hardening with scoped untrusted blocks looks solid.Using
tag_id(Line [109]) to scope untrusted sections and explicitly instructing the model to ignore in-band directives (Lines [115]-[120]) is a strong mitigation for prompt-injection attempts.Also applies to: 155-155
.github/workflows/reusable-bot-ci-failure.yml (2)
24-26:actorinput contract is correctly added as required.This aligns with downstream env usage and prevents missing-context runs.
130-132: Runtime env wiring for analysis is consistent.Passing
ACTORand sourcingGEMINI_MODELfrom workflow vars is consistent with the analyzer’s runtime resolution/fallback logic.
correct fix using the -f flag for query parameters, not URL embedding
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Inline comments:
In @.github/workflows/reusable-bot-ci-failure.yml:
- Around line 92-99: The gh api calls that populate FAILED_STEPS and write job
logs (the commands that set FAILED_STEPS via gh api
repos/$REPO/actions/jobs/$JOB_ID and gh api .../logs > job_logs.txt) can fail
under set -e and abort the whole step; wrap those calls so an API hiccup doesn't
exit (e.g., run them with a safe fallback like appending "|| true" or capture
the exit code and handle non-zero without exiting), then test for empty/invalid
FAILED_STEPS or job_logs.txt before proceeding and continue to the next job when
appropriate; update the logic around the FAILED_STEPS variable and the gh api
.../logs invocation to tolerate failures and still append "===== JOB $JOB_ID
=====" and any available partial logs to failed_logs.txt.
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: ASSERTIVE
Plan: Pro
Run ID: b0d97313-9860-4902-9d7a-c803cfdf159f
📒 Files selected for processing (1)
.github/workflows/reusable-bot-ci-failure.yml
📜 Review details
🧰 Additional context used
🧠 Learnings (4)
📓 Common learnings
Learnt from: pushpitkamboj
Repo: openwisp/openwisp-utils PR: 584
File: .github/workflows/reusable-bot-changelog.yml:49-49
Timestamp: 2026-03-05T09:38:03.232Z
Learning: In openwisp-utils, PR title prefixes are strictly limited to `[feature]`, `[fix]`, and `[change]` (exact bracketed tags, no scoping/sub-types). The regex `^\[(feature|fix|change)\]` in `.github/workflows/reusable-bot-changelog.yml` is intentional and correct — scoped variants like `[feature/bots]` are not valid and should not be matched.
Learnt from: nemesifier
Repo: openwisp/openwisp-utils PR: 584
File: .github/workflows/reusable-bot-changelog.yml:22-24
Timestamp: 2026-03-05T14:23:49.735Z
Learning: In `.github/workflows/reusable-bot-changelog.yml`, the maintainer (nemesifier) has explicitly decided that `github.event.review.author_association == 'COLLABORATOR'` should be allowed (alongside `OWNER` and `MEMBER`) to trigger the changelog bot workflow. The rationale is that the workflow is non-destructive and only posts a PR comment — it cannot make code changes. Do not flag `COLLABORATOR` as a security issue for this workflow.
📚 Learning: 2026-03-05T14:23:49.735Z
Learnt from: nemesifier
Repo: openwisp/openwisp-utils PR: 584
File: .github/workflows/reusable-bot-changelog.yml:22-24
Timestamp: 2026-03-05T14:23:49.735Z
Learning: In `.github/workflows/reusable-bot-changelog.yml`, the maintainer (nemesifier) has explicitly decided that `github.event.review.author_association == 'COLLABORATOR'` should be allowed (alongside `OWNER` and `MEMBER`) to trigger the changelog bot workflow. The rationale is that the workflow is non-destructive and only posts a PR comment — it cannot make code changes. Do not flag `COLLABORATOR` as a security issue for this workflow.
Applied to files:
.github/workflows/reusable-bot-ci-failure.yml
📚 Learning: 2026-03-05T09:38:03.232Z
Learnt from: pushpitkamboj
Repo: openwisp/openwisp-utils PR: 584
File: .github/workflows/reusable-bot-changelog.yml:49-49
Timestamp: 2026-03-05T09:38:03.232Z
Learning: In openwisp-utils, PR title prefixes are strictly limited to `[feature]`, `[fix]`, and `[change]` (exact bracketed tags, no scoping/sub-types). The regex `^\[(feature|fix|change)\]` in `.github/workflows/reusable-bot-changelog.yml` is intentional and correct — scoped variants like `[feature/bots]` are not valid and should not be matched.
Applied to files:
.github/workflows/reusable-bot-ci-failure.yml
📚 Learning: 2026-03-05T09:59:15.097Z
Learnt from: pushpitkamboj
Repo: openwisp/openwisp-utils PR: 584
File: .github/actions/bot-changelog-generator/generate_changelog.py:356-364
Timestamp: 2026-03-05T09:59:15.097Z
Learning: In `.github/actions/bot-changelog-generator/generate_changelog.py`, the `validate_changelog_output` function's purpose is to act as an output safety filter — ensuring no sensitive information or arbitrary LLM-generated text gets posted as a PR comment. It checks that the output starts with a valid tag ([feature]/[fix]/[change]) and contains a correctly structured PR reference pattern. It is NOT intended to strictly validate that the referenced PR number/URL matches the current PR.
Applied to files:
.github/workflows/reusable-bot-ci-failure.yml
🔇 Additional comments (3)
.github/workflows/reusable-bot-ci-failure.yml (3)
24-26: Actor propagation is correctly wired end-to-end.
actoris required at Line 24 and passed to runtime env at Line 130; this aligns with the workflow_call contract.Also applies to: 130-130
54-54: Keep this as a pre-merge reminder: switch off feature-branch ref.At Line 54,
ref: fix/ci-failure-botis still pinned to a PR branch, so this remains the same concern already raised earlier.
83-84: Pagination fix looks good.Using
--paginatewithper_page=100at Lines 83-84 correctly avoids dropping failed jobs in large/matrix runs.
|
temporary debugging for testing
fixed endpoint for fetching logs
There was a problem hiding this comment.
♻️ Duplicate comments (1)
.github/workflows/reusable-bot-ci-failure.yml (1)
101-107:⚠️ Potential issue | 🟠 MajorAPI failures will abort the entire log extraction.
With
set -eo pipefail(Line 90), failures in thegh apicalls at Lines 101-102 or 107 will terminate the step early, skipping partial logs for remaining jobs. This concern was raised in a previous review but remains unaddressed.Consider wrapping these calls to continue processing other jobs on failure:
🐛 Proposed fix
for JOB_ID in $JOB_IDS; do echo "Processing job $JOB_ID" - FAILED_STEPS=$(gh api repos/$REPO/actions/jobs/$JOB_ID \ - -q '.steps[] | select(.conclusion=="failure") | `@base64`') + if ! FAILED_STEPS=$(gh api repos/$REPO/actions/jobs/$JOB_ID \ + -q '.steps[]? | select(.conclusion=="failure") | `@base64`'); then + echo "Could not fetch steps for job $JOB_ID; skipping." >> failed_logs.txt + continue + fi if [ -z "$FAILED_STEPS" ]; then echo "No failed steps in job $JOB_ID" continue fi - gh api repos/$REPO/actions/jobs/$JOB_ID/logs > job_logs.txt + if ! gh api repos/$REPO/actions/jobs/$JOB_ID/logs > job_logs.txt; then + echo "Could not fetch logs for job $JOB_ID; skipping." >> failed_logs.txt + continue + fi echo "===== JOB $JOB_ID =====" >> failed_logs.txt🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In @.github/workflows/reusable-bot-ci-failure.yml around lines 101 - 107, The gh API calls that populate FAILED_STEPS and fetch job logs can cause the script to exit due to set -eo pipefail; wrap the two gh api invocations (the one producing FAILED_STEPS via "gh api repos/$REPO/actions/jobs/$JOB_ID -q ..." and the one writing job_logs.txt via "gh api repos/$REPO/actions/jobs/$JOB_ID/logs") with error-tolerant handling so a failing API call does not abort processing of subsequent jobs—e.g., run them in a guarded conditional or append a fallback (or || true) to capture the failure, log the error, and continue to the next job; ensure you still detect empty FAILED_STEPS and skip log download when appropriate.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Duplicate comments:
In @.github/workflows/reusable-bot-ci-failure.yml:
- Around line 101-107: The gh API calls that populate FAILED_STEPS and fetch job
logs can cause the script to exit due to set -eo pipefail; wrap the two gh api
invocations (the one producing FAILED_STEPS via "gh api
repos/$REPO/actions/jobs/$JOB_ID -q ..." and the one writing job_logs.txt via
"gh api repos/$REPO/actions/jobs/$JOB_ID/logs") with error-tolerant handling so
a failing API call does not abort processing of subsequent jobs—e.g., run them
in a guarded conditional or append a fallback (or || true) to capture the
failure, log the error, and continue to the next job; ensure you still detect
empty FAILED_STEPS and skip log download when appropriate.
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: ASSERTIVE
Plan: Pro
Run ID: ae0b82d7-6857-4862-8adc-b76a21f630f0
📒 Files selected for processing (1)
.github/workflows/reusable-bot-ci-failure.yml
📜 Review details
🧰 Additional context used
🧠 Learnings (4)
📓 Common learnings
Learnt from: pushpitkamboj
Repo: openwisp/openwisp-utils PR: 584
File: .github/workflows/reusable-bot-changelog.yml:49-49
Timestamp: 2026-03-05T09:38:03.232Z
Learning: In openwisp-utils, PR title prefixes are strictly limited to `[feature]`, `[fix]`, and `[change]` (exact bracketed tags, no scoping/sub-types). The regex `^\[(feature|fix|change)\]` in `.github/workflows/reusable-bot-changelog.yml` is intentional and correct — scoped variants like `[feature/bots]` are not valid and should not be matched.
Learnt from: nemesifier
Repo: openwisp/openwisp-utils PR: 584
File: .github/workflows/reusable-bot-changelog.yml:22-24
Timestamp: 2026-03-05T14:23:49.735Z
Learning: In `.github/workflows/reusable-bot-changelog.yml`, the maintainer (nemesifier) has explicitly decided that `github.event.review.author_association == 'COLLABORATOR'` should be allowed (alongside `OWNER` and `MEMBER`) to trigger the changelog bot workflow. The rationale is that the workflow is non-destructive and only posts a PR comment — it cannot make code changes. Do not flag `COLLABORATOR` as a security issue for this workflow.
📚 Learning: 2026-03-05T14:23:49.735Z
Learnt from: nemesifier
Repo: openwisp/openwisp-utils PR: 584
File: .github/workflows/reusable-bot-changelog.yml:22-24
Timestamp: 2026-03-05T14:23:49.735Z
Learning: In `.github/workflows/reusable-bot-changelog.yml`, the maintainer (nemesifier) has explicitly decided that `github.event.review.author_association == 'COLLABORATOR'` should be allowed (alongside `OWNER` and `MEMBER`) to trigger the changelog bot workflow. The rationale is that the workflow is non-destructive and only posts a PR comment — it cannot make code changes. Do not flag `COLLABORATOR` as a security issue for this workflow.
Applied to files:
.github/workflows/reusable-bot-ci-failure.yml
📚 Learning: 2026-03-05T09:38:03.232Z
Learnt from: pushpitkamboj
Repo: openwisp/openwisp-utils PR: 584
File: .github/workflows/reusable-bot-changelog.yml:49-49
Timestamp: 2026-03-05T09:38:03.232Z
Learning: In openwisp-utils, PR title prefixes are strictly limited to `[feature]`, `[fix]`, and `[change]` (exact bracketed tags, no scoping/sub-types). The regex `^\[(feature|fix|change)\]` in `.github/workflows/reusable-bot-changelog.yml` is intentional and correct — scoped variants like `[feature/bots]` are not valid and should not be matched.
Applied to files:
.github/workflows/reusable-bot-ci-failure.yml
📚 Learning: 2026-03-05T09:59:15.097Z
Learnt from: pushpitkamboj
Repo: openwisp/openwisp-utils PR: 584
File: .github/actions/bot-changelog-generator/generate_changelog.py:356-364
Timestamp: 2026-03-05T09:59:15.097Z
Learning: In `.github/actions/bot-changelog-generator/generate_changelog.py`, the `validate_changelog_output` function's purpose is to act as an output safety filter — ensuring no sensitive information or arbitrary LLM-generated text gets posted as a PR comment. It checks that the output starts with a valid tag ([feature]/[fix]/[change]) and contains a correctly structured PR reference pattern. It is NOT intended to strictly validate that the referenced PR number/URL matches the current PR.
Applied to files:
.github/workflows/reusable-bot-ci-failure.yml
🔇 Additional comments (5)
.github/workflows/reusable-bot-ci-failure.yml (5)
24-26: LGTM!The new
actorinput is properly defined and aligns with the downstream usage inanalyze_failure.py, which safely falls back topr_authorif the actor value is empty.
54-54: Reminder: Revertreftomasterbefore merging.This was flagged in a previous review and acknowledged by the maintainer. The feature branch
fix/ci-failure-botwill be deleted after merge, breaking this workflow.
119-127: LGTM!The AWK-based timestamp filtering correctly handles ISO 8601 timestamps using lexicographic comparison. The null
completed_atfallback at Line 115 properly handles steps that may still be running or have missing completion times.
139-141: LGTM!The
ACTORandGEMINI_MODELenvironment variables are properly configured. The Python analyzer safely handles missing or empty values with appropriate fallbacks (pr_authorfor actor,gemini-2.5-flash-litefor model).
81-88: Good defensive validation.The RUN_ID check with clear error messaging helps catch misconfigured caller workflows early. The debug group provides useful troubleshooting context.
Socket Address In Use ErrorHello @stktyagi, (Analysis for commit 1362613) It looks like your CI job failed due to a Explanation: Remediation: If the problem persists after re-running the job, it might indicate a more persistent issue with the CI environment's process management. In such cases, further investigation into the CI runner's configuration or state would be necessary. |
Modified workflow to see context ingested
Test new technique for fetching
fixed invalid -f per_page flag causing 404
|
https://github.com/openwisp/bot-testing-ground/actions/runs/22733913482 Context is now limited and tested |
|
@coderabbitai review |
✅ Actions performedReview triggered.
|
There was a problem hiding this comment.
Actionable comments posted: 1
♻️ Duplicate comments (1)
.github/workflows/reusable-bot-ci-failure.yml (1)
54-54:⚠️ Potential issue | 🔴 CriticalUse a durable checkout ref before merge.
At Line 54,
ref: fix/ci-failure-botis branch-scoped and will break once that branch is deleted. Switch to a stable ref (e.g.,master) before merging.🐛 Proposed fix
- ref: fix/ci-failure-bot + ref: master🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In @.github/workflows/reusable-bot-ci-failure.yml at line 54, The workflow currently pins the checkout to a branch-scoped ref (`ref: fix/ci-failure-bot`); replace that value in the reusable workflow's checkout step (the `ref` key) with a durable stable ref such as `master` or `main` (or a release tag or commit SHA) so the workflow won't break when the temporary branch is deleted.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Inline comments:
In @.github/workflows/reusable-bot-ci-failure.yml:
- Around line 89-95: The GH API call that populates JOB_IDS is unguarded and
will cause the script to exit under set -eo pipefail if gh fails; modify the
JOB_IDS assignment (the command invoking gh api --paginate
repos/$REPO/actions/runs/$RUN_ID/jobs -q '.jobs[] |
select(.conclusion=="failure") | .id') so failures are swallowed (e.g., append
|| true or use a conditional capture like if ! JOB_IDS=$(gh api ...); then
JOB_IDS=""; fi) so the later check if [ -z "$JOB_IDS" ] can run and produce
failed_logs.txt instead of aborting.
---
Duplicate comments:
In @.github/workflows/reusable-bot-ci-failure.yml:
- Line 54: The workflow currently pins the checkout to a branch-scoped ref
(`ref: fix/ci-failure-bot`); replace that value in the reusable workflow's
checkout step (the `ref` key) with a durable stable ref such as `master` or
`main` (or a release tag or commit SHA) so the workflow won't break when the
temporary branch is deleted.
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: ASSERTIVE
Plan: Pro
Run ID: a5fefa48-711c-4660-a6de-131266b12f36
📒 Files selected for processing (1)
.github/workflows/reusable-bot-ci-failure.yml
📜 Review details
🧰 Additional context used
🧠 Learnings (4)
📓 Common learnings
Learnt from: pushpitkamboj
Repo: openwisp/openwisp-utils PR: 584
File: .github/workflows/reusable-bot-changelog.yml:49-49
Timestamp: 2026-03-05T09:38:03.232Z
Learning: In openwisp-utils, PR title prefixes are strictly limited to `[feature]`, `[fix]`, and `[change]` (exact bracketed tags, no scoping/sub-types). The regex `^\[(feature|fix|change)\]` in `.github/workflows/reusable-bot-changelog.yml` is intentional and correct — scoped variants like `[feature/bots]` are not valid and should not be matched.
📚 Learning: 2026-03-05T14:23:49.735Z
Learnt from: nemesifier
Repo: openwisp/openwisp-utils PR: 584
File: .github/workflows/reusable-bot-changelog.yml:22-24
Timestamp: 2026-03-05T14:23:49.735Z
Learning: In `.github/workflows/reusable-bot-changelog.yml`, the maintainer (nemesifier) has explicitly decided that `github.event.review.author_association == 'COLLABORATOR'` should be allowed (alongside `OWNER` and `MEMBER`) to trigger the changelog bot workflow. The rationale is that the workflow is non-destructive and only posts a PR comment — it cannot make code changes. Do not flag `COLLABORATOR` as a security issue for this workflow.
Applied to files:
.github/workflows/reusable-bot-ci-failure.yml
📚 Learning: 2026-03-05T09:38:03.232Z
Learnt from: pushpitkamboj
Repo: openwisp/openwisp-utils PR: 584
File: .github/workflows/reusable-bot-changelog.yml:49-49
Timestamp: 2026-03-05T09:38:03.232Z
Learning: In openwisp-utils, PR title prefixes are strictly limited to `[feature]`, `[fix]`, and `[change]` (exact bracketed tags, no scoping/sub-types). The regex `^\[(feature|fix|change)\]` in `.github/workflows/reusable-bot-changelog.yml` is intentional and correct — scoped variants like `[feature/bots]` are not valid and should not be matched.
Applied to files:
.github/workflows/reusable-bot-ci-failure.yml
📚 Learning: 2026-03-05T09:59:15.097Z
Learnt from: pushpitkamboj
Repo: openwisp/openwisp-utils PR: 584
File: .github/actions/bot-changelog-generator/generate_changelog.py:356-364
Timestamp: 2026-03-05T09:59:15.097Z
Learning: In `.github/actions/bot-changelog-generator/generate_changelog.py`, the `validate_changelog_output` function's purpose is to act as an output safety filter — ensuring no sensitive information or arbitrary LLM-generated text gets posted as a PR comment. It checks that the output starts with a valid tag ([feature]/[fix]/[change]) and contains a correctly structured PR reference pattern. It is NOT intended to strictly validate that the referenced PR number/URL matches the current PR.
Applied to files:
.github/workflows/reusable-bot-ci-failure.yml
🔇 Additional comments (2)
.github/workflows/reusable-bot-ci-failure.yml (2)
24-26:actorworkflow input is correctly defined and required.This matches the downstream
ACTORenvironment usage and avoids silent empty-context behavior.
132-134: Environment wiring forACTORand repo-levelGEMINI_MODELlooks good.The variable source change is consistent with reusable-workflow configuration centralization.
Updated documentation and add guards in ci
|
@coderabbitai check for any potential security issues |
|
Tip For best results, initiate chat on the files or code changes. 🧩 Analysis chainLet me examine the relevant files for potential security issues. 🏁 Script executed: Length of output: 89 🏁 Script executed: Length of output: 15985
🔒 Security Analysis🔴 Critical — Hardcoded PR Branch for Trusted Script CheckoutFile: - name: Checkout Reusable Workflow
uses: actions/checkout@v6
with:
repository: openwisp/openwisp-utils
ref: fix/ci-failure-bot # ⚠️ PR branch, not master
path: trusted_scriptsThis is the most significant security issue. The workflow checks out
Fix: Change to ref: masterFor maximum supply-chain security, consider pinning to a specific commit SHA instead. 🟠 High — Expression Injection in Shell
|
| Severity | Finding | File |
|---|---|---|
| 🔴 Critical | Hardcoded PR branch fix/ci-failure-bot for trusted scripts |
reusable-bot-ci-failure.yml |
| 🟠 High | Expression injection pattern in run: blocks |
bot-ci-failure.yml |
| 🟡 Medium | Unpinned action versions (@v2, @v6) |
reusable-bot-ci-failure.yml |
| 🟡 Medium | Fork code + CI logs sent to external Gemini API | analyze_failure.py |
| 🟢 Low | 32-bit tag_id entropy for prompt injection isolation |
analyze_failure.py |
| 🟢 Low | greeting (user-controlled) embedded in system_instruction |
analyze_failure.py |
The most urgent fix before merging is changing ref: fix/ci-failure-bot → ref: master in the Checkout Reusable Workflow step.
🧠 Learnings used
Learnt from: pushpitkamboj
Repo: openwisp/openwisp-utils PR: 584
File: .github/workflows/reusable-bot-changelog.yml:49-49
Timestamp: 2026-03-05T09:38:03.232Z
Learning: In openwisp-utils, PR title prefixes are strictly limited to `[feature]`, `[fix]`, and `[change]` (exact bracketed tags, no scoping/sub-types). The regex `^\[(feature|fix|change)\]` in `.github/workflows/reusable-bot-changelog.yml` is intentional and correct — scoped variants like `[feature/bots]` are not valid and should not be matched.
Added checks for security
Removed debug statements from fetch ci logs
Updated ref to master and added comments for fetch step explanation
|
@coderabbitai are you happy now? |
|
Tip For best results, initiate chat on the files or code changes. 🧩 Analysis chainLet me check the current state of the files to see if the issues have been addressed. 🏁 Script executed: Length of output: 15844
Three smaller items remain open if you'd like to address them:
The two low-severity items are essentially cosmetic hardening improvements. The unpinned actions are the only remaining item worth addressing before merging if supply-chain hygiene is a concern, but they're not blockers for the core functionality fix. 🧠 Learnings used |
|
@coderabbitai: we are happy with using version tags and master for openwisp-utils, we know what we're doing and are fine with that, so please if there's nothing else approve and keep this in mind for our org (stop bringing this up all the time). |
|
✏️ Learnings added
🧠 Learnings used✅ Actions performedComments resolved and changes approved. |
3 participants
Previous method of fetching logs using gh view run was prone to weird context, whereas now it limits context and ensures it is correct.
Checklist
Description of Changes
Changed fetching logs step to be both correct and concise using gh api endpoints and filtering runs to jobs and jobs to steps.