Add AI Actions Report workflow using GitHub Models#165
Conversation
github-actions
bot
added
02
|
Warning Rate limit exceeded
⌛ How to resolve this issue?After the wait time has elapsed, a review can be triggered using the We recommend that you space out your commits to avoid hitting the rate limit. 🚦 How do rate limits work?CodeRabbit enforces hourly rate limits for each developer per organization. Our paid plans have higher rate limits than the trial, open-source and free plans. In all cases, we re-allow further reviews after a brief timeout. Please see our FAQ for further information. 📒 Files selected for processing (1)
WalkthroughThis PR introduces infrastructure for automated AI-generated documentation of GitHub workflows and actions. It adds a new GitHub Actions workflow that runs nightly and on manual dispatch, scanning workflow YAML files and generating structured JSON reports. The workflow fetches workflow execution metrics via GitHub API, uses AI providers (z.ai, Anthropic Claude, OpenAI) to generate descriptions, implements caching and rate-limit retry logic, and merges reports into a data branch. The README is simplified to reference this new system with a single link to explore all action workflows. Estimated code review effort🎯 4 (Complex) | ⏱️ ~45 minutes Possibly related PRs
Suggested labels
Suggested reviewers
🚥 Pre-merge checks | ✅ 2 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (2 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 0
🧹 Nitpick comments (5)
scripts/claude-actions-report.mjs (2)
36-52: Consider edge case with nested JSON or multiple objects.The fallback extraction using
indexOf("{")andlastIndexOf("}")works for single-object responses but could misbehave if Claude returns text like"Here's your JSON: {...} and another {...}". For this specific use case where Claude is instructed to return strict JSON, this is likely fine.
145-158: Add request timeout to prevent indefinite hangs.The
fetchcall lacks a timeout, which could cause the workflow to hang indefinitely if the API is unresponsive. Consider usingAbortControllerto add a timeout.♻️ Proposed fix
+ const controller = new AbortController(); + const timeout = setTimeout(() => controller.abort(), 60000); // 60s timeout + const res = await fetch(API_URL, { method: "POST", headers: { "content-type": "application/json", "x-api-key": apiKey, "anthropic-version": "2023-06-01", }, body: JSON.stringify(body), + signal: controller.signal, - }); + }).finally(() => clearTimeout(timeout));.github/workflows/generate-actions-report.yml (3)
3-14: Consider limiting push trigger to specific branches.The
pushtrigger without branch filters will run on every push to any branch, which could consume unnecessary API credits and workflow minutes. Consider limiting to the default branch or specific branches.♻️ Proposed fix
on: push: + branches: + - main workflow_dispatch:
36-44: Input defaults not applied on push trigger.When triggered by
push,inputs.scan_rootandinputs.default_branchwill be empty strings (not the defaults defined inworkflow_dispatch). This works because the script uses||which treats empty strings as falsy, but for clarity, you could use GitHub's expression syntax to provide defaults.♻️ Proposed fix for explicit defaults
- name: Generate report env: ANTHROPIC_API_KEY: ${{ secrets.ANTHROPIC_API_KEY }} CLAUDE_MODEL: claude-sonnet-4-5 - SCAN_ROOT: ${{ inputs.scan_root }} - REPO_DEFAULT_BRANCH: ${{ inputs.default_branch }} + SCAN_ROOT: ${{ inputs.scan_root || '.github' }} + REPO_DEFAULT_BRANCH: ${{ inputs.default_branch || 'main' }} run: | node scripts/claude-actions-report.mjs
31-34: Consider pinning dependency versions for reproducibility.Installing dependencies without version specifiers (
npm i fast-glob js-yaml) could lead to non-reproducible builds if packages introduce breaking changes.♻️ Proposed fix
- name: Install deps run: | npm init -y - npm i fast-glob js-yaml + npm i fast-glob@3 js-yaml@4
📜 Review details
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro
Disabled knowledge base sources:
- Jira integration is disabled by default for public repositories
You can enable these sources in your CodeRabbit configuration.
📒 Files selected for processing (2)
.github/workflows/generate-actions-report.ymlscripts/claude-actions-report.mjs
🔇 Additional comments (10)
scripts/claude-actions-report.mjs (8)
1-26: LGTM on imports and module setup.The shebang, documentation header, and imports are well-structured. The JSDoc-style header clearly documents the expected environment variables and output format.
27-34: LGTM!The
requiredEnvfunction correctly validates required environment variables and exits with an appropriate error code.
54-62: LGTM!Both file classification functions are correctly implemented with proper cross-platform path handling and case-insensitive checks where appropriate.
64-85: LGTM!The link generation correctly uses GitHub Actions environment variables with appropriate fallbacks for local execution.
87-109: LGTM!The execution method summarization covers the various formats for workflow triggers (string, array, object) and action run configurations.
162-176: LGTM on response parsing.The response parsing correctly handles Anthropic's content block format and provides useful error messages with truncated raw text for debugging.
178-244: LGTM on main function implementation.The main function has good error resilience:
- YAML parse failures gracefully fallback to null
- Claude API errors are caught and produce a descriptive fallback
- Content is truncated to avoid oversized payloads
- Sequential processing naturally avoids rate limiting
246-249: LGTM!Standard error handling pattern for CLI scripts with appropriate exit code.
.github/workflows/generate-actions-report.yml (2)
16-21: LGTM on job configuration.Good security practice with minimal
contents: readpermissions.
45-49: LGTM on artifact upload.The artifact upload step is correctly configured. Consider adding
retention-daysif you want to limit storage costs, but the default is reasonable.
igorpecovnik
changed the title
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Fix all issues with AI agents
In @scripts/claude-actions-report.mjs:
- Line 265: The chosen default Anthropic model identifier is invalid; update the
default in the model assignment so that when provider === "zai" is false the
Claude model uses a valid dated identifier (i.e., replace "claude-sonnet-4-5"
with a dated tag such as "claude-sonnet-4-5-20250929" or the latest valid
version), keeping the ability to override via process.env.AI_MODEL and
preserving the provider === "zai" branch logic that yields "glm-4.7".
🧹 Nitpick comments (1)
scripts/claude-actions-report.mjs (1)
162-174: Consider adding a request timeout.The fetch call has no timeout, which could cause the script to hang indefinitely if the API is slow or unresponsive. This is especially relevant in CI environments where job timeouts may be longer than desired.
♻️ Suggested improvement using AbortController
+ const controller = new AbortController(); + const timeout = setTimeout(() => controller.abort(), 60000); // 60s timeout + const res = await fetch(ZAI_API_URL, { method: "POST", headers: { "content-type": "application/json", "authorization": `Bearer ${apiKey}`, }, body: JSON.stringify(body), + signal: controller.signal, - }); + }).finally(() => clearTimeout(timeout));
📜 Review details
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro
Disabled knowledge base sources:
- Jira integration is disabled by default for public repositories
You can enable these sources in your CodeRabbit configuration.
📒 Files selected for processing (2)
.github/workflows/generate-actions-report.ymlscripts/claude-actions-report.mjs
🚧 Files skipped from review as they are similar to previous changes (1)
- .github/workflows/generate-actions-report.yml
🔇 Additional comments (10)
scripts/claude-actions-report.mjs (10)
1-27: LGTM!Clear documentation of the script's purpose, output schema, and environment variables. Imports are appropriate for the file scanning and YAML parsing tasks.
28-38: LGTM!Good use of distinct exit codes (2 for config errors vs 1 for runtime errors) and clear error messaging.
40-47: LGTM!Provider validation is correct with a sensible default and clear error messaging.
49-65: LGTM!The fallback JSON extraction handles common LLM output patterns where JSON is wrapped in extra text. The simple brace-matching approach is sufficient for the expected flat JSON schema.
67-75: LGTM!Correct file type detection with proper path normalization for cross-platform compatibility.
77-98: LGTM!Proper handling of GitHub context with graceful fallback for local execution. Link construction follows correct GitHub URL patterns.
100-122: LGTM!Comprehensive parsing of workflow triggers and action run configurations. Handles all common
onblock formats (string, array, object).
192-257: LGTM!Correct Anthropic Messages API integration with proper headers and response parsing. The same timeout recommendation from the z.ai call applies here as well.
276-325: LGTM!Robust per-file error handling ensures the report is generated even if some AI calls fail. The fallback to
parsedExecutionpreserves useful information when AI fails.
332-335: LGTM!Standard error handling pattern with appropriate exit code for runtime errors.
igorpecovnik
changed the title
igorpecovnik
force-pushed
the
ai
branch
8 times, most recently
from
430a8ad to
39ac92c
Compare
There was a problem hiding this comment.
Actionable comments posted: 1
🧹 Nitpick comments (2)
scripts/claude-actions-report.mjs (2)
156-163: Note on timestamp accuracy in CI environments.The
mtimefromfs.statreflects the filesystem modification time, which in CI environments typically equals the checkout time rather than the actual last commit time. For accurate "last edited" timestamps, consider using Git log:// Alternative using git log for accurate commit time async function getFileEditTime(filePath) { try { const { execSync } = await import('node:child_process'); const timestamp = execSync(`git log -1 --format=%aI -- "${filePath}"`, { encoding: 'utf8' }).trim(); return timestamp || null; } catch { // Fallback to mtime if git is unavailable const stats = await fs.stat(filePath); return stats.mtime.toISOString(); } }This may be acceptable if the current behavior is intentional.
262-331: Consider extracting shared prompt logic.The
callZaiandcallOpenAIfunctions share identical prompt construction (lines 263-288 duplicate lines 191-216). Consider extracting the shared logic to reduce maintenance burden.♻️ Suggested refactor
function buildPrompt(fileKind, relPath, content, parsedExecution) { const systemContent = [ "You are a senior DevOps engineer.", // ... rest of system prompt ].join("\n"); const userContent = [ `File kind: ${fileKind}`, `Path: ${relPath}`, `Parsed execution hint: ${parsedExecution}`, "", "File content:", content, ].join("\n"); return { systemContent, userContent }; }
📜 Review details
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro
Disabled knowledge base sources:
- Jira integration is disabled by default for public repositories
You can enable these sources in your CodeRabbit configuration.
📒 Files selected for processing (2)
.github/workflows/generate-actions-report.ymlscripts/claude-actions-report.mjs
🚧 Files skipped from review as they are similar to previous changes (1)
- .github/workflows/generate-actions-report.yml
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (1)
- GitHub Check: build-report
🔇 Additional comments (12)
scripts/claude-actions-report.mjs (12)
1-34: LGTM!Clear documentation header with env vars, output schema, and appropriate imports for the task.
35-57: LGTM!Exponential backoff implementation is correct. The retry logic properly distinguishes rate-limit errors from other failures and escalates wait times appropriately (4s → 8s → 16s → 32s → 64s).
59-70: LGTM!The
OPENAI_API_URLpoints to GitHub Models (Azure-hosted inference), which aligns with the PR description mentioning "GitHub Models (default: GPT-4o-mini via GITHUB_TOKEN)". The exit code 2 for missing env vars is appropriate for distinguishing configuration errors from runtime failures.
72-79: LGTM!Provider validation is correct with a sensible default.
81-97: LGTM!Robust JSON extraction that handles AI responses with extraneous text around JSON. The fallback strategy using first/last brace matching is practical given the strict JSON instruction in prompts.
99-107: LGTM!File type detection is correct for standard GitHub Actions layouts, with proper Windows path normalization.
109-130: LGTM!Link generation logic correctly handles both GitHub.com and GitHub Enterprise Server via
GITHUB_SERVER_URL, with appropriate fallbacks when repository context is unavailable.
132-154: LGTM!The execution method extraction handles the various forms of workflow triggers (string, array, object) and action runner configurations defensively.
190-260: LGTM!The Z.ai integration is well-structured with clear prompts, appropriate error handling, and correct use of the retry mechanism.
333-402: LGTM with minor note.The Anthropic API integration correctly uses the Messages API format with proper headers. The
anthropic-version: 2023-06-01is the stable version and should continue working.
404-488: Main function is well-structured.The flow is logical: glob files → parse YAML → call AI → collect metadata → write report. Good defensive handling with the try/catch around AI calls (lines 446-461) ensuring one file's failure doesn't break the entire report.
Note: The authentication issue at line 471 (passing AI provider key instead of
GITHUB_TOKENtogetLastWorkflowRunTime) was flagged in an earlier comment.
490-493: LGTM!Standard error handling pattern with appropriate exit code (1 for runtime errors, distinguishing from exit code 2 used for configuration errors).
igorpecovnik
force-pushed
the
ai
branch
2 times, most recently
from
60773a8 to
adb8b8b
Compare
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Fix all issues with AI agents
In @scripts/claude-actions-report.mjs:
- Around line 174-197: getLastWorkflowRunTime is being called with provider API
keys (ZAI_API_KEY/ANTHROPIC_API_KEY) so GitHub auth fails; update the call site
that invokes getLastWorkflowRunTime to pass process.env.GITHUB_TOKEN (not the
provider apiKey) and ensure the workflow identifier you pass is the workflow
filename (e.g., "build.yml") instead of the YAML `name`; you can also rename the
parameter in getLastWorkflowRunTime to githubToken for clarity and use that
token in the Authorization header when calling the GitHub Actions API.
🧹 Nitpick comments (1)
scripts/claude-actions-report.mjs (1)
271-411: LGTM!Both API client functions correctly implement their respective provider formats. The Anthropic function properly uses
x-api-keyheader and handles the content array response format.Optional: The system/user prompt construction is duplicated across all three
call*functions. Consider extracting abuildPrompt(fileKind, relPath, content, parsedExecution)helper to reduce duplication.
📜 Review details
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro
Disabled knowledge base sources:
- Jira integration is disabled by default for public repositories
You can enable these sources in your CodeRabbit configuration.
📒 Files selected for processing (2)
.github/workflows/generate-actions-report.ymlscripts/claude-actions-report.mjs
🚧 Files skipped from review as they are similar to previous changes (1)
- .github/workflows/generate-actions-report.yml
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (1)
- GitHub Check: build-report
🔇 Additional comments (12)
scripts/claude-actions-report.mjs (12)
1-43: LGTM!The header documentation is comprehensive and clearly documents the expected environment variables, output structure, and supported providers.
44-66: LGTM!The exponential backoff implementation is correct. The retry logic properly distinguishes rate-limit errors from other failures and re-throws non-rate-limit errors immediately.
68-79: LGTM!The API endpoints are correctly defined for each provider, and
requiredEnvprovides appropriate validation with clear error messages.
81-88: LGTM!Provider validation is straightforward and provides a helpful error message listing valid options.
90-106: LGTM!Robust JSON parsing with fallback extraction handles LLM responses that may include markdown wrapping or extra text.
108-116: LGTM!File classification logic correctly identifies workflow and action files with proper handling for Windows paths.
118-139: LGTM!Link construction is correct with appropriate fallbacks when repository information is unavailable.
141-163: LGTM!Comprehensive parsing of workflow triggers and action execution methods with appropriate fallbacks for all edge cases.
165-172: LGTM!Simple and correct implementation for retrieving file modification timestamps.
199-269: LGTM!Well-structured AI prompt with clear JSON schema and appropriate error handling. The
stream: falsesetting is correct for this use case.
413-513: LGTM overall with one note.The main function is well-structured with proper error handling. Content truncation at 20,000 characters is a sensible safeguard against oversized payloads.
Note: The fix suggested for
getLastWorkflowRunTimealso addresses the issue at line 488 wherename(the workflow display name from YAML) should befilenamefor the GitHub API call.
515-518: LGTM!Standard CLI error handling pattern with proper exit code.
There was a problem hiding this comment.
Actionable comments posted: 2
🤖 Fix all issues with AI agents
In @.github/workflows/generate-actions-report.yml:
- Around line 46-47: The workflow sets SCAN_ROOT and REPO_DEFAULT_BRANCH from
inputs that are only populated for workflow_dispatch, so on push they become
empty; update the assignment to use expression fallbacks so SCAN_ROOT and
REPO_DEFAULT_BRANCH default to the intended values when inputs.scan_root or
inputs.default_branch are empty (use the GitHub Actions expression fallback for
inputs.scan_root and inputs.default_branch), keeping the same variable names
SCAN_ROOT and REPO_DEFAULT_BRANCH so downstream steps continue to read the same
env vars.
- Around line 73-77: The REPOS assignment directly interpolates ${{
inputs.repositories }} into the shell, enabling command injection; instead read
the input into an environment variable and reference it safely (e.g., assign to
REPOS from a vetted env var), ensure you quote the variable when used (use
"$REPOS"), and validate/sanitize the value (allow only expected characters like
alphanumerics, commas, hyphens, underscores) before appending to GITHUB_OUTPUT;
update the lines that set and check REPOS (the REPOS variable and the echo
"list=..." write) to use the safe env var, quoting, and validation.
🧹 Nitpick comments (4)
.github/workflows/generate-actions-report.yml (4)
3-4: Consider adding branch or path filters to the push trigger.The
push:trigger without filters will run this workflow on every push to any branch. Since this workflow makes AI API calls and generates reports, this could be expensive and wasteful for feature branches.Suggested improvement
on: push: + branches: + - main + paths: + - '.github/workflows/**' + - 'scripts/claude-actions-report.mjs'
51-55: Wrap JSON output in a code block for proper rendering.Raw JSON piped to
$GITHUB_STEP_SUMMARYwill not render correctly. Wrap it in a fenced code block for readability.Suggested fix
- name: Display report run: | echo '# Actions Report' >> $GITHUB_STEP_SUMMARY echo '' >> $GITHUB_STEP_SUMMARY + echo '```json' >> $GITHUB_STEP_SUMMARY jq '.' actions-report.json >> $GITHUB_STEP_SUMMARY + echo '```' >> $GITHUB_STEP_SUMMARY
92-104: Missing error handling and hardcoded organization.
- If
git clonefails (e.g., repo doesn't exist or is private), the workflow will fail without a clear message.- The organization
armbianis hardcoded in the clone URL, limiting reusability.inputs.default_branchwill be empty onpushevents (same issue as build-report job).Suggested improvements
- name: Generate reports + env: + GITHUB_TOKEN: ${{ github.token }} + AI_PROVIDER: openai + AI_MODEL: gpt-4o-mini + REPO_DEFAULT_BRANCH: ${{ inputs.default_branch || 'main' }} + ORG_NAME: ${{ github.repository_owner }} run: | REPOS="${{ steps.repos.outputs.list }}" IFS=',' read -ra REPO_ARRAY <<< "$REPOS" for repo in "${REPO_ARRAY[@]}"; do echo "Processing $repo..." - git clone --depth 1 https://github.com/armbian/$repo.git temp-$repo + if ! git clone --depth 1 "https://github.com/${ORG_NAME}/${repo}.git" "temp-${repo}"; then + echo "::warning::Failed to clone $repo, skipping..." + continue + fi cd temp-$repo - GITHUB_REPOSITORY=armbian/$repo SCAN_ROOT=.github node ../scripts/claude-actions-report.mjs + GITHUB_REPOSITORY="${ORG_NAME}/${repo}" SCAN_ROOT=.github node ../scripts/claude-actions-report.mjs mv actions-report.json ../actions-report-$repo.json cd .. rm -rf temp-$repo done - env: - GITHUB_TOKEN: ${{ github.token }} - AI_PROVIDER: openai - AI_MODEL: gpt-4o-mini - REPO_DEFAULT_BRANCH: ${{ inputs.default_branch }}
20-21: Both jobs run in parallel, potentially duplicating work.On every trigger,
build-reportscans the current repository whilescan-organizationscans a list that may include the current repository (default includesarmbian.github.io). This could result in duplicate AI API calls and wasted compute.Consider either:
- Adding a condition to skip
scan-organizationon push events, or- Making
scan-organizationdepend onbuild-reportand excluding the current repo from its list.Option: Run scan-organization only on workflow_dispatch
scan-organization: runs-on: ubuntu-latest + if: github.event_name == 'workflow_dispatch' && inputs.repositories != '' permissions: contents: read models: readAlso applies to: 63-64
📜 Review details
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro
Disabled knowledge base sources:
- Jira integration is disabled by default for public repositories
You can enable these sources in your CodeRabbit configuration.
📒 Files selected for processing (2)
.github/workflows/generate-actions-report.ymlscripts/claude-actions-report.mjs
🚧 Files skipped from review as they are similar to previous changes (1)
- scripts/claude-actions-report.mjs
🔇 Additional comments (1)
.github/workflows/generate-actions-report.yml (1)
23-25: LGTM on permissions.The permissions are appropriately scoped to read-only for
contentsandmodels, following the principle of least privilege.Also applies to: 65-67
✏️ Tip: You can disable this entire section by setting review_details to false in your review settings.
igorpecovnik
force-pushed
the
ai
branch
3 times, most recently
from
6f517be to
b524909
Compare
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Fix all issues with AI agents
In @.github/workflows/generate-actions-report.yml:
- Around line 60-65: The workflow currently hardcodes the checkout ref to the
temporary branch "feature/ai-cache-optimization" in the Checkout step of
generate-actions-report.yml, which will fail once that branch is deleted; update
the Checkout step (the actions/checkout@v4 usage) to use a stable ref—e.g.,
replace the hardcoded ref with a workflow input or use a repository default like
github.ref_name or github.head_ref (or simply remove the ref to default to the
repository's default branch) so the action checks out a valid, non-temporary
branch at runtime.
♻️ Duplicate comments (2)
.github/workflows/generate-actions-report.yml (2)
27-37: Command injection vulnerability persists.The
inputs.repositoriesvalue is directly interpolated into shell (line 30) and then into Python code (line 35). This allows arbitrary command execution via crafted input.This was flagged in a previous review and remains unaddressed.
🛡️ Recommended fix using environment variables
- name: Set repository matrix id: set_repos + env: + INPUT_REPOS: ${{ inputs.repositories }} run: | - REPOS="${{ inputs.repositories }}" + REPOS="${INPUT_REPOS:-}" if [ -z "$REPOS" ]; then REPOS="armbian.github.io,build,imager,os,sdk,documentation,shallow,docker-armbian-build,configng" fi # Convert comma-separated to JSON array using python for simpler parsing - MATRIX=$(python3 -c "import json,sys; repos=[r.strip() for r in '$REPOS'.split(',') if r.strip()]; print(json.dumps(repos))") + MATRIX=$(python3 -c "import json,os; repos=[r.strip() for r in os.environ.get('INPUT_REPOS','').split(',') if r.strip()]; print(json.dumps(repos) if repos else json.dumps(['armbian.github.io','build','imager','os','sdk','documentation','shallow','docker-armbian-build','configng']))") echo "matrix=$MATRIX" >> $GITHUB_OUTPUT echo "Repositories to scan: $REPOS"
86-95: Command injection and empty input issues in report generation.Two issues in this step:
Command injection (line 89):
SCAN_ROOT=${{ inputs.scan_root }}directly interpolates user input into the shell command.Empty inputs on push (lines 89, 95):
inputs.scan_rootandinputs.default_branchare empty onpushevents since they're only populated duringworkflow_dispatch. This was flagged in a previous review.Suggested fix with env vars and fallbacks
- name: Generate report for ${{ matrix.repo }} + env: + GITHUB_TOKEN: ${{ github.token }} + AI_PROVIDER: openai + AI_MODEL: gpt-4o-mini + REPO_DEFAULT_BRANCH: ${{ inputs.default_branch || 'main' }} + INPUT_SCAN_ROOT: ${{ inputs.scan_root || '.github' }} run: | cd repo-source - GITHUB_REPOSITORY=armbian/${{ matrix.repo }} SCAN_ROOT=${{ inputs.scan_root }} node ../script-repo/scripts/claude-actions-report.mjs + GITHUB_REPOSITORY=armbian/${{ matrix.repo }} SCAN_ROOT="${INPUT_SCAN_ROOT}" node ../script-repo/scripts/claude-actions-report.mjs mv actions-report.json ../actions-report-${{ matrix.repo }}.json - env: - GITHUB_TOKEN: ${{ github.token }} - AI_PROVIDER: openai - AI_MODEL: gpt-4o-mini - REPO_DEFAULT_BRANCH: ${{ inputs.default_branch }}
🧹 Nitpick comments (5)
.github/workflows/generate-actions-report.yml (5)
3-5: Consider limiting push trigger scope.The workflow triggers on every push to any branch. This may cause excessive workflow runs and API usage. Consider restricting to specific branches or paths.
on: push: branches: [main] paths: - '.github/workflows/**' - 'scripts/claude-actions-report.mjs'
97-101: Duplicate cache save is unnecessary.The
actions/cache@v4action automatically saves the cache at the end of the job when usingrestore-keys. The explicit "Save AI description cache" step with the same key as the restore step is redundant.Suggested fix: Remove duplicate cache step
- - name: Save AI description cache - uses: actions/cache@v4 - with: - path: repo-source/.ai-cache - key: ai-descriptions-${{ matrix.repo }}-${{ hashFiles('repo-source/.github/**/*.yml', 'repo-source/.github/**/*.yaml') }} - - name: Upload artifact
133-150: Shell-based JSON generation is fragile.Building JSON by concatenating strings with
echois error-prone. If a filename contains special characters (quotes, backslashes), the output will be invalid JSON. Consider usingjqwhich is pre-installed on GitHub runners.Suggested fix using jq
- name: Generate index run: | cd data-branch/data/actions - # Create list of all JSON files - files=$(ls actions-report-*.json 2>/dev/null | sort) - # Generate index.json - echo "{\"generated\":\"$(date -u +%Y-%m-%dT%H:%M:%SZ)\",\"files\":[" > index.json - first=true - for f in $files; do - if [ "$first" = true ]; then - first=false - else - echo "," >> index.json - fi - echo -n "\"$f\"" >> index.json - done - echo "]}" >> index.json + # Generate index.json using jq for proper JSON handling + jq -n --arg ts "$(date -u +%Y-%m-%dT%H:%M:%SZ)" \ + '{generated: $ts, files: $ARGS.positional}' \ + --args $(ls actions-report-*.json 2>/dev/null | sort) > index.json cat index.json
152-163: Git push may fail on concurrent runs.If multiple workflow runs execute simultaneously (e.g., rapid pushes),
git pushcan fail due to divergent histories. Consider using--force-with-leaseor adding retry logic, or use a concurrency group to prevent parallel runs.Option 1: Add concurrency group at workflow level
Add at the top of the file after
on::concurrency: group: actions-report-${{ github.ref }} cancel-in-progress: trueOption 2: Add retry logic for git push
- name: Commit to data branch run: | cd data-branch git config user.name "GitHub Actions" git config user.email "actions@github.com" git add data/actions/ if git diff --staged --quiet; then echo "No changes to commit" else git commit -m "Update actions reports [skip ci]" - git push + # Retry push with pull-rebase on failure + git push || (git pull --rebase && git push) fi
39-51: Consider adding job timeout.The job lacks a
timeout-minutessetting. If the AI API becomes unresponsive, the job could hang indefinitely. Given the sequential processing (max-parallel: 1) and multiple repositories, consider adding a reasonable timeout.scan-repositories: name: "Scan" needs: setup runs-on: ubuntu-latest + timeout-minutes: 30 permissions:
📜 Review details
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro
Disabled knowledge base sources:
- Jira integration is disabled by default for public repositories
You can enable these sources in your CodeRabbit configuration.
📒 Files selected for processing (2)
.github/workflows/generate-actions-report.ymlscripts/claude-actions-report.mjs
🚧 Files skipped from review as they are similar to previous changes (1)
- scripts/claude-actions-report.mjs
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (1)
- GitHub Check: Scan (imager)
✏️ Tip: You can disable this entire section by setting review_details to false in your review settings.
This adds a GitHub Actions workflow and Node.js script that generates a JSON report of workflows/actions from multiple repositories in the Armbian organization using AI for descriptions. Features: - Matrix-based parallel scanning of multiple repositories - Defaults to armbian.github.io and build repositories - Supports GitHub Models (free GPT-4o-mini), Z.ai, and Anthropic Claude as AI providers - Uses GitHub's built-in GITHUB_TOKEN for free access to AI models - Generates hierarchical JSON with organization, repository, and actions data - Workflow names extracted from YAML (not filenames) - Automatic commit of reports to data branch at data/actions/ - Configurable via workflow_dispatch inputs (repositories, scan folder, default branch) JSON structure includes: - organization: GitHub organization name - repository: Repository name - actions: Array of workflow/action entries with name, filename, description (AI), execution_method, links, and timestamps
Enhance the GitHub Actions report script to track detailed workflow execution statistics: - Add retry_count: Number of re-run attempts for the last workflow run - Add total_run_time_seconds: Total execution time including all retries - Add last_run_duration_seconds: Duration of the last run attempt only - Add last_run_status: Status of the most recent run (success/failure/etc) - Add total_runs: Total number of runs in the sample (up to 10) The script now fetches up to 10 recent workflow runs to calculate these metrics by analyzing GitHub's run_number and run_attempt fields, where run_attempt > 1 indicates a manual retry. Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Fix all issues with AI agents
In @.github/workflows/generate-actions-report.yml:
- Around line 165-171: The inline comment "Run websire generation" contains a
typo; update that comment to "Run website generation" (refer to the workflow
step name "Generate website actions.armbian.com" and the surrounding step block
using peter-evans/repository-dispatch@v4) so the comment reads correctly.
♻️ Duplicate comments (3)
.github/workflows/generate-actions-report.yml (3)
27-37: Command injection vulnerability remains unaddressed.The
inputs.repositoriesvalue is directly interpolated into the shell (line 30) and into a Python string literal (line 35). This allows arbitrary command execution by anyone who can trigger the workflow_dispatch.🛡️ Recommended fix using environment variables
- name: Set repository matrix id: set_repos + env: + INPUT_REPOS: ${{ inputs.repositories }} run: | - REPOS="${{ inputs.repositories }}" + REPOS="${INPUT_REPOS:-}" if [ -z "$REPOS" ]; then REPOS="armbian.github.io,build,imager,os,sdk,documentation,shallow,docker-armbian-build,configng" fi # Convert comma-separated to JSON array using python for simpler parsing - MATRIX=$(python3 -c "import json,sys; repos=[r.strip() for r in '$REPOS'.split(',') if r.strip()]; print(json.dumps(repos))") + MATRIX=$(python3 -c "import json,os; repos=[r.strip() for r in os.environ.get('INPUT_REPOS','').split(',') if r.strip()]; print(json.dumps(repos) if repos else json.dumps(['armbian.github.io','build','imager','os','sdk','documentation','shallow','docker-armbian-build','configng']))") echo "matrix=$MATRIX" >> $GITHUB_OUTPUT echo "Repositories to scan: $REPOS"
86-95: Inputs are empty onpushevents.
inputs.scan_rootandinputs.default_branchare only populated duringworkflow_dispatch. Onpushevents, these will be empty strings, causing the script to receive empty environment variables instead of the intended defaults.Suggested fix using fallback syntax
- name: Generate report for ${{ matrix.repo }} run: | cd repo-source - GITHUB_REPOSITORY=armbian/${{ matrix.repo }} SCAN_ROOT=${{ inputs.scan_root }} node ../script-repo/scripts/claude-actions-report.mjs + GITHUB_REPOSITORY=armbian/${{ matrix.repo }} SCAN_ROOT=${{ inputs.scan_root || '.github' }} node ../script-repo/scripts/claude-actions-report.mjs mv actions-report.json ../actions-report-${{ matrix.repo }}.json env: GITHUB_TOKEN: ${{ github.token }} AI_PROVIDER: openai AI_MODEL: gpt-4o-mini - REPO_DEFAULT_BRANCH: ${{ inputs.default_branch }} + REPO_DEFAULT_BRANCH: ${{ inputs.default_branch || 'main' }}
60-65: Hardcoded branchaiwill break after merge.The checkout references the
aibranch, which appears to be the source branch of this PR. Once merged, this branch will likely be deleted, causing the workflow to fail.Suggested fix to use default branch
- name: Checkout script repo uses: actions/checkout@v4 with: repository: armbian/armbian.github.io path: script-repo - ref: ai + ref: main
🧹 Nitpick comments (2)
.github/workflows/generate-actions-report.yml (2)
3-5: Consider limiting push trigger to specific branches.The
pushtrigger without branch filters will run this workflow on every push to any branch, which may consume unnecessary compute resources and API quota for the AI descriptions.Suggested fix to limit to main branch
on: push: + branches: + - main workflow_dispatch:
133-150: Consider usingjqfor safer JSON generation.Building JSON manually with echo statements is fragile. While the filenames are controlled in this case, using
jqwould be more robust and handle edge cases better.Suggested fix using jq
- name: Generate index run: | cd data-branch/data/actions - # Create list of all JSON files - files=$(ls actions-report-*.json 2>/dev/null | sort) - # Generate index.json - echo "{\"generated\":\"$(date -u +%Y-%m-%dT%H:%M:%SZ)\",\"files\":[" > index.json - first=true - for f in $files; do - if [ "$first" = true ]; then - first=false - else - echo "," >> index.json - fi - echo -n "\"$f\"" >> index.json - done - echo "]}" >> index.json + # Generate index.json using jq for safe JSON handling + jq -n --arg ts "$(date -u +%Y-%m-%dT%H:%M:%SZ)" \ + --argjson files "$(ls -1 actions-report-*.json 2>/dev/null | jq -R -s 'split("\n") | map(select(length > 0))')" \ + '{generated: $ts, files: $files}' > index.json cat index.json
📜 Review details
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro
Disabled knowledge base sources:
- Jira integration is disabled by default for public repositories
You can enable these sources in your CodeRabbit configuration.
📒 Files selected for processing (1)
.github/workflows/generate-actions-report.yml
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (1)
- GitHub Check: Scan (os)
🔇 Additional comments (1)
.github/workflows/generate-actions-report.yml (1)
152-163: LGTM!Good practice to check for staged changes before committing and using
[skip ci]to prevent workflow loops.
✏️ Tip: You can disable this entire section by setting review_details to false in your review settings.
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Fix all issues with AI agents
In `@scripts/generate-actions-report.mjs`:
- Around line 271-275: The API URL construction in getWorkflowRunDetails is
fragile (server -> apiUrl) for GitHub Enterprise; replace the string replace
with proper URL parsing: create a URL object from server (const serverUrl = new
URL(server)), then set apiUrl to "https://api.github.com" when
serverUrl.hostname endsWith("github.com"), otherwise set apiUrl to
`${serverUrl.origin}/api/v3`; update any uses of apiUrl accordingly (refer to
function getWorkflowRunDetails and variables server and apiUrl).
♻️ Duplicate comments (4)
.github/workflows/generate-actions-report.yml (4)
28-38: Command injection vulnerability persists.The
inputs.repositoriesvalue is directly interpolated into shell (line 31) and Python code (line 36), allowing command injection by users with workflow dispatch access.Recommended fix using environment variable
- name: Set repository matrix id: set_repos + env: + INPUT_REPOS: ${{ inputs.repositories }} run: | - REPOS="${{ inputs.repositories }}" + REPOS="$INPUT_REPOS" if [ -z "$REPOS" ]; then REPOS="armbian.github.io,build,imager,os,sdk,documentation,shallow,docker-armbian-build,configng" fi # Convert comma-separated to JSON array using python for simpler parsing - MATRIX=$(python3 -c "import json,sys; repos=[r.strip() for r in '$REPOS'.split(',') if r.strip()]; print(json.dumps(repos))") + MATRIX=$(python3 -c "import json,sys,os; repos=[r.strip() for r in os.environ.get('INPUT_REPOS', '').split(',') if r.strip()]; print(json.dumps(repos) if repos else json.dumps(['armbian.github.io','build','imager','os','sdk','documentation','shallow','docker-armbian-build','configng']))") echo "matrix=$MATRIX" >> $GITHUB_OUTPUT echo "Repositories to scan: $REPOS"
61-66: Hardcoded feature branch reference will break after merge.The checkout references
ref: aiwhich is the PR's source branch. Once merged and deleted, this workflow will fail. Update to use the default branch or remove therefline entirely.Suggested fix
- name: Checkout script repo uses: actions/checkout@v4 with: repository: armbian/armbian.github.io path: script-repo - ref: ai
87-96: Inputs are empty on scheduled runs.On
scheduletriggers,inputs.scan_rootandinputs.default_branchare undefined (not the defaults from workflow_dispatch). The script will receive empty values instead of.githubandmain.Suggested fix using fallback syntax
- name: Generate report for ${{ matrix.repo }} run: | cd repo-source - GITHUB_REPOSITORY=armbian/${{ matrix.repo }} SCAN_ROOT=${{ inputs.scan_root }} node ../script-repo/scripts/generate-actions-report.mjs + GITHUB_REPOSITORY=armbian/${{ matrix.repo }} SCAN_ROOT=${{ inputs.scan_root || '.github' }} node ../script-repo/scripts/generate-actions-report.mjs mv actions-report.json ../actions-report-${{ matrix.repo }}.json env: GITHUB_TOKEN: ${{ github.token }} AI_PROVIDER: openai AI_MODEL: gpt-4o-mini - REPO_DEFAULT_BRANCH: ${{ inputs.default_branch }} + REPO_DEFAULT_BRANCH: ${{ inputs.default_branch || 'main' }}
166-167: Typo: "websire" → "website".- # Run websire generation + # Run website generation
🧹 Nitpick comments (4)
scripts/generate-actions-report.mjs (4)
318-320: Consider renamingtotal_runsfor clarity.The field name
total_runssuggests an all-time count, but it only reflects the fetched sample size (up to 10). Consider renaming tosampled_runsorrecent_runs_countto avoid confusion.
336-554: Consider extracting shared prompt logic to reduce duplication.The three AI provider functions (
callZai,callOpenAI,callAnthropic) contain nearly identical system prompts and user content construction (~20 lines each). Extracting these to shared helper functions would improve maintainability.Refactor suggestion
function buildSystemPrompt() { return [ "You are a senior DevOps engineer.", "You will analyze a GitHub Actions workflow or action definition file.", // ... rest of shared prompt ].join("\n"); } function buildUserContent({ fileKind, relPath, content, parsedExecution }) { return [ `File kind: ${fileKind}`, `Path: ${relPath}`, `Parsed execution hint: ${parsedExecution}`, "", "File content:", content, ].join("\n"); }
639-649: Caching AI failures may persist transient errors.When AI calls fail (lines 639-644), the failure message is cached (line 648). This prevents repeated API calls but also means transient errors (e.g., network issues, temporary outages) will persist until the source file changes.
Consider either:
- Not caching failed descriptions
- Using a shorter TTL for cached failures
- Marking failures differently in the cache to allow retry on next run
630-638: Consider logging when content is truncated.Large files are silently truncated at 20KB (line 636). Adding a log message when truncation occurs would help diagnose cases where important content at the end of a file is missed.
📜 Review details
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro
Disabled knowledge base sources:
- Jira integration is disabled by default for public repositories
You can enable these sources in your CodeRabbit configuration.
📒 Files selected for processing (3)
.github/workflows/generate-actions-report.ymlREADME.mdscripts/generate-actions-report.mjs
🚧 Files skipped from review as they are similar to previous changes (1)
- README.md
🔇 Additional comments (7)
scripts/generate-actions-report.mjs (3)
58-80: LGTM! Well-implemented exponential backoff.The retry mechanism with exponential backoff starting at 4s is appropriate for rate-limited APIs. The logic correctly distinguishes rate-limit errors from other failures.
86-167: Solid caching implementation with appropriate invalidation strategies.The cache correctly invalidates on version changes, content changes, and AI model changes. Graceful error handling ensures caching failures don't break the main workflow.
556-570: Clean provider and model configuration.The provider selection logic with sensible defaults for each AI provider is well-structured. The automatic extraction of organization/repository from
GITHUB_REPOSITORYis appropriate..github/workflows/generate-actions-report.yml (4)
3-19: Well-configured triggers with sensible defaults.The schedule runs every 2 days and manual dispatch provides flexibility with optional inputs that have reasonable defaults.
78-102: Good cache configuration with repository-specific keys.The cache strategy uses repo-specific keys with YAML file hashing for invalidation, and includes fallback restore keys for partial cache hits.
110-165: Well-structured merge and commit job.The job properly handles artifact merging with graceful error handling, generates a machine-readable index, and commits with
[skip ci]to prevent recursive triggers.
167-172: Cross-repository dispatch requires proper secret configuration.Ensure
secrets.DISPATCHis a PAT with appropriate permissions for thearmbian/actionsrepository. The dispatch will silently fail if the token lacks access.
✏️ Tip: You can disable this entire section by setting review_details to false in your review settings.
igorpecovnik
added
Ready to merge
github-actions
bot
added
Needs review
1 participant
Summary
.github/workflows/for workflow files and action definitionsFeatures
workflow_dispatchwith configurable parameters (scan folder, default branch)actions-report.jsonartifact with structured data about all automationGITHUB_TOKENfor free access to AI modelsedited(last file modification) andexecuted(last workflow run)Output Fields
name: Workflow/action filenamedescription: AI-generated description of what the workflow doesexecution_method: Triggers and entrypointsstatus_link: Link to workflow status page (for workflows)script_link: Link to the source fileedited: ISO 8601 timestamp of last file modificationexecuted: ISO 8601 timestamp of last workflow run (workflows only)Configuration
The workflow uses GitHub Models by default with the
GITHUB_TOKEN, which is automatically available in GitHub Actions and provides free access to GPT-4o-mini.Test plan
actions-report.jsonartifact is produced