feat(compliance-audit): add added/existing/removed issue count summary#255
Conversation
Track and surface the count of issues added (new), existing (updated), and removed (resolved) in each compliance audit run. Changes: - scripts/compliance-audit.sh: add ISSUES_ADDED/EXISTING/REMOVED global counters; increment them in create_issue_for_finding and close_resolved_issues; write issue-counts.json to REPORT_DIR; append an 'Issue Management' table to summary.md after issue processing. - .github/workflows/compliance-audit-and-improvement.yml: expose issues_added/existing/removed as job outputs by reading issue-counts.json; pass these values into the Claude Phase 6 prompt context and summary template so the step summary includes the count table. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
|
Warning Rate limit exceeded
You’ve run out of usage credits. Purchase more in the billing tab. ⌛ How to resolve this issue?After the wait time has elapsed, a review can be triggered using the We recommend that you space out your commits to avoid hitting the rate limit. 🚦 How do rate limits work?CodeRabbit enforces hourly rate limits for each developer per organization. Our paid plans have higher rate limits than the trial, open-source and free plans. In all cases, we re-allow further reviews after a brief timeout. Please see our FAQ for further information. ℹ️ Review info⚙️ Run configurationConfiguration used: Organization UI Review profile: ASSERTIVE Plan: Pro Run ID: 📒 Files selected for processing (2)
📝 WalkthroughWalkthroughThe PR extends the compliance audit workflow and script to track and report issue-management metrics. The script now initializes counters for added, existing, and removed issues, increments them during create/update/close operations, and outputs counts to JSON and a summary table. The workflow reads those counts from the JSON file, declares them as job outputs, includes them in the Claude analysis prompt, and reports them in the final step summary. ChangesIssue Metrics Tracking in Compliance Audit
Estimated code review effort🎯 2 (Simple) | ⏱️ ~10 minutes Possibly related PRs
🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
There was a problem hiding this comment.
Code Review
This pull request introduces issue management tracking to the compliance audit script by implementing counters for added, existing, and removed issues. These metrics are now exported to a JSON file and appended as a summary table to the final report. Feedback indicates that the new summary section is currently appended after the report's footer, which breaks the document structure; it is recommended to move the footer generation to the end of the execution to maintain a proper layout.
| cat >> "$SUMMARY_FILE" <<HEREDOC | ||
|
|
||
| ## Issue Management | ||
|
|
||
| | Action | Count | | ||
| |--------|-------| | ||
| | Added (new) | $ISSUES_ADDED | | ||
| | Existing (updated) | $ISSUES_EXISTING | | ||
| | Removed (resolved) | $ISSUES_REMOVED | | ||
| HEREDOC |
There was a problem hiding this comment.
The Issue Management section is appended to the end of the summary file. However, the generate_summary function (called on line 1399) already appends a footer containing a horizontal rule and a timestamp (lines 1309-1313). This results in the new section appearing after the report's footer, which breaks the document structure. Consider moving the footer generation logic out of generate_summary and placing it at the very end of the main function to ensure it remains the final element of the report.
There was a problem hiding this comment.
Pull request overview
Adds run-level issue management metrics to the compliance audit so downstream workflow steps (including the Claude analysis prompt and weekly report) can display how many issues were added, updated, and closed during an audit run.
Changes:
- Track
ISSUES_ADDED,ISSUES_EXISTING, andISSUES_REMOVEDinscripts/compliance-audit.shand emit them asissue-counts.json, plus append an “Issue Management” section tosummary.md. - Expose those counts as outputs from the
auditjob in.github/workflows/compliance-audit-and-improvement.ymland render them in the Phase 6 summary template and Claude prompt context.
Reviewed changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated 3 comments.
| File | Description |
|---|---|
| scripts/compliance-audit.sh | Adds global counters, increments them during issue create/update/close, writes issue-counts.json, and appends an Issue Management table to summary.md. |
| .github/workflows/compliance-audit-and-improvement.yml | Reads issue-counts.json into step/job outputs and includes the counts in the Claude Phase 6 prompt + report template. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| # Ensure claude label is present on pre-existing issues | ||
| gh issue edit "$existing" --repo "$ORG/$repo" --add-label "claude" 2>/dev/null || true | ||
| info "Updated existing issue #$existing in $repo for: $check" | ||
| ISSUES_EXISTING=$((ISSUES_EXISTING + 1)) |
| --comment "Resolved! This check is now passing as of $(date -u +%Y-%m-%d). Closing automatically." \ | ||
| 2>/dev/null || true | ||
| info "Closed resolved issue #$issue_num in $repo: $issue_title" | ||
| ISSUES_REMOVED=$((ISSUES_REMOVED + 1)) |
| FINDINGS_FILE="$REPORT_DIR/findings.json" | ||
| SUMMARY_FILE="$REPORT_DIR/summary.md" | ||
| ISSUES_FILE="$REPORT_DIR/issues.json" | ||
| ISSUE_COUNTS_FILE="$REPORT_DIR/issue-counts.json" |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 1
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: ASSERTIVE
Plan: Pro
Run ID: f0046637-c43b-46fb-9146-eb3f3a42a43c
📒 Files selected for processing (2)
.github/workflows/compliance-audit-and-improvement.ymlscripts/compliance-audit.sh
Superseded by automated re-review at
|
Enhance the step summary and Claude Phase 6 report with: 1. Grouping by compliance check type (not by repo) — the same check failing in N repos now appears once with all N repos listed, avoiding repeated rows for the same systemic problem. 2. Hyperlinks to every GitHub Issue and related open PR — the new 'Issues & Related PRs' section (shell script) and updated Phase 6 template (Claude) render each issue as [#N](url) and look up open PRs via closingIssuesReferences (one GraphQL call per affected repo). 3. Per-repo scorecard table — compact errors/warnings/total view so repos with the most debt are immediately visible. Scripts/compliance-audit.sh: - generate_summary: replace per-repo subsections with a 'Findings by Check Type' table (grouped by check, sorted by severity then alpha) and a new 'Per-Repo Scorecard' compact table. - append_issue_pr_links: new function called after issue creation; fetches linked PRs via GraphQL per affected repo and appends a '## Issues & Related PRs' section grouped by check type. - Footer and issue-management table moved to end of main() so they always appear after all appended sections. .github/workflows/compliance-audit-and-improvement.yml: - Phase 5: add grouping rule (same check type in N repos = 1 issue) and PR-lookup instruction before Phase 6 summary is written. - Phase 6 template: replace flat repo/issue table with per-check-type subsections (#### 'check' (severity), N repos, issue links, PR links). Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
donpetry-bot
left a comment
donpetry-bot
left a comment
There was a problem hiding this comment.
Automated review — APPROVED ✓
Risk: MEDIUM
Reviewed commit: e27d86d82624d1ba036be285157c3cf90ceb6ec2
Cascade: triage → deep (triage: haiku 4.5 → deep: sonnet 4.6 + duck: o4-mini → audit: opus 4.7)
Summary
All CI gates pass (ShellCheck, Lint, SonarCloud 0 new issues, CodeQL, AgentShield). The CodeRabbit CHANGES_REQUESTED is stale — it reviewed only the first commit; the second commit fixed the footer-ordering defect flagged by Gemini, and CodeRabbit's subsequent run hit a rate limit before completing. No security issues, no injection risks; minor robustness gaps around jq error handling are low-consequence.
Findings
- info: CodeRabbit CHANGES_REQUESTED was posted against the first commit (65e6b4b). The second commit (e27d86d) addressed the reported footer-ordering defect. CodeRabbit hit its rate limit before re-reviewing; its status context is SUCCESS but the PR-level review decision remains CHANGES_REQUESTED. This gate blocker is stale.
- info: CodeRabbit docstring-coverage warning (33.33% vs 80%) is its generic check applied to bash function headers — not a meaningful standard for shell scripts. No action required.
- minor: In the workflow,
jq '.added'/jq '.existing'/jq '.removed'lack the-rflag. For JSON integer values this is functionally safe (integers are not quoted), but ifissue-counts.jsonis malformed or a key is missing,jqexits non-zero and the variable becomes empty, writingissues_added=to GITHUB_OUTPUT. Adding|| echo 0would guard against this. - minor: In
append_issue_pr_links, the GraphQL query usesfirst:100for open PRs per repo. Repos with more than 100 open PRs will silently miss some PRs in the Related PRs column. Unlikely in practice but worth noting for future pagination. - info: The
gh search prsinstruction appears only inside the Claude AI prompt template (a multi-line string passed to the model), not as a shell command executed by the workflow runner. No automated enumeration occurs.
Reviewed by the PR-review cascade (triage: haiku 4.5 → deep: sonnet 4.6 + duck: o4-mini → audit: opus 4.7). Reply if you need a human review.
Two bugs in append_issue_pr_links caused the Issues & Related PRs section to render empty: 1. NDJSON slurp: ISSUES_FILE is newline-delimited JSON (one object per line) but jq was called without -n '[inputs]', so only the first record was ever processed. Fixed all ISSUES_FILE reads to use 'jq -rn/cn [inputs]' instead of 'jq -r/c'. 2. Missing severity field: the jq records written to ISSUES_FILE in create_issue_for_finding (both existing and new issue branches) omitted the severity field, causing sort_by severity to fail. Added --arg severity and included it in both jq writes. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…_links 1. repos_in_issues used '[.[].repo]' on NDJSON — only first repo was ever queried. Fixed to 'jq -rn [inputs | .repo] | unique[]'. 2. GraphQL --jq used $repo as an unbound jq variable (gh -f flags set GraphQL vars, not jq vars), causing every repo_prs to silently return []. Fixed by piping raw GraphQL response to 'jq --arg repo' so the repo name is properly available in the jq expression. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
|
Auto-rebase blocked — the base branch contains Please rebase this branch manually: |
donpetry-bot
left a comment
donpetry-bot
left a comment
There was a problem hiding this comment.
Automated review — APPROVED ✓
Risk: MEDIUM
Reviewed commit: 1d43ea553e06ff154c0a9dcd0e3876d9dc01b4cb
Cascade: triage → deep (triage: haiku 4.5 → deep: sonnet 4.6 + duck: o4-mini → audit: opus 4.7)
Summary
Non-trivial shell script and workflow changes adding issue-count tracking and a grouped summary section; all security checks pass (CodeQL, SonarCloud 0 new issues, ShellCheck) and no injection or secret risks are present. The CodeRabbit CHANGES_REQUESTED is stale — it reviewed only the first commit and was rate-limited before re-reviewing; the footer-ordering defect it flagged was fixed in commit 2. Branch is BEHIND main and requires a manual rebase before merge, but CI ran on the merge commit so test results are valid against a recent baseline.
Findings
- INFO [review-process]: CodeRabbit CHANGES_REQUESTED was posted against commit 65e6b4b (first commit only). Commit e27d86d fixed the footer-ordering defect it flagged. CodeRabbit hit its hourly rate limit before re-reviewing; its status context shows SUCCESS but the PR-level review decision remains CHANGES_REQUESTED. This gate blocker is stale and not a meaningful code-quality signal.
- INFO [merge-state]: Branch is BEHIND main (mergeStateStatus=BEHIND). Auto-rebase is blocked because the branch touches .github/workflows/ and the bot lacks the 'workflows' permission. A manual 'git fetch origin && git rebase origin/main && git push --force-with-lease' is required before merging. CI ran on merge commit 1d43ea5 (which already integrated main as of 2026-05-12T19:01Z), so existing test results are valid against a recent baseline.
- MINOR [robustness]: In the workflow's 'Parse issue management counts' block,
jq '.added',jq '.existing', andjq '.removed'lack the-rflag. For JSON integer values this is functionally safe (integers are unquoted), but if issue-counts.json is malformed or a key is absent, jq exits non-zero and the variable becomes empty, writingissues_added=to GITHUB_OUTPUT. Adding|| echo 0after each jq call would guard against this edge case. - MINOR [robustness]: In
append_issue_pr_links, the GraphQL query usesfirst:100for open PRs per repo. Repos with more than 100 open PRs will silently miss some entries in the Related PRs column. Unlikely in practice for this org, but worth noting if the audit is extended to larger orgs. - INFO [security]: The
gh search prscommand referenced in the diff appears only inside a heredoc string passed as a prompt to the Claude AI model (Phase 6 template), not as a shell command executed by the workflow runner. No automated enumeration of PRs across repos occurs at workflow runtime. - INFO [correctness]: NDJSON handling is correctly fixed in commits 3 and 4: all ISSUES_FILE reads now use
jq -rn '[inputs]'orjq -cn [inputs]instead of plainjq -r/c, and the missingseverityfield was added to both the existing-issue and new-issue branches ofcreate_issue_for_finding. GraphQL variables are passed via-fflags (not shell-interpolated into query strings), so no injection surface exists.
Reviewed by the PR-review cascade (triage: haiku 4.5 → deep: sonnet 4.6 + duck: o4-mini → audit: opus 4.7). Reply if you need a human review.
Automated review — human attention neededThis PR has been through 3 automated review cycles (cap: 3) without converging on an approval-and-merge state. Further automated review has been paused to avoid infinite loops. Please take a look manually, or close this PR if it's no longer needed. Once a human review resolves the situation, remove the Posted by the donpetry-bot PR-review cascade. |
|
@claude - Please review and address all comments, marking as resolved when fixed. Confirm CI is working. Request a re-review when ready. |
|
Claude finished @don-petry's task in 3m 12s —— View job Review Comments Addressed
What was changedAll changes are in Cannot address: The CI status: Push triggered a new CI run. The branch was previously BEHIND main; this commit includes the existing merge of main (74bdf11), so the rebase blocker noted by the auto-rebase bot should now be resolved. |
…d output - Add issue-counts.json to header Outputs comment (Copilot #36) - ISSUES_EXISTING: only increment when gh issue comment succeeds, not on || true failure (Copilot #1045) - ISSUES_REMOVED: only increment when gh issue close succeeds, not on || true failure (Copilot #1268) - Make issue-count JSON/summary conditional on issue management running; show skip notice when DRY_RUN=true or CREATE_ISSUES=false (CodeRabbit #1608) - Footer is now always the final element, written after the conditional Issue Management section Co-authored-by: Don Petry <don-petry@users.noreply.github.com>
|
@coderabbitai review |
✅ Actions performedReview triggered.
|
|
Summary
Surfaces a count of added, existing, and removed issues in every compliance audit run.
Changes
scripts/compliance-audit.shISSUES_ADDED,ISSUES_EXISTING,ISSUES_REMOVEDcreate_issue_for_finding: incrementsISSUES_ADDED(new issue) orISSUES_EXISTING(updated issue)close_resolved_issues: incrementsISSUES_REMOVEDwhen a resolved finding's issue is closedissue-counts.jsontoREPORT_DIRsummary.md.github/workflows/compliance-audit-and-improvement.ymlissues_added,issues_existing,issues_removedas job outputs (parsed fromissue-counts.json)Result
Both the Compliance Audit job step summary and the Claude-generated weekly report now include:
Summary by CodeRabbit