ci: rank review feedback guardrail backlog

[haasonsaas]

Summary

• add deterministic guardrail-class ranking on top of the review feedback ledger
• emit markdown and JSON guardrail backlog artifacts from the review feedback sentinel
• add a manual pr_limit input for longer backfills
• filter informational bot PR summaries and improve feedback headline extraction for Codex review bodies

Live dry-run

• ruby .github/scripts/sweep-recent-review-feedback.rb --owner evalops --since-hours 720 --pr-limit 200 --min-severity high --dry-run --json-output /tmp/evalops-feedback-backfill/ledger.json --guardrail-backlog-output /tmp/evalops-feedback-backfill/backlog.md --guardrail-backlog-json-output /tmp/evalops-feedback-backfill/backlog.json
• Produced 3 high+ findings clustered into 2 guardrail classes: runtime-smoke-coverage and workflow-shell-footgun

Tests

• ruby -c .github/scripts/check-pr-review-threads.rb
• ruby -c .github/scripts/sweep-recent-review-feedback.rb
• ruby -Itest test/check_pr_review_threads_test.rb
• ruby -Itest test/sweep_recent_review_feedback_test.rb
• ruby -Itest test/classify_agent_authorship_test.rb
• ruby -Itest test/publish_codex_structured_review_test.rb
• ruby -Itest test/validate_services_catalog_test.rb
• actionlint .github/workflows/review-feedback-sentinel.yml

[cursor]

PR Summary

Medium Risk
Moderate risk because it changes the CI sentinel’s detection/filtering and adds new report outputs, which could alter what feedback is flagged or emitted (false positives/negatives) and affect issue/backlog automation.

Overview
Adds a ranked “guardrail backlog” layer on top of the existing review-feedback ledger by classifying findings into predefined buckets (e.g., workflow, generated drift, security, tests) and scoring them by severity and repo spread, emitting both JSON and markdown outputs.

Updates the review-feedback sentinel workflow to accept a pr_limit input, generate/upload the new backlog artifacts, and append the backlog markdown to the GitHub step summary.

Tightens signal quality by skipping informational bot PR summaries when scanning top-level comments/reviews and improving body_first_line extraction to ignore Codex boilerplate/links and normalize leading blank lines; includes new tests for these behaviors.

^{Reviewed by Cursor Bugbot for commit 753dbad. Bugbot is set up for automated code reviews on this repo. Configure here.}

[cursor]

Cursor Bugbot has reviewed your changes and found 1 potential issue.

^{Bugbot Autofix is ON, but it could not run because the branch was deleted or merged before autofix could start.}

^{Reviewed by Cursor Bugbot for commit 753dbad. Configure here.}