Skip to content

feat(ci): add common factory guardrails#480

Merged
castrojo merged 3 commits into
mainfrom
feat/acmm-common-guardrails
Jun 4, 2026
Merged

feat(ci): add common factory guardrails#480
castrojo merged 3 commits into
mainfrom
feat/acmm-common-guardrails

Conversation

@castrojo
Copy link
Copy Markdown
Contributor

@castrojo castrojo commented Jun 4, 2026

What changed

  • Add the repo-local ACMM / AI-safety guardrails prepared in this branch
  • Update the matching docs/skills alongside the workflow or policy change

Validation

  • Ran the repo-appropriate existing validation commands locally for this change set

Summary by CodeRabbit

  • New Features

    • Added per-PR skill-drift validation checks
    • Added weekly/manual promotion-candidate E2E testing for candidate images
  • Improvements

    • Centralized and simplified E2E test orchestration across CI
    • Stricter image-reference validation to catch regressions early
    • Expanded CI and workflow documentation (README, agents/skills guides)

@dosubot dosubot Bot added size:L This PR changes 100-499 lines, ignoring generated files. area/testing Testing and QA kind/automation Automation and CI/CD kind/documentation Improvements or additions to documentation labels Jun 4, 2026
@coderabbitai
Copy link
Copy Markdown

coderabbitai Bot commented Jun 4, 2026

Review Change Stack

Warning

Review limit reached

@castrojo, we couldn't start this review because you've reached your PR review rate limit.

More reviews will be available in 52 minutes and 24 seconds. Learn how PR review limits work.

Your organization has run out of usage credits. Purchase more in the billing tab.

⌛ How to resolve this issue?

After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

We recommend that you space out your commits to avoid hitting the rate limit.

🚦 How do rate limits work?

CodeRabbit enforces hourly rate limits for each developer per organization.

Our paid plans include higher PR review limits than trial, open-source, and free plans. In all cases, reviews become available again over time. During sustained high-volume PR review activity, CodeRabbit may temporarily slow when the next review becomes available.

Please see our Fair Usage Limits Policy for further information.

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro Plus

Run ID: 63c73fc6-8f9f-4c18-9812-39e6dcfe89da

📥 Commits

Reviewing files that changed from the base of the PR and between 894f86a and 3da0061.

📒 Files selected for processing (15)
  • .github/workflows/e2e.yml
  • .github/workflows/pr-e2e.yml
  • .github/workflows/promotion-candidate-e2e.yml
  • .github/workflows/run-testsuite.yml
  • .github/workflows/skill-drift.yml
  • .github/workflows/validate.yml
  • AGENTS.md
  • README.md
  • docs/factory/README.md
  • docs/skills/INDEX.md
  • docs/skills/acmm-audit-level1.md
  • docs/skills/ci-tooling.md
  • docs/skills/e2e-ci.md
  • docs/skills/image-registry.md
  • docs/skills/workflow-map.md
📝 Walkthrough

Walkthrough

Refactors CI to centralize the external testsuite into a repo-local reusable wrapper, updates E2E jobs to call that wrapper, adds a promotion-candidate E2E workflow and a skill-drift PR gate, replaces the image-ref guard with a Python validator, and updates documentation to reflect the new three-layer model.

Changes

CI Workflow Refactoring and Testing Gates

Layer / File(s) Summary
Testsuite wrapper foundation
.github/workflows/run-testsuite.yml
Introduces a reusable run-testsuite.yml that accepts image and suites and forwards them to the pinned projectbluefin/testsuite E2E workflow.
E2E workflow refactoring
.github/workflows/e2e.yml, .github/workflows/pr-e2e.yml
Refactors post-merge and PR E2E jobs to call the local wrapper (matrix-driven), consolidating prior per-image reusable-workflow calls.
Testing promotion and image validation
.github/workflows/promotion-candidate-e2e.yml, .github/workflows/validate.yml, docs/skills/image-registry.md
Adds a weekly/manual promotion-candidate E2E gate (testing vs LTS images, suites: smoke,common) and replaces the grep guard with a Python scanner that allows only the two testing-candidate tags in the promotion workflow.
Skill drift enforcement workflow
.github/workflows/skill-drift.yml, docs/skills/ci-tooling.md
Adds a PR-triggered skill-drift workflow calling a pinned reusable check and documents its location and pinned-call usage in CI tooling docs.
CI/Testing public documentation
README.md, AGENTS.md, docs/skills/INDEX.md
Updates README to describe the three-layer CI model, expands AGENTS workflow roster, and refines skills index entries to reflect the new workflows.
E2E and factory automation documentation
docs/skills/e2e-ci.md, docs/factory/README.md
Documents the local wrapper pattern for pre-merge and promotion-candidate flows and updates factory guidance to reflect common's bonedigger lifecycle coverage.
Audit and capability parity tracking
docs/skills/acmm-audit-level1.md
Updates audit docs to reflect common gaining skill-drift.yml and bonedigger.yml, refreshes critical gaps, recommendations, parity snapshots, and issue-batch entries.
Workflow map documentation
docs/skills/workflow-map.md
Adds a new workflow-purpose map describing workflow categories, responsibilities, and an edit boundary rule.

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~25 minutes

Suggested labels

kind/automation

🐰 A wrapper hops in, neat and spry,
Calls pinned suites under watchful sky,
Promotion gates on weekly runs,
Skill checks prance until PRs are done,
Docs updated — CI takes a jaunty hop! 🥕

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Title check ✅ Passed The title 'feat(ci): add common factory guardrails' accurately and concisely describes the main change: adding factory guardrails to CI/automation.
Description check ✅ Passed The PR description adequately covers what changed (adding ACMM/AI-safety guardrails and updating docs) and validation approach, meeting the requirements despite minimal template usage.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch feat/acmm-common-guardrails

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

Copy link
Copy Markdown

@coderabbitai coderabbitai Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (1)
.github/workflows/validate.yml (1)

58-98: ⚠️ Potential issue | 🔴 Critical | ⚡ Quick win

Critical: Validation script scans itself and fails.

The Python validator scans all .github/workflows/*.yml files, including validate.yml itself. The regex pattern matches the hardcoded testing-tag strings in the allowlist definition (lines 69-70) and the error message (line 95). Since those matches are not from promotion-candidate-e2e.yml, they fail the allowlist check and trigger violations.

This is causing the pipeline failure.

🐛 Proposed fix: exclude validate.yml from scanning
 pattern = re.compile(r"ghcr\.io/projectbluefin/(bluefin|aurora|bazzite)(?::[A-Za-z0-9._-]+)?")
 candidates = list(Path(".github/workflows").rglob("*.yml"))
 candidates += list(Path(".github/workflows").rglob("*.yaml"))
+candidates = [p for p in candidates if p.name != "validate.yml"]
 candidates.append(Path("system_files/bluefin/usr/bin/ublue-rollback-helper"))
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In @.github/workflows/validate.yml around lines 58 - 98, The validator is
currently scanning its own workflow file and matching the allowlist strings;
update the candidate collection/filtering so
Path(".github/workflows/validate.yml") is excluded (or generally skip the
running script file) before iterating lines. In the inline script that builds
candidates (the code around candidates =
list(Path(".github/workflows").rglob("*.yml")) / candidates +=
list(Path(".github/workflows").rglob("*.yaml")) and the loop that checks
allowed_refs and pattern), filter out the validate.yml path (or any file equal
to the script's path) so the allowlist strings and error text in validate.yml
won't be treated as violations.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Outside diff comments:
In @.github/workflows/validate.yml:
- Around line 58-98: The validator is currently scanning its own workflow file
and matching the allowlist strings; update the candidate collection/filtering so
Path(".github/workflows/validate.yml") is excluded (or generally skip the
running script file) before iterating lines. In the inline script that builds
candidates (the code around candidates =
list(Path(".github/workflows").rglob("*.yml")) / candidates +=
list(Path(".github/workflows").rglob("*.yaml")) and the loop that checks
allowed_refs and pattern), filter out the validate.yml path (or any file equal
to the script's path) so the allowlist strings and error text in validate.yml
won't be treated as violations.

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro Plus

Run ID: b3f5c873-4bf5-4a2d-b7f5-a632d9baca83

📥 Commits

Reviewing files that changed from the base of the PR and between 3c91ae4 and d8960b4.

📒 Files selected for processing (14)
  • .github/workflows/e2e.yml
  • .github/workflows/pr-e2e.yml
  • .github/workflows/promotion-candidate-e2e.yml
  • .github/workflows/run-testsuite.yml
  • .github/workflows/skill-drift.yml
  • .github/workflows/validate.yml
  • AGENTS.md
  • README.md
  • docs/factory/README.md
  • docs/skills/INDEX.md
  • docs/skills/acmm-audit-level1.md
  • docs/skills/ci-tooling.md
  • docs/skills/e2e-ci.md
  • docs/skills/image-registry.md

castrojo and others added 3 commits June 4, 2026 15:56
Assisted-by: GPT-5.4 via Copilot CLI
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Assisted-by: GPT-5.4 via Copilot CLI

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Assisted-by: GPT-5.4 via GitHub Copilot CLI

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@castrojo castrojo force-pushed the feat/acmm-common-guardrails branch from 894f86a to 3da0061 Compare June 4, 2026 19:57
@castrojo castrojo merged commit e1e57e8 into main Jun 4, 2026
3 of 5 checks passed
@castrojo castrojo deleted the feat/acmm-common-guardrails branch June 4, 2026 19:57
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area/testing Testing and QA kind/automation Automation and CI/CD kind/documentation Improvements or additions to documentation size:L This PR changes 100-499 lines, ignoring generated files.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant