Skip to content

ci: remove Blacksmith runner usage - #856

Merged
haasonsaas merged 2 commits into
mainfrom
ci/remove-blacksmith
Jul 21, 2026
Merged

ci: remove Blacksmith runner usage#856
haasonsaas merged 2 commits into
mainfrom
ci/remove-blacksmith

Conversation

@haasonsaas

@haasonsaas haasonsaas commented Jul 21, 2026

Copy link
Copy Markdown
Contributor

Summary

  • Mirrored source-of-truth PR: evalops/maestro-internal#2944 (scripts/check-workflow-footguns.mjs, test/scripts/ci-guardrails.test.ts, and test/scripts/workflow-footguns.test.ts are identical in both repos; .github/actionlint.yaml and .github/workflows/* are edited per-repo since public CI already diverges from internal CI).
  • Org-wide decision (Jonathan, 2026-07-20): drop Blacksmith entirely. Tracking issue: evalops/deploy#8126.
  • This repo (maestro) is public, so ubuntu-latest is the correct, free target here -- unlike evalops/maestro-internal#2944, which is private and had to route to the org's owned Hetzner pool instead because GitHub-hosted Actions spend is currently hard-blocked on private repos (org budget exhausted, prevent_further_usage=true, will not be raised before it resets in August). Public-repo minutes are free and unaffected. If you're comparing the two PRs' diffs and wondering why they diverge on runner targets, that's why.
  • This repo's public-facing CI (ci.yml, rust.yml, release.yml) already routed through PUBLIC_PR_VALIDATION_RUNNER || 'ubuntu-latest' with no Blacksmith fallback, so the only live references were:
    • tag-release.yml / version-bump.yml's shared main-confirmation lane (used when github.repository == 'evalops/maestro-internal', i.e. dead in this repo's own runs but kept in sync with maestro-internal): INTERNAL_CONFIRMATION_RUNNER || BLACKSMITH_RUNNER || 'blacksmith-4vcpu-ubuntu-2404' -> INTERNAL_CONFIRMATION_RUNNER || 'ubuntu-latest'.
    • .github/actionlint.yaml's self-hosted-runner label registry, which listed the four blacksmith-* labels.
  • scripts/check-workflow-footguns.mjs no longer requires vars.BLACKSMITH_RUNNER as one of the acceptable PR runner failover vars, and gains a new guardrail (evaluateNoBlacksmithReferences) that recursively scans all of .github/ -- workflows, composite actions under .github/actions/**, and .github/actionlint.yaml -- for a blacksmith-* label or BLACKSMITH_* var, so the fleet can't silently creep back in from anywhere in .github/, not just workflow files.
  • test/scripts/ci-guardrails.test.ts and test/scripts/workflow-footguns.test.ts are updated to match (dead-branch assertions in this repo's test file that referenced Blacksmith -- inherited from the shared maestro-internal test content -- are cleaned up too; new tests cover the ban, including the composite-action/actionlint.yaml coverage).

require-internal-pr gate

This repo's public-source-provenance.yml requires PRs that touch mirrored source files to link the matching evalops/maestro-internal PR in the body (it is not a blanket ban on direct public PRs -- .github/workflows/* and .github/actionlint.yaml are explicitly exempt as public-only files; scripts/check-workflow-footguns.mjs and the two test files are the mirrored files that triggered it). Added the link above; gate passes.

Left in place (and why)

  • CHANGELOG.md mentions of Blacksmith (historical release notes) -- left untouched, it's a record of what was true at the time.

Test plan

  • node ./scripts/run-vitest.js --run test/scripts/ -- 43 files / 497 tests passed.
  • node scripts/check-workflow-footguns.mjs -- guardrails passed.
  • actionlint .github/workflows/tag-release.yml .github/workflows/version-bump.yml -- clean.
  • bunx @biomejs/biome@1.9.4 check on touched JS/TS files -- clean.

Co-authored by Claude Fable 5.


Open in Devin Review

Org is dropping Blacksmith entirely (owner decision 2026-07-20). This repo
already routed public CI through PUBLIC_PR_VALIDATION_RUNNER/ubuntu-latest,
so the only live references were the shared main-confirmation lane in
tag-release.yml and version-bump.yml (INTERNAL_CONFIRMATION_RUNNER now
falls back straight to ubuntu-latest instead of a blacksmith-* label) and
the self-hosted-runner label registry in actionlint.yaml.

check-workflow-footguns.mjs no longer requires BLACKSMITH_RUNNER as one of
the acceptable PR runner failover vars, and gains a new guardrail that
fails any workflow file referencing a blacksmith-* label or BLACKSMITH_*
var, so the fleet cannot silently creep back in. Test fixtures and
assertions in ci-guardrails.test.ts / workflow-footguns.test.ts are updated
to match (including new tests for the ban).

Ref evalops/deploy#8126 (org-wide Blacksmith removal tracking issue).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FsPX4PuGaFFajJEJcT1VuD
@chatgpt-codex-connector

Copy link
Copy Markdown

You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.

@github-actions

Copy link
Copy Markdown
Contributor

This PR changes mirrored Maestro source files in the public repo, but it does not link the matching private source-of-truth PR.

Add one of these to the PR body, then re-run the check:

  • https://github.com/evalops/maestro-internal/pull/<number>
  • evalops/maestro-internal#<number>
  • maestro-internal#<number>

Mirrored files touched:

  • scripts/check-workflow-footguns.mjs
  • test/scripts/ci-guardrails.test.ts
  • test/scripts/workflow-footguns.test.ts

@devin-ai-integration devin-ai-integration Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Devin Review found 2 potential issues.

Open in Devin Review

Comment thread scripts/check-workflow-footguns.mjs
Comment thread test/scripts/ci-guardrails.test.ts
Devin review on this PR flagged that evaluateNoBlacksmithReferences only
scanned .github/workflows/*.yml, so a blacksmith-* reference reintroduced
inside a composite action (.github/actions/**/action.yml) or back into
.github/actionlint.yaml's label registry wouldn't be caught. No live
reference exists today, but the guard should cover it. Scan all of
.github/ recursively for any .yml/.yaml file instead.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FsPX4PuGaFFajJEJcT1VuD
@haasonsaas
haasonsaas merged commit 681db6b into main Jul 21, 2026
15 of 17 checks passed
@haasonsaas
haasonsaas deleted the ci/remove-blacksmith branch July 21, 2026 02:55
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant