Skip to content

ci(gc): consume the setup-llvm22 action's LLVM 22 pair in the RS4GC job - #7384

Merged
proggeramlug merged 1 commit into
PerryTS:mainfrom
jdalton:fix/rs4gc-arm-clang-pair
Aug 4, 2026
Merged

ci(gc): consume the setup-llvm22 action's LLVM 22 pair in the RS4GC job#7384
proggeramlug merged 1 commit into
PerryTS:mainfrom
jdalton:fix/rs4gc-arm-clang-pair

Conversation

@jdalton

@jdalton jdalton commented Aug 4, 2026

Copy link
Copy Markdown
Contributor

Summary

The native-roots-rs4gc matrix's ubuntu-24.04-arm (aarch64/ELF) arm has been failing with no matched opt+clang pair under /usr/lib/llvm-22/bin — RS4GC cannot run. This is a toolchain-discovery gap in the workflow, not a GC-code problem: that runner image ships llvm-22's opt but no co-located clang, and the discovery only installed a toolchain when opt was missing — so on this arm it installed nothing, and the final opt && clang pair-check then failed. The x86-64 arm, where clang sits next to opt, was never affected.

Changes

  • .github/workflows/gc-native-roots.yml — in the Ubuntu branch of the opt/clang discovery: after resolving opt's directory, derive its LLVM major version and install the version-matched clang-<N> (falling back to the unversioned llvm/clang metapackages when the dir isn't a /usr/lib/llvm-<N>/bin) whenever either half of the pair is missing — not only when opt is absent. If apt lands the versioned clang at /usr/bin/clang-<N> instead of the llvm bin dir, symlink it in so the same-directory pair-check (and PERRY_LLVM_CLANG) resolves.

Ubuntu packages opt (in llvm-<N>) and clang (in clang-<N>) separately, so an image can genuinely have one without the other — which is exactly what the arm runner does.

Why the old logic couldn't recover, step by step

On ubuntu-24.04-arm (LLVM_SYS_221_PREFIX=/usr/lib/llvm-22):

  1. command -v opt finds no unversioned opt, so llvm_bin falls to the newest /usr/lib/llvm-*/bin glob → /usr/lib/llvm-22/bin.
  2. /usr/lib/llvm-22/bin/opt exists, so the install-fallback (guarded on ! -x opt) is skipped.
  3. /usr/lib/llvm-22/bin/clang does not exist, so the final [ ! -x opt ] || [ ! -x clang ] check trips and the job exits 1.

The install condition now also fires on a missing clang, and it installs clang-22 (matched to the opt already present) rather than an unversioned clang that could skew versions.

Related issue

n/a — CI gate fix for the RS4GC native-roots matrix. Unblocks the ubuntu-24.04-arm arm for every PR that inherits this workflow.

Test plan

Validated the changed shell locally (the arm runner itself is the real proof, on CI):

python3 -c "import yaml; yaml.safe_load(open('.github/workflows/gc-native-roots.yml'))"   # valid YAML
bash -n <the run block>                                                                   # syntax OK
# dry-run of the version/package selection:
#   /usr/lib/llvm-22/bin -> ver=22 -> apt install llvm-22 clang-22
#   /usr/bin | .         -> ver=<none> -> apt install llvm clang   (unversioned fallback)
  • cargo build --release clean — n/a (CI-workflow-only change, no Rust touched)
  • cargo test … — n/a
  • YAML + bash -n validated; package-selection dry-run correct on versioned and unversioned dirs

Checklist

  • I have NOT bumped the workspace version or edited CLAUDE.md / CHANGELOG.md
  • My commits follow the ci: prefix convention
  • CI-only churn — no crates/ touched, so no changelog fragment required (skip-changelog territory)

Summary by CodeRabbit

  • Bug Fixes
    • Improved Ubuntu build setup by consistently using the configured LLVM 22 toolchain.
    • Ensured the matching Clang 22 compiler is installed and available when needed.
    • Removed unreliable generic LLVM/Clang fallback behavior, improving native garbage-collection build compatibility.
    • Ensured LLVM tools are selected from the verified LLVM 22 installation for more reliable builds.
  • Documentation
    • Added release documentation describing the updated LLVM and Clang toolchain handling.

@coderabbitai

coderabbitai Bot commented Aug 4, 2026

Copy link
Copy Markdown

Review Change Stack

📝 Walkthrough

Walkthrough

The Ubuntu workflow now uses the setup-llvm22 prefix, provisions clang-22 when absent, and links it into the LLVM directory. The changelog records the toolchain discovery change.

Changes

LLVM toolchain setup

Layer / File(s) Summary
Provision matching LLVM 22 tools
.github/workflows/gc-native-roots.yml, changelog.d/7384-rs4gc-llvm22-prefix.md
The workflow uses the configured LLVM 22 prefix, installs clang-22 when needed, creates a symlink, and removes generic LLVM and Clang fallback packages. The changelog documents the updated behavior.

Estimated code review effort: 2 (Simple) | ~10 minutes

Possibly related PRs

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Title check ✅ Passed The title clearly identifies the CI workflow change to use the setup-llvm22 LLVM 22 pair in the RS4GC job.
Description check ✅ Passed The description includes the required summary, changes, issue status, test plan, and checklist sections with relevant technical details.
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@jdalton

jdalton commented Aug 4, 2026

Copy link
Copy Markdown
Contributor Author

What this unblocks

This fixes the native-roots-rs4gc (ubuntu-24.04-arm, aarch64, ELF) arm that was added to main in #7360. Because that image carries llvm-22's opt but no co-located clang, and the discovery only installed a toolchain when opt was missing, the arm has been red on every PR that inherits gc-native-roots.yml since #7360 merged — #7360's own run was cancelled, so this gate has never completed green anywhere.

It's not a required check, so it doesn't hard-block merges, but it's a red gate that should land first to clear the inherited failure on the open PRs that currently trip it:

x86-64, where clang is co-located with opt, was never affected.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In @.github/workflows/gc-native-roots.yml:
- Around line 220-229: Update the LLVM installation block around the versioned
apt-get command so the generic llvm clang installation runs whenever the
discovered versioned package install fails, rather than exiting under set -e.
After either installation path, re-resolve llvm_bin and the opt/clang
executables before the final same-directory pair check, preserving the existing
versioned clang co-location behavior.
🪄 Autofix

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: 33a0469d-6025-4a0e-8978-ce183733dd7f

📥 Commits

Reviewing files that changed from the base of the PR and between 0f3182d and e2f88d5.

📒 Files selected for processing (1)
  • .github/workflows/gc-native-roots.yml

Comment thread .github/workflows/gc-native-roots.yml Outdated
@proggeramlug

Copy link
Copy Markdown
Contributor

Your diagnosis is right — ubuntu-24.04-arm really does carry /usr/lib/llvm-22/bin/opt with no co-located clang, and a check that only looks for opt installs nothing and then fails the pair-check.

I think the cause sits one level up, so I've opened #7388 against the composite action instead: setup-llvm22 installs llvm-22-dev libpolly-22-dev libzstd-dev and no clang-22. apt ships opt and clang as separate packages, so that's what leaves the bin dir half-populated. #7388 installs the pair, symlinks clang under the llvm-config-22 prefix so directory-based resolution finds them co-located, and asserts clang's major.

There's also a sharper problem in the block you're editing, independent of the missing clang. This workflow already does uses: ./.github/actions/setup-llvm22 at line 167, then discards the pinned prefix at line 209 and re-discovers by hand:

llvm_bin="$(dirname "$(command -v opt || true)")"
...
sudo apt-get install -y -qq llvm clang      # Ubuntu 24.04 → LLVM 18

If command -v opt resolves to an unversioned /usr/bin/opt, that installs the distro's LLVM 18. Both binaries exist, the same-dir pair-check passes, and RS4GC runs opt 18 over IR emitted by Perry's linked LLVM 22 — green gate, wrong subject. Your ver="" fallback still reaches it. The action's own comment flags this ("Ubuntu 24.04 ships 18") and the workflow bypasses it forty lines below.

Once #7388 lands there's a guaranteed co-located 22 pair at $LLVM_SYS_221_PREFIX/bin, so the whole Linux discovery block here can collapse to that and the skew hazard goes with it. I deliberately didn't touch this file so as not to conflict with your diff — happy for you to take it, or I can push that simplification if you'd rather.

One other thing: there's no changelog.d/ fragment on this PR and changeset-gate is a required check, so it'll go red as-is.

proggeramlug added a commit that referenced this pull request Aug 4, 2026
* ci: install clang-22 in setup-llvm22 and assert the pair

apt ships opt and clang as separate packages, so llvm-22-dev alone leaves
/usr/lib/llvm-22/bin/opt with no clang beside it -- the state
ubuntu-24.04-arm runners land in. Consumers needing a matched opt+clang
pair (the RS4GC arm in gc-native-roots.yml) then fail the pair-check, or
fall back to hand-rolled discovery and install the distro's unversioned
llvm clang, which on Ubuntu 24.04 is LLVM 18 -- running opt 18 over IR
emitted by Perry's linked LLVM 22.

Install clang-22, symlink it under the llvm-config-22 prefix so
directory-based resolution finds a co-located pair, and assert clang's
major alongside the existing llvm-config check. A green setup step that
leaves a mismatched clang is the failure this action exists to prevent.

Fixes the root cause of #7384 for all 18 workflows rather than one.

* docs: name the fragment for its real PR (#7388)

---------

Co-authored-by: Ralph Küpper <ralph@skelpo.com>
@jdalton
jdalton force-pushed the fix/rs4gc-arm-clang-pair branch from e2f88d5 to d129943 Compare August 4, 2026 18:36
@jdalton

jdalton commented Aug 4, 2026

Copy link
Copy Markdown
Contributor Author

Thanks — this is the better framing, and you're right that the sharper bug is the skew, not the missing clang.

I took the workflow-block simplification. #7384 now drops the hand-discovery entirely and uses the $LLVM_SYS_221_PREFIX/bin the setup-llvm22 action pins, so it can never resolve an unversioned /usr/bin/opt or fall back to the distro's LLVM 18. I removed the generic llvm clang fallback (the skew vector) — a missing clang-22 now fails the pair-check instead of passing on the wrong LLVM.

Until #7388 lands, the block installs the matching clang-22 as a bridge and co-locates it under the prefix; once #7388 folds that into the action, this collapses to just llvm_bin="$LLVM_SYS_221_PREFIX/bin", exactly as you described. It's order-independent — the bridge is a no-op the moment the action installs clang-22, so merge in whatever order suits #7388.

Also added a changelog.d/ fragment for changeset-gate and rebased onto main.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🧹 Nitpick comments (1)
.github/workflows/gc-native-roots.yml (1)

219-224: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick win

Use one LLVM 22 provisioning contract.

The supplied .github/actions/setup-llvm22/action.yml already installs and verifies the matched LLVM 22 tools. The workflow duplicates that setup, and the changelog documents the old ownership model.

  • .github/workflows/gc-native-roots.yml#L219-L224: remove the duplicate clang-22 installation and symlink block. Keep the configured prefix and final pair check.
  • changelog.d/7384-rs4gc-llvm22-prefix.md#L1-L1: state that the workflow consumes the action-provided LLVM 22 pair. Remove the claim that the action omits clang-22 and the temporary #7388 bridge.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In @.github/workflows/gc-native-roots.yml around lines 219 - 224, Use the
setup-llvm22 action as the sole LLVM 22 provisioning contract: in
.github/workflows/gc-native-roots.yml lines 219-224, remove the duplicate
clang-22 installation and symlink block while preserving the configured prefix
and final compiler pair check. In changelog.d/7384-rs4gc-llvm22-prefix.md line
1, document that the workflow consumes the action-provided LLVM 22 pair and
remove the outdated claim that clang-22 is omitted and the temporary `#7388`
bridge is needed.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Nitpick comments:
In @.github/workflows/gc-native-roots.yml:
- Around line 219-224: Use the setup-llvm22 action as the sole LLVM 22
provisioning contract: in .github/workflows/gc-native-roots.yml lines 219-224,
remove the duplicate clang-22 installation and symlink block while preserving
the configured prefix and final compiler pair check. In
changelog.d/7384-rs4gc-llvm22-prefix.md line 1, document that the workflow
consumes the action-provided LLVM 22 pair and remove the outdated claim that
clang-22 is omitted and the temporary `#7388` bridge is needed.

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: 3b09fa5a-f6d5-49f1-a7ad-f84d2504eea0

📥 Commits

Reviewing files that changed from the base of the PR and between e2f88d5 and d129943.

📒 Files selected for processing (2)
  • .github/workflows/gc-native-roots.yml
  • changelog.d/7384-rs4gc-llvm22-prefix.md

@jdalton
jdalton force-pushed the fix/rs4gc-arm-clang-pair branch from d129943 to c9ecaf6 Compare August 4, 2026 18:50
@jdalton

jdalton commented Aug 4, 2026

Copy link
Copy Markdown
Contributor Author

Follow-up now that #7388 has merged: I dropped the clang-22 bridge as well. The Linux branch is just llvm_bin="${LLVM_SYS_221_PREFIX:-/usr/lib/llvm-22}/bin" plus the existing pair-check — it consumes the matched LLVM 22 pair the setup-llvm22 action already installs, co-locates, and verifies, with no duplicate install or symlink. That also addresses CodeRabbit's nitpick about the duplicated setup, and I updated the changelog to describe the action-provided pair rather than the old bridge. Rebased onto main (c9ecaf64b).

The native-roots-rs4gc job re-discovered opt by hand and could resolve an
unversioned /usr/bin/opt, then install the distro's LLVM 18 — both binaries
present, the same-directory pair-check passing, and RS4GC running opt 18 over
IR from Perry's linked LLVM 22 (a green gate on the wrong LLVM). Consume the
matched LLVM 22 opt+clang pair the setup-llvm22 action installs and verifies
under $LLVM_SYS_221_PREFIX instead, dropping the hand-discovery and its generic
llvm/clang fallback.
@jdalton
jdalton force-pushed the fix/rs4gc-arm-clang-pair branch from c9ecaf6 to 37ab182 Compare August 4, 2026 18:55
@jdalton jdalton changed the title ci(gc): install a version-matched clang so RS4GC finds an opt+clang pair on arm ci(gc): consume the setup-llvm22 action's LLVM 22 pair in the RS4GC job Aug 4, 2026
@proggeramlug

Copy link
Copy Markdown
Contributor

This is better than what I suggested — dropping the bridge entirely rather than keeping it alongside the prefix is the right call, and the nine lines of hand-discovery going away takes the LLVM-18 skew hazard with them.

Two things from my side that this now composes with:

With those in, the arm now compiles everything and runs probe 1 with real GC metrics, then segfaults in 02_survivor_promotion under forced evacuation — tracked as #7392.

One caveat on that issue worth knowing before anyone chases it: I checked ten consecutive gc-native-roots runs and the macos-14, ubuntu-latest and windows-latest arms were queued in all ten — never executed. Only the aarch64 arm has ever reached a runner. That was a missing concurrency group, fixed in #7393. Until the other arms actually run once, "the segfault is ELF-specific" can't be told apart from "macOS has never run the probe".

@proggeramlug
proggeramlug merged commit 8d58330 into PerryTS:main Aug 4, 2026
1 of 16 checks passed
@jdalton
jdalton deleted the fix/rs4gc-arm-clang-pair branch August 4, 2026 19:06
proggeramlug added a commit that referenced this pull request Aug 4, 2026
* docs(plan): fold in the 2026-08-04 findings

Two things this plan treated as measured were not.

Statepoints could not compile on aarch64-ELF at all -- a hard failure on
a default-on path, from two stacked bugs (#7390: the compact stack-map
parser did not model GNU-as `sym = expr`, emitted only at -O3 and only on
ELF; and the assembler was not told the -mcpu the code generator was
told, so Graviton-emitted SVE was rejected) behind two toolchain ones
(#7384, #7388).

And three of the four RS4GC matrix arms had NEVER executed, in any run,
for want of a concurrency group (#7393). Every "the ELF arm is the only
one red" conclusion rested on arms that never reached a runner. That is a
fifth way a gate cannot fail, and it is now written down.

Also folded in: nine Layer 3 rooting fixes and the rule they share
(ordering, not missing roots; a fault that MOVES is a real fix, one that
does not move by a byte was already dead before you rooted it); #7380's
type confusion and the `gc_type == GC_TYPE_OBJECT` generalisation; RSS
-69% (#7377); and the first honest performance measurement -- two
benchmarks that measure nothing (#7395) and the array-store guard's
siting cost (#7396).

The Layer 1 framing is corrected: lower_exprs_rooted already implements
the RFC's proposal for codegen operands, gated on
any_later_ref_may_trigger_gc, and all four arms of func_ref.rs use it. So
the gap is Layer 3, where #7389 supplies the first structural answer.

* docs: name the fragment for its real PR (#7397)

---------

Co-authored-by: Ralph Küpper <ralph@skelpo.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants