Skip to content

ci: pair macOS-amd64 tcc.exe rebuilds with a freshly-rebuilt, dynamically-linked libgc - #27982

Merged
JalonSolov merged 3 commits into
vlang:masterfrom
quaesitor-scientiam:fix/macos-amd64-libgc-dylib-pairing
Jul 30, 2026
Merged

ci: pair macOS-amd64 tcc.exe rebuilds with a freshly-rebuilt, dynamically-linked libgc#27982
JalonSolov merged 3 commits into
vlang:masterfrom
quaesitor-scientiam:fix/macos-amd64-libgc-dylib-pairing

Conversation

@quaesitor-scientiam

@quaesitor-scientiam quaesitor-scientiam commented Jul 30, 2026

Copy link
Copy Markdown
Contributor

Problem

thirdparty-macos-amd64_tcc.sh only ever rebuilds tcc.exe - it explicitly preserves whatever lib/libgc* was already committed, unchanged, on every automated update_tccbin.yml run. This produced a genuinely broken pair (vlang/tccbin commit d712d0f: new tcc.exe, old never-rebuilt libgc.a), which vlang/tccbin's CI correctly rejects - that check must stay blocking; this PR fixes the producer, not the check.

A freshly-rebuilt static libgc.a alone would not fix this either: vlang/tccbin PR #74 already proved tcc can't reliably link a modern-toolchain-built static archive on macOS-amd64 (GC_init well-defined, archive well-formed, tcc still reports it unresolved) - the same limitation macOS-arm64 already solved via libgc.dylib + rpath.

What this PR does

  • Adds thirdparty-macos-amd64_bdwgc.sh, mirroring the established arm64 pattern, with amd64-specific corrections grounded in PR suggestions for syntax & improvements #74's already-proven recipe (CC=clang, pinned MACOSX_DEPLOYMENT_TARGET, stale-artifact cleanup before staging, and explicit readlink/otool re-verification that libgc.dylib actually landed as a real symlink - confirmed via the GitHub API that the existing arm64 bundle's libgc.dylib is committed as a plain file, not a symlink, i.e. this exact failure mode has already happened once, silently).
  • Pins the deployment target on the tcc.exe half of the pair too, and records a toolchain-identity provenance file - neither half pinned this before.
  • Adds thirdparty-macos-amd64_bdwgc_validate.sh, porting the hardened dylib-blocking / no-GC-fallback / static-signature-XFAIL checks already proven in vlang/tccbin's build-and-test.yml, corrected to use V's actual full flag set (GC_THREADS, THREAD_LOCAL_ALLOC, not just the subset tccbin's own CI currently tests) and adapted to the current run.sh output format.
  • Expands update_tccbin.yml's rebuild-skip decision, for macos-amd64 only, to fingerprint the whole pair (TinyCC + bdwgc + libatomic_ops SHAs, deployment target, toolchain identity, a bumpable recipe version, required-file presence) rather than TinyCC alone.
  • After staging and committing the pair, clones that exact new commit into a genuinely independent, fresh checkout and re-validates against it - the check that actually catches "file exists in the build directory but was never git added" (the same class of gap that produced arm64's non-symlinked libgc.dylib).
  • Builds V itself in a separate, freshly-cloned workspace for a real compile-and-execute smoke test (-cc tcc -gc boehm) against the staged bundle. Deliberately never runs make against the staged checkout itself: GNUmakefile's default target depends on latest_tcc, which runs git clean -xf && git pull --rebase inside thirdparty/tcc unless invoked with local=1 - this would silently destroy the staged, uncommitted GC files before they're ever validated or committed.
  • Uploads the bundle as a tar (preserves symlinks/executable bits, unlike actions/upload-artifact on a raw directory) alongside the existing raw-directory upload.

What this PR deliberately does NOT do

Does not include the vlib/builtin/builtin_d_gcboehm.c.v selector change (amd64 still selects libgc.a there). That's a separate, later PR, held back until this pair is actually published and vlang/tccbin's checked-in-package gate (itself needing a coordinated follow-up: it currently hard-fails if that package unexpectedly starts passing, since today it's supposed to still be broken) is confirmed green under it. The smoke test above applies the selector collapse as an in-memory-only patch to the isolated workspace, never written back to this checkout or committed anywhere, specifically so this PR can validate the real end-to-end path without prematurely shipping the selector change.

Not touched: macOS-arm64 has the identical latent gap (its update_tccbin.yml entry also never rebuilds GC) but is out of scope here. linux-amd64's _tcc.sh has the same preserve-libgc pattern but wasn't reported broken. No credential/push-repair changes.

Verification plan

This needs to run on a real macos-15-intel GitHub Actions runner to validate - I can't do that from here. Suggested path:

gh workflow run update_tccbin.yml --ref <this-branch-once-its-a-real-vlang/v-ref> -f publish=false -f force_rebuild=true -f tcc_commit=mob -f libgc_commit=master

force_rebuild=true is deliberate here, not just a default: the current thirdparty-macos-amd64 tree has none of the new fingerprint files (libgc_build_source_hash.txt, etc.) this PR's expanded rebuild-skip check compares against, so relying on that check alone for the very first run against this new logic is untested territory - force it explicitly instead.

workflow_dispatch --ref needs a real ref in vlang/v itself (a fork branch won't resolve) - happy to have this validated on a test branch, or if a maintainer wants to dispatch it directly, I'll iterate on any failures.

Once this is merged and validated, the plan is: land the vlang/tccbin gate-conversion follow-up → publish the fix with force_rebuild=true → confirm vlang/tccbin CI is green → only then open the selector-change PR.

…ally-linked libgc

thirdparty-macos-amd64_tcc.sh only ever rebuilds tcc.exe - it
explicitly preserves whatever lib/libgc* was already committed,
unchanged, on every automated update_tccbin.yml run. This produced a
genuinely broken pair (vlang/tccbin commit d712d0f: new tcc.exe, old
never-rebuilt libgc.a), which vlang/tccbin's CI correctly rejects -
that check must stay blocking; this PR fixes the producer, not the
check.

A freshly-rebuilt STATIC libgc.a alone would not fix this either:
vlang/tccbin PR vlang#74 already proved tcc can't reliably link a modern-
toolchain-built static archive on macOS-amd64 (GC_init well-defined,
archive well-formed, tcc still reports it unresolved) - the same
limitation macOS-arm64 already solved via libgc.dylib + rpath.

This PR:
- Adds thirdparty-macos-amd64_bdwgc.sh, mirroring the established
  arm64 pattern, with amd64-specific corrections grounded in PR vlang#74's
  already-proven recipe (CC=clang, pinned MACOSX_DEPLOYMENT_TARGET,
  stale-artifact cleanup before staging, and explicit readlink/otool
  re-verification that libgc.dylib actually landed as a real symlink -
  confirmed via the GitHub API that the *existing* arm64 bundle's
  libgc.dylib is committed as a plain file, not a symlink, i.e. this
  exact failure mode has already happened once, silently).
- Pins the deployment target on the tcc.exe half of the pair too, and
  records a toolchain-identity provenance file - neither half pinned
  this before, so a rebuilt binary silently inherited the CI runner's
  own floor.
- Adds thirdparty-macos-amd64_bdwgc_validate.sh, porting the hardened
  dylib-blocking / no-GC-fallback / static-signature-XFAIL checks
  already proven in vlang/tccbin's build-and-test.yml, corrected to
  use V's actual full flag set (GC_THREADS, THREAD_LOCAL_ALLOC, not
  just the subset tccbin's own CI currently tests) and adapted to the
  current (not the old pinned) run.sh output format - the direct
  symbol/error-text checks stay authoritative either way.
- Expands update_tccbin.yml's rebuild-skip decision, for macos-amd64
  only, to fingerprint the whole pair (TinyCC + bdwgc + libatomic_ops
  SHAs, deployment target, toolchain identity, a bumpable recipe
  version, and required-file presence) rather than TinyCC alone -
  resolved SHAs are threaded through explicitly end to end, not
  re-resolved from a floating ref at build time.
- After staging and committing the pair, clones that exact new commit
  into a genuinely independent, fresh checkout and re-validates
  against it - the check that actually catches "file exists in the
  build directory but was never git-added" (the same class of gap
  that produced arm64's non-symlinked libgc.dylib).
- Builds V itself in a separate, freshly-cloned workspace for a real
  compile-and-execute smoke test (-cc tcc -gc boehm) against the
  staged bundle, deliberately never running `make` against the staged
  checkout itself - GNUmakefile's default target depends on
  latest_tcc, which runs `git clean -xf && git pull --rebase` inside
  thirdparty/tcc unless invoked with `local=1`, which would silently
  destroy the staged, uncommitted GC files before they're ever
  validated or committed.
- Uploads the bundle as a tar (preserves symlinks/executable bits,
  unlike actions/upload-artifact on a raw directory) alongside the
  existing raw-directory upload.

Does NOT include the vlib/builtin/builtin_d_gcboehm.c.v selector
change (amd64 still selects libgc.a there) - that's a deliberately
separate, later PR, held back until this pair is actually published
and vlang/tccbin's checked-in-package gate (itself needing a
coordinated follow-up: it currently hard-fails if that package
unexpectedly starts passing, since today it's supposed to still be
broken) is confirmed green under it. The smoke test above applies the
selector collapse as an in-memory-only patch to the isolated
workspace, never written back to this checkout or committed anywhere,
specifically so this PR can validate the real end-to-end path without
prematurely shipping the selector change itself.

Not touched: macOS-arm64 has the identical latent gap (its
update_tccbin.yml entry also never rebuilds GC) but is out of scope
here. linux-amd64's _tcc.sh has the same preserve-libgc pattern but
wasn't reported broken. No credential/push-repair changes.
@GGRei

GGRei commented Jul 30, 2026

Copy link
Copy Markdown
Contributor

Nice work — this matches the plan very closely. I only spotted three small items:

  • Pass the resolved libatomic_ops SHA into the build and explicitly check it out.
  • Include lib/libgc.dylib itself in bundle_checksums.txt.
  • Add force_rebuild=true to the first publish=false validation run.

Once those are addressed and the macos-15-intel run is green, it looks good to me. Thanks!

Reviewer feedback on vlang#27982:

- thirdparty-macos-amd64_bdwgc.sh cloned libatomic_ops's default
  branch and recorded whatever HEAD it happened to land on, without
  ever checking out the SHA update_tccbin.yml had already resolved and
  fingerprinted moments earlier - the same class of drift risk already
  fixed for LIBGC_COMMIT, just missed for libatomic_ops. Added
  LIBATOMIC_OPS_COMMIT (passed through from the workflow's resolved
  steps.bdwgc.outputs.libatomic_ops_hash) and an explicit `git checkout`
  before recording its hash.
- bundle_checksums.txt hashed lib/$libgc_dylib_target (the real
  versioned file) but not lib/libgc.dylib itself - the exact path V's
  own #flag references. shasum follows the symlink and produces the
  same hash either way, but recording it under the path that's
  actually used matters for anyone verifying the bundle.

Also updated the PR's suggested first validation dispatch to include
force_rebuild=true, since the current repo state has none of the new
provenance files this fingerprint check would otherwise compare
against - matching this same PR's own documented rule for the first
repair run.
@quaesitor-scientiam

Copy link
Copy Markdown
Contributor Author

Thanks for the quick review! All three addressed in c493456:

  • libatomic_ops SHA: thirdparty-macos-amd64_bdwgc.sh now accepts an explicit LIBATOMIC_OPS_COMMIT (same pattern as LIBGC_COMMIT) and does git -C libatomic_ops checkout $LIBATOMIC_OPS_COMMIT before recording its hash - it was previously cloning the default branch and just recording whatever HEAD it landed on, without the workflow's already-resolved SHA ever actually being passed through or checked out. update_tccbin.yml's "Build paired libgc" step now passes steps.bdwgc.outputs.libatomic_ops_hash through explicitly.
  • bundle_checksums.txt: now hashes lib/libgc.dylib itself (the exact path V's #flag references) alongside lib/$libgc_dylib_target - shasum follows the symlink so the hash values match, but the path a consumer would actually verify against is now recorded directly.
  • force_rebuild=true: added to the suggested first-dispatch command in the PR description, with a note on why it's needed for this specific run (no fingerprint files exist yet on the current tree for the new check to compare against).

Ready for the macos-15-intel validation run whenever you're able to dispatch it against a real vlang/v ref.

@GGRei

GGRei commented Jul 30, 2026

Copy link
Copy Markdown
Contributor

Thanks, this is very close and it covers our plan well, including the three earlier points. I found three blocking workflow issues: the fresh-checkout validator prefixes an already absolute path with $PWD; the isolated V clone uses make local=1 even though it has no vc/; and deletions of old versioned libgc dylibs are not staged if the SONAME changes. We should also prevent the monthly cron from publishing macOS-amd64 before the tccbin gate follow-up is ready. Once those are fixed, a green publish=false, force_rebuild=true run on macos-15-intel should complete this phase.

After that, I launch codex review.

… SONAME deletions, publish lock

- thirdparty-macos-amd64_bdwgc_validate.sh: don't blindly prefix $PWD onto
  TCC_FOLDER - one call site passes it relative (thirdparty/tcc), the other
  passes it already-absolute (/tmp/tccbin-fresh-checkout), and $PWD/<abs>
  collapsed into a nonexistent path for the latter.
- update_tccbin.yml isolated-workspace step: use plain `make -j4`, not
  `make local=1` - this is a genuinely fresh clone with no vc/ or
  thirdparty/tcc yet (both gitignored), and `local=1` skips exactly the
  git-clone bootstrap this workspace needs, unlike bootstrapping_ci.yml
  where a prior plain `make` already populated vc/ first.
- update_tccbin.yml commit step: git add -A against libgc*.dylib/libgc*.a
  globs instead of a hardcoded file list, so a SONAME bump's old versioned
  file (already rm -f'd from disk) gets its deletion staged too, instead of
  reappearing from history on every future checkout.
- update_tccbin.yml: gate the macos-amd64 tccbin push on a repo variable
  (MACOS_AMD64_LIBGC_PUBLISH_UNLOCKED) in addition to PUBLISH, so neither
  the monthly cron nor a manual publish=true dispatch can push before
  vlang/tccbin's gate conversion (deliverable B) is confirmed live.

Co-Authored-By: WOZCODE <contact@withwoz.com>
@quaesitor-scientiam

Copy link
Copy Markdown
Contributor Author

Thanks - all three fixed in fc2841e8c:

  • $PWD prefix on an already-absolute path: thirdparty-macos-amd64_bdwgc_validate.sh is called once with a relative TCC_FOLDER (thirdparty/tcc) and once with an already-absolute one (/tmp/tccbin-fresh-checkout). Prefixing $PWD unconditionally collapsed the fresh-checkout call into $PWD/tmp/tccbin-fresh-checkout/... (double-slash normalizes away), a path that doesn't exist. Now resolved via a case "$TCC_FOLDER" in /*) ... ;; esac that only prefixes $PWD for the relative case.
  • make local=1 in the isolated V clone: right catch - that workspace is a genuinely fresh git clone, and both vc/ and thirdparty/tcc are gitignored, so neither exists yet. local=1 makes latest_vc/latest_tcc skip their own git-bootstrap entirely (see GNUmakefile's ifndef local branches), which only works when vc/v.c already exists from an earlier step in the same job - true in bootstrapping_ci.yml (plain make runs first there), not true here. Switched to plain make -j4; latest_tcc cloning its own stock thirdparty/tcc under it is harmless since the next step wholesale-replaces it with the staged, fresh-checkout-validated bundle.
  • Stale versioned dylib deletions not staged on a SONAME bump: the commit step now does git add -A -- 'lib/libgc*.dylib' 'lib/libgc*.a' ... instead of a hardcoded file list, so an old versioned file already removed from disk gets its deletion staged too, instead of reappearing from history on every future checkout. One thing I caught while testing this: git add -A -- <pathspecs> fails its entire invocation (stages nothing at all, not just the unmatched one) if even one pathspec glob matches zero files on disk or in the index - confirmed empirically. I'd originally also included libgc.la/libgc.lai globs to mirror the build script's own defensive cleanup line, but those files are never actually staged in the first place (rsync only pulls from bdwgc/.libs/, and libtool's .la/.lai wrapper files live one level up), so including them would've turned every normal run into a hard failure, not just a SONAME-bump one. Dropped them from the glob list.

Also added the publish lock you flagged: the "Push tccbin branch" step now additionally requires a repo variable (MACOS_AMD64_LIBGC_PUBLISH_UNLOCKED) to be true for matrix.id == 'macos-amd64', on top of the existing PUBLISH check. It defaults unset/false, so neither the monthly cron nor a manual publish=true dispatch can push this platform until a maintainer flips it after confirming the vlang/tccbin gate conversion (deliverable B) is live and green - matches the "Sequence" section in the PR description.

Ready for another look, and for the macos-15-intel validation run whenever you're able to dispatch it against a real vlang/v ref.

@GGRei

GGRei commented Jul 30, 2026

Copy link
Copy Markdown
Contributor

Good job!

@codex review

@chatgpt-codex-connector

Copy link
Copy Markdown

Codex Review: Didn't find any major issues. What shall we delve into next?

Reviewed commit: fc2841e8cb

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

@GGRei

GGRei commented Jul 30, 2026

Copy link
Copy Markdown
Contributor

Please run the native validation from a temporary branch on your fork. Adjust only the github.repository guard so the workflow can run there, keep publish=false and force_rebuild=true, and share the Actions run link with us. The required result is the macos-15-intel job. Thanks!

@quaesitor-scientiam

Copy link
Copy Markdown
Contributor Author

Dispatched on the fork: https://github.com/quaesitor-scientiam/v/actions/runs/30544664222 (branch ci-validate-macos-amd64-fork, only the github.repository guard changed, publish=false force_rebuild=true). Currently running - will follow up here with the macos-15-intel job's result once it completes.

@quaesitor-scientiam

Copy link
Copy Markdown
Contributor Author

Run completed, all green: https://github.com/quaesitor-scientiam/v/actions/runs/30544664222

update-tccbin-hosted (macos-amd64, macos-15-intel, ...) succeeded. Key confirmations from that job's log:

  • Rebuild-fingerprint check correctly forced a rebuild (force_rebuild=true short-circuited the skip logic), so this genuinely exercised the full pipeline, not a no-op.
  • libgc.dylib -> libgc.1.dylib staged as a real symlink; install name of libgc.1.dylib: @rpath/libgc.dylib confirmed twice (once against the build directory, once independently against the fresh checkout).
  • Both validation passes: All gates passed: dylib lane blocking-green, no-GC fallback green, static lane is either green or the known signature-checked XFAIL.
  • fresh checkout is clean at dd69769189cf16d5ea0f2738cdbea1c39d38f5d4 - no untracked/uncommitted drift.
  • Isolated-workspace smoke test: confirmed: used the staged tcc.exe at /tmp/v-bootstrap-workspace/thirdparty/tcc/tcc.exe and confirmed: libgc.a does not appear in the link line - real V, real tcc.exe, real dylib, no static fallback.
  • Push tccbin branch step: skipped, as expected with publish=false.
  • update-tccbin-bsd job: skipped - confirms the repository-guard relaxation was scoped to exactly the one job, nothing else was touched.

linux-amd64 and macos-arm64 also came back green in the same run (unrelated to this change, but nice to have confirmation the guard relaxation didn't disturb them).

I'll delete the ci-validate-macos-amd64-fork branch now that the run is captured here - it was never intended to be merged.

@GGRei

GGRei commented Jul 30, 2026

Copy link
Copy Markdown
Contributor

@medvednikov Alex, it's ready to be merged. Thanks :)

@JalonSolov
JalonSolov merged commit b4ac5b4 into vlang:master Jul 30, 2026
60 of 81 checks passed
GGRei pushed a commit to vlang/tccbin that referenced this pull request Jul 30, 2026
…ass (#83)

* ci: flip checked-in tcc.exe/libgc gate from must-stay-broken to must-pass

Deliverable (B) of the macos-amd64 libgc-dylib-pairing plan, following
vlang/v#27982 (deliverable A, merged): vlang/v's update_tccbin.yml now
rebuilds libgc.dylib in lockstep with every tcc.exe rebuild for this
platform, mirroring macos-arm64's already-proven dylib+rpath pattern,
instead of silently preserving a stale libgc.a across every rebuild.

The "verify the checked-in tcc.exe (as distributed, before rebuild)" step
previously hard-failed if the checked-in pair unexpectedly passed the
conformance suite, because until now it genuinely could not (ancient
v0.9.27 tcc.exe, broken libc.dylib symlink, static-only libgc.a). That
assumption no longer holds and would immediately misfire the moment a
correct pair is published. This strengthens the gate rather than relaxing
it: split into five steps mirroring the already-proven shape used below
for the freshly-rebuilt copy - tcc.exe executable-bit check, independent
libgc.dylib symlink/install-name re-verification, libgc.a archive
validation (now fat-or-thin aware via `lipo -archs`, since a pair
published through vlang/v's amd64-native rebuild is expected to be a thin
x86_64-only archive, unlike the FAT universal binary this replaces), a
blocking dylib lane, a blocking no-GC fallback lane, and a signature-
checked XFAIL-only static lane - each one now required to actually PASS
instead of required to fail a specific known way.

Note: this PR's own CI run is expected to fail against the current tip of
thirdparty-macos-amd64, since the checked-in pair hasn't been replaced by
vlang/v#27982's producer-side fix yet (that publish is intentionally
gated behind vlang/v's MACOS_AMD64_LIBGC_PUBLISH_UNLOCKED repo variable
until this PR lands). Merging this PR is expected to keep CI red until
the real publish happens next.

Co-Authored-By: WOZCODE <contact@withwoz.com>

* ci: make checked-in libgc.a archive check independent of the dylib check

The libgc.dylib check's expected failure (pre-publish) was skipping this
step entirely by default, so the new lipo-archs-based fat-or-thin
detection never actually ran against the real checked-in archive in this
PR's own validation run. It validates a wholly separate file (libgc.a,
not libgc.dylib) - a dylib-check failure shouldn't hide whether the
archive is also fine or also broken, mirroring the !cancelled() pattern
already used below for the no-GC/static-XFAIL lanes.

Co-Authored-By: WOZCODE <contact@withwoz.com>

---------

Co-authored-by: Richard Wheeler <18647491+PythonWillRule@users.noreply.github.com>
Co-authored-by: WOZCODE <contact@withwoz.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants