Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

macOS arm64 CI runner is running an i386 shell by default & ARCHFLAGS isn't working #20790

Open
huonw opened this issue Apr 14, 2024 · 0 comments
Labels
bug category:internal CI, fixes for not-yet-released features, etc.

Comments

@huonw
Copy link
Contributor

huonw commented Apr 14, 2024

Describe the bug

In #20765, we discovered that the old psutil==5.9.0 dependency was being built to a wheel incorrectly on Apple Silicon: the native code was ending up as x86-64, rather than arm64. There's two layers to this bug:

  1. the CI steps attempt to set ARCHFLAGS='-arch arm64' but these seemed to stop working in release_2.17.0.dev5...release_2.17.0a0 (see ImportError: ... incompatible architecture on arm64 macOS for Pants 2.19.2, 2.20.0rc3, 2.21.0.dev5  #20765 (comment) for analysis)
  2. the runner is defaulting to i386/x86-64 shell (etc.) by default, rather than arm64 (a freshly installed self-hosted runner on my M1 laptop runs in arm64 shells by default: see ImportError: ... incompatible architecture on arm64 macOS for Pants 2.19.2, 2.20.0rc3, 2.21.0.dev5  #20765 (comment) for analysis)

Fixing 2 is arguably the better fix, as then the runner will be doing the right thing by default. If we fix this via 1, we need to make sure all uses of this runner apply the flags when they build things (this includes in scie-pants).

This can be confirmed by running commands like:

  • arch => outputs i386, should output arm64
  • echo 'int main() { return 0; }' > test.c && clang test.c -o test && file test (compile some C code) => outputs test: Mach-O 64-bit executable x86_64, should output ... arm64

Pants version
2.17.0a0

OS

arm64 macOS in Ci

Additional info

N/A

@huonw huonw added bug category:internal CI, fixes for not-yet-released features, etc. labels Apr 14, 2024
huonw added a commit to pantsbuild/scie-pants that referenced this issue Apr 16, 2024
This updates the hard-coded version that's used for the `PANTS_SOURCE`
test from 2.14.1 to 2.21.0.dev6, which is the current most recent `main`
release.

This is acutely motivated by the old version using dependencies without
appropriate wheels for macOS arm64 in particular, which causes CI to hit
pantsbuild/pants#20790, thus making
development here slower and dependent on fixing those infrastructure
issues.

It potentially also brings the Rust version used by that dependency to
be closer to that used by the scie-pants repo, thus making Rust upgrades
potentially easier (less juggling of rust versions required
#358).

Finally, the risk of this change is that we might regress `PANTS_SOURCE`
behaviour for old Pants versions, e.g. someone is using a local check
out of a 2.14.x-era release, and some future change to `scie-pants`
breaks this. I think we can treat this sort of out-of-date
`PANTS_SOURCE`s as semi-supported: if someone is affected, they can file
an issue and we (potentially) fix, and they have the easy work-around of
pinning to a lower/previously-working version of `scie-pants`.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug category:internal CI, fixes for not-yet-released features, etc.
Projects
None yet
Development

No branches or pull requests

1 participant