Skip to content

Use ARM runners for CPython and Numpy #1609

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 7 commits into
base: master
Choose a base branch
from

Conversation

AutonomicPerfectionist
Copy link
Contributor

This PR enables ARM builds for CPython and Numpy by using the Github Actions ARM runners. It also makes minor changes to the cppbuild.sh scripts for both to fix compilation issues on ARM. For numpy, this entailed removing the workaround implemented in #1386.

Note

I made the minimal changes necessary to fix ARM compilation. This means that the cpython build still essentially does cross-compilation to target ARM even when running on ARM hosts. It may be desirable to add a check for whether the target and host platforms match to improve build times.

The cpp build was verified to succeed on a Raspberry Pi 4, I have not yet verified full end-to-end success via maven due to a missing dependency on the pi.

@saudet
Copy link
Member

saudet commented Mar 25, 2025

Sounds good, but why are the CI actions not running? 🤔

@saudet
Copy link
Member

saudet commented Mar 29, 2025

Ok, the formatting of the workflows was incorrect. Now it still fails, so anyway please fix it!

@AutonomicPerfectionist
Copy link
Contributor Author

The numpy workflow seems to have failed because the ubuntu image has the wrong sources.list for apt, ARM64 packages are hosted at http://ports.ubuntu.com/ubuntu-ports, not http://archive.ubuntu.com/ubuntu. The cpython workflow is just showing me an fallocate error, which shouldn't have anything to do with us and is probably a transient error.

@AutonomicPerfectionist
Copy link
Contributor Author

It seems the github action tries to workaround an Actions issue by selecting a mirror, but the mirror selected is the archive.ubuntu.com one and not the ports.ubuntu.com:

Mirror: http://archive.ubuntu.com/ubuntu/ - UP
The closest mirror is: http://archive.ubuntu.com/ubuntu/
Updating /etc/apt/sources.list
Updated '/etc/apt/sources.list' with http://archive.ubuntu.com/ubuntu/

@AutonomicPerfectionist
Copy link
Contributor Author

Appears to be a problem with apt-spy2, the lines in the action that appear to select a mirror use it:

$SUDO apt-spy2 check
$SUDO apt-spy2 fix --commit

@saudet
Copy link
Member

saudet commented Mar 30, 2025

Ok, so could you try to fix it?

@saudet
Copy link
Member

saudet commented Mar 30, 2025

If I remember correctly, the checks for "amd64" is to detect cross compilation, so we should be able to skip all that when running actually running on ARM...

@AutonomicPerfectionist
Copy link
Contributor Author

I've attempted a fix by simply disabling mirror selection for arm64 runners

@AutonomicPerfectionist
Copy link
Contributor Author

Hmm, both runs failed again, with an error on fallocate... I do see that the fallocate command is being used to create a 4GB swap file, but I can't imagine the arm64 runners would be so resource constrained that they can't allocate an additional 4GB on the filesystem...

$SUDO fallocate -l 4GB /swapfile

I've pushed a new commit that just disables the fallocate call, but it didn't seem to change anything. I'll revert that commit and continue looking for the cause of the error

@saudet
Copy link
Member

saudet commented Apr 8, 2025

To test the actions, you'll need to change temporarily the workflows to point to your branch

if [[ ! "$ARCH" == "amd64" ]]; then
# arm64 runners don't work with apt-spy2,
# the selected mirrors don't have arm64 binaries
if [[ ! "$ARCH" == "amd64" && ! "$ARCH" == "arm64" ]]; then
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We need this when cross compiling though. Please disable it only when we're running on ARM

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Will do

@AutonomicPerfectionist
Copy link
Contributor Author

To test the actions, you'll need to change temporarily the workflows to point to your branch

Yeah I figured that out, been playing whackamole with some more issues on a different branch of mine. Slowly getting there

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants