Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add ARM64 support to repo2docker #1180

Closed
wants to merge 5 commits into from

Conversation

yuvipanda
Copy link
Collaborator

@yuvipanda yuvipanda commented Aug 26, 2022

I got a M1 Mac, what else does one do to procrastinate?

Fixes #1120

TODO:

  • Have frozen versions of the conda-lock files be architecture specific
  • Be platform specific in getting R
  • Setup arm tests for everything

- name: Set up QEMU
uses: docker/setup-qemu-action@v2
with:
platforms: 'arm64,arm'
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Running all the tests under qemu is going to take a long time.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For reference: in z2jh we do the following, where --platform is passed during docker build is run by chartpress.

https://github.com/jupyterhub/zero-to-jupyterhub-k8s/blob/d16d44aa44d43a3ac34a8fdec4a94328e47e2546/.github/workflows/test-docker-build.yaml#L47-L57


@yuvipanda I think you should only specify arm64 and not arm64,arm - at least unless you wish to go for arm32 support as well. Typically, the support for arm64 is quite more common, and in z2jh for example we provide arm64 + amd64 but not arm.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@consideRatio yay ty. Just switched it to arm64.

@minrk I'm trying to get it to at least run on arm64 now, and can't get it to run at all :( I think it's still just running on amd64 despite me passing platform, not sure why.

@yuvipanda
Copy link
Collaborator Author

I'm guessing this should be commandline param passed in.

@yuvipanda
Copy link
Collaborator Author

I see:

Warning: repo2docker:app.py:814  ---> [Warning] The requested image's platform (linux/arm64/v8) does not match the detected host platform (linux/amd64) and no specific platform was requested

everywhere, and am not sure if qemu is actually even being used? We don't use buildx, as we talk to it via the docker API (not buildx). Is it actually not possible to do this via the API? All the docs I can find seem to suggest using buildx

@manics
Copy link
Member

manics commented Aug 26, 2022

Looks like docker-py doesn't support buildx docker/docker-py#2230

@minrk
Copy link
Member

minrk commented Aug 26, 2022

everywhere, and am not sure if qemu is actually even being used?

If the build steps don't fail immediately, qemu's involved. You'll get some kind of exec error if there's no qemu. It is weird that it thinks no platform is requested, though.

@yuvipanda
Copy link
Collaborator Author

@minrk they currently fail with:

2022-08-26T10:38:55.0109683Z INFO     repo2docker:app.py:814 �[91m/tmp/install-base-env.bash: /srv/conda/envs/kernel/bin/ipython: /srv/conda/envs/kernel/bin/python: bad interpreter: No such file or directory

which I'd assume means they aren't using QEMU, and that's the arm binary trying to get started in an amd64 context.

It works on my local machine tho

@minrk
Copy link
Member

minrk commented Sep 2, 2022

Given how long our tests take, it may make sense to run arm tests on circle. I just did this on pyzmq, and it was ~10x faster than qemu on GHA.

Presumably we only need a pretty basic couple of tests, though.

@yuvipanda
Copy link
Collaborator Author

I dunno when I'll be able to work on this again though, would love for someone else to take it over <3

@consideRatio consideRatio mentioned this pull request Oct 31, 2022
3 tasks
@consideRatio
Copy link
Member

This is a PR related to #1055

@manics manics mentioned this pull request Jan 2, 2023
2 tasks
@yuvipanda
Copy link
Collaborator Author

A tarot card reader on the streets of los angeles told me that completion of this PR is not something she sees in my future given that Saturn is in retrograde, and I will hence close this in favor of the amazing #1228

@yuvipanda yuvipanda closed this Jan 2, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

arm64/aarch64 isn't supported
4 participants