Build multi-platform Docker images (amd64 + arm64)#149
Conversation
c52f580 to
d0e9077
Compare
The docker buildx build command had no --platform flag, so images were built only for the host architecture. This caused examples to fail on amd64 when the published image was arm64-only. - Add --platform linux/amd64,linux/arm64 to buildx commands in tox.ini - Add QEMU and Buildx setup steps to CI workflow - Simplify tox docker-push to push directly to Dockerhub - Update RELEASE.md to use tox -e docker-push - Switch base Docker image to python:3.14.1-slim and bump numpy version for 3.14 to avoid building numpy within amd64 container.
d0e9077 to
34ae879
Compare
henrikingo
left a comment
There was a problem hiding this comment.
Did you run this somewhere? I assume yes, but if so, where should I expect to see docker images coming through on the other side.
bump numpy version for 3.14 to avoid building numpy within amd64 container.
Why?
| steps: | ||
| - uses: actions/checkout@v4 | ||
|
|
||
| - name: Set up QEMU |
There was a problem hiding this comment.
Probably better in its own commit, but why is the workflow called python-app :-D
These are connected stories :) First, I added buildx to tox and CI to build the image for both architectures. The CI succeeded, but took 4 times longer (36 minutes instead of 9). While waiting for that CI job to finish, I decided to run it locally to find the potential issues sooner. On my personal laptop, Why did the docker-build build numpy instead of just using the wheels? Numpy provides wheels for Python 3.14 only starting with version The combination of these issues lead me to upgrading numpy for Python 3.14 which in turn enabled fast cross-arch builds. As a result of these changes, GH CI runs in 4.5 minutes instead of 9/36 (baseline/multiarch) and does not crash with cryptic errors on Macbook Air 2022.
We do not have yet a release CI job automating publishing to Dockerhub and PyPI. For the release process, you'll need to 1) check out the accepted version's tag and 2) build locally, then push to Dockerhub. I updated the release instructions in this PR. |
|
Ok thanks for the explanation.
I'll see what I can do about automating this. I don't think that we should publish release that someone built on their laptop. But I'd rather proceed with the release and designate the docker parts experimental. IMO the python parts deserve a releast now and shouldn''t be blocked |
|
I'm +1 on this if nobody else has any comments. Leaving to Alex to merge though. |
smccarthy788
left a comment
There was a problem hiding this comment.
Looks good, thanks @Gerrrr for keeping building time in mind.
The docker buildx build command had no
--platformflag, so images were built only for the host architecture. This caused examples to fail on amd64 when the published image was arm64-only.--platform linux/amd64,linux/arm64to buildx commands intox.initox -e docker-pushpython:3.14.1-slimand bump numpy versionfor 3.14 to avoid building numpy within amd64 container.