Skip to content

Add arm64 image #1

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 2 commits into from
Oct 20, 2025
Merged

Add arm64 image #1

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
6489fa6
Add arm64 image
SiarheiFedartsou Oct 19, 2025
5edadaf
Add arm64 image
SiarheiFedartsou Oct 19, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
56 changes: 45 additions & 11 deletions .github/workflows/build.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -13,19 +13,29 @@ on:
- "valhalla_python"
paths-ignore:
- '*.md'
pull_request:
Copy link
Member Author

@SiarheiFedartsou SiarheiFedartsou Oct 19, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Probably a good idea to have CI running for PRs...

Copy link
Member

@nilsnolde nilsnolde Oct 20, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah totally! Didn't ever expect PRs tbh😄 and I usually commit directly to master in this repo

Copy link
Member

@nilsnolde nilsnolde Oct 20, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you also ignore md files?

Copy link
Member Author

@SiarheiFedartsou SiarheiFedartsou Oct 20, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do you want me to create yet another PR? :) Now I understand that it was merged by mistake 😄

branches:
- "valhalla_python"

concurrency:
group: ${{ github.workflow }}-${{ github.event.pull_request.number || github.sha }}
cancel-in-progress: true

jobs:
build_manylinux:
name: manylinux_2_28_x86_64
runs-on: ubuntu-22.04
name: manylinux_2_28_${{ matrix.platform }}
runs-on: ${{ matrix.runner }}
strategy:
matrix:
include:
- platform: x86_64
runner: ubuntu-22.04
- platform: aarch64
runner: ubuntu-22.04-arm

env:
POLICY: manylinux_2_28
PLATFORM: x86_64
PLATFORM: ${{ matrix.platform }}
COMMIT_SHA: ${{ github.sha }}

steps:
Expand All @@ -39,13 +49,14 @@ jobs:
if: github.event_name != 'workflow_dispatch' || fromJSON(github.event.inputs.useCache)
uses: actions/cache/restore@v4
with:
path: .buildx-cache-manylinux_2_28_x86_64/
key: buildx-cache-manylinux_2_28_x86_64-${{ hashFiles('docker/**') }}
restore-keys: buildx-cache-manylinux_2_28_x86_64-
path: .buildx-cache-manylinux_2_28_${{ matrix.platform }}/
key: buildx-cache-manylinux_2_28_${{ matrix.platform }}-${{ hashFiles('docker/**') }}
restore-keys: buildx-cache-manylinux_2_28_${{ matrix.platform }}-

- name: Get branch name
run: |
echo "BRANCH=${GITHUB_REF##*/}" >> $GITHUB_ENV
echo "REPO_LOWERCASE=$(echo ${{ github.repository }} | tr '[:upper:]' '[:lower:]')" >> $GITHUB_ENV
Copy link
Member Author

@SiarheiFedartsou SiarheiFedartsou Oct 19, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Makes it properly working on forks: lowercase is needed because Docker doesn't allow any uppercase in repo name (SiarheiFedartsou/manylinux must be siarheifedartsou/manylinux)

Copy link
Member

@nilsnolde nilsnolde Oct 20, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah I thought that was only the "docker" docker registry.


- name: Build image
run: ./build.sh
Expand All @@ -54,12 +65,35 @@ jobs:
if: always()
uses: actions/cache/save@v4
with:
path: .buildx-cache-manylinux_2_28_x86_64/
key: buildx-cache-manylinux_2_28_x86_64-${{ hashFiles('docker/**') }}
path: .buildx-cache-manylinux_2_28_${{ matrix.platform }}/
key: buildx-cache-manylinux_2_28_${{ matrix.platform }}-${{ hashFiles('docker/**') }}

- name: Push image
if: github.event_name == 'push'
run: |
docker tag quay.io/pypa/manylinux_2_28_x86_64:${{ github.sha }} ghcr.io/valhalla/manylinux:2_28_${{ env.BRANCH }}
docker image rm quay.io/pypa/manylinux_2_28_x86_64:${{ github.sha }}
echo ${{ secrets.GITHUB_TOKEN }} | docker login ghcr.io -u ${{ github.actor }} --password-stdin
docker push ghcr.io/valhalla/manylinux:2_28_${{ env.BRANCH }}

docker tag quay.io/pypa/manylinux_2_28_${{ matrix.platform }}:${{ github.sha }} ghcr.io/${{ env.REPO_LOWERCASE }}:2_28_${{ matrix.platform }}_${{ env.BRANCH }}
docker push ghcr.io/${{ env.REPO_LOWERCASE }}:2_28_${{ matrix.platform }}_${{ env.BRANCH }}
docker image rm quay.io/pypa/manylinux_2_28_${{ matrix.platform }}:${{ github.sha }}

create_manifest:
Copy link
Member Author

@SiarheiFedartsou SiarheiFedartsou Oct 19, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Creates "magic" image which contains both architectures under the same name.

name: Create multi-arch manifest
runs-on: ubuntu-24.04
Copy link
Member

@nilsnolde nilsnolde Oct 20, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could do "-latest" where we can

needs: build_manylinux
if: github.event_name == 'push'

steps:
- name: Get branch name
run: |
echo "BRANCH=${GITHUB_REF##*/}" >> $GITHUB_ENV
echo "REPO_LOWERCASE=$(echo ${{ github.repository }} | tr '[:upper:]' '[:lower:]')" >> $GITHUB_ENV

- name: Create and push manifest
run: |
echo ${{ secrets.GITHUB_TOKEN }} | docker login ghcr.io -u ${{ github.actor }} --password-stdin

docker manifest create ghcr.io/${{ env.REPO_LOWERCASE }}:2_28_${{ env.BRANCH }} \
--amend ghcr.io/${{ env.REPO_LOWERCASE }}:2_28_x86_64_${{ env.BRANCH }} \
--amend ghcr.io/${{ env.REPO_LOWERCASE }}:2_28_aarch64_${{ env.BRANCH }}
docker manifest push ghcr.io/${{ env.REPO_LOWERCASE }}:2_28_${{ env.BRANCH }}
3 changes: 1 addition & 2 deletions docker/Dockerfile
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,5 +1,4 @@
# default to latest supported policy, x86_64
ARG BASEIMAGE=amd64/almalinux:9
ARG BASEIMAGE=almalinux:9
ARG POLICY=manylinux_2_34
ARG PLATFORM=x86_64
ARG DEVTOOLSET_ROOTPATH=/opt/rh/gcc-toolset-14/root
Expand Down
5 changes: 5 additions & 0 deletions docker/build_scripts/build-freexl.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,11 @@ check_var "${FREEXL_DOWNLOAD_URL}"
fetch_source "${FREEXL_ROOT}.tar.gz" "$FREEXL_DOWNLOAD_URL"
tar -xzf "${FREEXL_ROOT}.tar.gz"
pushd "${FREEXL_ROOT}"
# Update config.guess and config.sub for aarch64 support
if [ -f /usr/share/automake-*/config.guess ]; then
cp -f /usr/share/automake-*/config.guess .
Copy link
Member

@nilsnolde nilsnolde Oct 20, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This I don't really understand. We're copying some config stuff from automake?

Copy link
Member Author

@SiarheiFedartsou SiarheiFedartsou Oct 20, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Well, tbh it is something what I got as a tip from Claude Code when doing this(yes, these days it is how people actually do software engineering 😂 ). My understanding of what happens: these dependencies have these config.guess and config.sub in the root, but of quite old version which have no idea about aarch64, so we simply copy them from our own automake (which is modern enough to know about aarch64) to make it working. See this SO thread: https://stackoverflow.com/questions/4810996/how-to-resolve-configure-guessing-build-type-failure

cp -f /usr/share/automake-*/config.sub .
fi
./configure
make -j$(nproc)
DESTDIR=/manylinux-rootfs make install
Expand Down
5 changes: 5 additions & 0 deletions docker/build_scripts/build-spatialite.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,11 @@ check_var "${SPATIALITE_DOWNLOAD_URL}"
fetch_source "${SPATIALITE_ROOT}.tar.gz" "$SPATIALITE_DOWNLOAD_URL"
tar -xzf "${SPATIALITE_ROOT}.tar.gz"
pushd "${SPATIALITE_ROOT}"
# Update config.guess and config.sub for aarch64 support
if [ -f /usr/share/automake-*/config.guess ]; then
cp -f /usr/share/automake-*/config.guess .
cp -f /usr/share/automake-*/config.sub .
fi
./configure
make -j$(nproc)
DESTDIR=/manylinux-rootfs make install
Expand Down