Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

apk has errors when run on arm64 #35

Closed
kvrhdn opened this issue Oct 6, 2021 · 12 comments
Closed

apk has errors when run on arm64 #35

kvrhdn opened this issue Oct 6, 2021 · 12 comments

Comments

@kvrhdn
Copy link

kvrhdn commented Oct 6, 2021

Hi,

We noticed apk add commands fail with an error when run on arm64.

You can reproduce this locally if you have access to an ARM machine:

$ docker run -it --entrypoint=/bin/sh alpine/git
/git # apk --update --no-cache add bash
fetch https://dl-cdn.alpinelinux.org/alpine/v3.14/main/aarch64/APKINDEX.tar.gz
fetch https://dl-cdn.alpinelinux.org/alpine/v3.14/community/aarch64/APKINDEX.tar.gz
(1/2) Installing readline (8.1.0-r0)
(2/2) Installing bash (5.1.4-r0)
Executing bash-5.1.4-r0.post-install
Executing busybox-1.33.1-r3.trigger
1 error; 27 MiB in 34 packages
/git # echo $?
1

Our workaround is to run apk fix before any other apk command, this seems to repair busybox and ca-certificates:

$ docker run -it --entrypoint=/bin/sh alpine/git
/git # apk fix
fetch https://dl-cdn.alpinelinux.org/alpine/v3.14/main/aarch64/APKINDEX.tar.gz
fetch https://dl-cdn.alpinelinux.org/alpine/v3.14/community/aarch64/APKINDEX.tar.gz
(1/1) Reinstalling ca-certificates (20191127-r5)
Executing busybox-1.33.1-r3.trigger
Executing ca-certificates-20191127-r5.trigger
OK: 25 MiB in 32 packages
/git # apk --update --no-cache add bash
fetch https://dl-cdn.alpinelinux.org/alpine/v3.14/main/aarch64/APKINDEX.tar.gz
fetch https://dl-cdn.alpinelinux.org/alpine/v3.14/community/aarch64/APKINDEX.tar.gz
(1/2) Installing readline (8.1.0-r0)
(2/2) Installing bash (5.1.4-r0)
Executing bash-5.1.4-r0.post-install
Executing busybox-1.33.1-r3.trigger
OK: 27 MiB in 34 packages

Our builds use alpine/git:latest and they started failing from Oct 4, 2021, 13:15 UTC. This seems related to the latest merged PR #34.

@ozbillwang
Copy link
Contributor

ozbillwang commented Oct 6, 2021

thanks to report this issue. I don't have ARM environment to run the test. In fact, i only can test under "amd64"

The PR should not relate to the error, it is a fix to make sure pipeline build works. Maybe let's run some tests if you can help.

image

there are several tags recently we built with multi-arch, could you confirm any tags work or not work from above screenshot?

@ozbillwang
Copy link
Contributor

if can prove, we can add apx fix in Dockerfile

@ozbillwang
Copy link
Contributor

let me think about if I can add test cases in multi arch via Travis CI.

https://docs.travis-ci.com/user/multi-cpu-architectures/

@barthy1
Copy link
Contributor

barthy1 commented Oct 7, 2021

I can see the same behaviour for linux/s390x

docker run -it --entrypoint=/bin/sh alpine/git
/git # uname -a
Linux 014cd4d0f800 5.3.7-200.fc30.s390x #1 SMP Fri Oct 18 19:49:30 UTC 2019 s390x Linux
/git # apk --update --no-cache add bash
fetch https://dl-cdn.alpinelinux.org/alpine/v3.14/main/s390x/APKINDEX.tar.gz
fetch https://dl-cdn.alpinelinux.org/alpine/v3.14/community/s390x/APKINDEX.tar.gz
(1/2) Installing readline (8.1.0-r0)
(2/2) Installing bash (5.1.4-r0)
Executing bash-5.1.4-r0.post-install
Executing busybox-1.33.1-r3.trigger
2 errors; 28 MiB in 34 packages

The problem exists for latest, v2.32.0, and 1.0.31 and doesn't exist for v2.30.2 and 1.0.30

@kvrhdn
Copy link
Author

kvrhdn commented Oct 7, 2021

I have the same results on ARM (aarch64):

/git # uname -a
Linux abf6e669ba10 5.10.47-linuxkit #1 SMP PREEMPT Sat Jul 3 21:50:16 UTC 2021 aarch64 Linux

The problem exists for latest, v2.32.0 and 1.0.31
Does not exist for v2.30.2 and 1.0.30.

@ozbillwang
Copy link
Contributor

ozbillwang commented Oct 10, 2021

Finally I saw this issue by my way (with help by Travis CI multi-arch build support)

image

https://app.travis-ci.com/github/ozbillwang/multi-arch-docker-image-test-case/builds/239494444

@ozbillwang
Copy link
Contributor

ozbillwang commented Oct 10, 2021

I run the tests.

when built from my laptop, and push to hub.docker.com, it is fine, what ever I added apk fix in Dockerfile or not.

But if I built from Travis CI, it is always failed, I can't directly run apk add bash from the image, what ever I add apk fix in Dockerfile or not.

I did extra test to build the image via Circle CI, I got same issue. If you are interested on how I built multi-arch image in circle ci, please reference my another repo: https://github.com/ozbillwang/multi-arch-docker-build

I give up. If you meet this issue, just add a command apk fix before install any tools.

@kvrhdn
Copy link
Author

kvrhdn commented Oct 12, 2021

Running apk fix before any other apk commands is fine for us 👍

@kvrhdn
Copy link
Author

kvrhdn commented Jan 24, 2022

Hi, this issue is showing up again unfortunately. Running apk fix in the latest images fails now as well.

Logs from latest and v2.32.0 (the logs are identical):

$ docker run -it --entrypoint=/bin/sh alpine/git:v2.32.0
/git # apk fix
fetch https://dl-cdn.alpinelinux.org/alpine/v3.14/main/aarch64/APKINDEX.tar.gz
fetch https://dl-cdn.alpinelinux.org/alpine/v3.14/community/aarch64/APKINDEX.tar.gz
(1/1) [APK unavailable, skipped] Reinstalling ca-certificates (20191127-r5)
1 error; 25 MiB in 32 packages

v2.30.2 still works without apk fix:

$ docker run -it --entrypoint=/bin/sh alpine/git:v2.30.2
/git # apk --update --no-cache add bash
fetch https://dl-cdn.alpinelinux.org/alpine/v3.14/main/aarch64/APKINDEX.tar.gz
fetch https://dl-cdn.alpinelinux.org/alpine/v3.14/community/aarch64/APKINDEX.tar.gz
(1/2) Installing readline (8.1.0-r0)
(2/2) Installing bash (5.1.4-r0)
Executing bash-5.1.4-r0.post-install
Executing busybox-1.33.1-r3.trigger
OK: 27 MiB in 34 packages

I'm still running this from an ARM64 system (Apple M1)

$ docker run -it --entrypoint=/bin/sh alpine/git:v2.32.0
/git # uname -a
Linux 3dacf3f32152 5.10.76-linuxkit #1 SMP PREEMPT Mon Nov 8 11:22:26 UTC 2021 aarch64 Linux

For now, we will try to pin the image we use to v2.30.2.

@ozbillwang
Copy link
Contributor

Thanks for the information and detail.

There must be some reasons, but not sure anyone know the root cause.

@ozbillwang
Copy link
Contributor

@kvrhdn

Do we still have this issue with latest tag v2.36.2 ?

@ozbillwang
Copy link
Contributor

no reply, close it first

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Development

No branches or pull requests

3 participants