Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

4.0.0-5 / xattrs errors on Alpine (GCC) #74

Closed
pvizeli opened this issue Jul 6, 2019 · 15 comments · Fixed by #84
Closed

4.0.0-5 / xattrs errors on Alpine (GCC) #74

pvizeli opened this issue Jul 6, 2019 · 15 comments · Fixed by #84
Labels
bug enhancement

Comments

@pvizeli
Copy link

pvizeli commented Jul 6, 2019

Is this a bug report, feature (enhancement) request or question? (leave only one on its own line)

/kind bug

Description:

After I included 4.0.0-5 into my base images, some future builds like for python fails to install gcc with an xattrs set attributes errors. After I revert to previus version, it works.

Steps to reproduce the issue:

I saw the issue only on Azure Pipelines, but I had no time to investigate the issue deeper. I leaf only a issue that maybe other known that it could have an issue with this version.
@junaruga junaruga added the bug label Jul 6, 2019
@pvizeli
Copy link
Author

pvizeli commented Jul 7, 2019 •
edited
Loading

The log and output for 4.0.0-5:
https://dev.azure.com/home-assistant/Hass.io/_build/results?buildId=3377&view=results

ERROR: Failed to set xattrs on usr/libexec/gcc/armv6-alpine-linux-musleabihf/8.3.0/.apk.e5bdab84376a2fa35aedc80d2566e19a87cfa87b24e0ac6b: Function not implemented

@junaruga
Copy link
Member

Hi @pvizeli

After I revert to previus version, it works.

Do you know which version is the previous version of qemu? qemu 4.0.0-2?
You faced the issue with qemu 4.0.0-5.

Could you solve the issue by building images from qemu 4.0.0-2 by yourself?

I have a possible solution to improve this kind of issue.
That is to create version tagged images.
In my understanding, currently this repository's pipeline creates below images without qemu version number.

  • multiarch/qemu-user-static: (new)
  • multiarch/qemu-user-static:$from_arch-$to_arch
  • multiarch/qemu-user-static:$to_arch (same with multiarch/qemu-user-static:$from_arch-$to_arch)
  • multiarch/qemu-user-static:register

But for example, we change to add images with version tag, you have some choices when you face specific qemu version's image.

For example, when a used qemu's version is X.Y.Z-R

The pipeline program can set $version = X.Y.Z-R or X.Y in the process, and create below images.
I would prefer the $version is X.Y than X.Y.Z-R so that we can save the number of images.

  • multiarch/qemu-user-static
  • multiarch/qemu-user-static:$version
  • multiarch/qemu-user-static:$from_arch-$to_arch
  • multiarch/qemu-user-static:$from_arch-$to_arch-$version
  • multiarch/qemu-user-static:$to_arch
  • multiarch/qemu-user-static:$to_arch-$version
  • multiarch/qemu-user-static:register
  • multiarch/qemu-user-static:register-$version

You can choose one from version tagged images.

  • multiarch/qemu-user-static:arm-4.0 (the latest version is alias of multiarch/qemu-user-static:arm)
  • multiarch/qemu-user-static:arm-3.1.

What do you think?

meeDamian added a commit to meeDamian/docker-bitcoind that referenced this issue Aug 14, 2019
@meeDamian
specify version(?) multiarch/qemu-user-static#74
44fd6f7
@meeDamian
Copy link

I'm experiencing the same on v4.0.0-5

@meeDamian
Copy link

Versioning, as you've described above, would definitely be appreciated. I'd recommend triggering versioned builds on git tags.

@meeDamian meeDamian mentioned this issue Aug 15, 2019
Closed
@junaruga junaruga mentioned this issue Aug 16, 2019
Merged
@junaruga junaruga reopened this Aug 16, 2019
@junaruga
Copy link
Member

junaruga commented Aug 16, 2019 •
edited
Loading

Hi @pvizeli @meeDamian

Now the multiarch/qemu-user-static images started to provide version tag's images.
v4.0.0-5, v4.0.0-4 and v3.1.1-2 (Latest version of qemu version 3).

Below kind of commands work.
Could you try v4.0.0-4 or v3.1.1-2 images, and report us if it works or not on your environment?
If the versions are not enough, I will run the pipeline for other old qemu versions such as qemu v4.0.0-2.

It should also have a backward compatibility.

If you want to see the behavior of the containers without local cache, you can run below command to remove all the images at first.

$ docker system prune -a -f

$ docker run --rm --privileged multiarch/qemu-user-static --reset -p yes
$ docker run --rm --privileged multiarch/qemu-user-static:v4.0.0-5 --reset -p yes
$ docker run --rm --privileged multiarch/qemu-user-static:v4.0.0-4 --reset -p yes
$ docker run --rm --privileged multiarch/qemu-user-static:v3.1.1-2 --reset -p yes
$ cat /proc/sys/fs/binfmt_misc/qemu-aarch64

$ docker run --rm --privileged multiarch/qemu-user-static:register --reset
$ cat /proc/sys/fs/binfmt_misc/qemu-aarch64

$ docker run --rm -t multiarch/qemu-user-static:aarch64 /usr/bin/qemu-aarch64-static -version
$ docker run --rm -t multiarch/qemu-user-static:aarch64-v4.0.0-5 /usr/bin/qemu-aarch64-static -version
$ docker run --rm -t multiarch/qemu-user-static:aarch64-v4.0.0-4 /usr/bin/qemu-aarch64-static -version
$ docker run --rm -t multiarch/qemu-user-static:aarch64-v3.1.1-2 /usr/bin/qemu-aarch64-static -version
$ docker run --rm -t multiarch/qemu-user-static:x86_64-aarch64 /usr/bin/qemu-aarch64-static -version
$ docker run --rm -t multiarch/qemu-user-static:x86_64-aarch64-v4.0.0-5 /usr/bin/qemu-aarch64-static -version
$ docker run --rm -t multiarch/qemu-user-static:x86_64-aarch64-v4.0.0-4 /usr/bin/qemu-aarch64-static -version
$ docker run --rm -t multiarch/qemu-user-static:x86_64-aarch64-v3.1.1-2 /usr/bin/qemu-aarch64-static -version

$ docker run --rm -t multiarch/qemu-user-static:s390x-v4.0.0-4 /usr/bin/qemu-s390x-static -version

This new feature will be documented later.

@hoshsadiq
Copy link

Hello, I've come across the same issue recently. Annoyingly I did not come across this issue, however, after a lot of searching I came back with nothing so I reached out to the Alpine devs on IRC. This is the relevant issue https://gitlab.alpinelinux.org/alpine/apk-tools/issues/10657

@pvizeli could you confirm which version did work?

@hoshsadiq
Copy link

Tried it with all above mentioned versions, and it seems only 4.0.0-5 causes the issue.

$ docker run --rm --privileged multiarch/qemu-user-static:v4.0.0-4 --reset -p yes
Setting /usr/bin/qemu-alpha-static as binfmt interpreter for alpha
Setting /usr/bin/qemu-arm-static as binfmt interpreter for arm
Setting /usr/bin/qemu-armeb-static as binfmt interpreter for armeb
Setting /usr/bin/qemu-sparc32plus-static as binfmt interpreter for sparc32plus
Setting /usr/bin/qemu-ppc-static as binfmt interpreter for ppc
Setting /usr/bin/qemu-ppc64-static as binfmt interpreter for ppc64
Setting /usr/bin/qemu-ppc64le-static as binfmt interpreter for ppc64le
Setting /usr/bin/qemu-m68k-static as binfmt interpreter for m68k
Setting /usr/bin/qemu-mips-static as binfmt interpreter for mips
Setting /usr/bin/qemu-mipsel-static as binfmt interpreter for mipsel
Setting /usr/bin/qemu-mipsn32-static as binfmt interpreter for mipsn32
Setting /usr/bin/qemu-mipsn32el-static as binfmt interpreter for mipsn32el
Setting /usr/bin/qemu-mips64-static as binfmt interpreter for mips64
Setting /usr/bin/qemu-mips64el-static as binfmt interpreter for mips64el
Setting /usr/bin/qemu-sh4-static as binfmt interpreter for sh4
Setting /usr/bin/qemu-sh4eb-static as binfmt interpreter for sh4eb
Setting /usr/bin/qemu-s390x-static as binfmt interpreter for s390x
Setting /usr/bin/qemu-aarch64-static as binfmt interpreter for aarch64
Setting /usr/bin/qemu-aarch64_be-static as binfmt interpreter for aarch64_be
Setting /usr/bin/qemu-hppa-static as binfmt interpreter for hppa
Setting /usr/bin/qemu-riscv32-static as binfmt interpreter for riscv32
Setting /usr/bin/qemu-riscv64-static as binfmt interpreter for riscv64
Setting /usr/bin/qemu-xtensa-static as binfmt interpreter for xtensa
Setting /usr/bin/qemu-xtensaeb-static as binfmt interpreter for xtensaeb
Setting /usr/bin/qemu-microblaze-static as binfmt interpreter for microblaze
Setting /usr/bin/qemu-microblazeel-static as binfmt interpreter for microblazeel
Setting /usr/bin/qemu-or1k-static as binfmt interpreter for or1k

$ docker run --rm -it arm64v8/alpine:3.10 apk add gcc
fetch http://dl-cdn.alpinelinux.org/alpine/v3.10/main/aarch64/APKINDEX.tar.gz
fetch http://dl-cdn.alpinelinux.org/alpine/v3.10/community/aarch64/APKINDEX.tar.gz
(1/10) Installing binutils (2.32-r0)
(2/10) Installing gmp (6.1.2-r1)
(3/10) Installing isl (0.18-r0)
(4/10) Installing libgomp (8.3.0-r0)
(5/10) Installing libatomic (8.3.0-r0)
(6/10) Installing libgcc (8.3.0-r0)
(7/10) Installing mpfr3 (3.1.5-r1)
(8/10) Installing mpc1 (1.1.0-r0)
(9/10) Installing libstdc++ (8.3.0-r0)
(10/10) Installing gcc (8.3.0-r0)
Executing busybox-1.30.1-r2.trigger
OK: 77 MiB in 24 packages

@junaruga
Copy link
Member

junaruga commented Aug 26, 2019 •
edited
Loading

@hoshsadiq thank you for checking the issue.

Did actually below your report have error message on 4.0.0-5? Which kind of error did you face on 4.0.0-5?

$ docker run --rm -it arm64v8/alpine:3.10 apk add gcc
fetch http://dl-cdn.alpinelinux.org/alpine/v3.10/main/aarch64/APKINDEX.tar.gz
fetch http://dl-cdn.alpinelinux.org/alpine/v3.10/community/aarch64/APKINDEX.tar.gz
(1/10) Installing binutils (2.32-r0)
(2/10) Installing gmp (6.1.2-r1)
(3/10) Installing isl (0.18-r0)
(4/10) Installing libgomp (8.3.0-r0)
(5/10) Installing libatomic (8.3.0-r0)
(6/10) Installing libgcc (8.3.0-r0)
(7/10) Installing mpfr3 (3.1.5-r1)
(8/10) Installing mpc1 (1.1.0-r0)
(9/10) Installing libstdc++ (8.3.0-r0)
(10/10) Installing gcc (8.3.0-r0)
Executing busybox-1.30.1-r2.trigger
OK: 77 MiB in 24 packages

Actually we can reproduce the issue on qemu-user-static RPM on Fedora, we can report the issue to below page issues.
https://src.fedoraproject.org/rpms/qemu
But it seems the latest version qemu is already 4.1.0.

@hoshsadiq
Copy link

Pretty much same as @pvizeli said:

$ docker run --rm --privileged multiarch/qemu-user-static:v4.0.0-5 --reset -p yes
Setting /usr/bin/qemu-alpha-static as binfmt interpreter for alpha
Setting /usr/bin/qemu-arm-static as binfmt interpreter for arm
Setting /usr/bin/qemu-armeb-static as binfmt interpreter for armeb
Setting /usr/bin/qemu-sparc32plus-static as binfmt interpreter for sparc32plus
Setting /usr/bin/qemu-ppc-static as binfmt interpreter for ppc
Setting /usr/bin/qemu-ppc64-static as binfmt interpreter for ppc64
Setting /usr/bin/qemu-ppc64le-static as binfmt interpreter for ppc64le
Setting /usr/bin/qemu-m68k-static as binfmt interpreter for m68k
Setting /usr/bin/qemu-mips-static as binfmt interpreter for mips
Setting /usr/bin/qemu-mipsel-static as binfmt interpreter for mipsel
Setting /usr/bin/qemu-mipsn32-static as binfmt interpreter for mipsn32
Setting /usr/bin/qemu-mipsn32el-static as binfmt interpreter for mipsn32el
Setting /usr/bin/qemu-mips64-static as binfmt interpreter for mips64
Setting /usr/bin/qemu-mips64el-static as binfmt interpreter for mips64el
Setting /usr/bin/qemu-sh4-static as binfmt interpreter for sh4
Setting /usr/bin/qemu-sh4eb-static as binfmt interpreter for sh4eb
Setting /usr/bin/qemu-s390x-static as binfmt interpreter for s390x
Setting /usr/bin/qemu-aarch64-static as binfmt interpreter for aarch64
Setting /usr/bin/qemu-aarch64_be-static as binfmt interpreter for aarch64_be
Setting /usr/bin/qemu-hppa-static as binfmt interpreter for hppa
Setting /usr/bin/qemu-riscv32-static as binfmt interpreter for riscv32
Setting /usr/bin/qemu-riscv64-static as binfmt interpreter for riscv64
Setting /usr/bin/qemu-xtensa-static as binfmt interpreter for xtensa
Setting /usr/bin/qemu-xtensaeb-static as binfmt interpreter for xtensaeb
Setting /usr/bin/qemu-microblaze-static as binfmt interpreter for microblaze
Setting /usr/bin/qemu-microblazeel-static as binfmt interpreter for microblazeel
Setting /usr/bin/qemu-or1k-static as binfmt interpreter for or1k

$ docker run --rm -it arm64v8/alpine:3.10 apk add gcc
fetch http://dl-cdn.alpinelinux.org/alpine/v3.10/main/aarch64/APKINDEX.tar.gz
fetch http://dl-cdn.alpinelinux.org/alpine/v3.10/community/aarch64/APKINDEX.tar.gz
(1/10) Installing binutils (2.32-r0)
(2/10) Installing gmp (6.1.2-r1)
(3/10) Installing isl (0.18-r0)
(4/10) Installing libgomp (8.3.0-r0)
(5/10) Installing libatomic (8.3.0-r0)
(6/10) Installing libgcc (8.3.0-r0)
(7/10) Installing mpfr3 (3.1.5-r1)
(8/10) Installing mpc1 (1.1.0-r0)
(9/10) Installing libstdc++ (8.3.0-r0)
(10/10) Installing gcc (8.3.0-r0)
ERROR: Failed to set xattrs on usr/libexec/gcc/aarch64-alpine-linux-musl/8.3.0/.apk.e7f97aa03b8e515a97a9eb65f467457a2c02cba9c11c4651: Function not implemented
Executing busybox-1.30.1-r2.trigger
1 error; 77 MiB in 24 packages

If you could kick off a build for 4.1.0 I'm happy to test it.

@junaruga junaruga mentioned this issue Aug 26, 2019
Closed
@junaruga
Copy link
Member

junaruga commented Aug 26, 2019 •
edited
Loading

@hoshsadiq sure, thanks for the report of the error message.

I think maybe someone will upgrade qemu to latest version within a few days. In the meantime, of course below can be your temporary workflow.

$ docker run --rm --privileged multiarch/qemu-user-static:v4.0.0-4 --reset -p yes

If nobody upgrades it, I might do it this week after fixing #91 .

If you could kick off a build for 4.1.0 I'm happy to test it.

Thanks. I will ask you to test it on 4.1.0.

@pvizeli
Copy link
Author

pvizeli commented Sep 2, 2019

solve with 4.1.0

@junaruga
Copy link
Member

junaruga commented Sep 2, 2019

@pvizeli Oh did you already test the released 4.1.0? Thanks for checking.
Now as I mentioned above below version tag images are available too.

$ docker system prune -a -f

$ docker run --rm --privileged multiarch/qemu-user-static --reset -p yes
$ docker run --rm --privileged multiarch/qemu-user-static:4.1.0-1 --reset -p yes
$ cat /proc/sys/fs/binfmt_misc/qemu-aarch64

$ docker run --rm -t multiarch/qemu-user-static:aarch64 /usr/bin/qemu-aarch64-static -version
$ docker run --rm -t multiarch/qemu-user-static:aarch64-4.1.0-1 /usr/bin/qemu-aarch64-static -version
$ docker run --rm -t multiarch/qemu-user-static:x86_64-aarch64-4.1.0-1 /usr/bin/qemu-aarch64-static -version

$ docker run --rm -t multiarch/qemu-user-static:register

@junaruga
Copy link
Member

junaruga commented Sep 2, 2019

I would close this ticket.

@junaruga junaruga closed this as completed Sep 2, 2019
@hoshsadiq
Copy link

Thanks all.

@junaruga
Copy link
Member

junaruga commented Sep 5, 2019

This new feature will be documented later.

It is documented now by #96 .

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug enhancement
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Implement tag versioning. junaruga/qemu-user-static
4 participants
@junaruga @meeDamian @hoshsadiq @pvizeli