qemu-user-static executables from alpine are broken, use debian instead

[oflebbe]

Description
Run an emulated image, for instance ppc64le/debian:9 or ppc64le/fedora:26 . Try to use any program using dns, for instance curl or yum. Program does not find any host or fails.

As far as I could check, the qemu-user-*-static executables from alpine are broken.
Replacing them with debian ones fixes the issue.

Steps to reproduce the issue:
Fastest to see that qemu from alpine is broken can be done by fedora images:

docker run --rm -it ppc64le/fedora:26 /usr/libexec/system-python /usr/bin/yum update
Error: Failed to synchronize cache for repo 'updates'

Doublecheck that alpine-linux qemu- is the culprit:

docker -v $(pwd):/ws debian:9 bash -c "apt-get update && apt-get -y install qemu-user-static && cp /usr/bin/qemu-ppc64le-static /ws/qemu-debian
docker -v $(pwd):/ws alpine sh -c "apk update && apk add qemu-ppc64le &&/usr/bin/qemu-ppc64le /ws/qemu-alpine

docker run --rm -v $(pwd)/qemu-alpine:/qemu -it ppc64le/fedora:26 /qemu /usr/libexec/system-python /usr/bin/yum update
Error: Failed to synchronize cache for repo 'updates'

However debians qemu works:
docker run --rm -v $(pwd)/qemu-debian:/qemu -it ppc64le/fedora:26 /qemu /usr/libexec/system-python /usr/bin/yum update

Another one which fails

docker run --rm -it ppc64le/debian:9 bash -c "apt-get update && apt-get install -y curl && curl https://www.docker.com "
Return: curl: (6) Could not resolve host: www.docker.com

And btw: aarch64 is broken the same way.

Describe the results you received:

Describe the results you expected:

Additional information you deem important (e.g. issue happens only occasionally):

[oflebbe]

Proposing a patch like this one
PATCH.txt

[oflebbe]

I opened an upstream alpine ticket as well https://bugs.alpinelinux.org/issues/8131

[justincormack]

I would rather just fix upstream than switch; it was working previously... cc @ncopa

[ijc]

Was this fixed by the switch from Alpine 3.6 to 3.7 last year? Looks like we would have gone from 2.8.1-r1 to 2.10.1-r2.

[oflebbe]

The problem can be reproduced with alpine linux 3.7.0
See https://bugs.alpinelinux.org/issues/8131

Bug is still present in current docker edge for mac.

[ijc]

I see it with Edge too (commented on the Alpine rep).

Looking at the package tracker now its the same version as in 3.7, so hardly surprising.

/cc @ncopa

[oliviabarrick]

It seems like the patch would be easy enough to revert once this is fixed upstream. Is it really worth blocking this feature from working correctly just to force the first image in a multi-stage docker build to be alpine?

[deitch]

I agree with @justincormack that it should be fixed upstream. However, the ticket that @oflebbe opened has been untouched in half a year, since @ncopa reproduced it.

I don't think it I worth leaving this broken in the interim. If we knew a fix was going in in a week or two, sure, but I don't see that happening right now. In any case, that first stage in the multi-stage build is, indeed, just to get those binaries. The final image for binfmt is

FROM scratch
COPY --from=qemu usr/bin/qemu-* usr/bin/
# etc.

I think we should fix this for now and revert back later.

@oflebbe did you open a PR for this?

[oliviabarrick]

If he didn’t, I filed one here: #3071

[deitch]

FYI, if we really insist on staying with alpine for the first stage, there is an alternative. Do the following:

ARG QEMU_ARCHS="aarch64 ppc64le s390x"
RUN for i in ${QEMU_ARCHS}; do curl -L https://github.com/multiarch/qemu-user-static/releases/download/v${QEMU_VERSION}/qemu-${i}-static.tar.gz | tar zxvf - -C /usr/bin; done
RUN chmod +x /usr/bin/qemu-*

That is what we did over in Calico.