s390x images for nginx controller

[Cynerva]

Hey folks,

I see in the nginx controller Makefile that images are being built for multiple architectures, but s390x is not included.

What would it take to get s390x builds enabled and included in nginx controller releases? Is there any way I can help?

Thanks!

[aledbf]

@Cynerva we tried but running ARCH=s390x make docker-build just hangs running apt-get update
Can you help to diagnose why?

[aledbf]

@Cynerva you need to start with ubuntu-slim https://github.com/kubernetes/ingress/blob/master/images/ubuntu-slim/ then https://github.com/kubernetes/ingress/blob/master/images/nginx-slim before compiling the ingress controller

[Cynerva]

Thanks for the pointers! I'll give it a go :)

[aledbf]

@Cynerva awesome. Thanks!

[Cynerva]

Okay. I hit the issue while building ubuntu-slim, and I'm able to repro with a small Dockerfile:

FROM s390x/ubuntu:16.04

COPY qemu-s390x-static /usr/bin/qemu-s390x-static

RUN echo foo

The docker build hangs forever trying to run echo foo. Same occurs with s390x/ubuntu:16.10 and s390x/ubuntu:17.04, but for whatever reason, s390x/debian:jessie and s390x/busybox do not have the same issue.

I also could not get ubuntu s390x working with qemu-debootstrap, though in that scenario, it gives me a core dump instead of hanging. Nevermind, I was using the wrong version of qemu-s390x-static for this test. It does hang too.

I don't really have the expertise to troubleshoot further, but I get the feeling that qemu s390x support isn't mature enough to support recent Ubuntu images. A couple references lead me to think that:

https://wiki.qemu.org/Documentation/Platforms/S390X#QEMU_s390x_Guest_Support

When using tcg, the machine emulated is at an older level not supported on any of the recent enterprise distributions. Your best bet is probably using Debian.

https://wiki.ubuntu.com/S390X#Q:_Is_there_an_emulator_I_can_use_to_run_Ubuntu_s390x_on_non-mainframe_hardware.3F

Unfortunately there is currently no emulator support for Ubuntu s390x port - this includes current releases of Hercules, PDT, and QEMU.

[aledbf]

@Cynerva thanks for doing this. Please check https://lists.ubuntu.com/archives/foundations-bugs/2016-June/287498.html

[Cynerva]

Please check https://lists.ubuntu.com/archives/foundations-bugs/2016-June/287498.html

Looks like a similar symptom but I think it's a different problem -- sounds like it would apply more when Docker itself is built for s390x, rather than its containers. The fix in that link should be present in recent releases of Docker anyway.

FWIW I just tried upgrading from Docker 1.13.1 to Docker 17.06.0-ce. Still hangs.

[Cynerva]

I think we are hitting this bug: https://bugs.launchpad.net/qemu/+bug/1701798

From the description:

$ QEMU_LD_PREFIX=/usr/s390x-linux-gnu ~/inst-qemu/2.9.0/bin/qemu-s390x hello.s390x
<hangs>

[kolyshkin]

I am seeing the same bug when trying to testhttps://github.com/checkpoint-restore/criu on s390. Debian Jessie works, any newer distro either hangs on any command (except for binaries aiming in dynamic linking, like /sbin/ldconfig.real and /lib/ld64.so.1), or returns exit code 139. I tried with various distros and multiple versions of qemu-user-static.

[ixdy]

As @rphillips noted in multiarch/qemu-user-static#31, this should be fixed now.

[aledbf]

@ixdy sadly that PR does not fixes the issue in ubuntu