Use Debian instead of Alpine for multi-arch builds

[hiddeco]

Fixes #385

This PR changes the base image for the build and controller
container images to Debian slim.

Reason for this is that it has proven to be hard to produce working
executables for AMD64, ARM64 and ARMv7 at all times using Alpine,
due to them being dynamically linked and compiled using CGO, and
Alpine having constraints like musl that create an extra barrier,
especially in combination with our exotic set of dependency
constraints.

There are a number of trade-offs we have to live with by doing this,
not limited to:

• An increased build time, the full release pipeline used to take 25-35
  minutes, based on the images we have build for testing purposes this
  seems to have become 35-40 minutes.
• An increased image size of roughly two times the (compressed) size of
  the Alpine based image.

[stefanprodan]

Test image: docker.io/fluxcd/source-controller:debian--3f287d95

[kingdonb]

I am testing with a Debian install on armhf (armv7l kernel)

$ docker run -it docker.io/fluxcd/source-controller:debian--3f287d95
exec format error
Error response from daemon: Cannot start container 1617b01399696e1ee2d5dca950eb4e50defbbab2509bffe47155d0b884b879d6: [8] System error: exec format error

also uname -a

$ uname -a
Linux raspberrypi 4.14.70-v7+ #1144 SMP Tue Sep 18 17:34:46 BST 2018 armv7l GNU/Linux

[hiddeco]

@kingdonb can you run the following:

docker run -it --entrypoint /bin/sh docker.io/fluxcd/source-controller:debian--3f287d95
# ...in the container
ldd $(which source-controller)
export GOTRACEBACK=crash
source-controller

and share output for the two container commands?

[kingdonb]

Doesn't get quite that far:

pi@raspberrypi:~ $ docker run -it --entrypoint /bin/sh docker.io/fluxcd/source-controller:debian--3f287d95
exec format error
Error response from daemon: Cannot start container 5b8bf8149177d508fc827b42b1a8ec66c32305079a5258bbd58054909762651d: [8] System error: exec format error
pi@raspberrypi:~ $ docker run -it --entrypoint /bin/sh docker.io/fluxcd/source-controller:debian--3f287d95 -- source-controller
exec format error
Error response from daemon: Cannot start container 492c77a0258252802ad86c342e07387d0e121ad7003a73515fafc45c132ec60c: [8] System error: exec format error

Working on upgrades that may help with this (the Docker version and everything else is probably at least a year old)

$ dpkg -l|grep docker
ii  docker.io                             1.8.3~ds1-2                       armhf        Linux container runtime

[hiddeco]

As docker run -it --entrypoint /bin/sh debian:buster-slim produces the same error for @kingdonb, his OS is likely busted and the output not reliable and/or related to the contents of the image.

[kingdonb]

Better shape now:

$ docker run -it docker.io/fluxcd/source-controller:debian--3f287d95
Unable to find image 'fluxcd/source-controller:debian--3f287d95' locally
debian--3f287d95: Pulling from fluxcd/source-controller
3035777cd90a: Pull complete
e53f21b0472f: Pull complete
b980144b3d9a: Pull complete
cda93e6cdb3b: Pull complete
3048683fdf21: Pull complete
Digest: sha256:fd6f16a9a6a36c2eaa1e757987dac7b1045db8121cd5934ac3cbc0a3e9d4b7b5
Status: Downloaded newer image for fluxcd/source-controller:debian--3f287d95
{"level":"error","ts":"2021-06-17T13:58:48.146Z","logger":"controller-runtime.client.config","msg":"unable to get kubeconfig","error":"invalid configuration: no configuration has been provided, try setting KUBERNETES_MASTER environment variable","errorCauses":[{"error":"no configuration has been provided, try setting KUBERNETES_MASTER environment variable"}]}

I was using the baked-in docker.io package from raspbian, it is not new enough for multi-arch images. I upgraded to the actual current docker-ce package from Docker Inc's website. No more exec format errors.

[stefanprodan]

@kingdonb can you do the same on an ARM64 machine and post here please.

[hiddeco]

ubuntu@arm-hell:~$ uname -a
Linux arm-hell 5.4.0-1045-oracle #49-Ubuntu SMP Mon May 3 05:27:18 UTC 2021
aarch64 aarch64 aarch64 GNU/Linux
ubuntu@arm-hell:~$ docker run -it docker.io/fluxcd/source-controller:debian--3f287d95
Unable to find image 'fluxcd/source-controller:debian--3f287d95' locally
debian--3f287d95: Pulling from fluxcd/source-controller
fcad0c936ea5: Pull complete
689020739f42: Pull complete
08a40e62d58e: Pull complete
d3f454042be6: Pull complete
7a497846232c: Pull complete
Digest: sha256:fd6f16a9a6a36c2eaa1e757987dac7b1045db8121cd5934ac3cbc0a3e9d4b7b5
Status: Downloaded newer image for fluxcd/source-controller:debian--3f287d95
{"level":"error","ts":"2021-06-17T14:13:11.462Z","logger":"controller-runtime.client.config","msg":"unable to get kubeconfig","error":"invalid configuration: no configuration has been provided, try setting KUBERNETES_MASTER environment variable","errorCauses":[{"error":"no configuration has been provided, try setting KUBERNETES_MASTER environment variable"}]}

[kingdonb]

$ uname -a
Linux instance-20210526-1400 5.4.17-2102.201.3.el8uek.aarch64 #2 SMP Fri Apr 23 09:42:46 PDT 2021 aarch64 aarch64 aarch64 GNU/Linux
$ docker run -it docker.io/fluxcd/source-controller:debian--3f287d95
Unable to find image 'fluxcd/source-controller:debian--3f287d95' locally
debian--3f287d95: Pulling from fluxcd/source-controller
fcad0c936ea5: Pull complete
689020739f42: Pull complete
08a40e62d58e: Pull complete
d3f454042be6: Pull complete
7a497846232c: Pull complete
Digest: sha256:fd6f16a9a6a36c2eaa1e757987dac7b1045db8121cd5934ac3cbc0a3e9d4b7b5
Status: Downloaded newer image for fluxcd/source-controller:debian--3f287d95
{"level":"error","ts":"2021-06-17T14:13:43.260Z","logger":"controller-runtime.client.config","msg":"unable to get kubeconfig","error":"invalid configuration: no configuration has been provided, try setting KUBERNETES_MASTER environment variable","errorCauses":[{"error":"no configuration has been provided, try setting KUBERNETES_MASTER environment variable"}]}

Same as last time (but much faster)

[stefanprodan]

LGTM 🚀