Kubeadm broken on armv6l

[mitchhh22]

I tried to install kubeadm on my raspberry pi zero W, but get an "Illegal Instruction"
On a raspberry pi 3 (armv7) it works just fine.

[clabu]

I am facing the same issue with kubeadm 1.6.1 on a Raspberry Pi Model B+, also armv6.

$ kubelet --help
Illegal instruction

$ uname -a
Linux pi1 4.4.50-hypriotos+ #2 PREEMPT Sun Mar 19 14:44:01 UTC 2017 armv6l GNU/Linux

[clabu]

I downgraded to kubeadm 1.5.6 and it works. 1.6.0 gives the same error as 1.6.1.

[mitchhh22]

@clabu yeah downgrading to 1.5.6 works but can't join a 1.6+ cluster.

[luxas]

First off, thanks for using Kubernetes on ARM 😄!

This is a known issue; it was discussed in kubernetes/kubernetes#38067 and we dropped armel support (which part of RPi 1 uses when cross-compiling).

Basically armhf (GOARM=7) can't run on the Pi 1, so we used armel with GOARM=6 in -v1.5 to support RPi 1. However, we went all armhf in v1.6, hence it's not working on the Pi 1.

Deprecate armel and use armhf images instead and use GOARM=7 instead of GOARM=6
Motivation:

• The only GOARM=6 board Go will support in go1.8 is the Raspberry Pi 1 which is just too slow to run newer Kubernetes versions.
• Small performance improvements when using GOARM=7
• The armel (http://hub.docker.com/u/armel) images are not updated as often as the armhf (http://hub.docker.com/u/armhf) images are.

For example, https://hub.docker.com/r/armel/debian/ was updated 8 months ago which is really bad from a security standpoint, vs https://hub.docker.com/r/armhf/debian/ which was updated 3 days ago.

Also, with the armhf switch, we were able to use https://hub.docker.com/r/armhf/alpine, which is great.

Hope it helps, but sorry for not being able to support the RPi 1 anymore.

If you want to help with documenting it/spreading the word, please do or come with suggestions

[wsargent]

I'm having this same problem on a Pi Zero

Linux p1 4.9.59+ #1047 Sun Oct 29 11:47:10 GMT 2017 armv6l GNU/Linux

[mthkunze]

Is it possible to discuses a reintegration of armv6l Support. I found many Posts showing the interest for using Kubernetes on Pi Zero and other armv6l Pi Devices. P Zero is good for hosting Micro Services in Kubernetes or Swarm Cluster Environments. Docker Swarm works well for me. So it would be nice if anyone could recycle the discussion. Pi clusterhat is properly a nice demo infrastructure.

[kfox1111]

Looking at the current docker.io build for the pi zero,
Go version: go1.9.3
and docker version: 18.02.0-ce

It does seem to be using a recent version of go.

I agree that there is not enough ram to use k8s on it in a standalone mode, but having it be a slave on a bigger master, it should have enough resources to do some useful things still.

[juliancheal]

Does anyone know if it's possible just to build from source to use my pi zeros as k8s nodes?

[wxdao]

For example, https://hub.docker.com/r/armel/debian/ was updated 8 months ago which is really bad from a security standpoint, vs https://hub.docker.com/r/armhf/debian/ which was updated 3 days ago.

This is not true today since official images on different architectures are updated simultaneously. For example https://hub.docker.com/r/arm32v5/debian/, https://hub.docker.com/r/arm32v7/debian/ and https://hub.docker.com/r/amd64/debian/ were all updated 9 days ago.

Also, with the armhf switch, we were able to use https://hub.docker.com/r/armhf/alpine, which is great.

https://hub.docker.com/r/arm32v6/alpine/ runs well on Pi Zero.

I hope that you will reconsider it. Stopping Pi Zero from running latest k8s is so disappointing.

@luxas

[kfox1111]

+1. some confusion has happened as the hub was rearranged and the older repos are still around. The newer ones do seem to be getting frequent updates.

[shinichi-hashitani]

Hi @juliancheal ,

I am still in the middle of building k8s on ClusterHAT, but I was able to compile and build binaries for Pi Zero.

Basically, I have followed the below with some modifications:
https://povilasv.me/raspberrypi-kubelet/

I worked on wsl:
Linux DESKTOP-6GRDDIN 4.4.0-17134-Microsoft #48-Microsoft Fri Apr 27 18:06:00 PST 2018 x86_64 x86_64 x86_64 GNU/Linux

#1 install gcc-arm-linux-gnueabi instead of gcc-arm-linux-gnueabihf
sudo apt-get install gcc-arm-linux-gnueabi <- change

#2 before building for linux/arm, make two modifications to set_platform_envs() in hack/lib/golang.sh
*** add GOARM
export GOOS=${platform%/}
export GOARCH=${platform##/}
export GOARM=5 <- add
*** change CC
case "${platform}" in
"linux/arm")
export CGO_ENABLED=1
export CC=arm-linux-gnueabi-gcc <-change
;;

GOARM has to be 5. If you specify 6, you will get a linker error during the build. (Which I couldn't resolve.)

[wxdao]

@shinichi-hashitani It works for my Pi Zero! Thanks!

Also I resolved your problem on linker error. For Pi Zero, set GOARM=6 and keep gcc-arm-linux-gnueabihf. However for Pi 1 you gotta set GOARM=5 and use gcc-arm-linux-gnueabi instead.

[juliancheal]

@shinichi-hashitani this is great! I will give it a try thanks!