Replace references to echoserver with supported image from all documentation

[sschober]

Trying out minikube on my Mac mini apple silicon/m1.
But step 4 from minikube start is not working for me.

I am using Docker Desktop 3.3.1 (63152). I have activated the experimental feature:
"Use new virtualisation framework"

Enables the new Big Sur virtualization.framework instead of the hypervisor.framework. You must reset your Kubernetes cluster when enabling for the first time.

Please let me know if I can provide further information!

Steps to reproduce the issue:

1. brew install minikube
2. minikube start
3. kubectl create deployment hello-minikube --image=k8s.gcr.io/echoserver:1.10

Full output of failed command:

kubectl logs hello-minikube-5d9b964bfb-bscl4
Generating self-signed cert
Generating a 2048 bit RSA private key
.............+++
................................................................................................+++
writing new private key to '/certs/privateKey.key'
-----
Starting nginx
2021/04/15 19:56:10 [error] 14#14: failed to initialize Lua VM in /etc/nginx/nginx.conf:88
nginx: [error] failed to initialize Lua VM in /etc/nginx/nginx.conf:88

Full output of minikube start command used, if not already included:

😄 minikube v1.19.0 auf Darwin 11.2.3 (arm64)
✨ Automatically selected the docker driver
👍 Starting control plane node minikube in cluster minikube
🚜 Pulling base image ...
\ > gcr.io/k8s-minikube/kicbase...: 357.67 MiB / 357.67 MiB 100.00% 1.82 MiB
🔥 Creating docker container (CPUs=2, Memory=4000MB) ...
> kubectl.sha256: 64 B / 64 B [--------------------------] 100.00% ? p/s 0s
> kubelet.sha256: 64 B / 64 B [--------------------------] 100.00% ? p/s 0s
> kubeadm.sha256: 64 B / 64 B [--------------------------] 100.00% ? p/s 0s
> kubectl: 35.44 MiB / 35.44 MiB [------------] 100.00% 2.18 MiB p/s 16.44s
> kubeadm: 34.50 MiB / 34.50 MiB [-----------] 100.00% 1.72 MiB p/s 20.229s
> kubelet: 100.57 MiB / 100.57 MiB [---------] 100.00% 3.32 MiB p/s 30.452s

▪ Generating certificates and keys ...
▪ Booting up control plane ...
▪ Configuring RBAC rules ...

🔎 Verifying Kubernetes components...
▪ Using image gcr.io/k8s-minikube/storage-provisioner:v5
🌟 Enabled addons: storage-provisioner, default-storageclass
🏄 Done! kubectl is now configured to use "minikube" cluster and "default" namespace by default

Optional: Full output of minikube logs command:
minikube.log

[sschober]

I just deactivated the "new virtualisation framework" option and retried, with the same result.

[afbjorklund]

I'm not sure if k8s.gcr.io/echoserver:1.10 has any arm64 variant, it looks like amd64 only ?

kubernetes-retired/contrib#2991

[sschober]

Thanks for the hint! Might this be worth a note in the docs? Would have saved me some time. :) I could do that as well, if this would be welcome-

[afbjorklund]

I could do that as well, if this would be welcome-

That would be much welcome. I'm not sure if there is an alternative image available, but a note in the docs would be a start!

REPOSITORY              TAG       IMAGE ID       CREATED       SIZE
k8s.gcr.io/echoserver   1.10      365ec60129c5   3 years ago   95.4MB
k8s.gcr.io/echoserver   1.4       a90209bb39e3   4 years ago   140MB

https://minikube.sigs.k8s.io/docs/start/
https://kubernetes.io/docs/tutorials/hello-minikube/

It seems to be based on the "nginx" image, so it should be possible to make a new version of it that also supports arm64.

$ docker run -it --entrypoint "" k8s.gcr.io/echoserver:1.4 nginx -V
nginx version: nginx/1.10.0

cat /etc/os-release
PRETTY_NAME="Ubuntu 16.04 LTS"

$ docker run -it --entrypoint "" k8s.gcr.io/echoserver:1.10 nginx -V
nginx version: nginx/1.13.3

cat /etc/os-release
PRETTY_NAME="Ubuntu 16.04.2 LTS"

[afbjorklund]

I found it, it lived in k8s "ingress-nginx" - but it died in 2018.

https://github.com/kubernetes/ingress-nginx/tree/84a6e0c1ab54f8c8e374237b8d9215dcbaa29f0f/images/echoheaders

REPOSITORY                            TAG       IMAGE ID       CREATED       SIZE
gcr.io/google_containers/echoserver   1.10      365ec60129c5   3 years ago   95.4MB

Then it got bumped to 2.1, but nobody bothered updating docs.

kubernetes/ingress-nginx@77b922a | kubernetes/kubernetes@a2d94d9

REPOSITORY                                     TAG       IMAGE ID       CREATED       SIZE
gcr.io/kubernetes-e2e-test-images/echoserver   2.2       4081d9a83108   2 years ago   21.7MB
gcr.io/kubernetes-e2e-test-images/echoserver   2.1       76232f30d3c7   2 years ago   21.7MB

2.1
nginx version: nginx/1.12.2
PRETTY_NAME="Alpine Linux v3.7"

kubernetes/kubernetes@5e84dfb "Multiple same headers got wrong result"

2.2
nginx version: nginx/1.12.2
PRETTY_NAME="Alpine Linux v3.7"

kubectl create deployment hello-minikube --image=gcr.io/kubernetes-e2e-test-images/echoserver:2.2

[sschober]

Hey thank you for all this information! I am trying to digest what this means for me currently...

I tried deploying echoserver:2.2, but only got the following result:

Generating self-signed cert
Generating a 2048 bit RSA private key
..............................................+++
......+++
writing new private key to '/certs/privateKey.key'
-----
Starting nginx
PANIC: unprotected error in call to Lua API (bad light userdata pointer)

echoserver:2.1 yields the same.

I currently clone the Kubernetes repo and try to build the echoserver image on my machine... and yes this seems to be successful:

git clone <kubernetes-repo>
cd kubernetes/test/images/echoserver
vim Dockerfile # get rid of all that cross-build stuff I do not know :)
docker build .
docker tag <image id> echoserver:v2.2
minikube load echoserver:v2.2
kubectl create deployment hello-minikube --image=echoserver:v2.2
# success :)

This way, I can now continue with the getting started guide. :)

I can add a note to the docs if you like, with a reference to this issue maybe?

[afbjorklund]

My bad, I should have tested the image rather than assuming that it would work just because it is used in the k8s test.

It worked OK on amd64, but not on arm64...

Crashes the same way on Linux. Apparently they "forgot" to release a version of the echoserver, with arm64 support...

kubernetes/ingress-nginx#2802

ubuntu@ubuntu:~$ docker run gcr.io/kubernetes-e2e-test-images/echoserver:2.2
Generating self-signed cert
Generating a 2048 bit RSA private key
...............................+++
.............................................................................+++
writing new private key to '/certs/privateKey.key'
-----
Starting nginx
PANIC: unprotected error in call to Lua API (bad light userdata pointer)

Like you say, minikube doesn't need the LuaJIT support anyway.

But upgrading nginx to something less ancient, would fix it too...

The weird thing is that the git repository says nginx 1.15, but image says 1.12 ?
Anything newer than 2018 would work, the nginx-ingress-controller used 1.15.6.

[awarus]

+1, I've spent about 2 days running hello-minikube. I couldn't start hello-minikube on apple MacBook Pro with M1 chip(MacOS 11.4 BigSur (20F71)). Have always the same situation:
hello-minikube-6ddfcc9757-5x8bp 0/1 CrashLoopBackOff

[FLAGLORD]

@awarus Have you solved this problem?

[awarus]

@awarus Have you solved this problem?

No, as far I understand hello-minikube image is not available now for M1

[sharifelgamal]

What we're going to end up doing here is creating an image that support multiple archs and using that instead of echoserver for all these demos and guides.

[zhyyu]

Hey thank you for all this information! I am trying to digest what this means for me currently...

I tried deploying echoserver:2.2, but only got the following result:

Generating self-signed cert
Generating a 2048 bit RSA private key
..............................................+++
......+++
writing new private key to '/certs/privateKey.key'
-----
Starting nginx
PANIC: unprotected error in call to Lua API (bad light userdata pointer)

echoserver:2.1 yields the same.

I currently clone the Kubernetes repo and try to build the echoserver image on my machine... and yes this seems to be successful:

git clone <kubernetes-repo>
cd kubernetes/test/images/echoserver
vim Dockerfile # get rid of all that cross-build stuff I do not know :)
docker build .
docker tag <image id> echoserver:v2.2
minikube load echoserver:v2.2
kubectl create deployment hello-minikube --image=echoserver:v2.2
# success :)

This way, I can now continue with the getting started guide. :)

I can add a note to the docs if you like, with a reference to this issue maybe?

Hi, can you share the Dockerfile you changed with us?

[zhyyu]

@awarus Have you solved this problem?

No, as far I understand hello-minikube image is not available now for M1

e2eteam/echoserver:2.2-linux-arm64 worked for me on m1

[Tazovsky]

@preslavmihaylov thanks!

I have following service:

$ kubectl -n traefik get service   
NAME                      TYPE        CLUSTER-IP      EXTERNAL-IP   PORT(S)                       AGE
ggtips                    ClusterIP   10.96.41.7      <none>        80/TCP                        33m
traefik-ingress-service   NodePort    10.110.178.12   <none>        80:32194/TCP,8080:31621/TCP   33m
whoami                    ClusterIP   10.103.52.177   <none>        80/TCP                        33m

$ minikube ip
192.168.49.2

Can anyone explain why it is available only after port forwarding at http://localhost:8081 and http://localhost:8080:

kubectl -n traefik  port-forward service/traefik-ingress-service 8081:80 8080:8080

instead of 127.0.0.1:32194, 127.0.0.1:31621 or 192.168.49.2:32194, 192.168.49.2:31621 ?

And why minikube does not return any URL?

 $ minikube --namespace=traefik service traefik-ingress-service --url

🏃  Starting tunnel for service traefik-ingress-service.
❗  Because you are using a Docker driver on darwin, the terminal needs to be open to run it.

[preslavmihaylov]

@Tazovsky I believe this is because the addresses you see in there are part of minikube's internal network and you have to expose them for this to work locally.

When using AWS or similar as backend, usually, you'll get a hostname you can connect to directly

[drinkbeer]

kubectl port-forward service/hello-minikube 3000:3000

Thank you so much! It works in my M1 macbook.

[shorif2000]

@awarus Have you solved this problem?

No, as far I understand hello-minikube image is not available now for M1

e2eteam/echoserver:2.2-linux-arm64 worked for me on m1

This is showing all green but when running minikube service hello-minikube it fails with

panic: runtime error: index out of range [0] with length 0

goroutine 1 [running]:
k8s.io/minikube/cmd/minikube/cmd.startKicServiceTunnel({0x14000213450, 0x1, 0x1}, {0x0, 0x0, 0x0}, {0x1045f0cf3, 0x8}, {0x14000d87b1a, 0x6})
	/app/cmd/minikube/cmd/service.go:205 +0x340
k8s.io/minikube/cmd/minikube/cmd.glob..func35(0x106109aa0, {0x14000213450, 0x1, 0x1})
	/app/cmd/minikube/cmd/service.go:143 +0x594
github.com/spf13/cobra.(*Command).execute(0x106109aa0, {0x14000213420, 0x1, 0x1})
	/go/pkg/mod/github.com/spf13/cobra@v1.3.0/command.go:860 +0x640
github.com/spf13/cobra.(*Command).ExecuteC(0x1061095a0)
	/go/pkg/mod/github.com/spf13/cobra@v1.3.0/command.go:974 +0x410
github.com/spf13/cobra.(*Command).Execute(...)
	/go/pkg/mod/github.com/spf13/cobra@v1.3.0/command.go:902
k8s.io/minikube/cmd/minikube/cmd.Execute()
	/app/cmd/minikube/cmd/root.go:157 +0xdb8
main.main()
	/app/cmd/minikube/main.go:86 +0x2a0

[klaases]

Hi @sschober, did the comments above help resolve this issue in your case?

We see that @drinkbeer was able to get things working as well.

[silasb]

I ended up forking echoserver as https://github.com/silasb/echoserver and a ARM64 docker image is located here https://hub.docker.com/r/silasb/echoserver

[scubbo]

+1 to the request for a note in the docs that this image might not work for everyone. I only happened to recognize the error log of exec user process caused: exec format error as being related to architecture issues from previous experience, we shouldn't assume newbies would be able to infer that.

(FWIW, I found that echoserver-arm:1.8 worked fine on my Raspberry Pi 4 64-bit)

[klaases]

If anyone is interested in creating their own pod and service to use with the tutorial on Mac M1/ARM64, this comment might be helpful. Let me know if any questions.

[k8s-triage-robot]

The Kubernetes project currently lacks enough contributors to adequately respond to all issues and PRs.

This bot triages issues and PRs according to the following rules:

• After 90d of inactivity, lifecycle/stale is applied
• After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
• After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

• Mark this issue or PR as fresh with /remove-lifecycle stale
• Mark this issue or PR as rotten with /lifecycle rotten
• Close this issue or PR with /close
• Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle stale

[hanseltime]

I ended up forking echoserver as https://github.com/silasb/echoserver and a ARM64 docker image is located here https://hub.docker.com/r/silasb/echoserver

Thank you for pushing an image that works for the architecture. Would love to see this repo added to the k8s docks/registry

[spowelljr]

We've pushed an image kicbase/echo-server:1.0 that works on both amd64 & arm64, will make a PR shortly to update the documentation.