Unable to pull metallb image

[williamb541]

Summary

I am following the steps for installing the Metallb addon documented here: https://microk8s.io/docs/addon-metallb.

I have installed the ingress addon.

However, running the following command, microk8s enable metallb, results in ErrImagePull / ImagePullBackOff messages.

What Should Happen Instead?

The image for the Metallb addon should be successfully pulled.

Reproduction Steps

1. microk8s enable metallb
2. kubectl get all -n metallb-system

NAME READY STATUS RESTARTS AGE
pod/speaker-vk8cn 0/1 ImagePullBackOff 0 70m
pod/controller-66c86bddb9-rrvm9 0/1 ImagePullBackOff 0 70m
pod/speaker-7xplk 0/1 ImagePullBackOff 0 70m
pod/speaker-frm7b 0/1 ImagePullBackOff 0 70m
pod/speaker-bp2xm 0/1 ImagePullBackOff 0 70m
pod/speaker-2pxff 0/1 ImagePullBackOff 0 70m
pod/speaker-6gcr2 0/1 ImagePullBackOff 0 70m

NAME DESIRED CURRENT READY UP-TO-DATE AVAILABLE NODE SELECTOR AGE
daemonset.apps/speaker 6 6 0 6 0 beta.kubernetes.io/os=linux 70m

NAME READY UP-TO-DATE AVAILABLE AGE
deployment.apps/controller 0/1 1 0 70m

NAME DESIRED CURRENT READY AGE
replicaset.apps/controller-66c86bddb9 1 1 0 70m

[neoaggelos]

Hi @williamb541, which MicroK8s version are you using? Can you attach an inspection report tarball (generate with microk8s inspect). Thanks

[williamb541]

Hi @neoaggelos!
I'm using MicroK8s version 1.28.4.

Here is the link to the inspection report tarball:
[inspection-report-20231227_181851.tar.gz](https://github.com/canonical/microk8s/files/13788221/inspection-report-
20231227_181851.tar.gz)

[neoaggelos]

Hi @williamb541

I see this in the logs

Dec 26 18:53:58 microk8s-021 microk8s.daemon-containerd[877]: time="2023-12-26T18:53:58.074829897Z" level=info msg="PullImage \"metallb/speaker:v0.9.3\""

The metallb addon is trying to pull images from dockerhub, but these are no longer available. Newer microk8s versions use images from quay.io for this, and have for a long time now (see #3855)

Has this cluster been upgraded from older versions? If so, can you try:

# disable metallb
sudo microk8s disable metallb

# update the addons repository to the latest version
sudo microk8s addons repo update core --force

# enable metallb
sudo microk8s enable metallb

[williamb541]

I followed your steps for upgrading the addon repo and enabled metallb without any problem. Thank you!

To answer your question, a few weeks ago, I upgraded the microk8s version from 1.24 to 1.28.4.

Are there any other changes I need to make as a consequence of upgrading the addon repo?

[neoaggelos]

Hi @williamb541 (happy new year!). No, that should be all :)

As a side note, make sure to try and not jump minor versions when upgrading K8s clusters (e.g. upgrade should be 1.24 -> 1.25 -> 1.26 -> 1.27 -> 1.28, otherwise you might get breaking behaviour in the internal state of Kubernetes components).

[williamb541]

Hi @neoaggelos, happy new year to you too!
I did upgrade from 1.24 to 1.28. Would it be a good idea to reinstall microk8s?
Also, is there a clean way to shutdown a microk8s cluster?

[ktsakalozos]

Hi @williamb541, upgrading should fine, you do not need to reinstall your cluster if it works.

If you want to stop a single node cluster you can try sudo microk8s stop.