calicp-kube-controllers can't get API Server: context deadline exceeded #3085
Comments
|
I see these these in the I don't know if there is a need to load these manually. I suppose kube-proxy loads them on start thats why it triggered this... Then restart microk8s. I am not sure if this will work though. |
|
I have exactly the same probelm with my Raspberry Pi cluster running Microk8s. |
|
I too have the same issue, although mine is an existing cluster. I noticed the issue after installing the latest (5.13.0-40-generic) kernel and restarting the server. The logs of the I've tried to specify the calico interface in the Happy to help in any way. |
|
I am having the same issue on Ubuntu 21.10 running on multiple Raspberry Pi 4s with the latest kernel installed After long trail and error I at least found a solution to temporarily get the cluster sort of up and running again. Possible temporary mitigation/solution
Side note:
Hopefully this helps in either fixing the issue and restoring some smaller clusters. 😉 I am happy to provide more diagnostics if needed. |
|
Hey @balchua, the modules are loaded: It maybe the reason for RasPI, but I'm using Ubuntu Server 21.10 for amd64. |
|
Dear all, I have the same issue on a single node raspberry pi 4 deployment on ubuntu server 21.10. Regards Jörg |
|
Dear all, I have the same problem on an AMD Ubuntu server 21.10 deployment. Even tried to update Ubuntu to 22.04 and microk8s to 1.23.5. But the problem still exists. Best regards, |
|
Hey folks, we'll take a look at this right away. Thank you all for raising this! |
|
Can I please ask you to try our other channel? |
AlexsJones
added
kind/bug
|
What's the output of If its on... please try running This might be offloading related 🤔 |
|
Changed the channel to latest/edge and did an update of microk8s. A second attempt of the command showed no output |
|
Updating to edge did not make a difference for me. |
Thanks @blicknix the idea here is we sometimes come across issues with offloading being enabled. Most likely that interface doesn't exist because calico-node isn't running yet. |
Apologies I reworded the ask, this will disable offloading, if you restart the pods/microk8s lets see whether that brings back up the controller. I meant to ask for continuing to debug |
|
I did Here is the output of |
|
Hi all, I've got the same error on Ubuntu 21.10 on AMD64 with Linux 5.13.0-40-generic and MicroK8s 1.23/stable. Let me know if I should try something |
|
Similar to @nc-kab, I've updated to the The output of |
|
Here is a source of errors I would appreciate if you could please eliminate. When the calico CNI sets up the network it needs to select a network interface through which it will route traffic. In |
|
Changed |
Didn't work for me, too, but thank you |
|
@ktsakalozos I've tried that and had no effect on my cluster. I am also running as a single node. |
|
Could you also |
|
|
Same for me. I'm able to run all the modprobes in by terminal, but the error message is still there in the kubelite log |
|
Hey, @AlexsJones
I did it: The same problem:
|
|
Changing IP_AUTODETECTION_METHOD doesn't work for me as well. |
|
I just upgraded to Ubuntu 22.04 with kernel 5.15 and I still got the same error |
|
@dereisele could you share a |
|
Sorry for the maybe stupid question. Is there an easy way to share the tarball with you and without anybody getting the information? |
@blicknix you can find us in #microk8s on the Kubernetes slack. I am kjackal there, ping me. |
|
Hi could you please try this.
|
It looks like this works! 🎉 |
This worked for me, too. Thank you very much! 🎉 |
|
This worked for me as well. Thank you @ktsakalozos! Would it be possible to explain what happened, and why |
|
Yesterday I was having a hard time with a new install and came across this issue. I can confirm that this is also working for me too. |
|
I don't see any # Content of /var/snap/microk8s/current/args/kube-proxy
--kubeconfig=${SNAP_DATA}/credentials/proxy.config
--cluster-cidr=10.1.0.0/16
--healthz-bind-address=127.0.0.1
--profiling=false
|
The CNI used by default in MicroK8s is calico. Calico works best with the In any case, we will be shipping a patch in the following days for this issue. We would appreciate if you could verify that the edge channel of the track you are using works for you. You can test this by doing a fresh install or refreshing to the respective channel, eg assuming you are on the 1.23 track you can do |
|
I just validated this workaround on 3 different Fedora 36 microk8s clusters based on comment 1111290817. Thank you! |
|
I also had to make these changes on a fresh cluster under Rocky Linux 8. @ktsakalozos , do these changes (#3085 (comment)) made to |
|
For a bit of extra context, we are targeting |
Yes these changes will persist through snap refreshes. |
Awesome, thank you! |
|
This is happening to me in 1.24, ubuntu 22.04 on raspberry pi |
|
Yes, appears as if the error was reintroduced. Added ENVIRONMENT MicroK8s v1.26.0 revision 4390 on a NUC Intel Celeron N5095, 16GB RAM, 1TB SSD Distributor ID: Ubuntu Reproduce:
kc logs pod/calico-kube-controllers-7874bcdbb4-5ftc2 -n kube-system -f |
|
This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions. |
|
Following the steps here: To get simillar error: What could be the reason... |
I have the problem after having installed microk8s on my Ubuntu 21.10 server:
Checked the pods and saw that one crushed:
In logs , I see that something is wrong with the API Server:
Result of
microk8s.inspectinspection-report-20220422_132011.tar.gz
The text was updated successfully, but these errors were encountered: