-
Notifications
You must be signed in to change notification settings - Fork 1.3k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Calico 3.6-3.8 Pending forever on single-node Kubeadm #2699
Comments
the same problem. |
Could it be a resource request? Maybe your node isn't big enough? |
What does |
@fasaxc I don't think it is a resource problem, since it is a dedicated machine with 4 core and 16GB dedicated to it and nothing else is running. |
@fasaxc All kubectl describes:
|
OK, so most of the pods are failing to schedule because calico/node hasn't started yet, I think. What about logs for calico/node? That might tell us why the init container isn't finishing. |
@fasaxc I've posted the |
I think |
@fasaxc The only output I get (the other pending pods have no output):
|
This explains how to get the log: https://kubernetes.io/docs/tasks/debug-application-cluster/debug-init-containers/ |
The upgrade-ipam InitContainer has trillions of errors:
install-cni and flexvol-driver are waiting to start. |
Same behavior on my newborn cluster. My cluster is initialized using kubeadm, and applying calico manifests from either 3.6 or 3.8 leads to these errors in the As I'm on a fresh installation that doesn't need this "upgrade IPAM" stage (AFAIK), I've tried to delete this init container from the manifest (`kubectl edit daemonset -n kube-system calico-node) and everything went fine, issue resolved. |
I've found why the
I think it comes from a previous installation. Having done a Source : https://github.com/projectcalico/cni-plugin/blob/v3.8.0/pkg/upgrade/migrate.go#L66 |
@demikl can it be considered a Calico bug or kubeadm bug? I will try deleting it tomorrow. Thanks |
i am already fix it.
and then excute kubeadm init..... |
Deleting /var/lib/cni/ solves the problem. Are you doing a patch for migrate.go @withlin? |
no. re-install. it is ok. |
Maybe it's worth at least adding that to calico getting started documentation as a note. |
@staticdev Have you solved the problem? |
@withlin as I said yesterday: "Deleting /var/lib/cni/ solves the problem". =) |
ok. i think that you can close the issue. tks. |
@withlin Shouldn't this information be added in the documentation to prevent future issues like this one? |
@staticdev yes, that'd make a nice PR. |
Docs is probably good enough, though I feel we should be able to remove the need for a docs change here with some code adjustments. Ideally a I think the following sounds like a reasonable solution:
|
The reason for this happening is that the pod has no tolerations for running on master nodes. |
In case someone else stumbles upon this thread, I have found this can happen when you rename your host after installing parts of K8s subsystem. I have a calico node getting "assigned" to a node that is the name of my host before it was renamed ....while others are running on the node with the new name. (Microk8s is my distro) |
2024 same issue
I removed all taints, restarted kubeled, containerd everything, reset kubeadm removed cni0 did all the things but same result |
@ammirator-administrator what version do you use? Could you perhaps start this as a fresh issue? |
I am following the documentation on https://docs.projectcalico.org/v3.8/getting-started/kubernetes/ and pods are pending forever.
Expected Behavior
All pods running in
watch kubectl get pods --all-namespaces
.Current Behavio
When I
watch kubectl get pods --all-namespaces
over 15 minutes, it is still pending:Possible Solution
I don't know why, but version 3.5 just works:
I've tried 3.6, 3.7 and 3.8, with the same results.
Steps to Reproduce (for bugs)
Context
I can't create a cluster with newer versions of Calico.
Your Environment
The text was updated successfully, but these errors were encountered: