Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Kubelet --cloud-provider tag is not Supported on GCE with the CNI Plugin #35

Closed
denverwilliams opened this issue Jun 18, 2017 · 4 comments

Comments

@denverwilliams
Copy link
Contributor

commented Jun 18, 2017

When kubelet has the Cloud Provider Tag set on GCE and the CNI Plugin is being used the kube-controller-manager RouteController attempts to create routes, instead of delegating the responsibility to the CNI network plugin, making nodes unavailable for scheduling.

bash-4.3# kubectl describe pods kube-dns-v20-mzrgb --namespace=kube-system
Name:		kube-dns-v20-mzrgb
Namespace:	kube-system
Node:		/
Labels:		k8s-app=kube-dns
		version=v20
Status:		Pending
Tolerations:	CriticalAddonsOnly=:Exists
Events:
  FirstSeen	LastSeen	Count	From			SubObjectPath	Type		Reason			Message
  ---------	--------	-----	----			-------------	--------	------			-------
  50s		19s		7	{default-scheduler }			Warning		FailedScheduling	no nodes available to schedule pods

@denverwilliams denverwilliams created this issue from a note in cross-cloud ci v0.1.0 (Backlog ) Jun 18, 2017

@denverwilliams denverwilliams self-assigned this Jun 18, 2017

@denverwilliams

This comment has been minimized.

Copy link
Contributor Author

commented Jun 18, 2017

Looks like this is the related issue on Kubernetes kubernetes/kubernetes#44254

@denverwilliams

This comment has been minimized.

Copy link
Contributor Author

commented Jun 18, 2017

Looks like this is a unmerged fix https://github.com/kubernetes/kubernetes/pull/33573/files
@hh might be worth integrating this into our builds, but lets not do that yet.

@denverwilliams

This comment has been minimized.

Copy link
Contributor Author

commented Jun 19, 2017

Temporary work around for now is not to use the cloud-provider flag

bash-4.3# kubectl get nodes
NAME                                  STATUS                     AGE
test-master1.c.test-163823.internal   Ready,SchedulingDisabled   15s
test-master2.c.test-163823.internal   Ready,SchedulingDisabled   13s
test-worker1.c.test-163823.internal   Ready                      16s
test-worker2.c.test-163823.internal   Ready                      16s
test-worker3.c.test-163823.internal   Ready                      10s
bash-4.3# kubectl create -f ../kube-dns.yml 
service "kube-dns" created
replicationcontroller "kube-dns-v20" created
bash-4.3# kubectl get nodes
NAME                                  STATUS                     AGE
test-master1.c.test-163823.internal   Ready,SchedulingDisabled   29s
test-master2.c.test-163823.internal   Ready,SchedulingDisabled   27s
test-master3.c.test-163823.internal   Ready,SchedulingDisabled   12s
test-worker1.c.test-163823.internal   Ready                      30s
test-worker2.c.test-163823.internal   Ready                      30s
test-worker3.c.test-163823.internal   Ready                      24s
bash-4.3# kubectl get pods --namespace=kube-system
NAME                                                          READY     STATUS    RESTARTS   AGE
kube-apiserver-test-master1.c.test-163823.internal            1/1       Running   0          40s
kube-apiserver-test-master2.c.test-163823.internal            1/1       Running   0          39s
kube-apiserver-test-master3.c.test-163823.internal            1/1       Running   0          30s
kube-controller-manager-test-master1.c.test-163823.internal   1/1       Running   0          40s
kube-dns-v20-w1b9j                                            3/3       Running   0          15s
kube-proxy-test-master2.c.test-163823.internal                1/1       Running   0          39s
kube-proxy-test-worker1.c.test-163823.internal                1/1       Running   0          39s
kube-scheduler-test-master3.c.test-163823.internal            1/1       Running   0          30s

@hh hh removed this from Backlog in cross-cloud ci v0.1.0 Jul 5, 2017

@taylor

This comment has been minimized.

Copy link
Contributor

commented Nov 15, 2017

Fixed in 127b57f

@taylor taylor closed this Nov 15, 2017

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
2 participants
You can’t perform that action at this time.