Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

"kubectl get cells" shows cells from past incarnations of Controller #82

Closed
myechuri opened this issue May 8, 2020 · 3 comments
Closed

Comments

@myechuri
Copy link
Contributor

myechuri commented May 8, 2020

Setup: minikube on MacBook with VK shipping pods to GCE.

kubectl get pods does not show any nginx pods, but kubectl get cells shows nginx-deployment-66f967f649-9b8zj.

Madhuris-MacBook-Pro:gke myechuri$ kubectl get nodes
NAME              STATUS   ROLES    AGE   VERSION
m01               Ready    master   58d   v1.17.3
virtual-kubelet   Ready    agent    57d   
Madhuris-MacBook-Pro:gke myechuri$ kubectl get pods --all-namespaces
NAMESPACE     NAME                               READY   STATUS             RESTARTS   AGE
kube-system   aws-node-8glnz                     0/1     ImagePullBackOff   0          57d
kube-system   coredns-6955765f44-8ljwj           1/1     Running            4          58d
kube-system   coredns-6955765f44-kkbtq           1/1     Running            4          58d
kube-system   etcd-m01                           1/1     Running            4          58d
kube-system   kube-apiserver-m01                 1/1     Running            4          58d
kube-system   kube-controller-manager-m01        1/1     Running            11         58d
kube-system   kube-proxy-qx9rr                   1/1     Running            0          14d
kube-system   kube-proxy-ww8d9                   1/1     Running            4          58d
kube-system   kube-scheduler-m01                 1/1     Running            11         58d
kube-system   registry-creds-xth4n               1/1     Running            4          57d
kube-system   storage-provisioner                1/1     Running            6          58d
kube-system   virtual-kubelet-6685c575f9-hvdwd   1/1     Running            0          10m
Madhuris-MacBook-Pro:gke myechuri$ kubectl get cells
NAME                                   POD NAME                            POD NAMESPACE   NODE              LAUNCH TYPE   INSTANCE TYPE   INSTANCE ID                                                 IP
0dfd31cf-8d5a-494d-98e7-d8de2f4b4c9d   kube-proxy-rjspt                    kube-system     virtual-kubelet   On-Demand     t3.nano         i-0650833d57a649efd                                         172.31.74.58
373b3a15-a5e7-4715-9ead-c74d64cd5e84   kube-proxy-rjspt                    kube-system     virtual-kubelet   On-Demand     t3.nano         i-0f11c8ae89280f63e                                         172.31.74.83
61c44374-33aa-4636-8d09-23479025c3b4   kube-proxy-rjspt                    kube-system     virtual-kubelet   On-Demand     t3.nano         i-005c2fb81f90a1ede                                         172.31.77.142
66e5f0da-7f74-4a17-ac9b-fe6234ed8369   nginx-deployment-66f967f649-9b8zj   default         virtual-kubelet   On-Demand     t3.nano         i-0ec6eabbae4ca4c6b                                         172.31.72.109
9543d72d-3325-4356-8df0-c9067435b295   kube-proxy-qx9rr                    kube-system     virtual-kubelet   On-Demand     t3.nano         i-0df366d083e4d1507                                         172.31.68.151
ca2f2682-0a60-40e8-96f5-98a0972d3e61   kube-proxy-qx9rr                    kube-system     virtual-kubelet   On-Demand     e2-small        kip-lbom5vl74zhfzolegk3sbzkqiu-zixsnaqkmbaorfxvtcqjolj6me   10.168.15.202

Cell for nginx-deployment-66f967f649-9b8zj is listed with instance id i-0ec6eabbae4ca4c6b which corresponds to a cell running on AWS from a previous incarnation of vk which shipped pods to AWS.

Screen Shot 2020-05-08 at 4 51 22 PM

Verified that the current setup is indeed shipping pods to GCE and not AWS:

Madhuris-MacBook-Pro:gke myechuri$ kubectl create deployment nginx --image=nginx
deployment.apps/nginx created
Madhuris-MacBook-Pro:gke myechuri$ kubectl get pods
NAME                     READY   STATUS    RESTARTS   AGE
nginx-86c57db685-mzbjf   0/1     Pending   0          4s
Madhuris-MacBook-Pro:gke myechuri$ kubectl get cells
NAME                                   POD NAME                            POD NAMESPACE   NODE              LAUNCH TYPE   INSTANCE TYPE   INSTANCE ID                                                 IP
0dfd31cf-8d5a-494d-98e7-d8de2f4b4c9d   kube-proxy-rjspt                    kube-system     virtual-kubelet   On-Demand     t3.nano         i-0650833d57a649efd                                         172.31.74.58
373b3a15-a5e7-4715-9ead-c74d64cd5e84   kube-proxy-rjspt                    kube-system     virtual-kubelet   On-Demand     t3.nano         i-0f11c8ae89280f63e                                         172.31.74.83
61c44374-33aa-4636-8d09-23479025c3b4   kube-proxy-rjspt                    kube-system     virtual-kubelet   On-Demand     t3.nano         i-005c2fb81f90a1ede                                         172.31.77.142
66e5f0da-7f74-4a17-ac9b-fe6234ed8369   nginx-deployment-66f967f649-9b8zj   default         virtual-kubelet   On-Demand     t3.nano         i-0ec6eabbae4ca4c6b                                         172.31.72.109
9543d72d-3325-4356-8df0-c9067435b295   kube-proxy-qx9rr                    kube-system     virtual-kubelet   On-Demand     t3.nano         i-0df366d083e4d1507                                         172.31.68.151
d3abeb7c-243a-465d-8386-99fbbe05b7a7   kube-proxy-qx9rr                    kube-system     virtual-kubelet   On-Demand     e2-small        kip-lbom5vl74zhfzolegk3sbzkqiu-2ov6w7behjdf3a4gth534bnxu4   
ea3085c8-63c7-4cd6-9500-a946e8cc0480   nginx-86c57db685-mzbjf              default         virtual-kubelet   On-Demand     e2-small        kip-lbom5vl74zhfzolegk3sbzkqiu-5iyilsddy5gnnfiavfdortaeqa 
@myechuri
Copy link
Contributor Author

myechuri commented May 9, 2020

ControllerID of current incarnation:

Madhuris-MacBook-Pro:gke myechuri$ kubectl -n kube-system logs virtual-kubelet-6685c575f9-hvdwd | grep -i controllerid
I0508 23:33:13.759699       1 server.go:212] ControllerID: lbom5vl74zhfzolegk3sbzkqiu

ControllerID of the cells reported: 2qs4mvn2pza6teedd6hfz5hiky

@ldx
Copy link
Contributor

ldx commented May 16, 2020

Isn't this the same as #44?

@ldx ldx added the GCE label May 21, 2020
@justnoise justnoise removed the GCE label May 28, 2020
@justnoise justnoise changed the title GCE: "kubectl get cells" shows cells from past incarnations of Controller "kubectl get cells" shows cells from past incarnations of Controller May 28, 2020
@justnoise
Copy link
Contributor

Closing. Duplicate of #44.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants