Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Kubernetes stuck in "starting..." state after 18.02.0-ce-rc1-mac50 upgrade #2536

Closed
jarlefosen opened this issue Jan 30, 2018 · 49 comments
Closed

Comments

@jarlefosen
Copy link

Expected behavior

Docker Engine and Kubernetes should start up within a reasonable time.

Actual behavior

It's been 15 minutes and Kubernetes hasn't yet started up
Docker Engine started as expected.

Information

I'm on the Edge release and I just upgraded to 18.02.0-ce-rc1-mac50 when the issue started.

Diagnose output:

Docker for Mac: version: 18.02.0-ce-rc1-mac50 (e1912a8c46ae6f60a85d277f7b8d210dc9670b28)
macOS: version 10.12.6 (build: 16G1114)
logs: /tmp/5F73B1D4-3EBF-408F-95D6-E28BDE7A7CEF/20180130-131929.tar.gz
[OK]     db.git
[OK]     vmnetd
[OK]     dns
[OK]     driver.amd64-linux
[OK]     virtualization VT-X
[OK]     app
[OK]     moby
[OK]     system
[OK]     moby-syslog
[OK]     kubernetes
[OK]     env
[OK]     virtualization kern.hv_support
[OK]     slirp
[OK]     osxfs
[OK]     moby-console
[OK]     logs
[OK]     docker-cli
[OK]     menubar
[OK]     disk

Steps to reproduce the behavior

Not sure if reproducible, but these are the steps I went through.

  1. Early January switch from Stable to Edge releases of Docker to get 17.12 (if I remember correctly)
  2. Enable Kubernetes
  3. ... wait until today
  4. Upgrade Docker to 18.02
@akimd
Copy link
Contributor

akimd commented Jan 30, 2018

There's a lot to download. Is your connection good?

@jarlefosen
Copy link
Author

Yes, the connection here should be well above 200Mbps, so that shouldn't be the issue.

It's still in a starting state.

image

@akimd
Copy link
Contributor

akimd commented Jan 30, 2018

yes, actually looking at your logs the problem might be elsewhere.
Could you please look at your ~/.kube/config and pay special attention to the context? You should have something like

- context:
    cluster: docker-for-desktop-cluster
    user: docker-for-desktop
  name: docker-for-desktop
current-context: docker-for-desktop

and maybe you have something like this instead as a last line:

current-context: docker-for-desktop-context

Try removing the -context.

@akimd akimd self-assigned this Jan 30, 2018
@jarlefosen
Copy link
Author

jarlefosen commented Jan 30, 2018

Current context is something work related, a different provider. Switching to the local instance I see the expected outcome:

$ cat config | grep current-context
current-context: docker-for-desktop

As for the complete docker-for-desktop config itself:

- context:
    cluster: docker-for-desktop-cluster
    user: docker-for-desktop
  name: docker-for-desktop

kubectl isn't able to connect to the instance either so it's not an issue with the indicator in the status bar.

@akimd
Copy link
Contributor

akimd commented Jan 30, 2018

But I fear that your Docker is trying to use the wrong context. Your logs have

error: context "docker-for-desktop-context" does not exist

And it's probably in the way of Docker considering that Kube is ok.

What does your Kube item in the Docker menu display?

@MrMic
Copy link

MrMic commented Jan 30, 2018

Hi,

Same issue for me with the last update !
The app tells me that something seems wrong with Compose.

Version 18.02.0-ce-rc1-mac50 (22256)

@guillaumerose
Copy link
Contributor

Most of the time, sadly, the best is to do a factory reset. We currently have an issue with upgrading the Compose integration for Kubernetes.

You can also try to take a look at the kubelet logs : docker run -it --privileged --pid=host justincormack/nsenter1 /bin/sh -c "cat /var/log/kubelet.err.log". You should see kubeadm and kubelet logs.

@MrMic
Copy link

MrMic commented Jan 31, 2018

That does NOT solve anything.
I have removed completely Docker and reinstall it.
Kubernetes still do NOT start!
The kubelet.err.log is:
--- 8< -------------

I0131 10:04:34.458301    1193 kubelet_node_status.go:83] Attempting to register node docker-for-desktop
E0131 10:04:34.458760    1193 kubelet_node_status.go:107] Unable to register node "docker-for-desktop" with API server: Post https://192.168.65.3:6443/api/v1/nodes: dial tcp 192.168.65.3:6443: getsockopt: connection refused
E0131 10:04:35.403894    1193 reflector.go:205] k8s.io/kubernetes/pkg/kubelet/config/apiserver.go:47: Failed to list *v1.Pod: Get https://192.168.65.3:6443/api/v1/pods?fieldSelector=spec.nodeName%3Ddocker-for-desktop&resourceVersion=0: dial tcp 192.168.65.3:6443: getsockopt: connection refused
E0131 10:04:35.403894    1193 reflector.go:205] k8s.io/kubernetes/pkg/kubelet/kubelet.go:422: Failed to list *v1.Node: Get https://192.168.65.3:6443/api/v1/nodes?fieldSelector=metadata.name%3Ddocker-for-desktop&resourceVersion=0: dial tcp 192.168.65.3:6443: getsockopt: connection refused
E0131 10:04:35.404825    1193 reflector.go:205] k8s.io/kubernetes/pkg/kubelet/kubelet.go:413: Failed to list *v1.Service: Get https://192.168.65.3:6443/api/v1/services?resourceVersion=0: dial tcp 192.168.65.3:6443: getsockopt: connection refused
E0131 10:04:36.405465    1193 reflector.go:205] k8s.io/kubernetes/pkg/kubelet/config/apiserver.go:47: Failed to list *v1.Pod: Get https://192.168.65.3:6443/api/v1/pods?fieldSelector=spec.nodeName%3Ddocker-for-desktop&resourceVersion=0: dial tcp 192.168.65.3:6443: getsockopt: connection refused
E0131 10:04:36.405721    1193 reflector.go:205] k8s.io/kubernetes/pkg/kubelet/kubelet.go:422: Failed to list *v1.Node: Get https://192.168.65.3:6443/api/v1/nodes?fieldSelector=metadata.name%3Ddocker-for-desktop&resourceVersion=0: dial tcp 192.168.65.3:6443: getsockopt: connection refused
E0131 10:04:36.407611    1193 reflector.go:205] k8s.io/kubernetes/pkg/kubelet/kubelet.go:413: Failed to list *v1.Service: Get https://192.168.65.3:6443/api/v1/services?resourceVersion=0: dial tcp 192.168.65.3:6443: getsockopt: connection refused
E0131 10:04:37.406896    1193 reflector.go:205] k8s.io/kubernetes/pkg/kubelet/config/apiserver.go:47: Failed to list *v1.Pod: Get https://192.168.65.3:6443/api/v1/pods?fieldSelector=spec.nodeName%3Ddocker-for-desktop&resourceVersion=0: dial tcp 192.168.65.3:6443: getsockopt: connection refused
E0131 10:04:37.408009    1193 reflector.go:205] k8s.io/kubernetes/pkg/kubelet/kubelet.go:422: Failed to list *v1.Node: Get https://192.168.65.3:6443/api/v1/nodes?fieldSelector=metadata.name%3Ddocker-for-desktop&resourceVersion=0: dial tcp 192.168.65.3:6443: getsockopt: connection refused
E0131 10:04:37.409205    1193 reflector.go:205] k8s.io/kubernetes/pkg/kubelet/kubelet.go:413: Failed to list *v1.Service: Get https://192.168.65.3:6443/api/v1/services?resourceVersion=0: dial tcp 192.168.65.3:6443: getsockopt: connection refused
E0131 10:04:37.433355    1193 eviction_manager.go:238] eviction manager: unexpected err: failed to get node info: node 'docker-for-desktop' not found
E0131 10:04:38.408206    1193 reflector.go:205] k8s.io/kubernetes/pkg/kubelet/config/apiserver.go:47: Failed to list *v1.Pod: Get https://192.168.65.3:6443/api/v1/pods?fieldSelector=spec.nodeName%3Ddocker-for-desktop&resourceVersion=0: dial tcp 192.168.65.3:6443: getsockopt: connection refused
E0131 10:04:38.408492    1193 reflector.go:205] k8s.io/kubernetes/pkg/kubelet/kubelet.go:422: Failed to list *v1.Node: Get https://192.168.65.3:6443/api/v1/nodes?fieldSelector=metadata.name%3Ddocker-for-desktop&resourceVersion=0: dial tcp 192.168.65.3:6443: getsockopt: connection refused
E0131 10:04:38.409765    1193 reflector.go:205] k8s.io/kubernetes/pkg/kubelet/kubelet.go:413: Failed to list *v1.Service: Get https://192.168.65.3:6443/api/v1/services?resourceVersion=0: dial tcp 192.168.65.3:6443: getsockopt: connection refused

---- 8< ------------------------- >8 ----
And now ??
You release something that you even try on a Mac OSX?
Seriously, you will have the same issue, you too ...

@jarlefosen
Copy link
Author

I had previously allocated 2GB RAM and 2 CPU cores to the Docker Engine but needed to increase these limits today because a project needed more resources. This of course triggers a restart of Docker Engine and Kubernetes, but now nothing starts up.

For reference here's the logs and it's ID

Docker for Mac: version: 18.02.0-ce-rc1-mac50 (e1912a8c46ae6f60a85d277f7b8d210dc9670b28)
macOS: version 10.12.6 (build: 16G1114)
logs: /tmp/5F73B1D4-3EBF-408F-95D6-E28BDE7A7CEF/20180131-123457.tar.gz
failure: docker ps failed: (Failure "docker ps: timeout after 10.00s")
[OK]     db.git
[OK]     vmnetd
[OK]     dns
[OK]     driver.amd64-linux
[OK]     virtualization VT-X
[OK]     app
[OK]     moby
[OK]     system
[OK]     moby-syslog
[OK]     kubernetes
[OK]     env
[OK]     virtualization kern.hv_support
[OK]     slirp
[OK]     osxfs
[OK]     moby-console
[OK]     logs
[ERROR]  docker-cli
         docker ps failed
[OK]     menubar
[OK]     disk

I was about to do a complete reinstall of Docker when I noticed the issue but found that it didn't actually block my work; so I decided to keep it installed in case it would lead to some bug fixes.

@akimd: The Kube item (context I presume?) Displays all my gke_X environments as well as docker-for-desktop, where the gke_<project> is selected. I've tried swapping back and forth between these environments without any luck.

Now I will proceed to reinstall Docker for Mac, and hopefully the logs I've uploaded can help with discovering the upgrade issues.

@MrMic: Your comment is visually easier to comprehend if you surround the logs with the three ```

@jarlefosen
Copy link
Author

Aha, I was not aware of the Factory reset button inside the preferences and proceeded to remove the app from Applications/ before installing it again. Not entirely sure how one is supposed to actually uninstall apps from a Mac.

Anyways, once "reinstalled" Edge the naive way I still got the same issues as described above. Currently trying to do a factory reset from within the application, but it doesn't seem to be doing anything.

image

I tried quitting Docker mid-process since it took so long (5+ minutes without any changes) and then starting again. Nothing new happened.

I decided to open up the advanced features and click reset daemon settings in hopes that it would trigger some other resets and hopefully be able to start up again. No luck.

Last option I see is to try and do a Uninstall from within the preferences menu. Clicked it, looks a bit stuck as well. Could it be because Docker and Kubernetes aren't starting properly, or at least Docker Engine?
I chose a bold move and force quit the hyperkit process while the uninstall window was open, since it looked like it was stuck. That gave me a message almost immediately stating the uninstall was successful. Well, that was unexpected.

Now what? Well, let's install Docker Edge once again and see how it looks.

image

It asked for privileges, and asks me to log in with my Docker ID. That feels just about right! And look at that, Docker Engine is running! 🐳

image

Diagnose looks good. But it did initially as well when there were Kube issues. Let's check that commands are working properly.
Still looking good.

$ docker ps -a
CONTAINER ID        IMAGE               COMMAND             CREATED             STATUS              PORTS               NAMES

image

Kubernetes also works!

How I was able to re-install Docker

Go to Docker preferences, click Reset -> Uninstall. While the bar is stuck, open Activity Monitor and force quit hyperkit.

I wrote this as I went along, luckily it ended with a fixed issue. Not sure what went wrong along the way, but this is how I fixed my issue at least.

@MrMic
Copy link

MrMic commented Jan 31, 2018

I follow what @jarlefosen has written and finally, all is up: Docker + Kubernetes.
Thank you!
I need now to reinstall all my containers now ...

Rgds,

@akimd
Copy link
Contributor

akimd commented Feb 1, 2018

Hi guys,

It does appear that something was wrong in the way we did the diverse "Resets", and it could spin endlessly. The next Edge will do much better in this area.

I'm happy you managed to solve your issue. To bad though we could be pinpointed what it was.

@jarlefosen
Copy link
Author

Thank you for following up on this issue @akimd 👏

Feel free to close this issue, but I think it's fine to have it open and visible until the next Edge release in case more people are affected by this and want to continue the conversation.

@MrMic
Copy link

MrMic commented Feb 2, 2018

I concur too.

@akimd
Copy link
Contributor

akimd commented Feb 2, 2018

rc2 has just been released!

@akimd akimd closed this as completed Feb 2, 2018
@ajeetraina
Copy link

ajeetraina commented Feb 2, 2018 via email

@Danceiny
Copy link

Due to China's network environment, in this case, we can refer to this solution: https://github.com/maguowei/k8s-docker-for-mac

@shidongwa
Copy link

shidongwa commented Apr 30, 2018

it still doesn‘t work in step 4. @Danceiny

check the err log in the way:
docker run -it --privileged --pid=host justincormack/nsenter1 /bin/sh -c "cat /var/log/kubelet.err.log"

I have error message as below:

E0430 13:25:24.853253 2025 kubelet_node_status.go:106] Unable to register node "docker-for-desktop" with API server: Post https://192.168.65.3:6443/api/v1/nodes: dial tcp 192.168.65.3:6443: getsockopt: connection refused

@Danceiny
Copy link

Danceiny commented May 3, 2018

Emmmm, I think the best way should be configuring the docker proxies, I set manual proxy configuraiton, both http and https, point to 127.0.0.1:1080 (on this endpoint I opened http proxy by ssr). Actually I used both two ways(k8s-docker-for-mac and proxy) and finally I succeeded. @shidongwa

@shidongwa
Copy link

shidongwa commented May 19, 2018

thanks @Danceiny , it works for me using solution of minikube, ssr, privoxy and hyberkit. I document the steps at http://shidonghua.work/2018/05/01/mac-minikube-install/

@david618
Copy link

I had installed Docker 18.05.0 CE for Mac stable then decided to install edge; so I could try Kubernetes. I uninstalled Stable then installed Edge. I was seeing the same issue Kubernetes was stuck in state "Starting". Waited for 30+ minutes no change. Solution was to uninstall; restart my Mac; then re-install Docker Edge. After this I was able to start Kubernetes and it started in a couple of mins.

@MrMic
Copy link

MrMic commented Jun 12, 2018

Yes ! I have upgraded today to the latest version of Kubernetes edge and another time
it remains stuck in "Kubernetes starting".
I need to reset to factory. (No need to restart computer => see reset button at the
right of UI, and options in this panel)
One more time !!!!!

@eysi09
Copy link

eysi09 commented Jun 25, 2018

I had the exact same issue, namely Kubernetes being stuck in the "Kubernetes is starting..." state after upgrading to 18.05.0-ce-mac67 (25042).

What solved it for me was: Preferences => Reset => Reset Kubernetes cluster.

However, after that I got The connection to the server localhost:6443 was refused - did you specify the right host or port? when trying to use Kubectl. Simply restarting Docker for Mac from the main menu fixed that and now everything is back up and running.

So to summarize:

  1. Click on the Docker icon, select Preferences, select Reset, click on Reset Kubernetes cluster
  2. Click on the Docker icon, click on Restart

Hope this helps in other cases as well.

@massenz
Copy link

massenz commented Jun 29, 2018

FWIW, this is still happening (it may be indeed related to switching the current-context, but not really clear why):

Version 18.05.0-ce-mac67 (25042)

Resetting with the "nuclear option" and reinstalling seems to fix this; switching context and then back seems to "work as intended."

I believe that the root cause is related to current-context pointing to something other than docker-for-desktop during an update (or a restart?) then Docker for Mac gets confused and tries to install pods to the wrong cluster.

To verify this, after K8s started successfully, I switched context to another (work-related) cluster; did a Docker/Restart and observed the "Kubernetes starting..." hanging there for quite a while - switched back to the docker-for-desktop context, executed another restart, and now K8s is happily "Running".

My suggestion would be to execute a check on the current context upon install/update/restart and warn the user (a "forced" context switch may be undesirable, perhaps).

@WegDamit
Copy link

For me internet connection seemed to cause this issue.
The corporate firewall uses a squid proxy and a proxy pac and ... and kubernetes won't start. (Browsing etc. worked fine, just dockers kubernets not...)
switching to the open guest wlan solved this issue.

Not conclusive, as i reinstalled and did everything else in here before, but perhaps worth a try before trying the time consuming stuff above...

@marcellodesales
Copy link

Solution for 18.06 edge

  • I tried to restart
  • I tried switching context and restarting
    • At this point, docker ps would show only 2 containers running
    • cluster-info shows EOF
$ kubectl cluster-info
Kubernetes master is running at https://localhost:6443

To further debug and diagnose cluster problems, use 'kubectl cluster-info dump'.
Unable to connect to the server: EOF

$ docker ps
CONTAINER ID        IMAGE                        COMMAND                  CREATED              STATUS              PORTS               NAMES
94464862fb39        353b8f1d102e                 "kube-scheduler --ad…"   8 seconds ago        Up 7 seconds                            k8s_kube-scheduler_kube-scheduler-docker-for-desktop_kube-system_ea66a171667ec4aaf1b274428a42a7cf_1
5f4b58cb1733        k8s.gcr.io/pause-amd64:3.1   "/pause"                 About a minute ago   Up About a minute                       k8s_POD_kube-scheduler-docker-for-desktop_kube-system_ea66a171667ec4aaf1b274428a42a7cf_0
  • Nothing worked until I clicked in Preferences -> Reset -> Reset disk images
    • Then this resolved the problem, and I could see a huge list of docker containers
    • cluster-info returns a correct response

screen shot 2018-08-28 at 5 53 41 pm

$ kubectl cluster-info
Kubernetes master is running at https://localhost:6443
KubeDNS is running at https://localhost:6443/api/v1/namespaces/kube-system/services/kube-dns:dns/proxy

To further debug and diagnose cluster problems, use 'kubectl cluster-info dump'.

$ docker ps
CONTAINER ID        IMAGE                                      COMMAND                  CREATED             STATUS              PORTS               NAMES
1132bcbf3632        k8s.gcr.io/pause-amd64:3.1                 "/pause"                 2 seconds ago       Up 1 second                             k8s_POD_compose-api-6fbc44c575-lhm5r_docker_e0c9f7d5-ab25-11e8-8990-025000000001_0
4a5aa2e23a52        k8s.gcr.io/pause-amd64:3.1                 "/pause"                 3 seconds ago       Up 2 seconds                            k8s_POD_compose-7447646cf5-4jm8f_docker_e0cbf8c8-ab25-11e8-8990-025000000001_0
4af077f25a08        k8s.gcr.io/pause-amd64:3.1                 "/pause"                 3 seconds ago       Up 2 seconds                            k8s_POD_kube-proxy-w6zrb_kube-system_e0ce4a98-ab25-11e8-8990-025000000001_0
2e49b6e61708        k8s.gcr.io/pause-amd64:3.1                 "/pause"                 3 seconds ago       Up 2 seconds                            k8s_POD_kube-dns-86f4d74b45-q8ktw_kube-system_e0ca062d-ab25-11e8-8990-025000000001_0
eb2dcf80a676        k8s.gcr.io/etcd-amd64                      "etcd --client-cert-…"   22 seconds ago      Up 21 seconds                           k8s_etcd_etcd-docker-for-desktop_kube-system_6eb852021af3eb57c844b7b784a15d0c_0
a6d44c71f20c        k8s.gcr.io/kube-controller-manager-amd64   "kube-controller-man…"   30 seconds ago      Up 29 seconds                           k8s_kube-controller-manager_kube-controller-manager-docker-for-desktop_kube-system_df2760518b28fa45fd2f8ce7dd46edc2_0
af765142c5d9        k8s.gcr.io/kube-scheduler-amd64            "kube-scheduler --ku…"   36 seconds ago      Up 35 seconds                           k8s_kube-scheduler_kube-scheduler-docker-for-desktop_kube-system_6d5c9cb98205e46b85b941c8a44fc236_0
4c59af1c2cb2        k8s.gcr.io/kube-apiserver-amd64            "kube-apiserver --ad…"   40 seconds ago      Up 39 seconds                           k8s_kube-apiserver_kube-apiserver-docker-for-desktop_kube-system_a2d554b619ce823891ab5852be174add_0
5adbdf5dd774        k8s.gcr.io/pause-amd64:3.1                 "/pause"                 46 seconds ago      Up 45 seconds                           k8s_POD_etcd-docker-for-desktop_kube-system_6eb852021af3eb57c844b7b784a15d0c_0
80a0d57c1f6c        k8s.gcr.io/pause-amd64:3.1                 "/pause"                 47 seconds ago      Up 45 seconds                           k8s_POD_kube-scheduler-docker-for-desktop_kube-system_6d5c9cb98205e46b85b941c8a44fc236_0
f37395fca38d        k8s.gcr.io/pause-amd64:3.1                 "/pause"                 47 seconds ago      Up 45 seconds                           k8s_POD_kube-controller-manager-docker-for-desktop_kube-system_df2760518b28fa45fd2f8ce7dd46edc2_0
a029e881f986        k8s.gcr.io/pause-amd64:3.1                 "/pause"                 47 seconds ago      Up 45 seconds                           k8s_POD_kube-apiserver-docker-for-desktop_kube-system_a2d554b619ce823891ab5852be174add_0

@nabadger
Copy link

nabadger commented Sep 1, 2018

Same issue on18.06.1-ce-mac74 (26766)

Solution as above, although I actually actually did Preferences -> Reset -> Reset Kubernetes Cluster

@nanodocumet
Copy link

Running Version 18.06.1-ce-mac73 (26764) - channel: Stable. Wondering, is there a way to reset the kubernetes cluster using a command line from the terminal? Is the interface the only way?
Thanks!

@tonyxiao
Copy link

Still having this issue as of

image

@tsuna
Copy link
Contributor

tsuna commented Nov 8, 2018

Also kept having a similar looking issue (with Version 2.0.0.0-beta1-mac75 (27117)) and had to click the Reset Kubernetes cluster button in the Reset panel of the Docker for Mac settings.

edit: k8s apiserver logs from before the reset (not sure that'll shed any light on what was b0rked):

$ docker logs --follow 87170c4cc86e
Flag --admission-control has been deprecated, Use --enable-admission-plugins or --disable-admission-plugins instead. Will be removed in a future version.
Flag --insecure-port has been deprecated, This flag will be removed in a future version.
I1108 23:20:55.106027       1 server.go:135] Version: v1.10.3
I1108 23:20:55.106148       1 server.go:724] external host was not specified, using 192.168.65.3
I1108 23:20:56.685954       1 feature_gate.go:190] feature gates: map[Initializers:true]
I1108 23:20:56.686005       1 initialization.go:90] enabled Initializers feature as part of admission plugin setup
I1108 23:20:56.686311       1 plugins.go:149] Loaded 10 admission controller(s) successfully in the following order: NamespaceLifecycle,LimitRanger,ServiceAccount,NodeRestriction,DefaultTolerationSeconds,DefaultStorageClass,MutatingAdmissionWebhook,Initializers,ValidatingAdmissionWebhook,ResourceQuota.
I1108 23:20:56.686936       1 plugins.go:149] Loaded 10 admission controller(s) successfully in the following order: NamespaceLifecycle,LimitRanger,ServiceAccount,NodeRestriction,DefaultTolerationSeconds,DefaultStorageClass,MutatingAdmissionWebhook,Initializers,ValidatingAdmissionWebhook,ResourceQuota.
I1108 23:20:56.690172       1 master.go:228] Using reconciler: master-count
W1108 23:20:56.761621       1 genericapiserver.go:342] Skipping API batch/v2alpha1 because it has no resources.
W1108 23:20:56.768078       1 genericapiserver.go:342] Skipping API rbac.authorization.k8s.io/v1alpha1 because it has no resources.
W1108 23:20:56.769009       1 genericapiserver.go:342] Skipping API storage.k8s.io/v1alpha1 because it has no resources.
W1108 23:20:56.775355       1 genericapiserver.go:342] Skipping API admissionregistration.k8s.io/v1alpha1 because it has no resources.
[restful] 2018/11/08 23:20:56 log.go:33: [restful/swagger] listing is available at https://192.168.65.3:6443/swaggerapi
[restful] 2018/11/08 23:20:56 log.go:33: [restful/swagger] https://192.168.65.3:6443/swaggerui/ is mapped to folder /swagger-ui/
[restful] 2018/11/08 23:20:57 log.go:33: [restful/swagger] listing is available at https://192.168.65.3:6443/swaggerapi
[restful] 2018/11/08 23:20:57 log.go:33: [restful/swagger] https://192.168.65.3:6443/swaggerui/ is mapped to folder /swagger-ui/
I1108 23:20:57.777190       1 plugins.go:149] Loaded 10 admission controller(s) successfully in the following order: NamespaceLifecycle,LimitRanger,ServiceAccount,NodeRestriction,DefaultTolerationSeconds,DefaultStorageClass,MutatingAdmissionWebhook,Initializers,ValidatingAdmissionWebhook,ResourceQuota.
I1108 23:20:59.904317       1 serve.go:96] Serving securely on [::]:6443
I1108 23:20:59.904950       1 available_controller.go:262] Starting AvailableConditionController
I1108 23:20:59.904970       1 cache.go:32] Waiting for caches to sync for AvailableConditionController controller
I1108 23:20:59.904989       1 controller.go:84] Starting OpenAPI AggregationController
I1108 23:20:59.905049       1 apiservice_controller.go:90] Starting APIServiceRegistrationController
I1108 23:20:59.905055       1 cache.go:32] Waiting for caches to sync for APIServiceRegistrationController controller
I1108 23:20:59.907285       1 crdregistration_controller.go:110] Starting crd-autoregister controller
I1108 23:20:59.907316       1 controller_utils.go:1019] Waiting for caches to sync for crd-autoregister controller
I1108 23:20:59.908055       1 crd_finalizer.go:242] Starting CRDFinalizer
I1108 23:20:59.908081       1 customresource_discovery_controller.go:174] Starting DiscoveryController
I1108 23:20:59.908094       1 naming_controller.go:276] Starting NamingConditionController
I1108 23:21:11.405354       1 trace.go:76] Trace[239266152]: "Create /api/v1/nodes" (started: 2018-11-08 23:21:01.425047889 +0000 UTC m=+6.385318018) (total time: 10.001444538s):
Trace[239266152]: [10.001444538s] [10.000967845s] END
I1108 23:21:28.417364       1 trace.go:76] Trace[2036799729]: "Create /api/v1/nodes" (started: 2018-11-08 23:21:18.415487655 +0000 UTC m=+23.396933167) (total time: 10.001841347s):
Trace[2036799729]: [10.001841347s] [10.001689995s] END
I1108 23:21:29.958550       1 trace.go:76] Trace[1158884613]: "Patch /api/v1/namespaces/default/events/docker-for-desktop.1565496140ffe87f" (started: 2018-11-08 23:20:59.977832594 +0000 UTC m=+4.938102711) (total time: 30.001846396s):
Trace[1158884613]: [30.001846396s] [30.001812067s] END
I1108 23:21:45.409360       1 trace.go:76] Trace[865451581]: "Create /api/v1/nodes" (started: 2018-11-08 23:21:35.428327916 +0000 UTC m=+40.409773453) (total time: 10.001776139s):
Trace[865451581]: [10.001776139s] [10.00165253s] END
E1108 23:21:59.888304       1 reflector.go:205] k8s.io/kubernetes/vendor/k8s.io/client-go/informers/factory.go:87: Failed to list *v1beta1.VolumeAttachment: the server was unable to return a response in the time allotted, but may still be processing the request (get volumeattachments.storage.k8s.io)
E1108 23:21:59.924505       1 reflector.go:205] k8s.io/kubernetes/vendor/k8s.io/client-go/informers/factory.go:87: Failed to list *v1.Endpoints: the server was unable to return a response in the time allotted, but may still be processing the request (get endpoints)
E1108 23:21:59.924629       1 reflector.go:205] k8s.io/kubernetes/pkg/client/informers/informers_generated/internalversion/factory.go:86: Failed to list *rbac.ClusterRole: the server was unable to return a response in the time allotted, but may still be processing the request (get clusterroles.rbac.authorization.k8s.io)
E1108 23:21:59.924743       1 reflector.go:205] k8s.io/kubernetes/vendor/k8s.io/client-go/informers/factory.go:87: Failed to list *v1.Service: the server was unable to return a response in the time allotted, but may still be processing the request (get services)
E1108 23:21:59.928597       1 reflector.go:205] k8s.io/kubernetes/vendor/k8s.io/client-go/informers/factory.go:87: Failed to list *v1.Namespace: the server was unable to return a response in the time allotted, but may still be processing the request (get namespaces)
E1108 23:21:59.928655       1 reflector.go:205] k8s.io/kubernetes/pkg/client/informers/informers_generated/internalversion/factory.go:86: Failed to list *core.Pod: the server was unable to return a response in the time allotted, but may still be processing the request (get pods)
E1108 23:21:59.928671       1 reflector.go:205] k8s.io/kubernetes/pkg/client/informers/informers_generated/internalversion/factory.go:86: Failed to list *storage.StorageClass: the server was unable to return a response in the time allotted, but may still be processing the request (get storageclasses.storage.k8s.io)
E1108 23:21:59.930755       1 reflector.go:205] k8s.io/kubernetes/pkg/client/informers/informers_generated/internalversion/factory.go:86: Failed to list *core.PersistentVolume: the server was unable to return a response in the time allotted, but may still be processing the request (get persistentvolumes)
E1108 23:21:59.930845       1 reflector.go:205] k8s.io/kubernetes/pkg/client/informers/informers_generated/internalversion/factory.go:86: Failed to list *rbac.ClusterRoleBinding: the server was unable to return a response in the time allotted, but may still be processing the request (get clusterrolebindings.rbac.authorization.k8s.io)
E1108 23:21:59.930889       1 reflector.go:205] k8s.io/kubernetes/pkg/client/informers/informers_generated/internalversion/factory.go:86: Failed to list *core.LimitRange: the server was unable to return a response in the time allotted, but may still be processing the request (get limitranges)
E1108 23:21:59.930932       1 reflector.go:205] k8s.io/kubernetes/vendor/k8s.io/apiextensions-apiserver/pkg/client/informers/internalversion/factory.go:74: Failed to list *apiextensions.CustomResourceDefinition: the server was unable to return a response in the time allotted, but may still be processing the request (get customresourcedefinitions.apiextensions.k8s.io)
E1108 23:21:59.930973       1 reflector.go:205] k8s.io/kubernetes/vendor/k8s.io/kube-aggregator/pkg/client/informers/internalversion/factory.go:74: Failed to list *apiregistration.APIService: the server was unable to return a response in the time allotted, but may still be processing the request (get apiservices.apiregistration.k8s.io)
E1108 23:21:59.931072       1 reflector.go:205] k8s.io/kubernetes/vendor/k8s.io/client-go/informers/factory.go:87: Failed to list *v1beta1.MutatingWebhookConfiguration: the server was unable to return a response in the time allotted, but may still be processing the request (get mutatingwebhookconfigurations.admissionregistration.k8s.io)
E1108 23:21:59.938348       1 reflector.go:205] k8s.io/kubernetes/pkg/client/informers/informers_generated/internalversion/factory.go:86: Failed to list *rbac.RoleBinding: the server was unable to return a response in the time allotted, but may still be processing the request (get rolebindings.rbac.authorization.k8s.io)
E1108 23:21:59.938640       1 reflector.go:205] k8s.io/kubernetes/pkg/client/informers/informers_generated/internalversion/factory.go:86: Failed to list *core.ServiceAccount: the server was unable to return a response in the time allotted, but may still be processing the request (get serviceaccounts)
E1108 23:21:59.940043       1 reflector.go:205] k8s.io/kubernetes/vendor/k8s.io/client-go/informers/factory.go:87: Failed to list *v1beta1.ValidatingWebhookConfiguration: the server was unable to return a response in the time allotted, but may still be processing the request (get validatingwebhookconfigurations.admissionregistration.k8s.io)
E1108 23:21:59.940048       1 reflector.go:205] k8s.io/kubernetes/pkg/client/informers/informers_generated/internalversion/factory.go:86: Failed to list *core.Secret: the server was unable to return a response in the time allotted, but may still be processing the request (get secrets)
E1108 23:21:59.941027       1 reflector.go:205] k8s.io/kubernetes/pkg/client/informers/informers_generated/internalversion/factory.go:86: Failed to list *rbac.Role: the server was unable to return a response in the time allotted, but may still be processing the request (get roles.rbac.authorization.k8s.io)
I1108 23:21:59.941765       1 trace.go:76] Trace[589626804]: "Patch /api/v1/namespaces/default/events/docker-for-desktop.1565496140fff06d" (started: 2018-11-08 23:21:29.962147109 +0000 UTC m=+34.943592636) (total time: 30.000386067s):
Trace[589626804]: [30.000386067s] [30.000313233s] END
E1108 23:21:59.944058       1 reflector.go:205] k8s.io/kubernetes/pkg/client/informers/informers_generated/internalversion/factory.go:86: Failed to list *core.ResourceQuota: the server was unable to return a response in the time allotted, but may still be processing the request (get resourcequotas)
E1108 23:22:00.866784       1 storage_rbac.go:157] unable to initialize clusterroles: the server was unable to return a response in the time allotted, but may still be processing the request (get clusterroles.rbac.authorization.k8s.io)
I1108 23:22:02.423570       1 trace.go:76] Trace[569609368]: "Create /api/v1/nodes" (started: 2018-11-08 23:21:52.422488284 +0000 UTC m=+57.424722395) (total time: 10.001040147s):
Trace[569609368]: [10.001040147s] [10.000928175s] END
I1108 23:22:10.848848       1 trace.go:76] Trace[839106661]: "Create /api/v1/namespaces" (started: 2018-11-08 23:22:00.868208993 +0000 UTC m=+65.870443128) (total time: 10.001631581s):
Trace[839106661]: [10.001631581s] [10.001139739s] END
E1108 23:22:10.849859       1 client_ca_hook.go:78] namespaces "kube-system" is forbidden: not yet ready to handle request
E1108 23:22:14.429350       1 customresource_discovery_controller.go:177] timed out waiting for caches to sync
I1108 23:22:14.429383       1 customresource_discovery_controller.go:178] Shutting down DiscoveryController
I1108 23:22:14.429407       1 crd_finalizer.go:246] Shutting down CRDFinalizer
I1108 23:22:14.429431       1 naming_controller.go:280] Shutting down NamingConditionController
E1108 23:22:14.429517       1 controller_utils.go:1022] Unable to sync caches for crd-autoregister controller
E1108 23:22:14.429544       1 cache.go:35] Unable to sync caches for AvailableConditionController controller
E1108 23:22:14.429564       1 cache.go:35] Unable to sync caches for APIServiceRegistrationController controller
I1108 23:22:14.429609       1 controller.go:90] Shutting down OpenAPI AggregationController
I1108 23:22:14.429766       1 serve.go:136] Stopped listening on [::]:6443
I1108 23:22:14.430880       1 crdregistration_controller.go:115] Shutting down crd-autoregister controller
I1108 23:22:14.432058       1 available_controller.go:266] Shutting down AvailableConditionController
I1108 23:22:14.433209       1 apiservice_controller.go:94] Shutting down APIServiceRegistrationController
I1108 23:22:19.413432       1 trace.go:76] Trace[536056020]: "Create /api/v1/nodes" (started: 2018-11-08 23:22:09.433426125 +0000 UTC m=+74.435660258) (total time: 10.001024357s):
Trace[536056020]: [10.001024357s] [10.00092669s] END
I1108 23:22:29.923567       1 trace.go:76] Trace[445682212]: "Patch /api/v1/namespaces/default/events/docker-for-desktop.1565496140ffdc86" (started: 2018-11-08 23:21:59.943387509 +0000 UTC m=+64.945621623) (total time: 30.001187598s):
Trace[445682212]: [30.001187598s] [30.001149554s] END

after which the apiserver contained exited and another was started (and this was basically crash looping forever)

@andersonmorita
Copy link

Hi there,

Recently, I came across the same issue....Version 18.06.1-ce-mac73 (26764)...Docker version Engine: 18.06.1-ce and Kubernetes version : v1.10.3 release.
What I did to sort out it ....Preference->Reset to Factory defaults and then afterwards I Preference->Restart

Currently....both are running perfectly...

screen shot 2018-11-10 at 12 21 21 pm

Cheers...

@huineng
Copy link

huineng commented Nov 20, 2018

happened today for me with Version 2.0.0.0-mac78 (28905) had to reset kubernetes
(ps the reset kubernetes button could have used a confirmation window)

@unixorn
Copy link

unixorn commented Nov 24, 2018

👍 for a confirmation button for the reset kubernetes button, and it would also be nice if it gave some feedback that it actually did something. All I saw was Kubernetes is starting in the bottom of the dialog, then clicked the button and it still said Kubernetes is starting*, which I'd been seeing for 20 minutes already.

@ghost
Copy link

ghost commented Jan 16, 2019

image
image
Can't find kubernetes reset ...any help?

Sorry find it..
image

@YnievesDotNet
Copy link

I do Reset and all, mojave 10.14.1, and not work

same at the last post

@SivaKrishna05
Copy link

I have been seeing the similar issue in version 2.0.2.1(31274) nothing working for me.

@SivaKrishna05
Copy link

Deleting the ~/.kube/config and uninstall the edge and installed the stable version fixed the issue.

@YnievesDotNet
Copy link

YnievesDotNet commented Apr 17, 2019 via email

@robolivable
Copy link

This issue persists for me... Tried all suggestions to date.
Screen Shot 2019-05-31 at 12 06 45 PM

@sakethkoduru
Copy link

Had to reset to factory defaults, since nothing else worked. This is still an issue on MAC.

@dhawal55
Copy link

This issue should be re-opened. Problem continues in 19.03.2. I can't even reset to factory default since it is stuck in loading:

Screen Shot 2019-09-26 at 6 25 28 PM

Screen Shot 2019-09-26 at 6 25 20 PM

Screen Shot 2019-09-26 at 6 27 36 PM

@walkingsk
Copy link

Happend on 2.1.0.3 (38420) stable Windows version, the same issue

@da2r-20
Copy link

da2r-20 commented Dec 7, 2019

Same issue on 2.1.0.5
OSX 10.14.5

Restarting both docker and kubernetes didn't help

@YouveGotMeowxy
Copy link

So now what?

I'm new to Docker; just spent about a week figuring out how to learn Compose and get all my stuff tweaked and perfect, decided I'd "see what the Kubernetes hoopla is all about, checked the "enable Kubernetes" option, restarted, and now I get this:

image

This is on the latest Edge version as of today. I just want to disable the Kubernetes option and go back to normal again, but I can't even access the settings anymore because of the spinning icon, and grayed out settings. I have tried restarting both my computer and Docker several times and it's stuck at this. I don't want to delete anything other than the Kubernetes option. I reeeeeeally don't want to destroy all my containers and images! And also don't want to destroy all of my stacks. (btw, when it says that, does it mean it will destroy ALL stacks including the ones that were there for containers created before I enabled Kubernetes? Or does it mean just the one's created after I enabled Kubernetes?)

Please help! :/
Kacey

@STaRDoGG
Copy link

STaRDoGG commented Feb 4, 2020

@YouveGotMeowxy Have a look at this thread; it may help you to disable the Kubernetes option in the Docker GUI. It worked for me.

https://geekdrop.com/content/kubernetes-in-docker-stuck-showing-kubernetes-starting-forever

@scottming
Copy link

the problem continues in 19.03.5

@STaRDoGG
Copy link

STaRDoGG commented Mar 1, 2020

I'm not a mac user, so I don't know if it is running in a VM on a mac, but on Windows I found out my vm was out of space. Expanding the space resolved this issue for me.

@lucianrica
Copy link

lucianrica commented May 29, 2020

  • Windows 10
  • Docker Desktop
  • Docker Kubernetes
  • Proxy

If anyone facing this issue in may 2020, fixed by editing window environment variables

  1. Windows Search -> edit environment var...
  2. Environment Variables -> Variable -> no_proxy (create one if you don't have)
  3. Click -> Edit -> add .docker.internal
  4. Restart Docker

5.Not sure if required but i also have under no_proxy var: .company.com, localhost,

@docker-robott
Copy link
Collaborator

Closed issues are locked after 30 days of inactivity.
This helps our team focus on active issues.

If you have found a problem that seems similar to this, please open a new issue.

Send feedback to Docker Community Slack channels #docker-for-mac or #docker-for-windows.
/lifecycle locked

@docker docker locked and limited conversation to collaborators Jun 28, 2020
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

No branches or pull requests