Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

No networks found in /etc/cni/net.d #1704

Closed
RakeshNagarajan opened this issue Aug 2, 2019 · 3 comments
Closed

No networks found in /etc/cni/net.d #1704

RakeshNagarajan opened this issue Aug 2, 2019 · 3 comments

Comments

@RakeshNagarajan
Copy link

@RakeshNagarajan RakeshNagarajan commented Aug 2, 2019

I am configuring Kubernetes cluster in my local VMware. While initializing the kubeadm. i am facing the below error.

14086 cni.go:213] Unable to update cni config: No networks found in /etc/cni/net.d
Aug  2 07:26:53 node1 kubelet[14086]: E0802 07:26:53.752892   14086 kubelet.go:2169] Container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:docker: network plugin is not ready: cni config uninitialized

What could be the issue?

I am running this cluster behind proxy.

@neolit123

This comment has been minimized.

Copy link
Member

@neolit123 neolit123 commented Aug 2, 2019

please provide more detail following the issue template:
https://github.com/kubernetes/kubeadm/blob/master/.github/ISSUE_TEMPLATE.md

@RakeshNagarajan

This comment has been minimized.

Copy link
Author

@RakeshNagarajan RakeshNagarajan commented Aug 4, 2019

Is this a BUG REPORT or FEATURE REQUEST?

BUG, while initializing Kubernetes 3 node cluster in VMware.

Internet connection is achieved through Proxy.

Versions

kubeadm version (use kubeadm version):v1.15.1

Environment:

root@node3:/var/log# docker -v
Docker version 18.09.7, build 2d0083d

What happened?

i am trying to configure kubernetes 3 node cluster in vmware.
Initially i faced certificate issue, to resolve this issue, i have feeded my certificates under /usr/local/share/ca-certificates and ran update-ca-certificates

Also i have created these directories /etc/docker/certs.d/k8s.gcr.io and feeded my certificates here /etc/docker/certs.d/k8s.gcr.io. After that only i am able to run the commands kubeadm config images pull

kubeadm init --apiserver-advertise-address=192.168.228.131 --node-name=node3

But now i am facing the below issues.

  1. Hostname not found
Aug 04 09:07:35 node3.kubernetes kubelet[66141]: E0804 09:07:35.374187   66141 kubelet.go:2169] Container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:docker: network plugin is not ready: cni config uninitialized
Aug 04 09:07:35 node3.kubernetes kubelet[66141]: E0804 09:07:35.419463   66141 kubelet.go:2248] node "node3.kubernetes" not found
Aug 04 09:07:35 node3.kubernetes kubelet[66141]: E0804 09:07:35.520200   66141 kubelet.go:2248] node "node3.kubernetes" not found
Aug 04 09:07:35 node3.kubernetes kubelet[66141]: E0804 09:07:35.621013   66141 kubelet.go:2248] node "node3.kubernetes" not found
Aug 04 09:07:35 node3.kubernetes kubelet[66141]: E0804 09:07:35.721800   66141 kubelet.go:2248] node "node3.kubernetes" not found
Aug 04 09:07:35 node3.kubernetes kubelet[66141]: E0804 09:07:35.822629   66141 kubelet.go:2248] node "node3.kubernetes" not found
Aug 04 09:07:35 node3.kubernetes kubelet[66141]: E0804 09:07:35.923302   66141 kubelet.go:2248] node "node3.kubernetes" not found
Aug 04 09:07:36 node3.kubernetes kubelet[66141]: E0804 09:07:36.024055   66141 kubelet.go:2248] node "node3.kubernetes" not found
Aug 04 09:07:36 node3.kubernetes kubelet[66141]: E0804 09:07:36.124825   66141 kubelet.go:2248] node "node3.kubernetes" not found
Aug 04 09:07:36 node3.kubernetes kubelet[66141]: E0804 09:07:36.226415   66141 kubelet.go:2248] node "node3.kubernetes" not found
Aug 04 09:07:36 node3.kubernetes kubelet[66141]: E0804 09:07:36.327094   66141 kubelet.go:2248] node "node3.kubernetes" not found
Aug 04 09:07:36 node3.kubernetes kubelet[66141]: E0804 09:07:36.428262   66141 kubelet.go:2248] node "node3.kubernetes" not found
Aug 04 09:07:36 node3.kubernetes kubelet[66141]: E0804 09:07:36.528956   66141 kubelet.go:2248] node "node3.kubernetes" not found
Aug 04 09:07:36 node3.kubernetes kubelet[66141]: E0804 09:07:36.629665   66141 kubelet.go:2248] node "node3.kubernetes" not found
Aug 04 09:07:36 node3.kubernetes kubelet[66141]: E0804 09:07:36.730847   66141 kubelet.go:2248] node "node3.kubernetes" not found
Aug 04 09:07:36 node3.kubernetes kubelet[66141]: E0804 09:07:36.831616   66141 kubelet.go:2248] node "node3.kubernetes" not found
Aug 04 09:07:36 node3.kubernetes kubelet[66141]: E0804 09:07:36.931970   66141 kubelet.go:2248] node "node3.kubernetes" not found
Aug 04 09:07:37 node3.kubernetes kubelet[66141]: E0804 09:07:37.032423   66141 kubelet.go:2248] node "node3.kubernetes" not found
Aug 04 09:07:37 node3.kubernetes kubelet[66141]: W0804 09:07:37.099394   66141 cni.go:213] Unable to update cni config: No networks found in /etc/cni/net.d

  1. Namespace not found
Aug 04 09:07:39 node3.kubernetes kubelet[66141]: E0804 09:07:39.622550   66141 controller.go:115] failed to ensure node lease exists, will retry in 3.2s, error: namespaces "kube-node-lease" not found
Aug 04 09:07:39 node3.kubernetes kubelet[66141]: E0804 09:07:39.670429   66141 event.go:240] Server rejected event '&v1.Event{TypeMeta:v1.TypeMeta{Kind:"", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"node3.kubernetes.15b7c3f473cfb8cc", GenerateName:"", Namespace:"default", SelfLink:"", UID:"", ResourceVersion:"", Generation:0, CreationTimestamp:v1.Time{Time:time.Time{wall:0x0, ext:0, loc:(*time.Location)(nil)}}, DeletionTimestamp:(*v1.Time)(nil), DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Initializers:(*v1.Initializers)(nil), Finalizers:[]string(nil), ClusterName:"", ManagedFields:[]v1.ManagedFieldsEntry(nil)}, InvolvedObject:v1.ObjectReference{Kind:"Node", Namespace:"", Name:"node3.kubernetes", UID:"node3.kubernetes", APIVersion:"", ResourceVersion:"", FieldPath:""}, Reason:"Starting", Message:"Starting kubelet.", Source:v1.EventSource{Component:"kubelet", Host:"node3.kubernetes"}, FirstTimestamp:v1.Time{Time:time.Time{wall:0xbf49de908ceea4cc, ext:3285615597, loc:(*time.Location)(0x781f740)}}, LastTimestamp:v1.Time{Time:time.Time{wall:0xbf49de908ceea4cc, ext:3285615597, loc:(*time.Location)(0x781f740)}}, Count:1, Type:"Normal", EventTime:v1.MicroTime{Time:time.Time{wall:0x0, ext:0, loc:(*time.Location)(nil)}}, Series:(*v1.EventSeries)(nil), Action:"", Related:(*v1.ObjectReference)(nil), ReportingController:"", ReportingInstance:""}': 'namespaces "default" not found' (will not retry!)
Aug 04 09:07:39 node3.kubernetes kubelet[66141]: E0804 09:07:39.725591   66141 event.go:240] Server rejected event '&v1.Event{TypeMeta:v1.TypeMeta{Kind:"", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"node3.kubernetes.15b7c3f47c4fdf3c", GenerateName:"", Namespace:"default", SelfLink:"", UID:"", ResourceVersion:"", Generation:0, CreationTimestamp:v1.Time{Time:time.Time{wall:0x0, ext:0, loc:(*time.Location)(nil)}}, DeletionTimestamp:(*v1.Time)(nil), DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Initializers:(*v1.Initializers)(nil), Finalizers:[]string(nil), ClusterName:"", ManagedFields:[]v1.ManagedFieldsEntry(nil)}, InvolvedObject:v1.ObjectReference{Kind:"Node", Namespace:"", Name:"node3.kubernetes", UID:"node3.kubernetes", APIVersion:"", ResourceVersion:"", FieldPath:""}, Reason:"NodeHasSufficientMemory", Message:"Node node3.kubernetes status is now: NodeHasSufficientMemory", Source:v1.EventSource{Component:"kubelet", Host:"node3.kubernetes"}, FirstTimestamp:v1.Time{Time:time.Time{wall:0xbf49de90956ecb3c, ext:3428231777, loc:(*time.Location)(0x781f740)}}, LastTimestamp:v1.Time{Time:time.Time{wall:0xbf49de90956ecb3c, ext:3428231777, loc:(*time.Location)(0x781f740)}}, Count:1, Type:"Normal", EventTime:v1.MicroTime{Time:time.Time{wall:0x0, ext:0, loc:(*time.Location)(nil)}}, Series:(*v1.EventSeries)(nil), Action:"", Related:(*v1.ObjectReference)(nil), ReportingController:"", ReportingInstance:""}': 'namespaces "default" not found' (will not retry!)
Aug 04 09:07:39 node3.kubernetes kubelet[66141]: E0804 09:07:39.781658   66141 event.go:240] Server rejected event '&v1.Event{TypeMeta:v1.TypeMeta{Kind:"", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"node3.kubernetes.15b7c3f47c4ff23e", GenerateName:"", Namespace:"default", SelfLink:"", UID:"", ResourceVersion:"", Generation:0, CreationTimestamp:v1.Time{Time:time.Time{wall:0x0, ext:0, loc:(*time.Location)(nil)}}, DeletionTimestamp:(*v1.Time)(nil), DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Initializers:(*v1.Initializers)(nil), Finalizers:[]string(nil), ClusterName:"", ManagedFields:[]v1.ManagedFieldsEntry(nil)}, InvolvedObject:v1.ObjectReference{Kind:"Node", Namespace:"", Name:"node3.kubernetes", UID:"node3.kubernetes", APIVersion:"", ResourceVersion:"", FieldPath:""}, Reason:"NodeHasNoDiskPressure", Message:"Node node3.kubernetes status is now: NodeHasNoDiskPressure", Source:v1.EventSource{Component:"kubelet", Host:"node3.kubernetes"}, FirstTimestamp:v1.Time{Time:time.Time{wall:0xbf49de90956ede3e, ext:3428236646, loc:(*time.Location)(0x781f740)}}, LastTimestamp:v1.Time{Time:time.Time{wall:0xbf49de90956ede3e, ext:3428236646, loc:(*time.Location)(0x781f740)}}, Count:1, Type:"Normal", EventTime:v1.MicroTime{Time:time.Time{wall:0x0, ext:0, loc:(*time.Location)(nil)}}, Series:(*v1.EventSeries)(nil), Action:"", Related:(*v1.ObjectReference)(nil), ReportingController:"", ReportingInstance:""}': 'namespaces "default" not found' (will not retry!)
Aug 04 09:07:39 node3.kubernetes kubelet[66141]: E0804 09:07:39.837365   66141 event.go:240] Server rejected event '&v1.Event{TypeMeta:v1.TypeMeta{Kind:"", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"node3.kubernetes.15b7c3f47c4ffd5a", GenerateName:"", Namespace:"default", SelfLink:"", UID:"", ResourceVersion:"", Generation:0, CreationTimestamp:v1.Time{Time:time.Time{wall:0x0, ext:0, loc:(*time.Location)(nil)}}, DeletionTimestamp:(*v1.Time)(nil), DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Initializers:(*v1.Initializers)(nil), Finalizers:[]string(nil), ClusterName:"", ManagedFields:[]v1.ManagedFieldsEntry(nil)}, InvolvedObject:v1.ObjectReference{Kind:"Node", Namespace:"", Name:"node3.kubernetes", UID:"node3.kubernetes", APIVersion:"", ResourceVersion:"", FieldPath:""}, Reason:"NodeHasSufficientPID", Message:"Node node3.kubernetes status is now: NodeHasSufficientPID", Source:v1.EventSource{Component:"kubelet", Host:"node3.kubernetes"}, FirstTimestamp:v1.Time{Time:time.Time{wall:0xbf49de90956ee95a, ext:3428239488, loc:(*time.Location)(0x781f740)}}, LastTimestamp:v1.Time{Time:time.Time{wall:0xbf49de90956ee95a, ext:3428239488, loc:(*time.Location)(0x781f740)}}, Count:1, Type:"Normal", EventTime:v1.MicroTime{Time:time.Time{wall:0x0, ext:0, loc:(*time.Location)(nil)}}, Series:(*v1.EventSeries)(nil), Action:"", Related:(*v1.ObjectReference)(nil), ReportingController:"", ReportingInstance:""}': 'namespaces "default" not found' (will not retry!)
Aug 04 09:07:39 node3.kubernetes kubelet[66141]: E0804 09:07:39.900952   66141 event.go:240] Server rejected event '&v1.Event{TypeMeta:v1.TypeMeta{Kind:"", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"node3.kubernetes.15b7c3f47c4fdf3c", GenerateName:"", Namespace:"default", SelfLink:"", UID:"", ResourceVersion:"", Generation:0, CreationTimestamp:v1.Time{Time:time.Time{wall:0x0, ext:0, loc:(*time.Location)(nil)}}, DeletionTimestamp:(*v1.Time)(nil), DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Initializers:(*v1.Initializers)(nil), Finalizers:[]string(nil), ClusterName:"", ManagedFields:[]v1.ManagedFieldsEntry(nil)}, InvolvedObject:v1.ObjectReference{Kind:"Node", Namespace:"", Name:"node3.kubernetes", UID:"node3.kubernetes", APIVersion:"", ResourceVersion:"", FieldPath:""}, Reason:"NodeHasSufficientMemory", Message:"Node node3.kubernetes status is now: NodeHasSufficientMemory", Source:v1.EventSource{Component:"kubelet", Host:"node3.kubernetes"}, FirstTimestamp:v1.Time{Time:time.Time{wall:0xbf49de90956ecb3c, ext:3428231777, loc:(*time.Location)(0x781f740)}}, LastTimestamp:v1.Time{Time:time.Time{wall:0xbf49de9095e4017f, ext:3435913383, loc:(*time.Location)(0x781f740)}}, Count:2, Type:"Normal", EventTime:v1.MicroTime{Time:time.Time{wall:0x0, ext:0, loc:(*time.Location)(nil)}}, Series:(*v1.EventSeries)(nil), Action:"", Related:(*v1.ObjectReference)(nil), ReportingController:"", ReportingInstance:""}': 'namespaces "default" not found' (will not retry!)
Aug 04 09:07:39 node3.kubernetes kubelet[66141]: E0804 09:07:39.965153   66141 event.go:240] Server rejected event '&v1.Event{TypeMeta:v1.TypeMeta{Kind:"", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"node3.kubernetes.15b7c3f47c4ff23e", GenerateName:"", Namespace:"default", SelfLink:"", UID:"", ResourceVersion:"", Generation:0, CreationTimestamp:v1.Time{Time:time.Time{wall:0x0, ext:0, loc:(*time.Location)(nil)}}, DeletionTimestamp:(*v1.Time)(nil), DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Initializers:(*v1.Initializers)(nil), Finalizers:[]string(nil), ClusterName:"", ManagedFields:[]v1.ManagedFieldsEntry(nil)}, InvolvedObject:v1.ObjectReference{Kind:"Node", Namespace:"", Name:"node3.kubernetes", UID:"node3.kubernetes", APIVersion:"", ResourceVersion:"", FieldPath:""}, Reason:"NodeHasNoDiskPressure", Message:"Node node3.kubernetes status is now: NodeHasNoDiskPressure", Source:v1.EventSource{Component:"kubelet", Host:"node3.kubernetes"}, FirstTimestamp:v1.Time{Time:time.Time{wall:0xbf49de90956ede3e, ext:3428236646, loc:(*time.Location)(0x781f740)}}, LastTimestamp:v1.Time{Time:time.Time{wall:0xbf49de9095e414ae, ext:3435918293, loc:(*time.Location)(0x781f740)}}, Count:2, Type:"Normal", EventTime:v1.MicroTime{Time:time.Time{wall:0x0, ext:0, loc:(*time.Location)(nil)}}, Series:(*v1.EventSeries)(nil), Action:"", Related:(*v1.ObjectReference)(nil), ReportingController:"", ReportingInstance:""}': 'namespaces "default" not found' (will not retry!)
Aug 04 09:07:40 node3.kubernetes kubelet[66141]: E0804 09:07:40.026961   66141 event.go:240] Server rejected event '&v1.Event{TypeMeta:v1.TypeMeta{Kind:"", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"node3.kubernetes.15b7c3f47c4ffd5a", GenerateName:"", Namespace:"default", SelfLink:"", UID:"", ResourceVersion:"", Generation:0, CreationTimestamp:v1.Time{Time:time.Time{wall:0x0, ext:0, loc:(*time.Location)(nil)}}, DeletionTimestamp:(*v1.Time)(nil), DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Initializers:(*v1.Initializers)(nil), Finalizers:[]string(nil), ClusterName:"", ManagedFields:[]v1.ManagedFieldsEntry(nil)}, InvolvedObject:v1.ObjectReference{Kind:"Node", Namespace:"", Name:"node3.kubernetes", UID:"node3.kubernetes", APIVersion:"", ResourceVersion:"", FieldPath:""}, Reason:"NodeHasSufficientPID", Message:"Node node3.kubernetes status is now: NodeHasSufficientPID", Source:v1.EventSource{Component:"kubelet", Host:"node3.kubernetes"}, FirstTimestamp:v1.Time{Time:time.Time{wall:0xbf49de90956ee95a, ext:3428239488, loc:(*time.Location)(0x781f740)}}, LastTimestamp:v1.Time{Time:time.Time{wall:0xbf49de9095e41e6e, ext:3435920788, loc:(*time.Location)(0x781f740)}}, Count:2, Type:"Normal", EventTime:v1.MicroTime{Time:time.Time{wall:0x0, ext:0, loc:(*time.Location)(nil)}}, Series:(*v1.EventSeries)(nil), Action:"", Related:(*v1.ObjectReference)(nil), ReportingController:"", ReportingInstance:""}': 'namespaces "default" not found' (will not retry!)
Aug 04 09:07:40 node3.kubernetes kubelet[66141]: E0804 09:07:40.082848   66141 event.go:240] Server rejected event '&v1.Event{TypeMeta:v1.TypeMeta{Kind:"", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"node3.kubernetes.15b7c3f47d1697df", GenerateName:"", Namespace:"default", SelfLink:"", UID:"", ResourceVersion:"", Generation:0, CreationTimestamp:v1.Time{Time:time.Time{wall:0x0, ext:0, loc:(*time.Location)(nil)}}, DeletionTimestamp:(*v1.Time)(nil), DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Initializers:(*v1.Initializers)(nil), Finalizers:[]string(nil), ClusterName:"", ManagedFields:[]v1.ManagedFieldsEntry(nil)}, InvolvedObject:v1.ObjectReference{Kind:"Node", Namespace:"", Name:"node3.kubernetes", UID:"node3.kubernetes", APIVersion:"", ResourceVersion:"", FieldPath:""}, Reason:"NodeAllocatableEnforced", Message:"Updated Node Allocatable limit across pods", Source:v1.EventSource{Component:"kubelet", Host:"node3.kubernetes"}, FirstTimestamp:v1.Time{Time:time.Time{wall:0xbf49de90963583df, ext:3441255173, loc:(*time.Location)(0x781f740)}}, LastTimestamp:v1.Time{Time:time.Time{wall:0xbf49de90963583df, ext:3441255173, loc:(*time.Location)(0x781f740)}}, Count:1, Type:"Normal", EventTime:v1.MicroTime{Time:time.Time{wall:0x0, ext:0, loc:(*time.Location)(nil)}}, Series:(*v1.EventSeries)(nil), Action:"", Related:(*v1.ObjectReference)(nil), ReportingController:"", ReportingInstance:""}': 'namespaces "default" not found' (will not retry!)
Aug 04 09:07:40 node3.kubernetes kubelet[66141]: E0804 09:07:40.144352   66141 event.go:240] Server rejected event '&v1.Event{TypeMeta:v1.TypeMeta{Kind:"", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"node3.kubernetes.15b7c3f47c4fdf3c", GenerateName:"", Namespace:"default", SelfLink:"", UID:"", ResourceVersion:"", Generation:0, CreationTimestamp:v1.Time{Time:time.Time{wall:0x0, ext:0, loc:(*time.Location)(nil)}}, DeletionTimestamp:(*v1.Time)(nil), DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Initializers:(*v1.Initializers)(nil), Finalizers:[]string(nil), ClusterName:"", ManagedFields:[]v1.ManagedFieldsEntry(nil)}, InvolvedObject:v1.ObjectReference{Kind:"Node", Namespace:"", Name:"node3.kubernetes", UID:"node3.kubernetes", APIVersion:"", ResourceVersion:"", FieldPath:""}, Reason:"NodeHasSufficientMemory", Message:"Node node3.kubernetes status is now: NodeHasSufficientMemory", Source:v1.EventSource{Component:"kubelet", Host:"node3.kubernetes"}, FirstTimestamp:v1.Time{Time:time.Time{wall:0xbf49de90956ecb3c, ext:3428231777, loc:(*time.Location)(0x781f740)}}, LastTimestamp:v1.Time{Time:time.Time{wall:0xbf49de90a079e852, ext:3613509495, loc:(*time.Location)(0x781f740)}}, Count:3, Type:"Normal", EventTime:v1.MicroTime{Time:time.Time{wall:0x0, ext:0, loc:(*time.Location)(nil)}}, Series:(*v1.EventSeries)(nil), Action:"", Related:(*v1.ObjectReference)(nil), ReportingController:"", ReportingInstance:""}': 'namespaces "default" not found' (will not retry!)
Aug 04 09:07:40 node3.kubernetes kubelet[66141]: W0804 09:07:40.259405   66141 kubelet_getters.go:297] Path "/var/lib/kubelet/pods/06345808a8cebc32a64b3e5e2b314c0f/volumes" does not exist
Aug 04 09:07:40 node3.kubernetes kubelet[66141]: E0804 09:07:40.376226   66141 kubelet.go:2169] Container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:docker: network plugin is not ready: cni config uninitialized
Aug 04 09:07:40 node3.kubernetes kubelet[66141]: E0804 09:07:40.529232   66141 event.go:240] Server rejected event '&v1.Event{TypeMeta:v1.TypeMeta{Kind:"", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"node3.kubernetes.15b7c3f47c4ff23e", GenerateName:"", Namespace:"default", SelfLink:"", UID:"", ResourceVersion:"", Generation:0, CreationTimestamp:v1.Time{Time:time.Time{wall:0x0, ext:0, loc:(*time.Location)(nil)}}, DeletionTimestamp:(*v1.Time)(nil), DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Initializers:(*v1.Initializers)(nil), Finalizers:[]string(nil), ClusterName:"", ManagedFields:[]v1.ManagedFieldsEntry(nil)}, InvolvedObject:v1.ObjectReference{Kind:"Node", Namespace:"", Name:"node3.kubernetes", UID:"node3.kubernetes", APIVersion:"", ResourceVersion:"", FieldPath:""}, Reason:"NodeHasNoDiskPressure", Message:"Node node3.kubernetes status is now: NodeHasNoDiskPressure", Source:v1.EventSource{Component:"kubelet", Host:"node3.kubernetes"}, FirstTimestamp:v1.Time{Time:time.Time{wall:0xbf49de90956ede3e, ext:3428236646, loc:(*time.Location)(0x781f740)}}, LastTimestamp:v1.Time{Time:time.Time{wall:0xbf49de90a079fa78, ext:3613514140, loc:(*time.Location)(0x781f740)}}, Count:3, Type:"Normal", EventTime:v1.MicroTime{Time:time.Time{wall:0x0, ext:0, loc:(*time.Location)(nil)}}, Series:(*v1.EventSeries)(nil), Action:"", Related:(*v1.ObjectReference)(nil), ReportingController:"", ReportingInstance:""}': 'namespaces "default" not found' (will not retry!)
Aug 04 09:07:40 node3.kubernetes kubelet[66141]: E0804 09:07:40.937106   66141 event.go:240] Server rejected event '&v1.Event{TypeMeta:v1.TypeMeta{Kind:"", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"node3.kubernetes.15b7c3f47c4ffd5a", GenerateName:"", Namespace:"default", SelfLink:"", UID:"", ResourceVersion:"", Generation:0, CreationTimestamp:v1.Time{Time:time.Time{wall:0x0, ext:0, loc:(*time.Location)(nil)}}, DeletionTimestamp:(*v1.Time)(nil), DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Initializers:(*v1.Initializers)(nil), Finalizers:[]string(nil), ClusterName:"", ManagedFields:[]v1.ManagedFieldsEntry(nil)}, InvolvedObject:v1.ObjectReference{Kind:"Node", Namespace:"", Name:"node3.kubernetes", UID:"node3.kubernetes", APIVersion:"", ResourceVersion:"", FieldPath:""}, Reason:"NodeHasSufficientPID", Message:"Node node3.kubernetes status is now: NodeHasSufficientPID", Source:v1.EventSource{Component:"kubelet", Host:"node3.kubernetes"}, FirstTimestamp:v1.Time{Time:time.Time{wall:0xbf49de90956ee95a, ext:3428239488, loc:(*time.Location)(0x781f740)}}, LastTimestamp:v1.Time{Time:time.Time{wall:0xbf49de90a07a04da, ext:3613516798, loc:(*time.Location)(0x781f740)}}, Count:3, Type:"Normal", EventTime:v1.MicroTime{Time:time.Time{wall:0x0, ext:0, loc:(*time.Location)(nil)}}, Series:(*v1.EventSeries)(nil), Action:"", Related:(*v1.ObjectReference)(nil), ReportingController:"", ReportingInstance:""}': 'namespaces "default" not found' (will not retry!)
Aug 04 09:07:41 node3.kubernetes kubelet[66141]: E0804 09:07:41.326674   66141 event.go:240] Server rejected event '&v1.Event{TypeMeta:v1.TypeMeta{Kind:"", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"node3.kubernetes.15b7c3f47c4fdf3c", GenerateName:"", Namespace:"default", SelfLink:"", UID:"", ResourceVersion:"", Generation:0, CreationTimestamp:v1.Time{Time:time.Time{wall:0x0, ext:0, loc:(*time.Location)(nil)}}, DeletionTimestamp:(*v1.Time)(nil), DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Initializers:(*v1.Initializers)(nil), Finalizers:[]string(nil), ClusterName:"", ManagedFields:[]v1.ManagedFieldsEntry(nil)}, InvolvedObject:v1.ObjectReference{Kind:"Node", Namespace:"", Name:"node3.kubernetes", UID:"node3.kubernetes", APIVersion:"", ResourceVersion:"", FieldPath:""}, Reason:"NodeHasSufficientMemory", Message:"Node node3.kubernetes status is now: NodeHasSufficientMemory", Source:v1.EventSource{Component:"kubelet", Host:"node3.kubernetes"}, FirstTimestamp:v1.Time{Time:time.Time{wall:0xbf49de90956ecb3c, ext:3428231777, loc:(*time.Location)(0x781f740)}}, LastTimestamp:v1.Time{Time:time.Time{wall:0xbf49de90a162d4ff, ext:3628774437, loc:(*time.Location)(0x781f740)}}, Count:4, Type:"Normal", EventTime:v1.MicroTime{Time:time.Time{wall:0x0, ext:0, loc:(*time.Location)(nil)}}, Series:(*v1.EventSeries)(nil), Action:"", Related:(*v1.ObjectReference)(nil), ReportingController:"", ReportingInstance:""}': 'namespaces "default" not found' (will not retry!)
Aug 04 09:07:41 node3.kubernetes kubelet[66141]: E0804 09:07:41.727685   66141 event.go:240] Server rejected event '&v1.Event{TypeMeta:v1.TypeMeta{Kind:"", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"node3.kubernetes.15b7c3f47c4ff23e", GenerateName:"", Namespace:"default", SelfLink:"", UID:"", ResourceVersion:"", Generation:0, CreationTimestamp:v1.Time{Time:time.Time{wall:0x0, ext:0, loc:(*time.Location)(nil)}}, DeletionTimestamp:(*v1.Time)(nil), DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Initializers:(*v1.Initializers)(nil), Finalizers:[]string(nil), ClusterName:"", ManagedFields:[]v1.ManagedFieldsEntry(nil)}, InvolvedObject:v1.ObjectReference{Kind:"Node", Namespace:"", Name:"node3.kubernetes", UID:"node3.kubernetes", APIVersion:"", ResourceVersion:"", FieldPath:""}, Reason:"NodeHasNoDiskPressure", Message:"Node node3.kubernetes status is now: NodeHasNoDiskPressure", Source:v1.EventSource{Component:"kubelet", Host:"node3.kubernetes"}, FirstTimestamp:v1.Time{Time:time.Time{wall:0xbf49de90956ede3e, ext:3428236646, loc:(*time.Location)(0x781f740)}}, LastTimestamp:v1.Time{Time:time.Time{wall:0xbf49de90a162e302, ext:3628778025, loc:(*time.Location)(0x781f740)}}, Count:4, Type:"Normal", EventTime:v1.MicroTime{Time:time.Time{wall:0x0, ext:0, loc:(*time.Location)(nil)}}, Series:(*v1.EventSeries)(nil), Action:"", Related:(*v1.ObjectReference)(nil), ReportingController:"", ReportingInstance:""}': 'namespaces "default" not found' (will not retry!)
Aug 04 09:07:42 node3.kubernetes kubelet[66141]: W0804 09:07:42.099813   66141 cni.go:213] Unable to update cni config: No networks found in /etc/cni/net.d
Aug 04 09:07:42 node3.kubernetes kubelet[66141]: E0804 09:07:42.126569   66141 event.go:240] Server rejected event '&v1.Event{TypeMeta:v1.TypeMeta{Kind:"", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"node3.kubernetes.15b7c3f47c4ffd5a", GenerateName:"", Namespace:"default", SelfLink:"", UID:"", ResourceVersion:"", Generation:0, CreationTimestamp:v1.Time{Time:time.Time{wall:0x0, ext:0, loc:(*time.Location)(nil)}}, DeletionTimestamp:(*v1.Time)(nil), DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Initializers:(*v1.Initializers)(nil), Finalizers:[]string(nil), ClusterName:"", ManagedFields:[]v1.ManagedFieldsEntry(nil)}, InvolvedObject:v1.ObjectReference{Kind:"Node", Namespace:"", Name:"node3.kubernetes", UID:"node3.kubernetes", APIVersion:"", ResourceVersion:"", FieldPath:""}, Reason:"NodeHasSufficientPID", Message:"Node node3.kubernetes status is now: NodeHasSufficientPID", Source:v1.EventSource{Component:"kubelet", Host:"node3.kubernetes"}, FirstTimestamp:v1.Time{Time:time.Time{wall:0xbf49de90956ee95a, ext:3428239488, loc:(*time.Location)(0x781f740)}}, LastTimestamp:v1.Time{Time:time.Time{wall:0xbf49de90a162ee33, ext:3628780893, loc:(*time.Location)(0x781f740)}}, Count:4, Type:"Normal", EventTime:v1.MicroTime{Time:time.Time{wall:0x0, ext:0, loc:(*time.Location)(nil)}}, Series:(*v1.EventSeries)(nil), Action:"", Related:(*v1.ObjectReference)(nil), ReportingController:"", ReportingInstance:""}': 'namespaces "default" not found' (will not retry!)
Aug 04 09:07:42 node3.kubernetes kubelet[66141]: E0804 09:07:42.529749   66141 event.go:240] Server rejected event '&v1.Event{TypeMeta:v1.TypeMeta{Kind:"", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"node3.kubernetes.15b7c3f47c4fdf3c", GenerateName:"", Namespace:"default", SelfLink:"", UID:"", ResourceVersion:"", Generation:0, CreationTimestamp:v1.Time{Time:time.Time{wall:0x0, ext:0, loc:(*time.Location)(nil)}}, DeletionTimestamp:(*v1.Time)(nil), DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Initializers:(*v1.Initializers)(nil), Finalizers:[]string(nil), ClusterName:"", ManagedFields:[]v1.ManagedFieldsEntry(nil)}, InvolvedObject:v1.ObjectReference{Kind:"Node", Namespace:"", Name:"node3.kubernetes", UID:"node3.kubernetes", APIVersion:"", ResourceVersion:"", FieldPath:""}, Reason:"NodeHasSufficientMemory", Message:"Node node3.kubernetes status is now: NodeHasSufficientMemory", Source:v1.EventSource{Component:"kubelet", Host:"node3.kubernetes"}, FirstTimestamp:v1.Time{Time:time.Time{wall:0xbf49de90956ecb3c, ext:3428231777, loc:(*time.Location)(0x781f740)}}, LastTimestamp:v1.Time{Time:time.Time{wall:0xbf49de90a1625675, ext:3628742043, loc:(*time.Location)(0x781f740)}}, Count:5, Type:"Normal", EventTime:v1.MicroTime{Time:time.Time{wall:0x0, ext:0, loc:(*time.Location)(nil)}}, Series:(*v1.EventSeries)(nil), Action:"", Related:(*v1.ObjectReference)(nil), ReportingController:"", ReportingInstance:""}': 'namespaces "default" not found' (will not retry!)

  1. Unable to update cni config: No networks found in /etc/cni/net.d
Aug 04 09:13:37 node3.kubernetes kubelet[66141]: W0804 09:13:37.172066   66141 cni.go:213] Unable to update cni config: No networks found in /etc/cni/net.d
Aug 04 09:13:40 node3.kubernetes kubelet[66141]: E0804 09:13:40.534388   66141 kubelet.go:2169] Container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:docker: network plugin is not ready: cni config uninitialized
Aug 04 09:13:42 node3.kubernetes kubelet[66141]: W0804 09:13:42.173481   66141 cni.go:213] Unable to update cni config: No networks found in /etc/cni/net.d
Aug 04 09:13:45 node3.kubernetes kubelet[66141]: E0804 09:13:45.536455   66141 kubelet.go:2169] Container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:docker: network plugin is not ready: cni config uninitialized
Aug 04 09:13:47 node3.kubernetes kubelet[66141]: W0804 09:13:47.174524   66141 cni.go:213] Unable to update cni config: No networks found in /etc/cni/net.d
Aug 04 09:13:50 node3.kubernetes kubelet[66141]: E0804 09:13:50.538789   66141 kubelet.go:2169] Container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:docker: network plugin is not ready: cni config uninitialized
Aug 04 09:13:52 node3.kubernetes kubelet[66141]: W0804 09:13:52.175353   66141 cni.go:213] Unable to update cni config: No networks found in /etc/cni/net.d
Aug 04 09:13:55 node3.kubernetes kubelet[66141]: E0804 09:13:55.541579   66141 kubelet.go:2169] Container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:docker: network plugin is not ready: cni config uninitialized
  1. connection refused -- Its is taking NAT IP instead of Host-only IP.
Aug 04 09:07:31 node3.kubernetes kubelet[66141]: E0804 09:07:31.273409   66141 reflector.go:125] k8s.io/client-go/informers/factory.go:133: Failed to list *v1beta1.CSIDriver: Get https://192.168.188.131:6443/apis/storage.k8s.io/v1beta1/csidrivers?limit=500&resourceVersion=0: dial tcp 192.168.188.131:6443: connect: connection refused
Aug 04 09:01:20 node3.kubernetes kubelet[62059]: E0804 09:01:20.675013   62059 kubelet_node_status.go:94] Unable to register node "node3.kubernetes" with API server: Post https://192.168.188.131:6443/api/v1/nodes: dial tcp 192.168.188.131:6443: connect: connection refused
Aug 04 09:01:20 node3.kubernetes kubelet[62059]: E0804 09:01:20.884953   62059 kubelet_node_status.go:94] Unable to register node "node3.kubernetes" with API server: Post https://192.168.188.131:6443/api/v1/nodes: dial tcp 192.168.188.131:6443: connect: connection refused
Aug 04 09:01:21 node3.kubernetes kubelet[62059]: E0804 09:01:21.289299   62059 kubelet_node_status.go:94] Unable to register node "node3.kubernetes" with API server: Post https://192.168.188.131:6443/api/v1/nodes: dial tcp 192.168.188.131:6443: connect: connection refused
Aug 04 09:01:22 node3.kubernetes kubelet[62059]: E0804 09:01:22.109259   62059 kubelet_node_status.go:94] Unable to register node "node3.kubernetes" with API server: Post https://192.168.188.131:6443/api/v1/nodes: dial tcp 192.168.188.131:6443: connect: connection refused
Aug 04 09:07:30 node3.kubernetes kubelet[66141]: E0804 09:07:30.360350   66141 kubelet_node_status.go:94] Unable to register node "node3.kubernetes" with API server: Post https://192.168.188.131:6443/api/v1/nodes: dial tcp 192.168.188.131:6443: connect: connection refused
Aug 04 09:07:30 node3.kubernetes kubelet[66141]: E0804 09:07:30.564742   66141 kubelet_node_status.go:94] Unable to register node "node3.kubernetes" with API server: Post https://192.168.188.131:6443/api/v1/nodes: dial tcp 192.168.188.131:6443: connect: connection refused
Aug 04 09:07:30 node3.kubernetes kubelet[66141]: E0804 09:07:30.968193   66141 kubelet_node_status.go:94] Unable to register node "node3.kubernetes" with API server: Post https://192.168.188.131:6443/api/v1/nodes: dial tcp 192.168.188.131:6443: connect: connection refused
Aug 04 09:07:31 node3.kubernetes kubelet[66141]: E0804 09:07:31.777652   66141 kubelet_node_status.go:94] Unable to register node "node3.kubernetes" with API server: Post https://192.168.188.131:6443/api/v1/nodes: dial tcp 192.168.188.131:6443: connect: connection refused

What you expected to happen?

How to reproduce it (as minimally and precisely as possible)?

Anything else we need to know?

In VMware i have configured 2 Network adapters, Host-only and NAT.

Nslookup is successfull.

root@node3:~# nslookup node3.kubernetes
Server:		127.0.0.53
Address:	127.0.0.53#53

Non-authoritative answer:
Name:	node3.kubernetes
Address: 192.168.228.131

root@node3:~# 

Firewall disabled

root@node3:/var/log# ufw status 
Status: inactive

What am i missing ?

@neolit123

This comment has been minimized.

Copy link
Member

@neolit123 neolit123 commented Aug 4, 2019

by

In VMware

do you mean vSphere?

i am trying to configure kubernetes 3 node cluster in vmware.
Initially i faced certificate issue, to resolve this issue, i have feeded my certificates under /usr/local/share/ca-certificates and ran update-ca-certificates

for regular kubeadm setup on your own VMs (e.g. using Ubuntu), you do not need update-ca-certificates. i have no information why it was required for you to do that.

Also i have created these directories /etc/docker/certs.d/k8s.gcr.io and feeded my certificates here /etc/docker/certs.d/k8s.gcr.io. After that only i am able to run the commands kubeadm config images pull

the k8s.gcr.io is a public registry and does not require authentication!

if your setup requires TLS and certs under /etc/docker/certs.d/k8s.gcr.io to access the registry, you probably setup this correctly given you were able to pull using ...images pull.

but please mind that you are securing artifact transfer for artifacts that are publicly available.

In VMware i have configured 2 Network adapters, Host-only and NAT.
192.168.188.131:6443: connect: connection refused

this is not related to kubeadm, i suspect a networking problem.
it implies that a kubelet connect connect to the api server on the .131 machine.

network plugin is not ready: cni config uninitialized
Aug 04 09:13:42 node3.kubernetes kubelet[66141]: W0804 09:13:42.173481 66141 cni.go:213] Unable to update cni config: No networks found in /etc/cni/net.d

please make sure you install a networking plugin after you called kubeadm init....
https://kubernetes.io/docs/setup/production-environment/tools/kubeadm/create-cluster-kubeadm/#pod-network

it is better if you edit->copy the contents of your message here:
#1704 (comment)

into a new issue in the kuberentes/kubernetes repository and then go to the kubernetes slack channel provider-vsphere and mention your new kubernetes/kubernetes issue and ask for help.

i cannot figure out what your are not configuring correctly, but perhaps if you share more details in the new ticket about your VMs and "vmware" setup the VMware infrastructure experts might be able to help you.

thanks.

@neolit123 neolit123 closed this Aug 4, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
2 participants
You can’t perform that action at this time.