ExecStart=/usr/bin/kubelet $KUBELET_KUBECONFIG_ARGS $KUBELET_CONFIG_ARGS $KUBELET_KUBEADM_ARGS $KUBELET_EXTRA_ARGS (code=exited, status=255)

[sarath4]

After i have installed the kubelet am getting the exited and service never starts. Could you please help.

OS: CentOS Linux 7
Version of services:
[root@node system]# kubeadm version
kubeadm version: &version.Info{Major:"1", Minor:"16", GitVersion:"v1.16.1", GitCommit:"d647ddbd755faf07169599a625faf302ffc34458", GitTreeState:"clean", BuildDate:"2019-10-02T16:58:27Z", GoVersion:"go1.12.10", Compiler:"gc", Platform:"linux/amd64"}
[root@node system]# kubelet --version
Kubernetes v1.16.1
[root@node system]# docker -v
Docker version 1.13.1, build 7f2769b/1.13.1
[root@node system]#

Error:
[root@node system]# service kubelet status
Redirecting to /bin/systemctl status kubelet.service
● kubelet.service - kubelet: The Kubernetes Node Agent
Loaded: loaded (/usr/lib/systemd/system/kubelet.service; enabled; vendor preset: disabled)
Drop-In: /usr/lib/systemd/system/kubelet.service.d
└─10-kubeadm.conf
Active: activating (auto-restart) (Result: exit-code) since Tue 2019-10-15 01:54:06 EDT; 222ms ago
Docs: https://kubernetes.io/docs/
Process: 20919 ExecStart=/usr/bin/kubelet $KUBELET_KUBECONFIG_ARGS $KUBELET_CONFIG_ARGS $KUBELET_KUBEADM_ARGS $KUBELET_EXTRA_ARGS (code=exited, status=255)
Main PID: 20919 (code=exited, status=255)

Oct 15 01:54:06 node systemd[1]: kubelet.service: main process exited, code=exited, status=255/n/a
Oct 15 01:54:06 node systemd[1]: Unit kubelet.service entered failed state.
Oct 15 01:54:06 node systemd[1]: kubelet.service failed.
[root@node system]#

[neolit123]

hi, the kubelet that is deployed next to the kubeadm package will crashloop until you start kubeadm init. check the status of the kubelet after kubeadm init finishes.

/triage support

[neolit123]

/remove-kind bug
/sig cluster-lifecycle

[neolit123]

/close
please re-open if needed.

[k8s-ci-robot]

@neolit123: Closing this issue.

In response to this:

/close
please re-open if needed.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[dramdayal]

hello I think this needs to be re-open. I am attempting a 4 node cluster with 1 master 3 workers. 2 workers can join master, however 3 worker cannot get kubelet to start. we have attempted to re-image and install several times with different packages but same issue.

Here is the info from this problem node.

All are CentOS 7

docker -v
Docker version 1.13.1, build 7f2769b/1.13.1

kubelet --version
Kubernetes v1.16.2

[root@centos-7-k8s-worker-01 ~]# systemctl status kubelet
● kubelet.service - kubelet: The Kubernetes Node Agent
Loaded: loaded (/usr/lib/systemd/system/kubelet.service; enabled; vendor preset: disabled)
Drop-In: /usr/lib/systemd/system/kubelet.service.d
└─10-kubeadm.conf
Active: activating (auto-restart) (Result: exit-code) since Fri 2019-10-25 16:05:02 UTC; 4s ago
Docs: https://kubernetes.io/docs/
Process: 1928 ExecStart=/usr/bin/kubelet $KUBELET_KUBECONFIG_ARGS $KUBELET_CONFIG_ARGS $KUBELET_KUBEADM_ARGS $KUBELET_EXTRA_ARGS (code=exited, status=255)
Main PID: 1928 (code=exited, status=255)

Oct 25 16:05:02 centos-7-k8s-worker-01.novalocal systemd[1]: kubelet.service: main process exited, code=exited, status=255/n/a
Oct 25 16:05:02 centos-7-k8s-worker-01.novalocal systemd[1]: Unit kubelet.service entered failed state.
Oct 25 16:05:02 centos-7-k8s-worker-01.novalocal systemd[1]: kubelet.service failed.

Master node:
[root@centos-7-k8s-master-01 ~]# kubectl get nodes
NAME STATUS ROLES AGE VERSION
centos-7-k8s-master-01.novalocal Ready master 3d19h v1.16.2
centos-7-k8s-worker-02.novalocal Ready 3d19h v1.16.2
centos-7-k8s-worker-03.novalocal Ready 3d19h v1.16.2

we are trying to add worker centos-7-k8s-worker-01.novalocal to this cluster.

If you run kubadm init it will complete and provide token info to have nodes join. So it basically becomes a master.

[neolit123]

Oct 25 16:05:02 centos-7-k8s-worker-01.novalocal systemd[1]: kubelet.service: main process exited, code=exited, status=255/n/a
Oct 25 16:05:02 centos-7-k8s-worker-01.novalocal systemd[1]: Unit kubelet.service entered failed state.
Oct 25 16:05:02 centos-7-k8s-worker-01.novalocal systemd[1]: kubelet.service failed.

the kubelet logs e.g. using journalctl -xeu kubelet should provide more details about the failure.

[dramdayal]

okay will run those. thanks.

[dramdayal]

I ended up re-installing fresh setup and all good now. We can close ticket.

thanks

[msrinivascharan]

i am using CentOS8 as node, works fine after disabling swap.

[alexluicwhk]

check also if the control plane node /etc/kubernetes/kubelet.conf is valid and up-to-date. if it is expired, the service kubelet start may fail.

[lgfabres]

hi, i'm having the same problems starting kubelet.
I am working with OL8.
My installation script is:

I.- presettings in OL8
1.- Upgrade the system
dnf -y upgrade

2. Disable SELinux enforcement.
   setenforce 0
   sed -i --follow-symlinks 's/SELINUX=enforcing/SELINUX=disabled/g' /etc/sysconfig/selinux

3. Enable transparent masquerading and facilitate Virtual Extensible LAN (VxLAN) traffic for communication between Kubernetes pods across the cluster.
   modprobe br_netfilter
   firewall-cmd --add-masquerade --permanent
   firewall-cmd --reload

4. Set bridged packets to traverse iptables rules.
   cat < /etc/sysctl.d/k8s.conf
   net.bridge.bridge-nf-call-ip6tables = 1
   net.bridge.bridge-nf-call-iptables = 1
   EOF

sysctl --system

5. Disable all memory swaps to increase performance.
   swapoff -a

II.- Installing Docker

1. Add the repository for the docker installation package.
   dnf config-manager --add-repo=https://download.docker.com/linux/centos/docker-ce.repo

2. Install container.io which is not yet provided by the package manager before installing docker.
   dnf install https://download.docker.com/linux/centos/7/x86_64/stable/Packages/containerd.io-1.2.6-3.3.el7.x86_64.rpm

3. Then install Docker from the repositories.
   dnf install docker-ce --nobest -y

4. Start the docker service.
   systemctl start docker

5. Make it also start automatically on server restart.
   systemctl enable docker

III.- Installing Kubernetes

1. Add the Kubernetes repository to your package manager by creating the following file.
   cat < /etc/yum.repos.d/kubernetes.repo
   [kubernetes]
   name=Kubernetes
   baseurl=https://packages.cloud.google.com/yum/repos/kubernetes-el7-\$basearch
   enabled=1
   gpgcheck=1
   repo_gpgcheck=1
   gpgkey=https://packages.cloud.google.com/yum/doc/yum-key.gpg https://packages.cloud.google.com/yum/doc/rpm-package-key.gpg
   exclude=kubelet kubeadm kubectl
   EOF

2. Then update the repo info.
   dnf upgrade -y

3. Install all the necessary components for Kubernetes.
   dnf install -y kubelet kubeadm kubectl --disableexcludes=kubernetes

4.- Start the Kubernetes services and enable them to run at startup.
systemctl enable kubelet
systemctl start kubelet

When checking the kubelet service
systemctl status kubelet
this is the result

[K8s.master01.ejto.mil]-[root]# systemctl status kubelet
● kubelet.service - kubelet: The Kubernetes Node Agent
Loaded: loaded (/usr/lib/systemd/system/kubelet.service; enabled; vendor preset: disabled)
Drop-In: /usr/lib/systemd/system/kubelet.service.d
└─10-kubeadm.conf
Active: activating (auto-restart) (Result: exit-code) since Fri 2021-08-13 10:54:41 EDT; 3s ago
Docs: https://kubernetes.io/docs/
Process: 108552 ExecStart=/usr/bin/kubelet $KUBELET_KUBECONFIG_ARGS $KUBELET_CONFIG_ARGS $KUBELET_KUBEADM_ARGS $KUBELET_EXTRA_ARGS (code=exited, status=1/FAILURE)
Main PID: 108552 (code=exited, status=1/FAILURE)
[K8s.master01.ejto.mil]-[root]#

journalctl -xeu kubelet
ago 13 10:57:39 k8s.master01.ejto.mil systemd[1]: kubelet.service: Service RestartSec=10s expired, scheduling restart.
ago 13 10:57:39 k8s.master01.ejto.mil systemd[1]: kubelet.service: Scheduled restart job, restart counter is at 265.
-- Subject: Automatic restarting of a unit has been scheduled
-- Defined-By: systemd
-- Support: https://support.oracle.com

-- Automatic restarting of the unit kubelet.service has been scheduled, as the result for
-- the configured Restart= setting for the unit.
ago 13 10:57:39 k8s.master01.ejto.mil systemd[1]: Stopped kubelet: The Kubernetes Node Agent.
-- Subject: Unit kubelet.service has finished shutting down
-- Defined-By: systemd
-- Support: https://support.oracle.com

-- Unit kubelet.service has finished shutting down.
ago 13 10:57:39 k8s.master01.ejto.mil systemd[1]: Started kubelet: The Kubernetes Node Agent.
-- Subject: Unit kubelet.service has finished start-up
-- Defined-By: systemd
-- Support: https://support.oracle.com

-- Unit kubelet.service has finished starting up.

-- The start-up result is done.
ago 13 10:57:39 k8s.master01.ejto.mil kubelet[90148]: E0813 10:57:39.113885 90148 server.go:206] "Failed to load kubelet config file" err="failed to load Kubelet config file /var/lib/kubelet/config.yaml, error failed to read kubelet config file "/var/lib/kubelet/config.yaml", error: open /var/lib/kubelet/config.yaml: no such file or directory" path="/var/lib/kubelet/config.yaml"
ago 13 10:57:39 k8s.master01.ejto.mil systemd[1]: kubelet.service: Main process exited, code=exited, status=1/FAILURE
ago 13 10:57:39 k8s.master01.ejto.mil systemd[1]: kubelet.service: Failed with result 'exit-code'.
-- Subject: Unit failed
-- Defined-By: systemd
-- Support: https://support.oracle.com

-- The unit kubelet.service has entered the 'failed' state with result 'exit-code'.

here it indicates that the file /var/lib/kubelet/config.yaml does not exist, which generates the error.

my question is, why is this file not created with the indicated installation? the installation was done with the root user, so there should be no permission problems to create the file.
thanks in advance

[neolit123]

my question is, why is this file not created with the indicated installation? the installation was done with the root user, so there should be no permission problems to create the file.
thanks in advance

the kubelet DEB/RPM packages setup the systemd unit files to have the kubelet "crash loop" (repeatedly try to start), until it has all the needed files (pointed by flags) ready. /var/lib/kubelet/config.yaml is one of them. you can stop the kubelet if you want it to not crash-loop.

https://kubernetes.io/docs/setup/production-environment/tools/kubeadm/install-kubeadm/

The kubelet is now restarting every few seconds, as it waits in a crashloop for kubeadm to tell it what to do.

if you are using "kubeadm init/join" it will provided the required files and it should start properly.

this is something that we want to change in the future:
kubernetes/kubeadm#2178

[123MwanjeMike]

This is usually because you are lacking the /var/lib/kubelet/config.yaml file

[zikkuratti]

sup googlers) error code

[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp [::1]:10248: connect: connection refused.
ExecStart=/usr/bin/kubelet $KUBELET_KUBECONFIG_ARGS $KUBELET_CONFIG_ARGS $KUBELET_KUBEADM_ARGS $KUBELET_EXTRA_ARGS (code=exited, status=1/FAILURE)

for me 22.04 help this cascade worked, but if u get troubles whith keys use(only):

( sudo apt-key adv --keyserver keyserver.ubuntu.com --recv-keys 40976EAF437D05B5 <--- your last 8 code here & sudo cp /etc/apt/trusted.gpg /etc/apt/trusted.gpg.d)

sudo apt-get update
sudo apt-get install -y apt-transport-https ca-certificates curl

curl -fsSL https://dl.k8s.io/apt/doc/apt-key.gpg | sudo gpg --dearmor -o /etc/apt/keyrings/kubernetes-archive-keyring.gpg

echo "deb [signed-by=/etc/apt/keyrings/kubernetes-archive-keyring.gpg] https://apt.kubernetes.io/ kubernetes-xenial main" | sudo tee /etc/apt/sources.list.d/kubernetes.list

sudo apt-get update
sudo apt remove kubelet kubeadm kubectl
sudo apt-get install -y kubelet kubeadm kubectl
sudo apt-mark hold kubelet kubeadm kubectl
systemctl start docker kubelet && systemctl enable docker kubelet

after that

kubeadm init --pod-network-cidr=10.244.0.0/16 --cri-socket=unix:///var/run/crio/crio.sock

do his job right!

if trouble in JOIN node !!!!
rm -rf kubelet.service.d/
rm /etc/kubernetes/pki/ca.crt
apt-get remove --purge kubelet kubeadm kubectl
apt-get install kubelet kubeadm kubectl
systemctl start docker kubelet && systemctl enable docker kubelet
systemctl status kubelete

kubeadm join 10.10.14.89:6443 --token flXXXXXXXXXXXXXXXx --discovery-token-ca-cert-hash sha256:649dXXXXXXXXXXXXXXXX7a11c94 --cri-socket=unix:///var/run/crio/crio.sock

THX ME