Kubelet v1.28.6: Unable to fetch container log stats err= failed to get fsstats

[cpsrujana]

We have installed K8s 1.28.6 on a single node Deiban. with docker as container manager. Below are the version details.
I am seeing an error in kubelet service logs. What is the reason for this error. How to resolve this issue.
Please let me know if providing any further mode details can help in understanding this issue.

$ sudo service kubelet status
● kubelet.service - Kubernetes Kubelet Server
Loaded: loaded (/etc/systemd/system/kubelet.service; enabled; vendor preset: enabled)
Active: active (running) since Mon 2024-04-15 06:52:21 UTC; 3 days ago
Docs: https://github.com/GoogleCloudPlatform/kubernetes
Main PID: 988432 (kubelet)
Tasks: 17 (limit: 35749)
Memory: 90.0M
CGroup: /system.slice/kubelet.service
└─988432 /usr/local/bin/kubelet --v=2 --node-ip=172.27.173.227 --hostname-override=k8snode --bootstrap-kubeconfig=/etc/kubernetes/bootstrap>

Apr 19 05:36:46 k8snode kubelet[988432]: E0419 05:36:46.772752 988432 cri_stats_provider.go:675] "Unable to fetch container log stats" err="failed to g>
Apr 19 05:37:02 k8snode kubelet[988432]: E0419 05:37:02.207654 988432 cri_stats_provider.go:675] "Unable to fetch container log stats" err="failed to g>
Apr 19 05:37:02 k8snode kubelet[988432]: E0419 05:37:02.207958 988432 cri_stats_provider.go:675] "Unable to fetch container log stats" err="failed to g>
Apr 19 05:37:17 k8snode kubelet[988432]: E0419 05:37:17.364224 988432 cri_stats_provider.go:675] "Unable to fetch container log stats" err="failed to g>
Apr 19 05:37:17 k8snode kubelet[988432]: E0419 05:37:17.366166 988432 cri_stats_provider.go:675] "Unable to fetch container log stats" err="failed to g>
Apr 19 05:37:28 k8snode kubelet[988432]: I0419 05:37:28.875143 988432 kubelet_getters.go:187] "Pod status updated" pod="kube-system/kube-scheduler-k8sn>
Apr 19 05:37:28 k8snode kubelet[988432]: I0419 05:37:28.875264 988432 kubelet_getters.go:187] "Pod status updated" pod="kube-system/kube-apiserver-k8sn>
Apr 19 05:37:28 k8snode kubelet[988432]: I0419 05:37:28.875318 988432 kubelet_getters.go:187] "Pod status updated" pod="kube-system/kube-controller-man>
Apr 19 05:37:32 k8snode kubelet[988432]: E0419 05:37:32.654805 988432 cri_stats_provider.go:675] "Unable to fetch container log stats" err="failed to g>
Apr 19 05:37:32 k8snode kubelet[988432]: E0419 05:37:32.655182 988432 cri_stats_provider.go:675] "Unable to fetch container log stats" err="failed to g>

sudo kubelet --version
Kubernetes v1.28.6

sudo docker --version
Docker version 20.10.20, build 9fdeb9c

sudo containerd --version
containerd containerd.io 1.6.16 31aa4358a36870b21a992d3ad2bef29e1d693bec

sudo runc --version
runc version 1.1.4
commit: v1.1.4-0-g5fd4c4d
spec: 1.0.2-dev
go: go1.18.10
libseccomp: 2.5.1

[tamilselvan1102]

/sig node

[k8s-triage-robot]

The Kubernetes project currently lacks enough contributors to adequately respond to all issues.

This bot triages un-triaged issues according to the following rules:

• After 90d of inactivity, lifecycle/stale is applied
• After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
• After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

• Mark this issue as fresh with /remove-lifecycle stale
• Close this issue with /close
• Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle stale

[k8s-triage-robot]

The Kubernetes project currently lacks enough active contributors to adequately respond to all issues.

This bot triages un-triaged issues according to the following rules:

• After 90d of inactivity, lifecycle/stale is applied
• After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
• After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

• Mark this issue as fresh with /remove-lifecycle rotten
• Close this issue with /close
• Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle rotten

[k8s-triage-robot]

The Kubernetes project currently lacks enough active contributors to adequately respond to all issues and PRs.

This bot triages issues according to the following rules:

• After 90d of inactivity, lifecycle/stale is applied
• After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
• After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

• Reopen this issue with /reopen
• Mark this issue as fresh with /remove-lifecycle rotten
• Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/close not-planned

[k8s-ci-robot]

@k8s-triage-robot: Closing this issue, marking it as "Not Planned".

In response to this:

The Kubernetes project currently lacks enough active contributors to adequately respond to all issues and PRs.

This bot triages issues according to the following rules:

• After 90d of inactivity, lifecycle/stale is applied
• After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
• After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

• Reopen this issue with /reopen
• Mark this issue as fresh with /remove-lifecycle rotten
• Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/close not-planned

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.