Failed to delete cgroup paths

[shaneing]

What happened:

The kubelet occur error and the error logs as follows:

Dec 24 16:56:15 k8s-node kubelet: I1224 16:56:15.947112    7140 setters.go:77] Using node IP: "10.202.41.216"
Dec 24 16:56:17 k8s-node kubelet: W1224 16:56:17.618290    7140 kubelet_getters.go:300] Path "/var/lib/kubelet/pods/395553c4-7fa8-44d6-b3eb-462be9115c9b/volumes" does not exist
Dec 24 16:56:17 k8s-node kubelet: W1224 16:56:17.618327    7140 kubelet_getters.go:300] Path "/var/lib/kubelet/pods/d5d909cd-bf2f-4534-ab88-ad1be5d778e5/volumes" does not exist
Dec 24 16:56:17 k8s-node kubelet: W1224 16:56:17.618348    7140 kubelet_getters.go:300] Path "/var/lib/kubelet/pods/67b4f5a2-f08d-460d-95c2-fe64f7ff63b4/volumes" does not exist
Dec 24 16:56:17 k8s-node kubelet: W1224 16:56:17.618369    7140 kubelet_getters.go:300] Path "/var/lib/kubelet/pods/793ee7b8-c915-4e98-8553-1928ac626c70/volumes" does not exist
Dec 24 16:56:17 k8s-node kubelet: W1224 16:56:17.618512    7140 kubelet_getters.go:300] Path "/var/lib/kubelet/pods/9fd24c82-9635-4ac3-ba35-e20cbe756d75/volumes" does not exist
Dec 24 16:56:17 k8s-node kubelet: W1224 16:56:17.618534    7140 kubelet_getters.go:300] Path "/var/lib/kubelet/pods/10bb1377-ac8f-4089-8b60-5cb7b9886f76/volumes" does not exist
Dec 24 16:56:17 k8s-node kubelet: W1224 16:56:17.818980    7140 pod_container_manager_linux.go:200] failed to delete cgroup paths for [kubepods burstable pod10bb1377-ac8f-4089-8b60-5cb7b9886f76] : unable to destroy cgroup paths for cgroup [kubepods burstable pod10bb1377-ac8f-4089-8b60-5cb7b9886f76] : Failed to remove paths: map[blkio:/sys/fs/cgroup/blkio/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod10bb1377_ac8f_4089_8b60_5cb7b9886f76.slice cpu:/sys/fs/cgroup/cpu,cpuacct/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod10bb1377_ac8f_4089_8b60_5cb7b9886f76.slice cpuacct:/sys/fs/cgroup/cpu,cpuacct/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod10bb1377_ac8f_4089_8b60_5cb7b9886f76.slice cpuset:/sys/fs/cgroup/cpuset/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod10bb1377_ac8f_4089_8b60_5cb7b9886f76.slice devices:/sys/fs/cgroup/devices/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod10bb1377_ac8f_4089_8b60_5cb7b9886f76.slice freezer:/sys/fs/cgroup/freezer/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod10bb1377_ac8f_4089_8b60_5cb7b9886f76.slice hugetlb:/sys/fs/cgroup/hugetlb/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod10bb1377_ac8f_4089_8b60_5cb7b9886f76.slice memory:/sys/fs/cgroup/memory/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod10bb1377_ac8f_4089_8b60_5cb7b9886f76.slice net_cls:/sys/fs/cgroup/net_cls,net_prio/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod10bb1377_ac8f_4089_8b60_5cb7b9886f76.slice net_prio:/sys/fs/cgroup/net_cls,net_prio/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod10bb1377_ac8f_4089_8b60_5cb7b9886f76.slice perf_event:/sys/fs/cgroup/perf_event/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod10bb1377_ac8f_4089_8b60_5cb7b9886f76.slice pids:/sys/fs/cgroup/pids/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod10bb1377_ac8f_4089_8b60_5cb7b9886f76.slice systemd:/sys/fs/cgroup/systemd/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod10bb1377_ac8f_4089_8b60_5cb7b9886f76.slice]
Dec 24 16:56:17 k8s-node kubelet: W1224 16:56:17.832408    7140 pod_container_manager_linux.go:200] failed to delete cgroup paths for [kubepods burstable pod793ee7b8-c915-4e98-8553-1928ac626c70] : unable to destroy cgroup paths for cgroup [kubepods burstable pod793ee7b8-c915-4e98-8553-1928ac626c70] : Failed to remove paths: map[blkio:/sys/fs/cgroup/blkio/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod793ee7b8_c915_4e98_8553_1928ac626c70.slice cpu:/sys/fs/cgroup/cpu,cpuacct/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod793ee7b8_c915_4e98_8553_1928ac626c70.slice cpuacct:/sys/fs/cgroup/cpu,cpuacct/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod793ee7b8_c915_4e98_8553_1928ac626c70.slice cpuset:/sys/fs/cgroup/cpuset/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod793ee7b8_c915_4e98_8553_1928ac626c70.slice devices:/sys/fs/cgroup/devices/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod793ee7b8_c915_4e98_8553_1928ac626c70.slice freezer:/sys/fs/cgroup/freezer/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod793ee7b8_c915_4e98_8553_1928ac626c70.slice hugetlb:/sys/fs/cgroup/hugetlb/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod793ee7b8_c915_4e98_8553_1928ac626c70.slice memory:/sys/fs/cgroup/memory/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod793ee7b8_c915_4e98_8553_1928ac626c70.slice net_cls:/sys/fs/cgroup/net_cls,net_prio/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod793ee7b8_c915_4e98_8553_1928ac626c70.slice net_prio:/sys/fs/cgroup/net_cls,net_prio/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod793ee7b8_c915_4e98_8553_1928ac626c70.slice perf_event:/sys/fs/cgroup/perf_event/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod793ee7b8_c915_4e98_8553_1928ac626c70.slice pids:/sys/fs/cgroup/pids/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod793ee7b8_c915_4e98_8553_1928ac626c70.slice systemd:/sys/fs/cgroup/systemd/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod793ee7b8_c915_4e98_8553_1928ac626c70.slice]
Dec 24 16:56:17 k8s-node kubelet: W1224 16:56:17.835573    7140 pod_container_manager_linux.go:200] failed to delete cgroup paths for [kubepods burstable pod395553c4-7fa8-44d6-b3eb-462be9115c9b] : unable to destroy cgroup paths for cgroup [kubepods burstable pod395553c4-7fa8-44d6-b3eb-462be9115c9b] : Failed to remove paths: map[blkio:/sys/fs/cgroup/blkio/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod395553c4_7fa8_44d6_b3eb_462be9115c9b.slice cpu:/sys/fs/cgroup/cpu,cpuacct/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod395553c4_7fa8_44d6_b3eb_462be9115c9b.slice cpuacct:/sys/fs/cgroup/cpu,cpuacct/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod395553c4_7fa8_44d6_b3eb_462be9115c9b.slice cpuset:/sys/fs/cgroup/cpuset/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod395553c4_7fa8_44d6_b3eb_462be9115c9b.slice devices:/sys/fs/cgroup/devices/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod395553c4_7fa8_44d6_b3eb_462be9115c9b.slice freezer:/sys/fs/cgroup/freezer/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod395553c4_7fa8_44d6_b3eb_462be9115c9b.slice hugetlb:/sys/fs/cgroup/hugetlb/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod395553c4_7fa8_44d6_b3eb_462be9115c9b.slice memory:/sys/fs/cgroup/memory/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod395553c4_7fa8_44d6_b3eb_462be9115c9b.slice net_cls:/sys/fs/cgroup/net_cls,net_prio/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod395553c4_7fa8_44d6_b3eb_462be9115c9b.slice net_prio:/sys/fs/cgroup/net_cls,net_prio/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod395553c4_7fa8_44d6_b3eb_462be9115c9b.slice perf_event:/sys/fs/cgroup/perf_event/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod395553c4_7fa8_44d6_b3eb_462be9115c9b.slice pids:/sys/fs/cgroup/pids/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod395553c4_7fa8_44d6_b3eb_462be9115c9b.slice systemd:/sys/fs/cgroup/systemd/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod395553c4_7fa8_44d6_b3eb_462be9115c9b.slice]
Dec 24 16:56:17 k8s-node kubelet: W1224 16:56:17.843944    7140 pod_container_manager_linux.go:200] failed to delete cgroup paths for [kubepods burstable podd5d909cd-bf2f-4534-ab88-ad1be5d778e5] : unable to destroy cgroup paths for cgroup [kubepods burstable podd5d909cd-bf2f-4534-ab88-ad1be5d778e5] : Failed to remove paths: map[blkio:/sys/fs/cgroup/blkio/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd5d909cd_bf2f_4534_ab88_ad1be5d778e5.slice cpu:/sys/fs/cgroup/cpu,cpuacct/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd5d909cd_bf2f_4534_ab88_ad1be5d778e5.slice cpuacct:/sys/fs/cgroup/cpu,cpuacct/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd5d909cd_bf2f_4534_ab88_ad1be5d778e5.slice cpuset:/sys/fs/cgroup/cpuset/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd5d909cd_bf2f_4534_ab88_ad1be5d778e5.slice devices:/sys/fs/cgroup/devices/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd5d909cd_bf2f_4534_ab88_ad1be5d778e5.slice freezer:/sys/fs/cgroup/freezer/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd5d909cd_bf2f_4534_ab88_ad1be5d778e5.slice hugetlb:/sys/fs/cgroup/hugetlb/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd5d909cd_bf2f_4534_ab88_ad1be5d778e5.slice memory:/sys/fs/cgroup/memory/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd5d909cd_bf2f_4534_ab88_ad1be5d778e5.slice net_cls:/sys/fs/cgroup/net_cls,net_prio/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd5d909cd_bf2f_4534_ab88_ad1be5d778e5.slice net_prio:/sys/fs/cgroup/net_cls,net_prio/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd5d909cd_bf2f_4534_ab88_ad1be5d778e5.slice perf_event:/sys/fs/cgroup/perf_event/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd5d909cd_bf2f_4534_ab88_ad1be5d778e5.slice pids:/sys/fs/cgroup/pids/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd5d909cd_bf2f_4534_ab88_ad1be5d778e5.slice systemd:/sys/fs/cgroup/systemd/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd5d909cd_bf2f_4534_ab88_ad1be5d778e5.slice]
Dec 24 16:56:17 k8s-node kubelet: W1224 16:56:17.843939    7140 pod_container_manager_linux.go:200] failed to delete cgroup paths for [kubepods burstable pod9f8fcdb3-7a00-4a37-aedc-39e2301358c8] : unable to destroy cgroup paths for cgroup [kubepods burstable pod9f8fcdb3-7a00-4a37-aedc-39e2301358c8] : Failed to remove paths: map[blkio:/sys/fs/cgroup/blkio/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9f8fcdb3_7a00_4a37_aedc_39e2301358c8.slice cpu:/sys/fs/cgroup/cpu,cpuacct/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9f8fcdb3_7a00_4a37_aedc_39e2301358c8.slice cpuacct:/sys/fs/cgroup/cpu,cpuacct/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9f8fcdb3_7a00_4a37_aedc_39e2301358c8.slice cpuset:/sys/fs/cgroup/cpuset/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9f8fcdb3_7a00_4a37_aedc_39e2301358c8.slice devices:/sys/fs/cgroup/devices/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9f8fcdb3_7a00_4a37_aedc_39e2301358c8.slice freezer:/sys/fs/cgroup/freezer/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9f8fcdb3_7a00_4a37_aedc_39e2301358c8.slice hugetlb:/sys/fs/cgroup/hugetlb/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9f8fcdb3_7a00_4a37_aedc_39e2301358c8.slice memory:/sys/fs/cgroup/memory/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9f8fcdb3_7a00_4a37_aedc_39e2301358c8.slice net_cls:/sys/fs/cgroup/net_cls,net_prio/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9f8fcdb3_7a00_4a37_aedc_39e2301358c8.slice net_prio:/sys/fs/cgroup/net_cls,net_prio/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9f8fcdb3_7a00_4a37_aedc_39e2301358c8.slice perf_event:/sys/fs/cgroup/perf_event/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9f8fcdb3_7a00_4a37_aedc_39e2301358c8.slice pids:/sys/fs/cgroup/pids/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9f8fcdb3_7a00_4a37_aedc_39e2301358c8.slice systemd:/sys/fs/cgroup/systemd/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9f8fcdb3_7a00_4a37_aedc_39e2301358c8.slice]
Dec 24 16:56:17 k8s-node kubelet: W1224 16:56:17.849472    7140 pod_container_manager_linux.go:200] failed to delete cgroup paths for [kubepods burstable pod67b4f5a2-f08d-460d-95c2-fe64f7ff63b4] : unable to destroy cgroup paths for cgroup [kubepods burstable pod67b4f5a2-f08d-460d-95c2-fe64f7ff63b4] : Failed to remove paths: map[blkio:/sys/fs/cgroup/blkio/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod67b4f5a2_f08d_460d_95c2_fe64f7ff63b4.slice cpu:/sys/fs/cgroup/cpu,cpuacct/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod67b4f5a2_f08d_460d_95c2_fe64f7ff63b4.slice cpuacct:/sys/fs/cgroup/cpu,cpuacct/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod67b4f5a2_f08d_460d_95c2_fe64f7ff63b4.slice cpuset:/sys/fs/cgroup/cpuset/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod67b4f5a2_f08d_460d_95c2_fe64f7ff63b4.slice devices:/sys/fs/cgroup/devices/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod67b4f5a2_f08d_460d_95c2_fe64f7ff63b4.slice freezer:/sys/fs/cgroup/freezer/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod67b4f5a2_f08d_460d_95c2_fe64f7ff63b4.slice hugetlb:/sys/fs/cgroup/hugetlb/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod67b4f5a2_f08d_460d_95c2_fe64f7ff63b4.slice memory:/sys/fs/cgroup/memory/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod67b4f5a2_f08d_460d_95c2_fe64f7ff63b4.slice net_cls:/sys/fs/cgroup/net_cls,net_prio/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod67b4f5a2_f08d_460d_95c2_fe64f7ff63b4.slice net_prio:/sys/fs/cgroup/net_cls,net_prio/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod67b4f5a2_f08d_460d_95c2_fe64f7ff63b4.slice perf_event:/sys/fs/cgroup/perf_event/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod67b4f5a2_f08d_460d_95c2_fe64f7ff63b4.slice pids:/sys/fs/cgroup/pids/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod67b4f5a2_f08d_460d_95c2_fe64f7ff63b4.slice systemd:/sys/fs/cgroup/systemd/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod67b4f5a2_f08d_460d_95c2_fe64f7ff63b4.slice]
Dec 24 16:56:17 k8s-node kubelet: W1224 16:56:17.850218    7140 pod_container_manager_linux.go:200] failed to delete cgroup paths for [kubepods burstable pod9fd24c82-9635-4ac3-ba35-e20cbe756d75] : unable to destroy cgroup paths for cgroup [kubepods burstable pod9fd24c82-9635-4ac3-ba35-e20cbe756d75] : Failed to remove paths: map[blkio:/sys/fs/cgroup/blkio/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9fd24c82_9635_4ac3_ba35_e20cbe756d75.slice cpu:/sys/fs/cgroup/cpu,cpuacct/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9fd24c82_9635_4ac3_ba35_e20cbe756d75.slice cpuacct:/sys/fs/cgroup/cpu,cpuacct/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9fd24c82_9635_4ac3_ba35_e20cbe756d75.slice cpuset:/sys/fs/cgroup/cpuset/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9fd24c82_9635_4ac3_ba35_e20cbe756d75.slice devices:/sys/fs/cgroup/devices/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9fd24c82_9635_4ac3_ba35_e20cbe756d75.slice freezer:/sys/fs/cgroup/freezer/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9fd24c82_9635_4ac3_ba35_e20cbe756d75.slice hugetlb:/sys/fs/cgroup/hugetlb/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9fd24c82_9635_4ac3_ba35_e20cbe756d75.slice memory:/sys/fs/cgroup/memory/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9fd24c82_9635_4ac3_ba35_e20cbe756d75.slice net_cls:/sys/fs/cgroup/net_cls,net_prio/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9fd24c82_9635_4ac3_ba35_e20cbe756d75.slice net_prio:/sys/fs/cgroup/net_cls,net_prio/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9fd24c82_9635_4ac3_ba35_e20cbe756d75.slice perf_event:/sys/fs/cgroup/perf_event/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9fd24c82_9635_4ac3_ba35_e20cbe756d75.slice pids:/sys/fs/cgroup/pids/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9fd24c82_9635_4ac3_ba35_e20cbe756d75.slice systemd:/sys/fs/cgroup/systemd/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9fd24c82_9635_4ac3_ba35_e20cbe756d75.slice]

What you expected to happen:

Can delete the cgroup paths.

How to reproduce it (as minimally and precisely as possible):

Anything else we need to know?:

Environment:

Kubernetes version:

Client Version: v1.19.4
Server Version: v1.19.6

OS:

NAME="CentOS Linux"
VERSION="7 (Core)"
ID="centos"
ID_LIKE="rhel fedora"
VERSION_ID="7"
PRETTY_NAME="CentOS Linux 7 (Core)"
ANSI_COLOR="0;31"
CPE_NAME="cpe:/o:centos:centos:7"
HOME_URL="https://www.centos.org/"
BUG_REPORT_URL="https://bugs.centos.org/"

CENTOS_MANTISBT_PROJECT="CentOS-7"
CENTOS_MANTISBT_PROJECT_VERSION="7"
REDHAT_SUPPORT_PRODUCT="centos"
REDHAT_SUPPORT_PRODUCT_VERSION="7"

Kernel:

4.15.6-1.el7.elrepo.x86_64

Install tools:

• Kubespray

Network plugin and version:

• v3.16.5

[pacoxu]

kubernetes/vendor/github.com/opencontainers/runc/libcontainer/cgroups/utils.go

Lines 214 to 237 in cea1d4e

	func RemovePaths(paths map[string]string) (err error) {
	delay := 10 * time.Millisecond
	for i := 0; i < 5; i++ {
	if i != 0 {
	time.Sleep(delay)
	delay *= 2
	}
	for s, p := range paths {
	os.RemoveAll(p)
	// TODO: here probably should be logging
	_, err := os.Stat(p)
	// We need this strange way of checking cgroups existence because
	// RemoveAll almost always returns error, even on already removed
	// cgroups
	if os.IsNotExist(err) {
	delete(paths, s)
	}
	}
	if len(paths) == 0 {
	return nil
	}
	}
	return fmt.Errorf("Failed to remove paths: %v", paths)
	}

As the comment L223 says, errors of os.RemoveAll(p) should be logging.
You may try to delete those paths manually to see the error.

Currently kubernetes 1.20 use runc v1.0.0-rc92.

The log format would be better in runc next release. See opencontainers/runc#2506 & https://github.com/opencontainers/runc/blob/master/libcontainer/cgroups/utils.go#L262-L271 for more.

[neolit123]

/sig node

[ehashman]

/priority important-soon
/triage accepted
/assign

[tharun208]

@ehashman I like to work on this issue.

[ehashman]

We think this may have been fixed by #101524.

[haircommander]

I believe a combo of #101524 plus runc bump should have fixed this. If someone runs into it again, please reopen
/close

[k8s-ci-robot]

@haircommander: Closing this issue.

In response to this:

I believe a combo of #101524 plus runc bump should have fixed this. If someone runs into it again, please reopen
/close

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.