promtail and grafana agent operator are stuck in CreateContainerError

[cwrx777]

Hi,

What's common with Promtail and Grafana agent operator that are both stuck in CreateContainerError?
Environment:

1. RHEL 8.9
2. k3s 1.29, selinux enabled
3. installed promtail using helm. (promtail-6.15.5)
4. installed grafana-agent-operator as part of loki installation using helm loki-5.47.1, in singlebinary installation.
   • other pods, such as loki, loki-canary are running.
5. installed grafana using helm grafana-7.3.7, - running

error message:
grafana- agent operator:

Warning  Failed  8m52s (x1473 over 7h41m)  kubelet  (combined from similar events): Error: failed to create containerd container: error unpacking image: failed to extract layer sha256:da6ffece931cbcf6c3b04a1a9ab1251003332cd553c121ed174973020a796682: failed to unmount /k3s/data/agent/containerd/tmpmounts/containerd-mount3607510181: failed to unmount target /k3s/data/agent/containerd/tmpmounts/containerd-mount3607510181: device or resource busy: unknown

promtail:

Warning  Failed  15m (x1518 over 8h)  kubelet  (combined from similar events): Error: failed to create containerd container: error unpacking image: failed to extract layer sha256:3c8879ab2cf25eb049af95711715b5b99d97490ab4eedbe8d0f160f53ef5a645: failed to unmount /k3s/data/agent/containerd/tmpmounts/containerd-mount3648689416: failed to unmount target /k3s/data/agent/containerd/tmpmounts/containerd-mount3648689416: device or resource busy: unknown

[JStickler]

Questions have a better chance of being answered if you ask them on the community forums.