Join GitHub today
Slow CNI cmdDel processing causes infra container to be deleted prematurely #89440
What happened: When a pod is deleted, it calls cmdDel on multus which in turn calls sriov-cni plugin multiple times for all the interfaces. This process takes time, but in the middle of it the infra container gets killed - and network namespace gets deleted.
What you expected to happen: Network namespace should continue to exist until CNI plugin is done with the cmdDel command. i.e. do not kill infra container out of band.
How to reproduce it (as minimally and precisely as possible):
Anything else we need to know?:
Suggestion: A fix where filtering out of pods based on termination status should also have some grace period. Likely here: https://github.com/kubernetes/kubernetes/blob/master/pkg/kubelet/kubelet_pods.go#L987
Environment: Master and previous versions
This issue addresses "make sure pause container is alive while cni is detaching devices from pause container"
I tested the PR #89667 and I observe that pause container is deleted before cni can delete all the devices. It does not fix this issue.
@thockin This is a different issue (sounds similar, I agree).
This one is critical: pod spec's containers are deleted, infra container gets deleted but CNI may still be working. The desire is not to remove the infra container until CNI is done.
This addresses a different problem but will automatically address issue #88543.
@kmala Your opinion will be useful here.
The issue i fixed #89667 is different from this and i looked into the PR #89541 and it is not the correct fix for my issue because it introduces a grace period to update the status of pod to status manager and after that grace period pod will be deleted irrespective of the network resources getting removed.