Document that pods hanging in terminating if DRA driver is truly gone is WAI (re: 129402 discussion)

[pohly]

If the DRA driver is well and truly gone, despite all the retry and reconciliation loops, a pod will be stuck in Terminating for as long as its NodeUnprepareResources call has not been fulfilled without error, which is (currently) impossible without a kubelet connection to the driver.

This is also true for networking plugins (as discussed in kubernetes/kubernetes#129402 (comment)), and volumes/CSI drivers (kubernetes/kubernetes#129402 (comment)) which have external services that handle the cleanup asynchronously, and sometimes untracked, by the pod phasing. Device Plugins don't have this issue (though they are at risk of leaving stuff lying around -- per kubernetes/kubernetes#129402 (comment)).

This issue is to document this behavior as it pertains to DRA and describe how it is WAI and what the mediation steps available to a cluster administrator are.

[k8s-ci-robot]

This issue is currently awaiting triage.

SIG Docs takes a lead on issue triage for this website, but any Kubernetes member can accept issues by applying the triage/accepted label.

The triage/accepted label can be added by org members by writing /triage accepted in a comment.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

[pohly]

/wg device-management
/assign

[pohly]

Discussion of alternatives in https://kubernetes.slack.com/archives/C0409NGC1TK/p1746001550655339.

TLDR: skipping under certain conditions is bound to also sometimes skip incorrectly when the driver gets reinstalled later, which breaks a fundamental guarantee of DRA and thus could have unknown and worse consequences.