RFC: dra: support skipping NodePrepareResource and NodeUnprepareResource

[pohly]

What type of PR is this?

/kind feature
/kind api-change

What this PR does / why we need it:

This enables writing DRA drivers where the part local to a node isn't using CDI and CRI. For example, an NRI plugin might act upon the allocated resource.

Special notes for your reviewer:

Using a new special value for the DriverName is simple. However, like any alternative (for example, a separate field) it will cause problems when the kubelet handling the pod does not know about it: then the kubelet will try to find a non-existent "nop.k8s.io" plugin and fail to prepare.

Does this PR introduce a user-facing change?

DRA: when using the new kubelet on a node, allocated ResourceClaims can skip the local prepare/unprepare operations and thus run without a DRA driver on the node. This enables additional use cases for DRA beyond the traditional injection of resources into the container.

Additional documentation e.g., KEPs (Kubernetes Enhancement Proposals), usage docs, etc.:

- [KEP]: https://github.com/kubernetes/enhancements/issues/3063

[k8s-ci-robot]

Skipping CI for Draft Pull Request.
If you want CI signal for your change, please convert it to an actual PR.
You can still manually trigger a test run with /test all

[k8s-ci-robot]

This issue is currently awaiting triage.

If a SIG or subproject determines this is a relevant issue, they will accept it by applying the triage/accepted label and provide further guidance.

The triage/accepted label can be added by org members by writing /triage accepted in a comment.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[pohly]

The alternative that I had considered was something like:

Usage []string

where the empty set is the default behavior (call plugin). The same could also be specified explicitly with usage: [ cdi ].

usage: [ nop ] could indicate "do nothing".

If we ever do something with CNI, usage: [ cni ] could be added, perhaps even in combination (usage: [ cdi, cni ]).

I got stuck a bit when wondering whether all strings need to be pre-defined. Probably, otherwise we have a problem when a future release adds a new value that an old kubelet doesn't support. We want that old kubelet to fail when encountering an unknown value instead of silently ignoring it.

[kad]

NRI doesn't know anything about the high level objects like claims or allocation. To pass information to NRI something on kubelet should get info from claim object and convert it to something that will be passed down to runtime.
So, even were the cdi/cni ways of using might be feasible (I'd prefer to call it more container/pod resources rather than cdi/cni), but for full skip of kubelet part it doesn't make sense.

[kad]

/cc

[k8s-ci-robot]

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: mimani68, pohly
Once this PR has been reviewed and has the lgtm label, please assign liggitt for approval. For more information see the Kubernetes Code Review Process.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

• pkg/apis/OWNERS
• pkg/generated/openapi/OWNERS
• pkg/kubelet/cm/OWNERS
• staging/src/k8s.io/api/OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[alexzielenski]

/remove-sig api-machinery
/sig scheduling

[bart0sh]

@pohly do we still need this PR? If not, please consider to close it, thanks.

[pohly]

There is an alternative that should work without any changes: just declaring pod.spec.resourceClaims without referencing them in pod.spec.containers[].resources.claims should also cause kubelet to not call NodePrepareResource. I had suggested that to some colleagues, but haven't heard back.

Before we close this PR, let me add an E2E test for this and a KEP update which documents this.

[pohly]

See #118786

[pohly]

/close

We agreed that just declaring pod.spec.resourceClaims without referencing them in pod.spec.containers[].resources.claims should work, and the pending PR restores that behavior for Kubernetes 1.28.

[k8s-ci-robot]

@pohly: Closed this PR.

In response to this:

/close

We agreed that just declaring pod.spec.resourceClaims without referencing them in pod.spec.containers[].resources.claims should work, and the pending PR restores that behavior for Kubernetes 1.28.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.