dra: update for Kubernetes 1.28 #41856

pohly · 2023-07-03T14:56:25Z

Related-to: kubernetes/enhancements#3063
Fixes: #38841

sftim · 2023-07-05T10:43:51Z

/sig node

content/en/docs/concepts/scheduling-eviction/dynamic-resource-allocation.md

katcosgrove · 2023-07-24T16:29:57Z

Hello @pohly 👋 please take a look at Documenting for a release - PR Ready for Review to get your PR ready for review before Tuesday 25th July 2023. Thank you!

Several improvements under the hood which don't need to be documented here. API changes (like storing generated resource claim names in the pod status) are part of the generated API documentation. What is worth mentioning because it was listed as "limitation" before is that pre-scheduled pods are now supported better.

netlify · 2023-08-02T14:11:31Z

👷 Deploy Preview for kubernetes-io-vnext-staging processing.

Name	Link
🔨 Latest commit	`6c926cb`
🔍 Latest deploy log	https://app.netlify.com/sites/kubernetes-io-vnext-staging/deploys/64ca640d8f8c030008a310bf

pohly · 2023-08-02T14:12:26Z

PR updated. Sorry, I was on vacation and had to catch up this week.

kannon92 · 2023-08-02T14:21:29Z

content/en/docs/concepts/scheduling-eviction/dynamic-resource-allocation.md

+However, it is better to avoid this because a Pod which is assigned to a node
+blocks normal resources (RAM, CPU) that then cannot be used for other Pods
+while the Pod is stuck. To make a Pod run on a specific node while still going
+through the normal scheduling flow, create the Pod with a node selector that


Suggested change

through the normal scheduling flow, create the Pod with a node selector that

through the normal scheduling flow, create the Pod with a `nodeSelector` that

Not quite sure if this is correct but you capitalize other api fields or used code style for those. Thought it looked a bit off.

"a node selector" refers to the general concept here, not the specific field. That is then shown in the example. I think using plain English is fine here and also used elsewhere.

bart0sh · 2023-08-03T11:42:15Z

/cc

sftim

Looks ready for tech review. The suggestions here wouldn't block a merge.

sftim · 2023-08-04T23:34:30Z

content/en/docs/concepts/scheduling-eviction/dynamic-resource-allocation.md

+blocks normal resources (RAM, CPU) that then cannot be used for other Pods
+while the Pod is stuck. To make a Pod run on a specific node while still going
+through the normal scheduling flow, create the Pod with a node selector that
+matches exactly the desired node:


Suggested change

matches exactly the desired node:

exactly matches the desired node:

sftim · 2023-08-04T23:37:05Z

content/en/docs/concepts/scheduling-eviction/dynamic-resource-allocation.md

+detects this and tries to make the Pod runnable by triggering allocation and/or
+reserving the required ResourceClaims.
+
+However, it is better to avoid this because a Pod which is assigned to a node


(nit)

Suggested change

However, it is better to avoid this because a Pod which is assigned to a node

However, it is better to avoid this because a Pod that is assigned to a node

sftim · 2023-08-04T23:37:34Z

content/en/docs/concepts/scheduling-eviction/dynamic-resource-allocation.md

-future.
+## Pre-scheduled Pods
+
+When creating a Pod with `nodeName` already set, the scheduler gets bypassed.


Suggested change

When creating a Pod with `nodeName` already set, the scheduler gets bypassed.

When you - or another API client - create a Pod with `.spec.nodeName` already set, the

scheduler gets bypassed.

sftim · 2023-08-04T23:40:15Z

content/en/docs/concepts/scheduling-eviction/dynamic-resource-allocation.md

+  nodeSelector:
+    kubernetes.io/hostname: name-of-the-intended-node
+  ...
+```



Optionally:

Suggested change

You may also be able to mutate the incoming Pod, at admission time, to unset the `.spec.nodeName`

field and to use a node selector instead.

SergeyKanzhelev · 2023-08-07T19:45:46Z

content/en/docs/concepts/scheduling-eviction/dynamic-resource-allocation.md

+
+When creating a Pod with `nodeName` already set, the scheduler gets bypassed.
+If some ResourceClaim needed by that Pod does not exist yet, is not allocated
+or not reserved for the Pod, then the kubelet will fail to run the Pod and


it will be great to add instruction on how to unstuck it. I think simply deleting the pod will do. The only thing - mention that if Pod is part of the replicaset, another instance may be created with the same issue.

There are numerous reasons for what might be wrong, so listing all possible remediation here will not be possible. The goal in this section is more about raising awareness of the problem ("don't do this"!) and explain that some automatic mitigation is available now through the kube-controller-manager changes.

SergeyKanzhelev

lgtm overall, thanks

sftim · 2023-08-07T21:42:51Z

Thanks

/lgtm
/approve

Fixup PRs welcome!

k8s-ci-robot · 2023-08-07T21:42:58Z

LGTM label has been added.

Git tree hash: 9f280718ce19836c8db17404415c3fe8328bca25

k8s-ci-robot · 2023-08-07T21:42:58Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: SergeyKanzhelev, sftim

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

~~content/en/OWNERS~~ [sftim]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

This is a follow-up to kubernetes#41856 with the suggested enhancements.

pohly · 2023-08-08T07:05:17Z

Fixup PRs welcome!

Darn, too slow! Sorry, I should have update the PR yesterday. I was still catching after my vacation. See #42445 for the follow-up.

dra: update for Kubernetes 1.28

k8s-ci-robot added this to the 1.28 milestone Jul 3, 2023

k8s-ci-robot added do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. size/XS Denotes a PR that changes 0-9 lines, ignoring generated files. labels Jul 3, 2023

k8s-ci-robot requested a review from klueska July 3, 2023 14:56

k8s-ci-robot added the language/en Issues or PRs related to English language label Jul 3, 2023

k8s-ci-robot requested a review from shannonxtreme July 3, 2023 14:56

k8s-ci-robot added the sig/docs Categorizes an issue or PR as relevant to SIG Docs. label Jul 3, 2023

pohly mentioned this pull request Jul 3, 2023

dynamic resource allocation kubernetes/enhancements#3063

Open

34 tasks

k8s-ci-robot added the sig/node Categorizes an issue or PR as relevant to SIG Node. label Jul 5, 2023

sftim reviewed Jul 5, 2023

View reviewed changes

content/en/docs/concepts/scheduling-eviction/dynamic-resource-allocation.md Outdated Show resolved Hide resolved

bart0sh added this to WIP in SIG Node PR Triage Jul 12, 2023

pohly force-pushed the dra-update branch from 679d489 to 6c926cb Compare August 2, 2023 14:11

k8s-ci-robot removed the size/XS Denotes a PR that changes 0-9 lines, ignoring generated files. label Aug 2, 2023

pohly changed the title ~~WIP: dra: update for Kubernetes 1.28~~ dra: update for Kubernetes 1.28 Aug 2, 2023

k8s-ci-robot added size/M Denotes a PR that changes 30-99 lines, ignoring generated files. and removed do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. labels Aug 2, 2023

pohly mentioned this pull request Aug 2, 2023

Document using node selector for dynamic resource allocation delayed claim resolution #38841

Closed

kannon92 reviewed Aug 2, 2023

View reviewed changes

bart0sh moved this from WIP to Needs Reviewer in SIG Node PR Triage Aug 3, 2023

k8s-ci-robot requested a review from bart0sh August 3, 2023 11:42

sftim reviewed Aug 4, 2023

View reviewed changes

SergeyKanzhelev reviewed Aug 7, 2023

View reviewed changes

SergeyKanzhelev approved these changes Aug 7, 2023

View reviewed changes

k8s-ci-robot assigned sftim Aug 7, 2023

k8s-ci-robot added the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Aug 7, 2023

k8s-ci-robot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Aug 7, 2023

k8s-ci-robot merged commit d9091f3 into kubernetes:dev-1.28 Aug 7, 2023
7 checks passed

SIG Node PR Triage automation moved this from Needs Reviewer to Done Aug 7, 2023

pohly pushed a commit to pohly/website that referenced this pull request Aug 8, 2023

dra: cleanup of Kubernetes 1.28 update

98b52d5

This is a follow-up to kubernetes#41856 with the suggested enhancements.

pohly mentioned this pull request Aug 8, 2023

dra: cleanup of Kubernetes 1.28 update #42445

Merged

Rishit-dagli pushed a commit to Rishit-dagli/website that referenced this pull request Aug 12, 2023

Merge pull request kubernetes#41856 from pohly/dra-update

20d4caa

dra: update for Kubernetes 1.28

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

dra: update for Kubernetes 1.28 #41856

dra: update for Kubernetes 1.28 #41856

pohly commented Jul 3, 2023 •

edited

sftim commented Jul 5, 2023

katcosgrove commented Jul 24, 2023

netlify bot commented Aug 2, 2023

pohly commented Aug 2, 2023

kannon92 Aug 2, 2023

pohly Aug 8, 2023

bart0sh commented Aug 3, 2023

sftim left a comment

sftim Aug 4, 2023

sftim Aug 4, 2023

sftim Aug 4, 2023

sftim Aug 4, 2023

SergeyKanzhelev Aug 7, 2023

pohly Aug 8, 2023

SergeyKanzhelev left a comment

sftim commented Aug 7, 2023

k8s-ci-robot commented Aug 7, 2023

k8s-ci-robot commented Aug 7, 2023

pohly commented Aug 8, 2023

	through the normal scheduling flow, create the Pod with a node selector that
	through the normal scheduling flow, create the Pod with a `nodeSelector` that

	matches exactly the desired node:
	exactly matches the desired node:

	However, it is better to avoid this because a Pod which is assigned to a node
	However, it is better to avoid this because a Pod that is assigned to a node

	When creating a Pod with `nodeName` already set, the scheduler gets bypassed.
	When you - or another API client - create a Pod with `.spec.nodeName` already set, the
	scheduler gets bypassed.



	You may also be able to mutate the incoming Pod, at admission time, to unset the `.spec.nodeName`
	field and to use a node selector instead.

dra: update for Kubernetes 1.28 #41856

dra: update for Kubernetes 1.28 #41856

Conversation

pohly commented Jul 3, 2023 • edited

sftim commented Jul 5, 2023

katcosgrove commented Jul 24, 2023

netlify bot commented Aug 2, 2023

👷 Deploy Preview for kubernetes-io-vnext-staging processing.

pohly commented Aug 2, 2023

Choose a reason for hiding this comment

Choose a reason for hiding this comment

bart0sh commented Aug 3, 2023

sftim left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

SergeyKanzhelev left a comment

Choose a reason for hiding this comment

sftim commented Aug 7, 2023

k8s-ci-robot commented Aug 7, 2023

k8s-ci-robot commented Aug 7, 2023

pohly commented Aug 8, 2023

pohly commented Jul 3, 2023 •

edited