Skip to content

Adding Alpha criteria for KEP to support IPPR & Pod Level Resources #5423

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
wants to merge 1 commit into
base: master
Choose a base branch
from
Open

Adding Alpha criteria for KEP to support IPPR & Pod Level Resources #5423

wants to merge 1 commit into from
+965 −0

Conversation

ndixita
Copy link
Contributor

@ndixita ndixita commented Jun 18, 2025 •
edited
Loading

  • One-line PR description: Split KEP from Pod level resources #2837 to add the alpha criteria for In place pod-level resources resize

@k8s-ci-robot
Copy link
Contributor

Skipping CI for Draft Pull Request.
If you want CI signal for your change, please convert it to an actual PR.
You can still manually trigger a test run with /test all

@k8s-ci-robot k8s-ci-robot added do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. labels Jun 18, 2025
@k8s-ci-robot k8s-ci-robot added kind/kep Categorizes KEP tracking issues and PRs modifying the KEP directory sig/node Categorizes an issue or PR as relevant to SIG Node. size/XL Denotes a PR that changes 500-999 lines, ignoring generated files. labels Jun 18, 2025
@ndixita ndixita force-pushed the pod-level-resources-ippr-alpha branch from 4e712ae to 71e896c Compare June 19, 2025 04:57
@ndixita ndixita mentioned this pull request Jun 19, 2025
Open
4 tasks
@ndixita ndixita force-pushed the pod-level-resources-ippr-alpha branch from 71e896c to c561b22 Compare June 19, 2025 05:05
@ndixita ndixita marked this pull request as ready for review June 19, 2025 05:17
@k8s-ci-robot k8s-ci-robot removed the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Jun 19, 2025
@ndixita
Copy link
Contributor Author

ndixita commented Jun 19, 2025

/assign @tallclair
/assign @soltysh

@ndixita ndixita force-pushed the pod-level-resources-ippr-alpha branch from c561b22 to 287a985 Compare June 19, 2025 06:05
ndixita
ndixita commented Jun 19, 2025
View reviewed changes
keps/sig-node/5419-pod-level-resources-in-place-resize/README.md
3. Decrease pod-level cgroup (if needed)
4. Increase container resources

Note: As part of a resize operation, users will now be permitted to add or modify pod-level resources within the Pod specification. Upon successful update, the Kubelet will ensure its internal checkpoint for the Pod is updated to reflect these new resource definitions. Importantly, to optimize performance and prevent redundant operations, the Kubelet will only trigger an actual cgroup resize for the Pod's sandbox if the specified pod-level resources are not equal to the aggregated sum of the individual container-level resources.
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@tallclair I couldn't find your comment because I changed the origin of my branch. I tried to address your question about allowing adding pod-level resources as a part of resize. PTAL

@ndixita ndixita mentioned this pull request Jun 19, 2025
Merged
@ndixita
Copy link
Contributor Author

ndixita commented Jun 19, 2025

/assign @mrunalp

soltysh
soltysh reviewed Jun 19, 2025
View reviewed changes
Copy link
Contributor

@soltysh soltysh left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is mostly ok, the two bits that I'd like to get clarified before approving are:

  1. defaulting
  2. Resoruces field in PodStatus.

keps/sig-node/5419-pod-level-resources-in-place-resize/README.md
a pod with pod-level resources demonstrates several key aspects of this behavior,
showing how containers without explicit limits (which inherit pod-level limits) interact
with resize policy, and how containers with specified resources remain unaffected by
pod-level resizes.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What happens if the resizePolicy is not specified at all? Can you describe the defaulting mechanism being applied here as well? I believe NotRequired is the default based on looking at our API.

@k8s-ci-robot
Copy link
Contributor

@ndixita: Can not set label lead-opted-in: Must be member in one of these teams: [release-team-enhancements release-team-leads sig-api-machinery-leads sig-apps-leads sig-architecture-leads sig-auth-leads sig-autoscaling-leads sig-cli-leads sig-cloud-provider-leads sig-cluster-lifecycle-leads sig-contributor-experience-leads sig-docs-leads sig-instrumentation-leads sig-k8s-infra-leads sig-multicluster-leads sig-network-leads sig-node-leads sig-release-leads sig-scalability-leads sig-scheduling-leads sig-security-leads sig-storage-leads sig-testing-leads sig-windows-leads]

In response to this:

/label lead-opted-in

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

@ndixita ndixita force-pushed the pod-level-resources-ippr-alpha branch from 287a985 to 0aacd00 Compare June 19, 2025 17:34
@k8s-ci-robot
Copy link
Contributor

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: ndixita
Once this PR has been reviewed and has the lgtm label, please ask for approval from mrunalp, soltysh. For more information see the Code Review Process.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

mrunalp
mrunalp reviewed Jun 19, 2025
View reviewed changes
@ndixita ndixita force-pushed the pod-level-resources-ippr-alpha branch 2 times, most recently from 7b5a42b to 9445a26 Compare June 19, 2025 18:48
@ndixita
Copy link
Contributor Author

ndixita commented Jun 19, 2025

This is mostly ok, the two bits that I'd like to get clarified before approving are:

  1. defaulting
  2. Resoruces field in PodStatus.

@soltysh added some details in #### Surfacing Pod Resource Requirements to add some clarity. PTAL

@ndixita ndixita force-pushed the pod-level-resources-ippr-alpha branch from 9445a26 to 01f01ff Compare June 19, 2025 23:56
toVersus
toVersus reviewed Jun 20, 2025
View reviewed changes
keps/sig-node/5419-pod-level-resources-in-place-resize/README.md

## Summary

This KEP proposes extending the existing "In-Place Pod Resize" (IPPR) functionality to support resources specified at the Pod level, building upon the foundations laid by KEP-2837: Pod Level Resource Specifications. Currently, IPPR primarily focuses on dynamically adjusting container-level resource allocations. With the introduction of pod-level resource limits and requests (as proposed in ((KEP#2387)[https://github.com/kubernetes/enhancements/blob/master/keps/sig-node/2837-pod-level-resource-spec/README.md]), it becomes essential to enable the in-place resizing of these aggregate pod-level resources without requiring a full pod restart. This enhancement will provide greater flexibility and efficiency in managing the overall resource consumption of multi-container pods.
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: broken link

Suggested change
This KEP proposes extending the existing "In-Place Pod Resize" (IPPR) functionality to support resources specified at the Pod level, building upon the foundations laid by KEP-2837: Pod Level Resource Specifications. Currently, IPPR primarily focuses on dynamically adjusting container-level resource allocations. With the introduction of pod-level resource limits and requests (as proposed in ((KEP#2387)[https://github.com/kubernetes/enhancements/blob/master/keps/sig-node/2837-pod-level-resource-spec/README.md]), it becomes essential to enable the in-place resizing of these aggregate pod-level resources without requiring a full pod restart. This enhancement will provide greater flexibility and efficiency in managing the overall resource consumption of multi-container pods.
This KEP proposes extending the existing "In-Place Pod Resize" (IPPR) functionality to support resources specified at the Pod level, building upon the foundations laid by KEP-2837: Pod Level Resource Specifications. Currently, IPPR primarily focuses on dynamically adjusting container-level resource allocations. With the introduction of pod-level resource limits and requests (as proposed in ([KEP#2387](https://github.com/kubernetes/enhancements/blob/master/keps/sig-node/2837-pod-level-resource-spec/README.md)), it becomes essential to enable the in-place resizing of these aggregate pod-level resources without requiring a full pod restart. This enhancement will provide greater flexibility and efficiency in managing the overall resource consumption of multi-container pods.

keps/sig-node/5419-pod-level-resources-in-place-resize/README.md
Comment on lines +137 to +138
Update]
(https://github.com/kubernetes/enhancements/blob/master/keps/sig-node/1287-in-place-update-pod-resources/README.md#api-changes)
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: broken link

Suggested change
Update]
(https://github.com/kubernetes/enhancements/blob/master/keps/sig-node/1287-in-place-update-pod-resources/README.md#api-changes)
Update](https://github.com/kubernetes/enhancements/blob/master/keps/sig-node/1287-in-place-update-pod-resources/README.md#api-changes)

keps/sig-node/5419-pod-level-resources-in-place-resize/README.md
apiVersion: v1
kind: Pod
metadata:
name: pod-level-resources
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit:

Suggested change
name: pod-level-resources
name: pod-level-resources

keps/sig-node/5419-pod-level-resources-in-place-resize/README.md
capacity is unrealistic and cumbersome.

To address this, this KEP proposes reusing the field called `Resources`
from the Pod Status. This field will provide a clear and readily available
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The Resources field is an aggregate of the values specified by the user, so it doesn't always reflect whether those values can actually be applied. On the other hand, the AllocatedResources field likely reflects the values that have actually been applied by the kubelet. Will components like the kube-scheduler and the Cluster Autoscaler refer to the Resources field instead of AllocatedResources field?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@mrunalp mrunalp mrunalp left review comments

@soltysh soltysh soltysh left review comments

@toVersus toVersus toVersus left review comments

@derekwaynecarr derekwaynecarr Awaiting requested review from derekwaynecarr

@johnbelamaric johnbelamaric Awaiting requested review from johnbelamaric

@kikisdeliveryservice kikisdeliveryservice Awaiting requested review from kikisdeliveryservice

Assignees

@mrunalp mrunalp

@soltysh soltysh

@tallclair tallclair

Labels
cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. kind/kep Categorizes KEP tracking issues and PRs modifying the KEP directory sig/node Categorizes an issue or PR as relevant to SIG Node. size/XL Denotes a PR that changes 500-999 lines, ignoring generated files.
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
6 participants
@ndixita @k8s-ci-robot @mrunalp @soltysh @toVersus @tallclair