Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

mark QOSReserved as beta and enable by default #130022

Open
wants to merge 1 commit into
base: master
Choose a base branch
from
Open

mark QOSReserved as beta and enable by default #130022

wants to merge 1 commit into from
+5 −0

Conversation

pacoxu
Copy link
Member

@pacoxu pacoxu commented Feb 7, 2025 •
edited
Loading

What type of PR is this?

/kind cleanup

What this PR does / why we need it:

deprecate the FG as it is alpha since v1.11.

When the FG is enabled, kubelet will respect qosReserved setting. So I just deprecate the FG and respect the qosReserved.

Which issue(s) this PR fixes:

Fixes kubernetes/enhancements#5046

Special notes for your reviewer:

I think there are two choices

  1. deprecated it: remove the qosReserved setting.
  2. deprecated it: respect the qosReserved setting.
  3. promote it to beta and then GA: respect the qosReserved setting [This PR now]
  4. remove the fg: keep the qosReserved setting
  5. remove the fg: remove the qosReserved setting

I think 2\3\4 are OK. I prefer 2 > 4 > 3. I am also OK for 3.

Currently, I prefer to the second choice since this FG is merely a gate and you need to additionally specify the qosReserved to use this feature.

kubernetes/pkg/kubelet/cm/qos_container_manager_linux.go

Lines 348 to 377 in d36322f

if utilfeature.DefaultFeatureGate.Enabled(kubefeatures.QOSReserved) {
for resource, percentReserve := range m.qosReserved {
switch resource {
case v1.ResourceMemory:
m.setMemoryReserve(qosConfigs, percentReserve)
}
}
updateSuccess := true
for _, config := range qosConfigs {
err := m.cgroupManager.Update(config)
if err != nil {
updateSuccess = false
}
}
if updateSuccess {
klog.V(4).InfoS("Updated QoS cgroup configuration")
return nil
}
// If the resource can adjust the ResourceConfig to increase likelihood of
// success, call the adjustment function here. Otherwise, the Update() will
// be called again with the same values.
for resource, percentReserve := range m.qosReserved {
switch resource {
case v1.ResourceMemory:
m.retrySetMemoryReserve(qosConfigs, percentReserve)
}
}
}

Does this PR introduce a user-facing change?

Promote feature gate QOSReserved to beta and enable by default

Additional documentation e.g., KEPs (Kubernetes Enhancement Proposals), usage docs, etc.:

- [KEP]: https://github.com/kubernetes/enhancements/issues/5046

@k8s-ci-robot k8s-ci-robot added release-note Denotes a PR that will be considered when it comes time to generate release notes. size/XS Denotes a PR that changes 0-9 lines, ignoring generated files. kind/cleanup Categorizes issue or PR as related to cleaning up code, process, or technical debt. cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. do-not-merge/needs-sig Indicates an issue or PR lacks a `sig/foo` label and requires one. needs-triage Indicates an issue or PR lacks a `triage/foo` label and requires one. labels Feb 7, 2025
@k8s-ci-robot
Copy link
Contributor

This issue is currently awaiting triage.

If a SIG or subproject determines this is a relevant issue, they will accept it by applying the triage/accepted label and provide further guidance.

The triage/accepted label can be added by org members by writing /triage accepted in a comment.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

@k8s-ci-robot k8s-ci-robot added the needs-priority Indicates a PR lacks a `priority/foo` label and requires one. label Feb 7, 2025
@pacoxu pacoxu mentioned this pull request Feb 7, 2025
Open
4 tasks
@pacoxu
Copy link
Member Author

pacoxu commented Feb 7, 2025

/cc @haircommander @mtaufen
/assign @derekwaynecarr @sjenning

@pacoxu
Copy link
Member Author

pacoxu commented Feb 11, 2025

/sig node

@k8s-ci-robot k8s-ci-robot added sig/node Categorizes an issue or PR as relevant to SIG Node. and removed do-not-merge/needs-sig Indicates an issue or PR lacks a `sig/foo` label and requires one. labels Feb 11, 2025
bart0sh
bart0sh reviewed Feb 27, 2025
View reviewed changes
pkg/features/versioned_kube_features.go Outdated
@@ -609,6 +609,7 @@ var defaultVersionedKubernetesFeatureGates = map[featuregate.Feature]featuregate

QOSReserved: {
{Version: version.MustParse("1.11"), Default: false, PreRelease: featuregate.Alpha},
{Version: version.MustParse("1.33"), Default: true, PreRelease: featuregate.Deprecated}, // remove in 1.35
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does this trick mean that we're enabling alpha feature without going through Beta to GA, i.e. without any usage? Why deprecation is better than promoting to Beta?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
{Version: version.MustParse("1.33"), Default: true, PreRelease: featuregate.Deprecated}, // remove in 1.35
{Version: version.MustParse("1.33"), Default: true, PreRelease: featuregate.Beta},

I am OK to promote it.

  • I have concerns regarding the absence of a Kubernetes Enhancement Proposal (KEP) for this feature gate. According to the KEP process, a KEP is mandatory for all enhancements. To promote this feature to beta, it seems necessary to have a KEP and undergo a Production Readiness Review (PRR).

Deprecating this feature gate could expedite its removal; however, this approach might be somewhat aggressive.

I will update to promote it the beta then.

@pacoxu pacoxu changed the title mark QOSReserved as deprecated and enable by default mark QOSReserved as beta and enable by default Mar 4, 2025
@k8s-ci-robot
Copy link
Contributor

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: pacoxu
Once this PR has been reviewed and has the lgtm label, please ask for approval from derekwaynecarr. For more information see the Code Review Process.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@pacoxu pacoxu force-pushed the deprecate-QOSReserved branch from eb86998 to b32e5b3 Compare March 5, 2025 03:32
@pacoxu pacoxu force-pushed the deprecate-QOSReserved branch from b32e5b3 to 21e01aa Compare March 5, 2025 05:49
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@bart0sh bart0sh

@smarterclayton smarterclayton

@soltysh soltysh

@haircommander haircommander

@mtaufen mtaufen

Assignees

@sjenning sjenning

@derekwaynecarr derekwaynecarr

Labels
cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. kind/cleanup Categorizes issue or PR as related to cleaning up code, process, or technical debt. needs-priority Indicates a PR lacks a `priority/foo` label and requires one. needs-triage Indicates an issue or PR lacks a `triage/foo` label and requires one. release-note Denotes a PR that will be considered when it comes time to generate release notes. sig/node Categorizes an issue or PR as relevant to SIG Node. size/XS Denotes a PR that changes 0-9 lines, ignoring generated files.
Projects
SIG Node: code and documentation PRs
Status: Waiting on Author
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

Allows resource reservations at the QoS level preventing pods at lower QoS levels
5 participants
@pacoxu @k8s-ci-robot @bart0sh @sjenning @derekwaynecarr