Support memory qos with cgroups v2

[xiaoxubeii]

Enhancement Description

• One-line enhancement description (can be used as a release note): Support memory qos with cgroups v2
• Kubernetes Enhancement Proposal: https://github.com/kubernetes/enhancements/tree/master/keps/sig-node/2570-memory-qos
• Discussion Link: https://docs.google.com/document/d/1r9Fx2omdMLP8SJ59np-C6uBqeuS4TSbQc5ChmQWMYJ8/edit?usp=sharing
• Primary contact (assignee): @xiaoxubeii
• Responsible SIGs: node
• Enhancement target (which target equals to which milestone):
  • Alpha release target (x.y): v1.22
  • Beta release target (x.y):
  • Stable release target (x.y):
• Alpha
  • KEP (k/enhancements) update PR(s):
    • KEP-2570: Support memory qos with cgroups v2 #2571
  • Code (k/k) update PR(s):
    • Feature: add unified on CRI to support cgroup v2 kubernetes#102578
    • Feature: Support memory qos with cgroups v2 kubernetes#102970
  • Docs (k/website) update PR(s):
    • Support Memory QoS with cgroups v2 for 1.22 #2570 website#28566

[xiaoxubeii]

/sig node

[xiaoxubeii]

/assign @xiaoxubeii

[MadhavJivrajani]

Hi! This sounds really interesting and I'd love to help out, please let me know how I can help out with this!

[ehashman]

/stage alpha
/milestone v1.22

[gracenng]

Hi @xiaoxubeii 👋 1.22 Enhancements Shadow here.

This enhancement is in good shape, some minor change requests in light of Enhancement Freeze on Thursday May 13th:

• Update kep.yaml file to the latest template
• In kep.yaml, status is currently provisional instead of implementable
• Alpha graduation criteria missing
• KEP not merged to master

Thanks!

[gracenng]

Hi @xiaoxubeii 👋 1.22 Enhancements shadow here.

To help SIG's be aware of their workload, I just wanted to check to see if SIG-Node will need to do anything for this enhancement and if so, are they OK with it?
Thanks!

[xiaoxubeii]

@gracenng Hey grace, I have updated necessary contents as follows:

• update kep.yaml for prr approval
• add Alpha graduation criteria

sig-node approvers @derekwaynecarr @mrunalp are reviewing for that. I am waiting for lgtm/approve and merge as implementable.

[xiaoxubeii]

@gracenng sig-node approvers(Derek and Mrunal) have gave lgtm/approve. There are few prr review requests, I have updated and am waiting for next review round. I think we can catch up with the freeze day :)

[gracenng]

Hi @xiaoxubeii , looks like your PRR was approved and the requested changes are all here. I have updated the status of this enhancement to tracked
Thank you for keeping me updated!

[xiaoxubeii]

Thanks for your help. Also thanks very much to a lot of valuable review suggestions and helps from @derekwaynecarr @mrunalp @bobbypage @giuseppe @odinuge @johnbelamaric @ehashman
Really appreciate that :)

[ritpanjw]

Hello @xiaoxubeii 👋 , 1.22 Docs Shadow here.

This enhancement is marked as Needs Docs for 1.22 release.
Please follow the steps detailed in the documentation to open a PR against dev-1.22 branch in the k/website repo. This PR can be just a placeholder at this time and must be created before Fri July 9, 11:59 PM PDT.
Also, take a look at Documenting for a release to familiarize yourself with the docs requirement for the release.

Thank you!

[xiaoxubeii]

@ritpanjw OK, thanks for reminding.

[gracenng]

Hi @xiaoxubeii 🌞 1.22 enhancements shadow here.

In light of Code Freeze on July 8th, this enhancement current status is tracked, and we're currently tracking kubernetes/kubernetes#102578 kubernetes/kubernetes/pull/102970

Please let me know if there is other code PR associated with this enhancement.

Thanks

[xiaoxubeii]

Hi @xiaoxubeii 🌞 1.22 enhancements shadow here.

In light of Code Freeze on July 8th, this enhancement current status is tracked, and we're currently tracking kubernetes/kubernetes#102578 kubernetes/kubernetes/pull/102970

Please let me know if there is other code PR associated with this enhancement.

Thanks

@gracenng It is all here, thanks.

[k8s-triage-robot]

The Kubernetes project currently lacks enough contributors to adequately respond to all issues and PRs.

This bot triages issues and PRs according to the following rules:

• After 90d of inactivity, lifecycle/stale is applied
• After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
• After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

• Mark this issue or PR as fresh with /remove-lifecycle stale
• Mark this issue or PR as rotten with /lifecycle rotten
• Close this issue or PR with /close
• Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle stale

[pacoxu]

I opened kubernetes/website#40513 and I will update it ASAP.

[SergeyKanzhelev]

as per the sig node meeting on 5/2/2023 this will be worked on in 1.28

/milestone v1.28

[SergeyKanzhelev]

/stage beta

[SergeyKanzhelev]

/label lead-opted-in

[npolshakova]

Hi @xiaoxubeii 👋, Enhancements team here!

Just checking in as we approach enhancements freeze on 01:00 UTC Friday, 16th June 2023.

This enhancement is targeting for stage beta for 1.28 (correct me, if otherwise.)

Here's where this enhancement currently stands:

• KEP readme using the latest template has been merged into the k/enhancements repo.
• KEP status is marked as implementable for latest-milestone: 1.28
• KEP readme has a updated detailed test plan section filled out
• KEP readme has up to date graduation criteria
• KEP has a production readiness review that has been completed and merged into k/enhancements.

For this KEP, we would just need to update the following:

• update the latest milestone to 1.28
• update beta to 1.28

The status of this enhancement is marked as at risk. Please keep the issue description up-to-date with appropriate stages as well. Thank you!

[ndixita]

Thanks @npolshakova for bringing this up. This KEP is targeting beta in 1.28. I will go ahead and take care of the action items here on Monday.

cc: @xiaoxubeii

[johnbelamaric]

Please be sure to update the PRR questions for beta.

[npolshakova]

Hi @xiaoxubeii 👋, just checking in before the enhancements freeze on 01:00 UTC Friday, 16th June 2023. The status for this enhancement is at risk.

For this KEP, we would just need to update the following:

• update the latest milestone to 1.28
• update beta to 1.28
• answer the PRR questions for beta and get an approval

Let me know if I missed anything. Thanks!

[SergeyKanzhelev]

@npolshakova this should be good for 1.28 now

[AdminTurnedDevOps]

Hey @xiaoxubeii

1.28 Docs Shadow here.

Does this enhancement work planned for 1.28 require any new docs or modification to existing docs?

If so, please follows the steps here to open a PR against dev-1.28 branch in the k/website repo. This PR can be just a placeholder at this time and must be created before Thursday 20th July 2023.

Also, take a look at Documenting for a release to get yourself familiarize with the docs requirement for the release.

Thank you!

[npolshakova]

Hey again @xiaoxubeii 👋

Just checking in as we approach Code freeze at 01:00 UTC Friday, 19th July 2023 .

Here’s the enhancement’s state for the upcoming code freeze:

• All the PRs that are related to your enhancement are linked in the above issue description (for tracking purposes). This includes code, tests, and documentation related PR/s.
• All code related PR/s are merged or are in merge-ready state ( i.e they have approved and lgtm labels applied) by the code freeze deadline. This includes any tests related PR/s too.
  • Feature: add unified on CRI to support cgroup v2 kubernetes#102578
  • Feature: Support memory qos with cgroups v2 kubernetes#102970

For this enhancement, it looks like the following code related PR/s are open and they need to be merged or should be in merge-ready state before the code freeze commences :

• [postponed]promote MemoryQoS to Beta kubernetes#118699

Also please let me know if there are other PRs in k/k we should be tracking for this KEP.
As always, we are here to help if any questions come up. Thanks!

[Rishit-dagli]

Hey @xiaoxubeii , could you please create a docs PR even if it is a draft PR with no content yet against dev-1.28 branch in the k/website repo. The deadline to create this draft PR is Thursday 20th July 2023.

[pacoxu]

Thanks for @ndixita detailed test: https://docs.google.com/document/d/1mY0MTT34P-Eyv5G1t_Pqs4OWyIH-cg9caRKWmqYlSbI/edit?usp=sharing.

🚨🚨 Sometimes the application pod is stuck for throttling memory. This is a worse behavior than OOM kill.
So we decided to postpone promoting this feature until we can gracefully handle this issue.

See kubernetes/kubernetes#118699 (comment) as well.

So KEP needs an update.

[Atharva-Shinde]

Thanks for @ndixita detailed test: https://docs.google.com/document/d/1mY0MTT34P-Eyv5G1t_Pqs4OWyIH-cg9caRKWmqYlSbI/edit?usp=sharing.
🚨🚨 Sometimes the application pod is stuck for throttling memory. This is a worse behavior than OOM kill.
So we decided to postpone promoting this feature until we can gracefully handle this issue.
See kubernetes/kubernetes#118699 (comment) as well.
So KEP needs an update.

Hello @ndixita @pacoxu with reference to above comment, I am removing this KEP from the current milestone.
/milestone clear
/remove label lead-opted-in

[npolshakova]

Hello @ndixita, 1.29 Enhancements team here! Is this enhancement targeting 1.29? If it is, can you follow the instructions here to opt in the enhancement and make sure the lead-opted-in label is set so it can get added to the tracking board? Thanks!