Quotas for Ephemeral Storage

[RobertKrawitz]

Enhancement Description

• Use filesystem quotas to monitor local ephemeral storage utilization
• Kubernetes Enhancement Proposal: 1029-ephemeral-storage-quotas
• Primary contact (assignee): @RobertKrawitz @pacoxu
• Responsible SIGs: SIG-Node
• Reviewer(s) - @dashpole @derekwaynecarr
• Approver: #derekwaynecarr @dchen1107
• Enhancement target (which target equals to which milestone):
  • Alpha release target (x.y): 1.15
  • Beta release target (x.y): 1.31 (User Namespace Support User Namespaces in pods #127 is beta in v1.30)
    • promoted in v1.25.0 and then reverted in v1.25.1
  • Stable release target (x.y): v1.33 or later
• Alpha
  • KEP (k/enhancements) update PR(s): Updates from SIG-Node meeting 20181204 #646
  • Code (k/k) update PR(s): Ephemeral storage monitoring via filesystem quotas kubernetes#66928
  • Docs (k/website) update(s): Add support for quotas for ephemeral storage monitoring. website#14268
• Reverted Beta (first-try: promoted in v1.25.0 and then reverted in v1.25.1)
  • KEP (k/enhancements) update PR(s): promote ephemeral-storage-quotas to beta in 1.25 #2697
  • Code (k/k) update PR(s):
    • promote LocalStorageCapacityIsolationFSQuotaMonitoring to beta kubernetes#107329
    • add VolumeStatCalDuration metrics for fsquato monitoring benchmark kubernetes#107201
    • assignQuota checks if the underlying medium supports quotas and if so setting it kubernetes#107302
    • add warning log if volume calculation took too long than 1 second kubernetes#107490
  • Docs(k/website) updates: Revert: promote LocalStorageCapacityIsolationFSQuotaMonitoring to beta website#36401
  • Revert PR: Revert "promote LocalStorageCapacityIsolationFSQuotaMonitoring to beta" kubernetes#112076, Changelog-1.25: add 1.25.0 known issues section kubernetes#112106, LocalStorageCapacityIsolationFSQuotaMonitoring revert to alpha for regression #3485
• Beta (second-try) 1.31
  • KEP (k/enhancements) update PR(s): [v1.29] ephemeral-storage-quotas: repromote to beta #3821
    • sig-node/1029-ephemeral-storage-quotas/*: #4699
  • Code (k/k) update PR(s):
    • Add CI for quota monitoring test-infra#28614
    • 1.27 fsquota: only generate pod uuid is nil kubernetes#112624
    • 1.27 kubelet: Fix fs quota monitoring on volumes kubernetes#115314
    • [pending UserNamespace beta]promote LocalStorageCapacityIsolationFSQuotaMonitoring to beta kubernetes#112626
    • [pending UserNamespace beta]promote LocalStorageCapacityIsolationFSQuotaMonitoring to beta kubernetes#112626 (comment) kernel @haircommander pointed out that it's in a user namespace it can't change the projid and if not, project ID could be changed. So we want to wait for User Namespace beta before promoting this to beta.
    • 1.31 pkg/volume/*: Enable quotas in user namespace kubernetes#126014
  • Docs (k/website) update(s):
    • [WIP] Update LocalStorageCapacityIsolationFSQuotaMonitoring to beta website#43455
    • Promote ephemeral storage quota to beta(1.31) website#46967

[RobertKrawitz]

/sig node

[derekwaynecarr]

per sig-node discussion, this is good.

the kep will merge here: #646

/milestone v1.15

[kacole2]

@RobertKrawitz This is OK to track for 1.15. However, please update the KEP with established graduation criteria for Beta and Alpha when possible.

/stage alpha

[makoscafee]

Hey, @RobertKrawitz @derekwaynecarr 👋 I'm the v1.15 docs Lead.
Does this enhancement require any new docs (or modifications)?

Just a friendly reminder we're looking for a PR against k/website (branch dev-1.15) due by Thursday, May 30th. It would be great if it's the start of the full documentation, but even a placeholder PR is acceptable. Let me know if you have any questions!

[RobertKrawitz]

Yes, this will require new documentation.

I will open the website PR today.

[kacole2]

Hi @RobertKrawitz @derekwaynecarr . Code Freeze is Thursday, May 30th 2019 @ EOD PST. All enhancements going into the release must be code-complete, including tests, and have docs PRs open.

Please list all current k/k PRs so they can be tracked going into freeze. If the PRs aren't merged by freeze, this feature will slip for the 1.15 release cycle. Only release-blocking issues and PRs will be allowed in the milestone.

If you know this will slip, please reply back and let us know. Thanks!

[makoscafee]

Hey, @RobertKrawitz @derekwaynecarr .Just a friendly reminder we're looking for at least a draft/placeholder PR against k/website (branch dev-1.15) due by Thursday, May 30th 2019 @ EOD PST.

[RobertKrawitz]

@makoscafee The k/website PR is kubernetes/website#14268

[kacole2]

Hi @RobertKrawitz @derekwaynecarr , today is code freeze. I do not see a reply for any k/k PRs to track for this merge. It's now being marked as At Risk in the 1.15 Enhancement Tracking Sheet. If there is no response, or you respond with PRs to track and they are not merged by EOD PST, this will be dropped from the 1.15 Milestone. After this point, only release-blocking issues and PRs will be allowed in the milestone with an exception.

[RobertKrawitz]

The PR for this is kubernetes/kubernetes#66928

[RobertKrawitz]

/hold cancel

[kacole2]

Hi @RobertKrawitz , I'm the 1.16 Enhancement Lead. Is this feature going to be graduating alpha/beta/stable stages in 1.16? Please let me know so it can be added to the 1.6 Tracking Spreadsheet. If not's graduating, I will remove it from the milestone and change the tracked label.

Once coding begins or if it already has, please list all relevant k/k PRs in this issue so they can be tracked properly.

Milestone dates are Enhancement Freeze 7/30 and Code Freeze 8/29.

Thank you.

[kcmartin]

Hello @RobertKrawitz -- 1.17 Enhancement Shadow here! 🙂

I wanted to reach out to see if this enhancement will be graduating to alpha/beta/stable in 1.17?


Please let me know so that this enhancement can be added to 1.17 tracking sheet.

Thank you!

🔔Friendly Reminder

The current release schedule is

• Monday, September 23 - Release Cycle Begins
• Tuesday, October 15, EOD PST - Enhancements Freeze
• Thursday, November 14, EOD PST - Code Freeze
• Tuesday, November 19 - Docs must be completed and reviewed
• Monday, December 9 - Kubernetes 1.17.0 Released

[fejta-bot]

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/lifecycle stale

[palnabarun]

/remove-lifecycle stale

[Princesso]

Hi @RobertKrawitz @pacoxu @PannagaRao ,
1.31 Doc lead here. I noticed that this Enhancement work already has a draft PR here, however, the PR is closed. Can you re-open the PR against dev-1.31 branch in the k/website repo. This deadline for having a draft PR up is Thursday June 27, 2024 18:00 PDT.

[Princesso]

Hi @RobertKrawitz @pacoxu @PannagaRao, this is a gentle reminder that today is the deadline for having the doc draft PR opened against dev-1.31.

[haircommander]

here's the PR kubernetes/website#46967

[a-mccarthy]

hi @haircommander, friendly reminder about the upcoming blog opt-in and placeholder deadline on July 3rd. Please open a blog placeholder PR if you are interested in contributing a blog.

[dipesh-rawat]

Hey again @RobertKrawitz @pacoxu @PannagaRao 👋, 1.31 Enhancements team here,

Just checking in as we approach code freeze at at 02:00 UTC Wednesday 24th July 2024 / 19:00 PDT Tuesday 23rd July 2024.

Here's where this enhancement currently stands:

• All PRs to the Kubernetes repo that are related to your enhancement are linked in the above issue description (for tracking purposes).
• All PR/s are ready to be merged (they have approved and lgtm labels applied) by the code freeze deadline. This includes tests.

For this enhancement, it looks like the following PRs are open and need to be merged before code freeze (and we need to update the Issue description to include all the related PRs of this KEP):

• pkg/volume/*: Enable quotas in user namespace kubernetes#126014

Additionally, please let me know if there are any other PRs in k/k not listed in the descrption that we should track for this KEP, so that we can maintain accurate status.

For this KEP, we would need to do the following:

• Ensure all PRs are prepared for merging (they have approved and lgtm labels applied) by the code freeze deadline. This includes tests.

If you anticipate missing code freeze, you can file an exception request in advance.

The status of this enhancement is marked as at risk for code freeze.

Edited: To include the PR related to the KEP

[dipesh-rawat]

Hey again @RobertKrawitz @pacoxu @PannagaRao 👋, 1.31 Enhancements team here,

Just a quick friendly reminder as we approach code freeze in around two weeks time, at 02:00 UTC Wednesday 24th July 2024 / 19:00 PDT Tuesday 23rd July 2024.

The current status of this enhancement is marked as at risk for code freeze. There are a few requirements mentioned in the comment #1029 (comment) that still need to be completed.

If you anticipate missing code freeze, you can file an exception request in advance.

[dipesh-rawat]

Hey again @RobertKrawitz @pacoxu @PannagaRao 👋, 1.31 Enhancements team here,

Just a quick friendly reminder as we approach code freeze next week, at 02:00 UTC Wednesday 24th July 2024 / 19:00 PDT Tuesday 23rd July 2024.

The current status of this enhancement is marked as at risk for code freeze. There are a few requirements mentioned in the comment #1029 (comment) that still need to be completed.

If you anticipate missing code freeze, you can file an exception request in advance.

[dipesh-rawat]

Hey again @RobertKrawitz @pacoxu @PannagaRao 👋, 1.31 Enhancements team here,

Just a quick friendly reminder as we approach code freeze in less than 2 days, at 02:00 UTC Wednesday 24th July 2024 / 19:00 PDT Tuesday 23rd July 2024.

The current status of this enhancement is marked as at risk for code freeze. There are a few requirements mentioned in the comment #1029 (comment) that still need to be completed.

If you anticipate missing code freeze, you can file an exception request in advance.

[dipesh-rawat]

Hey again @RobertKrawitz @pacoxu @PannagaRao 👋, 1.31 Enhancements team here,

Just checking in as we approach code freeze at 02:00 UTC Wednesday 24th July 2024 / 19:00 PDT Tuesday 23rd July 2024.

Here's where this enhancement currently stands:

• All PRs to the Kubernetes repo that are related to your enhancement are linked in the above issue description (for tracking purposes).
• All PR/s are ready to be merged (they have approved and lgtm labels applied) by the code freeze deadline. This includes tests.

With all the implementation(code related) PRs merged as per the issue description:

• pkg/volume/*: Enable quotas in user namespace kubernetes#126014

Additionally, please let me know if there are any other PRs in k/k not listed in the description that we should track for this KEP, so that we can maintain accurate status.

This enhancement is now marked as tracked for code freeze for the 1.31 Code Freeze!

[kannon92]

Hi, what are the plans for 1.32?

[haircommander]

this only just went to beta so I presume nothing until 1.33

[tjons]

Inadvertently added this to the 1.32 tracking board

/remove-label lead-opted-in

[pacoxu]

this only just went to beta so I presume nothing until 1.33

As this depends on the User Namespace feature and is beta but by default false/disabled, I assume that the next step is beta 2, default true which needs to be done after UserNamespacesSupport is beta and by default true.