[pending UserNamespace beta]promote LocalStorageCapacityIsolationFSQuotaMonitoring to beta

[pacoxu]

This reverts commit 32a90f5.
This feature gate was promoted to beta only in v1.25.0 and was reverted in v1.25.1 for a regression issue.

See the history in #107329 and #112076

The KEP can be found in kubernetes/enhancements#2697

Action Items:

• add some metrics or make more visibility add VolumeStatCalDuration metrics for fsquato monitoring benchmark #107201
• benchmark the feature ⬆️ the comments above.
• add some warning log for long time cost volume calculation add warning log if volume calculation took too long than 1 second #107490
• check e2e testings: e2e evolution (LocalStorageCapacityIsolationQuotaMonitoring [Slow] [Serial] [Disruptive] [Feature:LocalStorageCapacityIsolationQuota][NodeFeature:LSCIQuotaMonitoring])
• need some feedbacks
• Ephemeral storage doesn't account for deleted files with open handles #83107 （I tested it. It will work immediately if we enable this feature gate ）
• fix a bug: during kubelet restart. assignQuota checks if the underlying medium supports quotas and if so setting it #107302 (This may be the cause of below issue.
• rending configmap regression: when updated ConfigMaps to pods，pod's filesystem by the Kubelet get errors #112081
• new issue that is reported: failed attempt to reassign project ID #114506
• kubelet issues when LocalStorageCapacityIsolationFSQuotaMonitoring is enabled #115309 (may be dup with when updated ConfigMaps to pods，pod's filesystem by the Kubelet get errors #112081)
• [pending UserNamespace beta]promote LocalStorageCapacityIsolationFSQuotaMonitoring to beta #112626 (comment) kernel @haircommander pointed out that it's in a user namespace it can't change the projid and if not, project ID could be changed. So we want to wait for User Namespace beta before promoting this to beta.

/kind feature
Xrefs kubernetes/enhancements#1029

promote LocalStorageCapacityIsolationFSQuotaMonitoring to beta

[pacoxu]

/test pull-kubernetes-node-kubelet-serial-crio-cgroupv1
/test pull-kubernetes-node-kubelet-serial-crio-cgroupv2
/test pull-kubernetes-node-crio-cgrpv2-e2e
/test pull-kubernetes-node-crio-cgrpv2-e2e-kubetest2
/test pull-kubernetes-node-crio-e2e
/test pull-kubernetes-node-crio-e2e-kubetest2

[SergeyKanzhelev]

is the isue fix in this pr: #112625? Is this PR just WIP than?

[pacoxu]

I think this should be pending until #112625 is merged to fix the regression.

I still need more verifying for the fix.

[pacoxu]

Since the fsquota related bug is fixed, I think we can promote it again. I will update the KEP as soon as possible in kubernetes/enhancements#3821

[k8s-triage-robot]

The Kubernetes project currently lacks enough contributors to adequately respond to all issues and PRs.

This bot triages issues and PRs according to the following rules:

• After 90d of inactivity, lifecycle/stale is applied
• After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
• After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

• Mark this issue or PR as fresh with /remove-lifecycle stale
• Mark this issue or PR as rotten with /lifecycle rotten
• Close this issue or PR with /close
• Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle stale

[pacoxu]

/remove-lifecycle stale
pending on #112624 and maybe #114506

[SergeyKanzhelev]

/assign @rphillips

Let's evaluate whether we can really promote it to beta as there seems to be bugs still that needs to be looked at.

/hold

[pacoxu]

/unhold
/triage accepted
/priority important-soon
/assign @dchen1107 @mrunalp

[pacoxu]

Any further comments on this? @SergeyKanzhelev @rphillips as kubernetes/enhancements#3821 was merged for v1.29.

@rphillips @SergeyKanzhelev @derekwaynecarr Do you have any comments on this feature promotion?

[pacoxu]

/assign @derekwaynecarr

[pacoxu]

Any further comments on this? @SergeyKanzhelev @rphillips as kubernetes/enhancements#3821 was merged for v1.29.

@rphillips @SergeyKanzhelev @derekwaynecarr Do you have any comments on this feature promotion?

Ping again before the code freeze.

[pacoxu]

Kernel issue. Other bugs reported
#112626
Mrunal and Peter to check on kernel issue

@mrunalp @haircommander do you have the kernel issue link?

[haircommander]

It's not so much an issue as expected behavior, and actually I may have been wrong. Any process in the root user namespace can change the project ID of a path, effectively nullifying the quota for a malicious user. In some of the documentation, users need CAP_SYS_RESOURCE to change the hard limit, and I'm not sure if that also applies to changing the project ID. there seems to have been a discussion on this but it's not clear where this went.

I am trying to investigate further. If a user does need CAP_SYS_RESOURCE to change a project ID, then we're good to move forward with this beta IMO. Otherwise, we will need to wait until user namespaces are more prevalent to move forward with this feature

[haircommander]

Okay I confirmed with kernel engineer. It's not actually documented other than in the code, but a process just needs to own the file or have CAP_FOWNER. If it's in a user namespace it can't change the projid, so we would need to require user namespace support. Theoretically we could pursue beta for this in 1.30, since we're pushing for user namespace beta then, but we may want to wait for the feature to be on by default (maybe 1.31?)

[pacoxu]

Okay I confirmed with kernel engineer. It's not actually documented other than in the code, but a process just needs to own the file or have CAP_FOWNER. If it's in a user namespace it can't change the projid, so we would need to require user namespace support. Theoretically we could pursue beta for this in 1.30, since we're pushing for user namespace beta then, but we may want to wait for the feature to be on by default (maybe 1.31?)

I may update the KEP based on this.

• If user namespace FG is off and LocalStorageCapacityIsolationFSQuotaMonitoring is on, should we give a warning to user about this?

As a risk, we'd better to wait for User Namespace beta before promoting this to beta in my opinion.

[haircommander]

If user namespace FG is off and LocalStorageCapacityIsolationFSQuotaMonitoring is on, should we give a warning to user about this?

yeah probably

As a risk, we'd better to wait for kubernetes/enhancements#127 beta before promoting this to beta in my opinion.

works for me!

[pacoxu]

/remove-priority important-soon

[k8s-triage-robot]

The Kubernetes project currently lacks enough contributors to adequately respond to all PRs.

This bot triages PRs according to the following rules:

• After 90d of inactivity, lifecycle/stale is applied
• After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
• After 30d of inactivity since lifecycle/rotten was applied, the PR is closed

You can:

• Mark this PR as fresh with /remove-lifecycle stale
• Close this PR with /close
• Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle stale