Fine-grained SupplementalGroups control

[everpeace]

Enhancement Description

• One-line enhancement description (can be used as a release note): Add new API surface to control and track how supplemental groups are applied in the container.
• Kubernetes Enhancement Proposal: https://github.com/kubernetes/enhancements/tree/master/keps/sig-node/3619-supplemental-groups-policy
• Discussion Link: Can bypass PodSecurityContext.SupplementalGroups by custom container image although PSP(or other policy engines) enforces the field kubernetes#112879
• Primary contact (assignee):　@everpeace
• Responsible SIGs: sig-node
• Enhancement target (which target equals to which milestone):
  • Alpha release target (x.y): 1.31
  • Beta release target (x.y): 1.32 --> 1.33 or later (depends on Containerd v2 release and it's adoption)
  • Stable release target (x.y):
• Alpha
  • KEP (k/enhancements) update PR(s):
    • KEP-3169: Fine-grained SupplementalGroups control #3620
    • KEP-3619: update Test Plan and Graduation Criteria for KEP freeze #3862
    • KEP-3619: Cleanup After Freeze #3874
    • KEP-3619: update the latest milestone to v1.31 #4628
    • KEP-3619: Add SupplementalGroupsPolicy feature fields in Kubernetes API(Node.Status) and CRI(RuntimeStatusResponse) #4728
  • Code (k/k) update PR(s):
    • KEP-3619: Fine-grained SupplementalGroups control kubernetes#117842
    • KEP-3619: Add NodeStatus.Features.SupplementalGroupsPolicy API and e2e kubernetes#125470
  • Docs (k/website) update PR(s):
    • [dev-1.31] KEP-3619: Fine-grained SupplementalGroups control website#46920
    • blog post for KEP-3619: Fine-grained SupplementalGroups control website#46921
  • contained (optional):
    • Fine-grained SupplementalGroups control containerd/containerd#9737
    • Implement RuntimeStatus.features.supplemental_groups_policy from KEP-3619 containerd/containerd#10410
  • CRI-O (optional):
    • server/*: add fine-grained SupplementalGroups control for enhanced security cri-o/cri-o#8268
    • KEP-3619: implement RuntimeStatus.features.supplemental_groups_policy cri-o/cri-o#8386
  • cri-tools (optional):
    • KEP-3619: Fine-grained SupplementalGroups control kubernetes-sigs/cri-tools#1438
• Beta
  • KEP (k/enhancements) update PR(s):
    • KEP-3619: updated KEP with alpha release status, and added haircommander as a reviewer #4809
    • KEP-3619: updated Production Readiness Review Questionnaire for beta release #4895
  • Code (k/k) update PR(s):
  • Docs (k/website) update(s):

Please keep this description up to date. This will help the Enhancement Team to track the evolution of the enhancement efficiently.

/assign
/sig node
/kind feature

[k8s-ci-robot]

@everpeace: The label(s) /remove-label sig/scheduling cannot be applied. These labels are supported: api-review, tide/merge-method-merge, tide/merge-method-rebase, tide/merge-method-squash, team/katacoda, refactor, lead-opted-in, tracked/no, tracked/out-of-tree, tracked/yes. Is this label configured under labels -> additional_labels or labels -> restricted_labels in plugin.yaml?

In response to this:

/remove-label sig/scheduling

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[k8s-triage-robot]

The Kubernetes project currently lacks enough contributors to adequately respond to all issues and PRs.

This bot triages issues and PRs according to the following rules:

• After 90d of inactivity, lifecycle/stale is applied
• After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
• After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

• Mark this issue or PR as fresh with /remove-lifecycle stale
• Mark this issue or PR as rotten with /lifecycle rotten
• Close this issue or PR with /close
• Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle stale

[everpeace]

/remove-lifecycle stale

[SergeyKanzhelev]

/milestone v1.27

let's see if we can start it in 1.27

[SergeyKanzhelev]

/label lead-opted-in

[SergeyKanzhelev]

/stage alpha

[npolshakova]

Hello @everpeace 👋, 1.27 Enhancements team here.

Just checking in as we approach enhancements freeze on 18:00 PDT Thursday 9th February 2023.

This enhancement is targeting for stage alpha for 1.27 (correct me, if otherwise)

Here's where this enhancement currently stands:

• KEP readme using the latest template has been merged into the k/enhancements repo.
• KEP status is marked as implementable for latest-milestone: 1.27
• KEP readme has a updated detailed test plan section filled out
• KEP readme has up to date graduation criteria
• KEP has a production readiness review that has been completed and merged into k/enhancements.

It looks like #3620 will address most of these issues.

The status of this enhancement is marked as at risk. Please keep the issue description up-to-date with appropriate stages as well. Thank you!

[haircommander]

@everpeace do you plan on making progress on this in 1.32?

[kannon92]

Sounds like there is plans to bring this up for beta graduation in 1.32. Moving it to "Proposed for consideration".

[everpeace]

@haircommander

do you plan on making progress on this in 1.32?

Yes, I'm planning some improvements in 1.32 toward beta:

• error handling
  • rejecting pods with the SupplementalGroupsPolicy field on the node with Node.Status.Features.SupplementalGroupsPolicy: false (currently, silently falls back to Merge behavior)
• observability
  • add kube-state-metrics node_status_features{feature="SupplementalGroupsPolicy"}???

@kannon92

Moving it to "Proposed for consideration".

Thanks!

---

By the way, about the actual timing of its beta promotion, I think it would be better after both containerd and CRI-O supports SupplementalGroupsPolicy feature as defined in Beta graduation criteria.

• CRI-O already supports this feature in v1.31, but
• containerd has not supported this feature yet (we have to wait for containerd v2)

Thus, I'm not sure v1.32 is good timing for its beta promotion because I don't have information about containerd v2 release date even the expected one.

[tjons]

Hi, enhancements lead here - I inadvertently added this to the 1.32 tracking board 😀. Please readd it if you wish to progress this enhancement in 1.32.

/remove-label lead-opted-in

[everpeace]

@haircommander I'm now preparing PRR of this KEP for beta release in #4895. May this KEP lead-opt-in for v1.32?? (Sorry for the last minutes)

[haircommander]

/milestone v1.32
/stage beta

Thanks for the heads up @everpeace !

[haircommander]

/label lead-opted-in

[tjons]

Hello @everpeace 👋, Enhancements team here.

Just checking in as we approach enhancements freeze on 02:00 UTC Friday 11th October 2024 / 19:00 PDT Thursday 10th October 2024.

This enhancement is targeting for stage beta for v1.32 (correct me, if otherwise).

Here's where this enhancement currently stands:

• KEP readme using the latest template has been merged into the k/enhancements repo.
• KEP status is marked as implementable for latest-milestone: v1.32.
• KEP readme has up-to-date graduation criteria
• KEP has a production readiness review that has been completed and merged into k/enhancements. (For more information on the PRR process, check here). If your production readiness review is not completed yet, please make sure to fill the production readiness questionnaire in your KEP by the PRR Freeze deadline on Thursday 3rd October 2024 so that the PRR team has enough time to review your KEP.

To track this KEP for enhancements freeze:

• the KEP readme needs to have the rest of the PRR review questions filled out: see https://github.com/kubernetes/enhancements/tree/master/keps/NNNN-kep-template for beta requirements.
• KEP-3619: updated Production Readiness Review Questionnaire for beta release #4895 will need to merge
• Please mark the status of the KEP as implementable for latest-milestone: v1.32.

The status of this enhancement is marked as at risk for enhancement freeze. Please keep the issue description up-to-date with appropriate stages as well. Thank you!

If you anticipate missing enhancements freeze, you can file an exception request in advance. Thank you!

[tjons]

Hello 👋, v1.32 Enhancements team here.

Unfortunately, this enhancement did not meet requirements for enhancements freeze.

If you still wish to progress this enhancement in 1.32, please file an exception request as soon as possible, within three days. If you have any questions, you can reach out in the #release-enhancements channel on Slack and we'll be happy to help. Thanks!

[tjons]

/milestone clear