Add support for user namespaces

[derekwaynecarr]

Enhancement Description

• One-line enhancement description (can be used as a release note): Add support for user namespaces in pods
• Kubernetes Enhancement Proposal: https://github.com/kubernetes/enhancements/blob/master/keps/sig-node/127-user-namespaces/README.md
• Discussion Link: This was discussed in several sig-node meetings, most notable Nov 2 2021 and in this PR, that feedback was incorporated already by this KEP is taking: keps/127: Support User Namespaces #2101.
• Primary contact (assignee): @rata @giuseppe
• Responsible SIGs: sig-node
• Enhancement target (which target equals to which milestone):
  • Alpha release target (x.y): 1.25
  • Beta release target (x.y):
  • Stable release target (x.y):
• Alpha
  • KEP (k/enhancements) update PR(s):
  • Code (k/k) update PR(s):
  • Docs (k/website) update PR(s):

Please keep this description up to date. This will help the Enhancement Team to track the evolution of the enhancement efficiently.

[derekwaynecarr]

This work is being done by @pweil- and is reviewed by @derekwaynecarr, it is sponsored by @kubernetes/sig-node

[mdshuai]

@derekwaynecarr Could you help create a user story card for this feature?

[idvoretskyi]

@derekwaynecarr can you confirm that this feature targets alpha for 1.5?

[pweil-]

@derekwaynecarr can you confirm that this feature targets alpha for 1.5?

Yes, this feature is experimental only so it would be considered alpha.

[idvoretskyi]

@derekwaynecarr @pweil- can you confirm that this item targets beta in 1.6?

[adelton]

@derekwaynecarr, the proposal kubernetes/kubernetes#34569 was closed by bot due to inactivity.

@pweil-, in kubernetes/kubernetes#34569 (comment) you've proposed the approach pweil-/kubernetes@16f29eb which changes the group of /var/lib/kubelet/pods to the remapped root group. Do I understand it correctly that this is currently not tracked in any pull request?

[adelton]

@pweil-, I also wonder if similar to docker's /var/lib/docker/<uid>.<gid> approach when --userns-remap is used, it might make sense to use /var/lib/kubelet/pods-<uid>.<gid> and just chown/chgroup everything in those subdirectories to the remapped <uid>.<gid>. Why did you opt for just the chgrp and not the full chown?

[pweil-]

@adelton in the end, I think having this be transparent to Kubernetes is the right approach. Whether that be something like shiftfs or implementation in the CRI (moby/moby#28593). You are correct that my existing proposal is not currently tracked in an open PR anymore.

The reasoning behind using the chgrp was to follow our fsgroup strategy where we just ensure group access instead of uid access.

[adelton]

Thanks @pweil-.

When you say transparent, you mean that nothing should be needed to be added to code or to configuration on Kubernetes' side to allow running under docker with userns-remap?

As for the fsgroup strategy, do you mean https://kubernetes.io/docs/concepts/policy/pod-security-policy/#fsgroup or some generic methodology within Kubernetes?

I have now filed kubernetes/kubernetes#55707 as an alternative approach where I make the remapped uid/gid an explicit option, and use those values to chown/chgrp the necessary directories.

[pweil-]

When you say transparent, you mean that nothing should be needed to be added to code or to configuration on Kubernetes' side to allow running under docker with userns-remap?

that would be ideal. Whether that is feasible (or more likely, feasible in an acceptable time frame) is another question 😄

As for the fsgroup strategy, do you mean https://kubernetes.io/docs/concepts/policy/pod-security-policy/#fsgroup or some generic methodology within Kubernetes?

Yes

I have now filed kubernetes/kubernetes#55707 as an alternative approach where I make the remapped uid/gid an explicit option, and use those values to chown/chgrp the necessary directories.

👍 subscribed

[adelton]

When you say transparent, you mean that nothing should be needed to be added to code or to configuration on Kubernetes' side to allow running under docker with userns-remap?

that would be ideal. Whether that is feasible (or more likely, feasible in an acceptable time frame) is another question

Ideally, the pod would specify how many distinct uids/gids it would require / list of uids it wants to see inside of the containers, and docker or different container runtime would setup the user namespace accordingly. But unless docker also changes ownership of the volumes mounted to the containers, Kubernetes will have to do that as part of the setup.

[adelton]

@pwel-, what is the best way to get some review and comments on kubernetes/kubernetes#55707, to get it closer to mergeable state?

[kargakis]

@pweil- ^

[pweil-]

@adelton I would try to engage the sig-node folks either at their Tuesday meeting or on slack: https://github.com/kubernetes/community/tree/master/sig-node

[adelton]

@derekwaynecarr, could you please bring kubernetes/kubernetes#55707 to sig-node's radar?

[idvoretskyi]

@pweil- @derekwaynecarr any progress on this feature is expected?

[wojtek-t]

/milestone v1.25

[parul5sahoo]

Hello @rata 👋, 1.25 Enhancements team here.

Just checking in as we approach enhancements freeze on 18:00 PST on Thursday June 16, 2022.

For note, This enhancement is targeting for stage alpha for 1.25 (correct me, if otherwise)

Here's where this enhancement currently stands: (updated on June 9, 2022)

• KEP file using the latest template has been merged into the k/enhancements repo.
• KEP status is marked as implementable
• KEP has a updated detailed test plan section filled out
• KEP has up to date graduation criteria
• KEP has a production readiness review that has been completed and merged into k/enhancements.

KEP PR #3275 addressed all checkboxes!

For note, the status of this enhancement is marked as tracked. Please keep the issue description up-to-date with appropriate stages as well. Thank you!

[rata]

@parul5sahoo I think the place-holders weren't replaced in your message. So, to avoid confusions, let me tell you that:

• We are targeting alpha stage for 1.25
• It is correct that none of the check boxes are ticked now. But all should be ticked probably by today EOD. I'll update you when that is done

Thanks!

[rata]

@parul5sahoo well, it happened faster. Now all the check boxes you mentioned are done :)

[parul5sahoo]

@rata apologies for the misunderstanding created because of that, it was the first time I did reach out to a KEP owner as an Enhancements shadow, thank you for understanding and verifying it.

[rata]

@parul5sahoo no problem, I understood everything! And thanks again :)

[giuseppe]

opened a PR to add the CRI changes: kubernetes/kubernetes#110535

[sethmccombs]

Hello @rata! 👋,

1.25 Release Docs Shadow here.

Does this enhancement work planned for 1.25 require any new docs or modification to existing docs?

If so, please make sure to open a PR against the dev-1.25 branch in the k/website repo (follow the steps here)

This PR can be just a placeholder at this time and must be created before August 4.

Also, take a look at Documenting for a release to get yourself familiarize with the docs requirement for the release.
Thank you!

[rata]

k/k PR is open for a few days now: kubernetes/kubernetes#111090

[Priyankasaggu11929]

Hi @rata 👋

Checking in once more as we approach 1.25 code freeze at 01:00 UTC on Wednesday, 3rd August 2022.

Please ensure the following items are completed:

• All PRs to the Kubernetes repo that are related to your enhancement are linked in the above issue description (for tracking purposes).
• All PRs are fully merged by the code freeze deadline.
  • Add support for user namespaces phase 1 (KEP 127) kubernetes#111090

Please verify, if there are any additional k/k PRs besides the ones listed above.

Please plan to get the open PRs merged by the code freeze deadline. The status of the enhancement is currently marked as at-risk.

Also kindly update the issue description with the relevant links for tracking purposes. Thank you so much!

[rata]

@wojtek-t can you please update the PR description (or maybe I can join some team and do it myself? I'm part of the k8s org for several years) with this? Thanks!

k/k PRs:

• Add support for user namespaces phase 1 (KEP 127) kubernetes#111090
• kubelet: add CRI definitions for user namespaces kubernetes#110535

docs PR:

• Add docs for user namespaces in pods - phase 1 (KEP-127) website#35235

enhancements PRs:

• 127-user-namespaces: round userns size to 65536 #3430

blog post PR:

• blog: Add blogpost for user namespaces in 1.25 website#35483

[rata]

@wojtek-t friendly ping?

[wojtek-t]

I don't think we need to have them in the description - it's enough for them to be linked to this issue.

[Priyankasaggu11929]

Hello @rata 👋

Just a gentle reminder from the enhancement team as we approach 1.25 code freeze at 01:00 UTC on Wednesday, 3rd August 2022 (which is two days from now).

Please plan to have the open k/k PR merged before then.

The status of this enhancement is currently marked as at risk. Thank you.

[Priyankasaggu11929]

Hello 👋, 1.25 Enhancements Lead here.

Unfortunately, this enhancement did not meet the code freeze criteria because there are still unmerged k/k code PRs.

The RT leads are evaluating the exception request & will get back! Thank you!

/milestone clear