-
Notifications
You must be signed in to change notification settings - Fork 39.1k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Prefer non-mutating PSPs, then order by name #52849
Prefer non-mutating PSPs, then order by name #52849
Conversation
8463ac9
to
83aebfe
Compare
cc @kubernetes/sig-auth-pr-reviews |
// all containers in a single pod must validate under a single provider or we will reject the request | ||
validationErrs := field.ErrorList{} | ||
var ( | ||
allowedPod *api.Pod | ||
allowingProvider psp.Provider |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Do we really need to hold an entire provider if we need only its name?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
the goal was to keep the existing code intact and move it out of the loop
// but continue to see if another PSP allows without mutating | ||
allowedPod = podCopy | ||
allowingProvider = provider | ||
} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Could you add more logging for all these branches? Even if they will be on level 6, it's better to have them for easy debugging.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
added logging
@@ -142,27 +144,62 @@ func (c *podSecurityPolicyPlugin) Admit(a admission.Attributes) error { | |||
return nil | |||
} | |||
|
|||
// sort by name to make order deterministic | |||
// TODO: add priority field to allow admins to bucket differently |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
open issue or something for this? Sorting on name leads people do weird things.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It's already here: #45358
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
+1 For this change. I think this should be considered a cherry-pick candidate for 1.8 patch release, since this is fixing an open bug in a beta feature.
allowedPod = podCopy | ||
allowingProvider = provider | ||
break loop | ||
case specMutationAllowed && allowedPod == nil: |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It should be possible to tell whether a PSP is mutating without applying it. Maybe add a TODO here to optimize that in the future? I would prefer it NOT to go into this PR though.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
definitely agree... started down that road with #50169 and it got... large. The preemptive copy here is less than ideal, but this was focused on the minimal change to get the external behavior we want
if allowedPod != nil { | ||
*pod = *allowedPod | ||
// annotate and accept the pod | ||
glog.V(4).Infof("pod %s (generate: %s) validated against provider %s", pod.Name, pod.GenerateName, allowingProvider.GetPSPName()) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
nit: include namespace
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
^^
@liggitt - Are you waiting for anything on this? If it's just a matter of finding the time, I'd be happy to help. |
Got distracted by a couple 1.8.0 fires. Plan to add tests today/tomorrow, then will be ready |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I like this. I think it's a clever solution that also allows people to use less restrictive PSPs if they have access without having to manipulate the pod spec or sorting.
83aebfe
to
0889d7b
Compare
0889d7b
to
32e5147
Compare
updated with tests, removed one unnecessary defaulter, included a commit fixing tolerating of non-pod-spec mutations done by GC as part of delete |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks for working on this.
providers, errs := c.createProvidersFromPolicies(matchedPolicies, pod.Namespace) | ||
logProviders(pod, providers, errs) | ||
|
||
if len(providers) == 0 { | ||
return admission.NewForbidden(a, fmt.Errorf("no providers available to validate pod request")) | ||
} | ||
|
||
// TODO(liggitt): allow spec mutation during initializing updates? |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
/cc @caesarxuchao
if allowedPod != nil { | ||
*pod = *allowedPod | ||
// annotate and accept the pod | ||
glog.V(4).Infof("pod %s (generate: %s) validated against provider %s", pod.Name, pod.GenerateName, allowingProvider.GetPSPName()) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
^^
for _, provider := range providers { | ||
if errs := assignSecurityContext(provider, pod, field.NewPath(fmt.Sprintf("provider %s: ", provider.GetPSPName()))); len(errs) > 0 { | ||
podCopy := pod.DeepCopy() |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
As long as you're doing the copying here, you might as well remove the copying logic from assignSecurityContext
(it does a bunch of stuff to make sure it doesn't modify the pod if it's invalid).
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
done (which greatly simplified that function), and added lots of tests to make sure pods that pass validation as-is are not mutated by the top-level Admit() call
// the entire pod validated | ||
|
||
switch { | ||
case apiequality.Semantic.DeepEqual(pod, podCopy): |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This might be a premature optimization, but you can skip this check if len(providers) == 1
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
only if specMutationAllowed
is true... I'd rather keep this straightforward and deal with optimizations when we profile where the hotspots are
testPSPAdmitAdvanced(testCaseName, kadmission.Create, psps, pod, nil, shouldPass, true, expectedPSP, t) | ||
} | ||
|
||
func testPSPAdmitAdvanced(testCaseName string, op kadmission.Operation, psps []*extensions.PodSecurityPolicy, pod, oldPod *kapi.Pod, shouldPass bool, canMutate bool, expectedPSP string, t *testing.T) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
oldPod
looks unused?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
fixed
oldPod *kapi.Pod | ||
psps []*extensions.PodSecurityPolicy | ||
shouldPass bool | ||
canMutate bool |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Can you rename this to expectMutation
?
911772d
to
8c5b013
Compare
/assign @dchen1107 @smarterclayton |
/approve no-issue Looks good as well to me |
/lgtm |
[APPROVALNOTIFIER] This PR is APPROVED This pull-request has been approved by: liggitt, smarterclayton, tallclair Associated issue: 36184 The full list of commands accepted by this bot can be found here.
Needs approval from an approver in each of these OWNERS Files:
You can indicate your approval by writing |
Automatic merge from submit-queue (batch tested with PRs 48665, 52849, 54006, 53755). If you want to cherry-pick this change to another branch, please follow the instructions here. |
Thank you for the PR! Are there any plans on cherry-picking this for the v1.8.x branch as @tallclair mentioned earlier? |
Pick is open in #54029 pending review and soak time |
w.containerSC.AllowPrivilegeEscalation = v | ||
} | ||
|
||
func NewEffectiveContainerSecurityContextAccessor(podSC PodSecurityContextAccessor, containerSC ContainerSecurityContextMutator) ContainerSecurityContextAccessor { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@liggitt Why do we need ContainerSecurityContextMutator
here? It looks weird to require mutate for an accessor.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
just because it was reusing the concrete type which required it. I wouldn't object to taking an accessor and wrapping in a no-op mutator impl
…9-upstream-release-1.8 Automatic merge from submit-queue. Automated cherry pick of #52849 Cherry pick of #52849 on release-1.8. #52849: PodSecurityPolicy: Do not mutate nil privileged field This is a larger change than usual for a cherry-pick, but changes are isolated to the PSP component, the cherry-pick was clean, and this fixes severe usability issues with systems with more than one PSP. Would like to get feedback from community users making use of PSP and let the master PR soak as we consider this. ```release-note PodSecurityPolicy: when multiple policies allow a submitted pod, priority is given to ones which do not require any fields in the pod spec to be defaulted. If the pod must be defaulted, the first policy (ordered by name) that allows the pod is used. ```
Automatic merge from submit-queue (batch tested with PRs 52367, 53363, 54989, 54872, 54643). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>. Basic GCE PodSecurityPolicy Config **What this PR does / why we need it**: This PR lays the foundation for enabling PodSecurityPolicy in GCE and other default deployments. The 3 commits are: 1. Add policies, roles & bindings for the default addons on GCE. 2. Enable the PSP admission controller & load the addon policies when the`ENABLE_POD_SECURITY_POLICY=true` environment variable is set. 3. Support the PodSecurityPolicy in the E2E environment & add PSP tests. NOTES: - ~~Depends on #52301 for privileged capabilities~~ - ~~Depends on #52849 for sane mutations~~ - ~~Depends on #53479 for aggregator tests to pass~~ - ~~Depends on #54175 for dedicated fluentd service~~ account - This PR is a fork of #46064, credit to @Q-Lee **Which issue this PR fixes**: #43538 **Release note**: ```release-note Add support for PodSecurityPolicy on GCE: `ENABLE_POD_SECURITY_POLICY=true` enables the admission controller, and installs policies for default addons. ```
Automatic merge from submit-queue (batch tested with PRs 55557, 55504, 56269, 55604, 56202). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>. Create{Container,Pod}SecurityContext: modify a pod and don't return the annotations **What this PR does / why we need it**: Prior #52849 we couldn't modify a pod and had to return annotations from the methods. But now, as we always working with a copy of a pod, we can modify it directly and we don't need to copy&return annotations separately. This PR simplifies the code by modifying a pod directly. Also it renames these methods and replaces returning of the `SecurityContext` by in-place modification. In fact it reverts the changes from #30257 **Release note**: ```release-note NONE ``` PTAL @liggitt @timstclair CC @simo5
…ring_pod_update Automatic merge from submit-queue (batch tested with PRs 17856, 16934, 17979, 17993, 18001). Improve the process of pod updates by preferring non-mutating SCCs and reducing pod mutations This is adaptation of kubernetes/kubernetes#52849 to SCC. Details (mostly copy&pasted from original PR): - SCCs which allow the pod as-is (no defaulting/mutating) are preferred - During update operations, when mutations to pod specs are disallowed, only non-mutating SCCs are used to validate the pod - Removes unnecessary mutation of pods: - Determines effective security context for pods using a wrapper containing the pod and container security context, rather than building/setting a combined struct on every admission - Does not set `privileged: &false` on security contexts with `privileged: nil` - Does not set `runAsNonRoot: &true` on security contexts that already have a non-nil, non-0 runAsUser - Does not mutate/normalize container capabilities unless changes are required (missing `defaultAddCapabilities` or `requiredDropCapabilities`) Fixes #16467
Fixes #36184
Fixes #23217
Related to #23217
Removes unnecessary mutation of pods:
privileged:&false
on security contexts withprivileged:nil
runAsNonRoot:&true
on security contexts that already have a non-nil, non-0runAsUser
Defines behavior when multiple PSP objects allow a pod: