Bug - disallowing privileged mode also sets SecurityContextDeny

[paulczar]

What happened:

Many Kubernetes manifests and helm charts require setting securityContext.runAsUser in order to have the same permissions as when the image was built to ensure access to files/directories/etc works.

Unfortunately SecurityContextDeny is being added as an admission controller when a cluster is created without privileged mode. This breaks the ability to set a runAsUser and thus breaks many helm charts etc which in turn causes operators to deploy clusters with privileged mode enabled and reducing the overall security of the cluster.

see

kubo-release/jobs/kube-apiserver/templates/config/bpm.yml.erb

Lines 24 to 26 in fdad0de

	if !p('allow_privileged')
	admission_control_options.push("SecurityContextDeny")
	end

  if !p('allow_privileged')
    admission_control_options.push("SecurityContextDeny")
  end

What you expected to happen:

SecurityContextDeny should be selectable as a admission controller and not be tied to privileged.

How to reproduce it (as minimally and precisely as possible):

helm install redis

Anything else we need to know?:

It looks like this Pull Request will solve the issue - #206

and this will help general security concerns that might cause a person to want SecurityContextDeny in the first place - https://github.com/cloudfoundry-incubator/kubo-release/pull/216/files

Environment:

• Deployment Info (bosh -d <deployment> deployment):
• Environment Info (bosh -e <environment> environment):
• Kubernetes version (kubectl version):
• Cloud provider (e.g. aws, gcp, vsphere):

[cf-gitbot]

We have created an issue in Pivotal Tracker to manage this:

https://www.pivotaltracker.com/story/show/159152591

The labels on this github issue will be updated when the story is started.

[making]

I met this issue when deploying ingress-nginx following the doc
https://kubernetes.github.io/ingress-nginx/deploy/

[seanos11]

Hi @paulczar, SecurityContextDeny was added based on a recommendation to secure clusters by default.
We see the problem with this restricting other security context features that are not related to priviledged containers

We are looking at replacing SecurityContextDeny with PodSecurityPolicy, I see you have provided input into the design doc. Thanks for that

[youreddy]

Hi @paulczar, I've added the fix to decouple privilege containers from security context. The default behavior of the release is to set allow_privileged to false and not to set the SecurityContextDeny admission controller. On a related note, we provide an ops-file in kubo-deployment that will enable SecurityContextDeny when privilege containers are desired.

The commits are making their way through our CI right now and should be available in the next release.

Related commits: b7dc91b, 3eac3f

Thanks for bringing this issue to our attention. I'm going to close it out but please feel free to re-open it if you feel it has not been addressed.