Tutorial for pod security admission

Refer blog post for v1.23 + suggestions from code review
kubernetes · Dec 8, 2021 · d29e93a · d29e93a
1 parent e808eda
commit d29e93a
Show file tree

Hide file tree

Showing 4 changed files with 489 additions and 0 deletions.
diff --git a/content/en/docs/tutorials/_index.md b/content/en/docs/tutorials/_index.md
@@ -57,6 +57,11 @@ Before walking through each tutorial, you may want to bookmark the
 
 * [Using Source IP](/docs/tutorials/services/source-ip/)
 
+## Security
+
+* [Applying Pod Security Standards at Cluster level](/docs/tutorials/security/cluster-level-pss/)
+* [Applying Pod Security Standards at Namespace level](/docs/tutorials/security/ns-level-pss/)
+
 ## {{% heading "whatsnext" %}}
 
 If you would like to write a tutorial, see

diff --git a/content/en/docs/tutorials/security/_index.md b/content/en/docs/tutorials/security/_index.md
@@ -0,0 +1,5 @@
+---
+title: "Security"
+weight: 40
+---
+
diff --git a/content/en/docs/tutorials/security/cluster-level-pss.md b/content/en/docs/tutorials/security/cluster-level-pss.md
@@ -0,0 +1,319 @@
+---
+title: Applying Pod Security Standards at the cluster level
+content_type: tutorial
+weight: 10
+---
+
+{{% alert title="Note" %}}
+This tutorial applies only for new clusters.
+{{% /alert %}}
+
+Pod Security admission (PSA) is enabled by default in v1.23 and later, as it [graduated
+to beta](/blog/2021/12/15/pod-security-admission-beta/). Pod Security Admission
+is an admission controller that applies Pod Security Standards when pods are 
+created. This tutorial shows you how to enforce the `baseline` Pod Security 
+Standard at the cluster level which applies a standard configuration 
+to all namespaces in a cluster.
+
+For applying pod security standards one namespace at a time, please [follow this
+tutorial](/docs/tutorials/security/ns-level-pss).
+
+## {{% heading "prerequisites" %}}
+
+Install the following on your workstation:
+
+- [KinD](https://kind.sigs.k8s.io/docs/user/quick-start/#installation)
+- [kubectl](https://kubernetes.io/docs/tasks/tools/)
+
+## Choose the right Pod Security Standard to apply
+
+[Pod Security Admission](/docs/concepts/security/pod-security-admission/)
+lets you apply built-in [Pod Security Standards](/docs/concepts/security/pod-security-standards/)
+with the following modes: `enforce`, `audit`, and `warn`.
+
+To gather information that helps you to choose the Pod Security Standards
+that are most appropriate for your configuration, do the following:
+
+1. Create a cluster with no Pod Security Standards applied:
+
+    ```shell
+    kind create cluster --name psa-wo-cluster-pss --image kindest/node:latest
+    ```
+    The output is similar to this:
+    ```
+    Creating cluster "psa-wo-cluster-pss" ...
+    ✓ Ensuring node image (kindest/node:latest) 🖼
+    ✓ Preparing nodes 📦  
+    ✓ Writing configuration 📜
+    ✓ Starting control-plane 🕹️
+    ✓ Installing CNI 🔌
+    ✓ Installing StorageClass 💾
+    Set kubectl context to "kind-psa-wo-cluster-pss"
+    You can now use your cluster with:
+
+    kubectl cluster-info --context kind-psa-wo-cluster-pss
+
+    Thanks for using kind! 😊
+
+    ```
+
+2. Set the kubectl context to the new cluster:
+
+    ```shell
+    kubectl cluster-info --context kind-psa-wo-cluster-pss
+    ```
+    The output is similar to this:
+
+    ```
+     Kubernetes control plane is running at https://127.0.0.1:61350
+
+    CoreDNS is running at https://127.0.0.1:61350/api/v1/namespaces/kube-system/services/kube-dns:dns/proxy
+
+    To further debug and diagnose cluster problems, use 'kubectl cluster-info dump'.
+    ```
+
+3. Get a list of namespaces in the cluster:
+
+    ```shell
+    kubectl get ns
+    ```
+    The output is similar to this:
+    ```      
+    NAME                 STATUS   AGE
+    default              Active   9m30s
+    kube-node-lease      Active   9m32s
+    kube-public          Active   9m32s
+    kube-system          Active   9m32s
+    local-path-storage   Active   9m26s
+    ```
+
+4. Use `--dry-run=server` to understand what happens when different Pod Security Standards
+    are applied:
+
+   1. Privileged
+       ```shell
+       kubectl label --dry-run=server --overwrite ns --all \                    
+       pod-security.kubernetes.io/enforce=privileged
+       ```
+       The output is similar to this:
+       ```      
+       namespace/default labeled
+       namespace/kube-node-lease labeled
+       namespace/kube-public labeled
+       namespace/kube-system labeled
+       namespace/local-path-storage labeled
+       ```
+   2. Baseline
+       ```shell    
+       kubectl label --dry-run=server --overwrite ns --all \
+       pod-security.kubernetes.io/enforce=baseline
+       ```
+       The output is similar to this:
+       ```   
+       namespace/default labeled
+       namespace/kube-node-lease labeled
+       namespace/kube-public labeled
+       Warning: existing pods in namespace "kube-system" violate the new PodSecurity enforce level "baseline:latest"
+       Warning: etcd-psa-wo-cluster-pss-control-plane (and 3 other pods): host namespaces, hostPath volumes
+       Warning: kindnet-vzj42: non-default capabilities, host namespaces, hostPath volumes
+       Warning: kube-proxy-m6hwf: host namespaces, hostPath volumes, privileged
+       namespace/kube-system labeled
+       namespace/local-path-storage labeled
+       ```   
+
+   3. Restricted
+      ```shell
+       kubectl label --dry-run=server --overwrite ns --all \
+       pod-security.kubernetes.io/enforce=restricted
+       ```
+       The output is similar to this:
+       ```   
+       namespace/default labeled
+       namespace/kube-node-lease labeled
+       namespace/kube-public labeled
+       Warning: existing pods in namespace "kube-system" violate the new PodSecurity enforce level "restricted:latest"
+       Warning: coredns-7bb9c7b568-hsptc (and 1 other pod): unrestricted capabilities, runAsNonRoot != true, seccompProfile
+       Warning: etcd-psa-wo-cluster-pss-control-plane (and 3 other pods): host namespaces, hostPath volumes, allowPrivilegeEscalation != false, unrestricted capabilities, restricted volume types, runAsNonRoot != true
+       Warning: kindnet-vzj42: non-default capabilities, host namespaces, hostPath volumes, allowPrivilegeEscalation != false, unrestricted capabilities, restricted volume types, runAsNonRoot != true, seccompProfile
+       Warning: kube-proxy-m6hwf: host namespaces, hostPath volumes, privileged, allowPrivilegeEscalation != false, unrestricted capabilities, restricted volume types, runAsNonRoot != true, seccompProfile
+       namespace/kube-system labeled
+       Warning: existing pods in namespace "local-path-storage" violate the new PodSecurity enforce level "restricted:latest"
+       Warning: local-path-provisioner-d6d9f7ffc-lw9lh: allowPrivilegeEscalation != false, unrestricted capabilities, runAsNonRoot != true, seccompProfile
+       namespace/local-path-storage labeled
+       ```
+
+From the previous output, you'll notice that applying the `privileged` Pod Security Standard shows no warnings
+for any namespaces. However, `baseline` and `restricted` standards both have
+warnings, specifically in the `kube-system` namespace.
+
+## Set modes, versions and standards
+
+In this tutorial, you apply the following Pod Security Standards to the `latest` version:
+
+  * `baseline` standard in `enforce` mode.
+  * `restricted` standard in `warn` and `audit` mode.
+
+The `baseline` Pod Security Standard provides a convenient
+middle ground that allows keeping the exemption list short and prevents known
+privilege escalations.
+
+Additionally, to prevent pods from failing in `kube-system`, you'll exempt the namespace
+from having Pod Security Standards applied.
+
+When you implement Pod Security Admission in your own environment, consider the
+following:
+
+1. Based on the risk posture applied to a cluster, a stricter Pod Security
+   Standard like `restricted` might be a better choice.
+1. Exempting the `kube-system` namespace allows pods to run as
+   `privileged` in this namespace. We recommend that you apply strict RBAC
+   policies that limit access to `kube-system`, following the principle of least
+   privilege.
+
+1. Create a configuration file that can be consumed by the Pod Security
+Admission Controller to implement these Pod Security Standards:
+
+```
+mkdir -p /tmp/pss
+cat <<EOF > /tmp/pss/cluster-level-pss.yaml 
+apiVersion: apiserver.config.k8s.io/v1
+kind: AdmissionConfiguration
+plugins:
+- name: PodSecurity
+  configuration:
+    apiVersion: pod-security.admission.config.k8s.io/v1beta1
+    kind: PodSecurityConfiguration
+    defaults:
+      enforce: "baseline"
+      enforce-version: "latest"
+      audit: "restricted"
+      audit-version: "latest"
+      warn: "restricted"
+      warn-version: "latest"
+    exemptions:
+      usernames: []
+      runtimeClasses: []
+      namespaces: [kube-system]
+EOF
+```
+
+
+1. Configure the API server to consume this file during cluster creation:
+
+    ```
+    cat <<EOF > /tmp/pss/cluster-config.yaml 
+    kind: Cluster
+    apiVersion: kind.x-k8s.io/v1alpha4
+    nodes:
+    - role: control-plane
+      kubeadmConfigPatches:
+      - |
+        kind: ClusterConfiguration
+        apiServer:
+            extraArgs:
+              admission-control-config-file: /etc/config/cluster-level-pss.yaml
+            extraVolumes:
+              - name: accf
+                hostPath: /etc/config
+                mountPath: /etc/config
+                readOnly: false
+                pathType: "DirectoryOrCreate"
+      extraMounts:
+      - hostPath: /tmp/pss
+        containerPath: /etc/config
+        # optional: if set, the mount is read-only.
+        # default false
+        readOnly: false
+        # optional: if set, the mount needs SELinux relabeling.
+        # default false
+        selinuxRelabel: false
+        # optional: set propagation mode (None, HostToContainer or Bidirectional)
+        # see https://kubernetes.io/docs/concepts/storage/volumes/#mount-propagation
+        # default None
+        propagation: None
+    EOF
+    ```
+
+    {{<note>}}
+    If you use Docker Desktop with KinD, the `/tmp`
+    directory is added as a Shared Directory under
+    **Preferences > Resources > File Sharing** on Mac OS.
+    {{</note>}}
+
+2. Create a cluster that uses Pod Security Admission to apply
+   these Pod Security Standards:
+
+   ```shell
+    kind create cluster --name psa-with-cluster-pss --image kindest/node:latest --config /tmp/pss/cluster-config.yaml
+   ```
+   The output is similar to this:
+   ```
+    Creating cluster "psa-with-cluster-pss" ...
+     ✓ Ensuring node image (kindest/node:latest) 🖼 
+     ✓ Preparing nodes 📦  
+     ✓ Writing configuration 📜 
+     ✓ Starting control-plane 🕹️ 
+     ✓ Installing CNI 🔌 
+     ✓ Installing StorageClass 💾 
+    Set kubectl context to "kind-psa-with-cluster-pss"
+    You can now use your cluster with:
+
+    kubectl cluster-info --context kind-psa-with-cluster-pss
+
+    Have a question, bug, or feature request? Let us know! https://kind.sigs.k8s.io/#community 🙂
+
+3. Point kubectl to the cluster 
+   ```shell
+    kubectl cluster-info --context kind-psa-with-cluster-pss
+    Kubernetes control plane is running at https://127.0.0.1:63855
+    CoreDNS is running at https://127.0.0.1:63855/api/v1/namespaces/kube-system/services/kube-dns:dns/proxy
+    
+    To further debug and diagnose cluster problems, use 'kubectl cluster-info dump'.
+    ```
+
+4. Create a Pod with minimal configuration in the default namespace:
+
+    ```
+    cat <<EOF > /tmp/pss/nginx-pod.yaml
+    apiVersion: v1
+    kind: Pod
+    metadata:
+      name: nginx
+    spec:
+      containers:
+        - image: nginx
+          name: nginx
+          ports:
+            - containerPort: 80
+    EOF
+    ```
+5. Create Pod after pod security is enabled at cluster level:
+    
+   ```shell
+    kubectl apply -f /tmp/pss/nginx-pod.yaml
+   ```
+   The output is similar to this:
+   ```
+    Warning: would violate PodSecurity "restricted:latest": allowPrivilegeEscalation != false (container "nginx" must set securityContext.allowPrivilegeEscalation=false), unrestricted capabilities (container "nginx" must set securityContext.capabilities.drop=["ALL"]), runAsNonRoot != true (pod or container "nginx" must set securityContext.runAsNonRoot=true), seccompProfile (pod or container "nginx" must set securityContext.seccompProfile.type to "RuntimeDefault" or "Localhost")
+    pod/nginx created
+    ```
+## Clean up
+
+Run `kind delete cluster -name psa-with-cluster-pss` and 
+`kind delete cluster -name psa-wo-cluster-pss` to delete the clusters you 
+created.
+
+## {{% heading "whatsnext" %}}
+
+- Run a 
+  [gist](https://gist.github.com/PushkarJ/9f7a0045f4bec31097bdd1e9db0f2f6e) 
+  to perform all the preceding steps at once:
+     1. Create a Pod Security Standards based cluster level Configuration
+     2. Create a file to let API server consumes this configuration
+     3. Create a cluster that creates an API server with this configuration
+     4. Set kubectl context to this new cluster
+     5. Create a minimal pod yaml file
+     6. Apply this file to create a Pod in the new cluster
+- [Pod Security Admission](/docs/concepts/security/pod-security-admission/)
+- [Pod Security Standards](/docs/concepts/security/pod-security-standards/)
+- [Applying Pod Security Standards at the namespace level](/docs/tutorials/security/ns-level-pss/)