Fixes for 1.10 docs (#903)

* Update version link to 1.9 (#880) * set branch version Signed-off-by: Chip Zoller <chipzoller@gmail.com> * update version link for 1.9 Signed-off-by: Chip Zoller <chipzoller@gmail.com> --------- Signed-off-by: Chip Zoller <chipzoller@gmail.com> * collapse roles and add self-service note for Contributors (#883) * collapse roles and add self-service note for Contributors Signed-off-by: Jim Bugwadia <jim@nirmata.com> * fix typos Signed-off-by: Jim Bugwadia <jim@nirmata.com> * flip columns and revert contributor responsibilities Signed-off-by: Jim Bugwadia <jim@nirmata.com> --------- Signed-off-by: Jim Bugwadia <jim@nirmata.com> Signed-off-by: Chip Zoller <chipzoller@gmail.com> * blog: Let's Play Kyverno (#890) Signed-off-by: Frank Jogeleit <frank.jogeleit@lovoo.com> Signed-off-by: Chip Zoller <chipzoller@gmail.com> * fix: update broken link to background scans page (#897) * fix: update broken link to background scans page Signed-off-by: emmanuel-ferdman <35470921+emmanuel-ferdman@users.noreply.github.com> * fix: update broken link to background scans page Signed-off-by: emmanuel-ferdman <emmanuelferdman@gmail.com> Signed-off-by: emmanuel-ferdman <35470921+emmanuel-ferdman@users.noreply.github.com> * fix: update broken link to background scans page Signed-off-by: emmanuel-ferdman <emmanuelferdman@gmail.com> Signed-off-by: emmanuel-ferdman <35470921+emmanuel-ferdman@users.noreply.github.com> --------- Signed-off-by: emmanuel-ferdman <35470921+emmanuel-ferdman@users.noreply.github.com> Signed-off-by: emmanuel-ferdman <emmanuelferdman@gmail.com> * add missing auto-gen note Signed-off-by: Chip Zoller <chipzoller@gmail.com> * fix link Signed-off-by: Chip Zoller <chipzoller@gmail.com> * clarify description of exceptionNamespace Signed-off-by: Chip Zoller <chipzoller@gmail.com> * improved description of skipResourceFilters Signed-off-by: Chip Zoller <chipzoller@gmail.com> * New PSA blog (#895) * save Signed-off-by: Chip Zoller <chipzoller@gmail.com> * newline Signed-off-by: Chip Zoller <chipzoller@gmail.com> --------- Signed-off-by: Chip Zoller <chipzoller@gmail.com> * Docfixes (#898) * fix flag description Signed-off-by: Chip Zoller <chipzoller@gmail.com> * updates Signed-off-by: Chip Zoller <chipzoller@gmail.com> * PSA blog updates (#900) * Merge branch 'main' into flagfix * fix: update broken link to "Disallow Default Namespace" page (#902) Signed-off-by: emmanuel-ferdman <35470921+emmanuel-ferdman@users.noreply.github.com> * Merge branch 'main' into flagfix * fix flag description (#899) --------- Signed-off-by: Chip Zoller <chipzoller@gmail.com> Signed-off-by: Jim Bugwadia <jim@nirmata.com> Signed-off-by: Frank Jogeleit <frank.jogeleit@lovoo.com> Signed-off-by: emmanuel-ferdman <35470921+emmanuel-ferdman@users.noreply.github.com> Signed-off-by: emmanuel-ferdman <emmanuelferdman@gmail.com> Co-authored-by: Jim Bugwadia <jim@nirmata.com> Co-authored-by: Frank Jogeleit <frank.jogeleit@web.de> Co-authored-by: Emmanuel Ferdman <35470921+emmanuel-ferdman@users.noreply.github.com> Co-authored-by: Emmanuel Ferdman <emmanuelferdman@gmail.com>
kyverno · Jun 13, 2023 · 17d99dc · 17d99dc
1 parent dddd64c
commit 17d99dc
Show file tree

Hide file tree

Showing 5 changed files with 151 additions and 6 deletions.
diff --git a/content/en/blog/general/psa-better-together/index.md b/content/en/blog/general/psa-better-together/index.md
@@ -0,0 +1,143 @@
+---
+date: 2023-06-12
+title: Using Kyverno with Pod Security Admission
+linkTitle: Using Kyverno with Pod Security Admission
+description: Using Pod Security Admission with Kyverno for the best of both worlds.
+draft: false
+---
+
+[Pod Security Admission](https://kubernetes.io/docs/concepts/security/pod-security-admission/) (PSA) is the built-in successor to Kubernetes [PodSecurityPolicy](https://kubernetes.io/docs/concepts/security/pod-security-policy/) (PSP) and is enabled by default starting in v1.23, graduating to stable in v1.25, the same version where PSP was finally removed. PSA is different from PSP in [many respects](/blog/2023/05/24/podsecuritypolicy-migration-with-kyverno/#comparison), however one of the most important--and central to how PSA operates--is that it is focuses on implementing **standards** and not individual checks like PSP did. The standards in this case are the [Pod Security Standards](https://kubernetes.io/docs/concepts/security/pod-security-standards/). While many might see technologies like Kyverno and other admission controllers in competition PSA, the two can actually be highly complementary and deliver increased value through tighter security guarantees while catering to the flexibility demanded by modern Kubernetes operations. In this blog, we'll explain some of these use cases for how Kyverno can be used alongside PSA to get the best of both worlds.
+
+## Introduction
+
+First, let's explain a bit about how PSA can be used for those not familiar. Although this won't be a full tutorial on how to use the technology, it'll at least establish a good set of boundaries to help you know where it begins and ends.
+
+PSA can basically be used in two ways: at the cluster level and at the Namespace level. Although it is "on" by default, it isn't "active" by default. Since PSA does not require an API resource similar to how PSP worked, it is "activated" in different ways. To activate PSA at the cluster level, one must create and install a specific file local to the control plane of the kind `AdmissionConfiguration` the contents of which will dictate how it works. Many Kubernetes PaaS providers have already done the work at the cluster level for you and, in many cases, there's nothing you can configure yourself. For example, in cases like EKS, PSA is enabled at the cluster level but set to the privileged mode which allows insecure Pod configurations. To activate PSA at the Namespace level, one need only label a Namespace. The two methods aren't mutually exclusive. In the case of the Namespace method, using PSA is extremely quick and simple.
+
+Now on to the important point about profiles mentioned at the outset. PSA is the technology which implements the Pod Security Standards (PSS). Those standards are broken up into three **profiles**: [privileged](https://kubernetes.io/docs/concepts/security/pod-security-standards/#privileged), which is effectively the lack of any security, [baseline](https://kubernetes.io/docs/concepts/security/pod-security-standards/#baseline), and [restricted](https://kubernetes.io/docs/concepts/security/pod-security-standards/#restricted). Each profile is a collection of different **controls**. Each control establishes which fields of a Pod's spec should or should not be set and with what value(s). The restricted profile is cumulative of the baseline profile, so by enabling restricted you implicitly get baseline.
+
+Regardless of how PSA is activated and at what scope, it implements these profiles. Critical to understand is that **this is an all-or-nothing proposition**: a profile is implemented in its entirety and there is no possibility to cherry pick which controls in the profile are checked. Although each profile can be checked in either `audit`, `warn`, or `enforce` mode (even multiple modes simultaneously), you either get all of that profile or you get none of it.
+
+Lastly, there is the matter of exclusions. It is extremely common for users to need the ability to tactically exclude certain Pods from the scope of policy because of some special needs they have. PSA allows exclusions via exemptions but only when PSA is activated cluster wide. The exemptions available are `usernames`, `runtimeClasses` and `namespaces`. All exemptions must be specified in the `AdmissionConfiguration` file mentioned earlier.
+
+With some of the mechanics of PSA out of the way, let's get into the use cases for PSA and Kyverno.
+
+## Use Cases
+
+Although there are many possible points of intersection between PSA and Kyverno, here are four we've identified and commonly see employed. For some, there can be many different variations which we won't all cover.
+
+### Use Kyverno to require PSA Namespace labels
+
+As mentioned in the beginning, one of PSA's methods of operation is through assignment of labels to Namespaces. Without at least one PSA-specific label, nothing will happen. Kyverno can assist in this process in a couple ways:
+
+1. Validate that labels are assigned where and how they need to be
+2. Automatically assign the labels based upon your criteria
+
+On the first, Kyverno can ensure that every Namespace has _some_ PSA label attached to it helping to prevent either typos or just forgetfulness. What that/those label(s) is/are and with what values is entirely up to you, but it is all driven by policy-as-code (PaC). This is, indeed, convenient, but it can be a more serious issue when you have PSA activated at, for example, the baseline profile across the cluster. Should a user be able to assign the Namespace label `pod-security.kubernetes.io/enforce: privileged` in these cases, it will effectively deactivate PSA on that Namespace. That's obviously not good and could compromise the security of that Namespace. But since Kubernetes RBAC doesn't allow for such granular permissions, you can outsource that job to Kyverno to ensure this act cannot be performed.
+
+On the second, Kyverno can actually assign (i.e., mutate) the desired PSA labels for you based upon whatever criteria you establish and define in a policy. Even if you've done so in git, Kyverno can double-check your work and assign them if not present. For example, you might want to assign the `pod-security.kubernetes.io/enforce: baseline` label to all Namespaces by default but assign `pod-security.kubernetes.io/enforce: restricted` to those which begin with the name `pci-` or have another label called `env: prod`. Kyverno's [mutation](/docs/writing-policies/mutate/) ability gives you very flexible definitions which can make these things happen.
+
+These policies have already been written and are provided to you [here](/policies/?policytypes=Pod%2520Security%2520Admission).
+
+### Use PSA and Kyverno both to enforce
+
+In this use case, which is the most common, you use both PSA and Kyverno combined to provide comprehensive protection for your cluster where each does something different. Due to the limited nature of exclusions, we often find that PSA is used right up until the point that its flexibility can go no further. The most common examples are with some monitoring/observability tools and service meshes. These types of tools often have Pods that need to run with privileges not afforded by sometimes even the baseline profile. Rather than deactivate Pod security in those Namespaces--which for many is a non-starter--Kyverno can be used because of its great flexibility.
+
+Kyverno has the ability to exclude a Pod based on just about any combination you can think up. Whether it's Pod name, image, user, label, or what have you, Kyverno can exempt it from controls.
+
+When it comes to how Kyverno enforces, there are generally two options. First, Kyverno [publishes and maintains](/policies/pod-security/) the complete Pod Security Standards as individual policies (one per control). These are also conveniently packaged into their own Helm chart for easy installation. Second, Kyverno directly integrates the PSA libraries into the engine and exposes a simplified rule type we call a validate "subrule" known as [podSecurity](/docs/writing-policies/validate/#pod-security). This rule type, powered under the hood by PSA, allows a consolidated, simplified way to enforce the Pod Security Standards. But in addition, it also allows more capabilities not found in PSA. For example, using this podSecurity subrule, it is possible to exempt specific controls by name and even specific images in addition to the [standard exclusions](/docs/writing-policies/match-exclude/) provided by all Kyverno rule types. This is incredibly powerful yet easy to use.
+
+For example, below is a Kyverno policy which applies to Pods in all Namespaces beginning with the prefix `prod-` and enforces the restricted profile except for the Seccomp check and specifically on containers which use the `cadvisor` image.
+
+```yaml
+apiVersion: kyverno.io/v1
+kind: ClusterPolicy
+metadata:
+  name: psa
+spec:
+  background: true
+  validationFailureAction: Enforce
+  rules:
+  - name: restricted
+    match:
+      any:
+      - resources:
+          kinds:
+          - Pod
+          namespaces:
+          - prod-*
+    validate:
+      podSecurity:
+        level: restricted
+        version: latest
+        exclude:
+        - controlName: Seccomp
+        - controlName: Seccomp
+          images:
+          - cadvisor*
+```
+
+This is just one example and there are many variations for how these two technologies can be used simultaneously, each to enforce some aspect of Pod security on your cluster. For more information as well as a list of those variations, see the [documentation](/docs/writing-policies/validate/#psa-interoperability). In addition to the documentation, there are pre-built samples ready to go [here](/policies/?policytypes=Pod%2520Security).
+
+And, by the way, one more benefit you get when using Kyverno to enforce its Pod Security Standards over PSA is that it allows you to block Pod controllers directly as a result of its [rule auto-generation](/docs/writing-policies/autogen/) capability. This is not something PSA allows and only non-compliant Pods will be blocked.
+
+### Use Kyverno reporting for PSA
+
+PSA allows for assigning multiple labels to a Namespace (or configuring the same via the `AdmissionConfiguration` file) to get a union of the behaviors. For example, PSA can `enforce` at the baseline profile and `audit` and/or `warn` at the restricted profile. The audit mode means any violations will be allowed yet will be written in the Kubernetes audit log. There are a few down sides with this approach. The audit log is not accessible from inside the cluster because it's not a native Kubernetes resource, it's literally a log file written to the control plane or, optionally, sent to an external server. Audit logs also commonly cannot be enabled retroactively or to do so is cumbersome and disruptive. In the case of some cloud PaaS offerings, if audit logging wasn't enabled at the time the cluster was created then it can't be subsequently enabled, potentially forcing you to build another cluster just to gain these insights.
+
+This is yet another opportunity where Kyverno can step in and help. In cases where PSA is not enforcing at the most restrictive level, Kyverno can be layered in to generate its Policy Reports. A [Policy Report](/docs/policy-reports/) is an in-cluster, Kubernetes-native resource which is an open standard developed by the upstream [Kubernetes policy working group](https://github.com/kubernetes-sigs/wg-policy-prototypes/tree/master/policy-report) and adopted by many other tools. Policy reports give you a leg up over audit logs because they empower self-service and compliment multi-tenant approaches. Since they're just another Kubernetes resource, RBAC can be wrapped around them allowing, for example, developers to see violations in their own Namespaces so they know what they're responsible for addressing. They can also help you as a cluster admin or operator know which resources in the cluster are non-compliant at any point in time since Policy Reports are not historical and always reflect the current cluster state.
+
+It's important to understand, however, that Kyverno's Policy Reporting feature will only work on Pods which have _not_ been blocked by PSA. But, provided that is true, the process to enable and begin viewing Policy Reports is simple: load any Kyverno policy, for example the [restricted profile of the PSS](/policies/pod-security/subrule/restricted/restricted-latest/restricted-latest/), in `audit` mode with the flag `background: true`. The `audit` mode means Kyverno will not block any Pods, and the `true` setting on the `background` field means it will periodically scan the cluster to compare existing resources against the policy giving you an extra layer of comfort that nothing got missed. Policy Reports will then be created in each Namespace where Pods are located.
+
+Reporting isn't just restricted to Pods either but can be used to check for those Namespace labels we discussed earlier. For example, you may want to know which Namespaces have no PSA labels whatsoever which could key you in to someone or something doing things it perhaps shouldn't. By introducing the below policy, Kyverno will generate and maintain and a Cluster Policy Report in the `default` Namespace with the result of these continual checks. The results can be conveniently viewed by cluster admins or scraped and alerted on by other tools such as the [Policy Reporter](https://kyverno.github.io/policy-reporter/) and more.
+
+```yaml
+apiVersion: kyverno.io/v1
+kind: ClusterPolicy
+metadata:
+  name: psa-reporting
+spec:
+  validationFailureAction: audit
+  background: true
+  rules:
+  - name: check-namespace-labels
+    match:
+      any:
+      - resources:
+          kinds:
+            - Namespace
+    validate:
+      message: This Namespace is missing a PSA label.
+      pattern:
+        metadata:
+          labels:
+            pod-security.kubernetes.io/*: "?*"
+```
+
+### Use Kyverno CLI in pipelines to test against PSS profiles
+
+If none of the above use cases piqued your interests or you are unwilling to install anything additional in your cluster, there is **still** an opportunity for Kyverno to provide value. Kyverno comes with a CLI which can be used in CI pipelines to check resource manifests against those Pod Security Standards enforced by PSA. But since PSA itself is in-line to the Kubernetes API with no external tooling, it takes standing up a full, running cluster to perform the same checks otherwise. As we mentioned earlier, Kyverno integrates the PSA libraries directly into its engine. And since the CLI is really just a different form factor of the same Kyverno engine, you get the same benefits of Kyverno outside the cluster as in.
+
+Take one of the earlier policies we've pointed out along the way and see a sample GitHub Action workflow we publish in the [documentation](/docs/testing-policies/#github-actions) and you're up and running. The `kyverno apply` command will naturally exit with an error code if the resource(s) do not pass. The CLI combined with Kyverno's rule auto-generation capability mentioned earlier means you can be informed of those Deployments, StatefulSets, and others before they need to touch a cluster.
+
+Below you can see the result of the Kyverno CLI comparing a standard Deployment with no security settings defined to the restricted profile of the PSS showed previously. As you can see, the Deployment failed the checks and a message is printed with exactly which controls failed and what must be done to address them.
+
+```sh
+$ kubectl-kyverno apply restricted.yaml -r deploy.yaml 
+
+Applying 1 policy rule to 1 resource...
+
+policy psa -> resource default/Deployment/busybox failed: 
+1. autogen-restricted: Validation rule 'autogen-restricted' failed. It violates PodSecurity "restricted:latest":
+({Allowed:false ForbiddenReason:allowPrivilegeEscalation != false ForbiddenDetail:container "busybox" must set securityContext.allowPrivilegeEscalation=false})
+({Allowed:false ForbiddenReason:unrestricted capabilities ForbiddenDetail:container "busybox" must set securityContext.capabilities.drop=["ALL"]})
+({Allowed:false ForbiddenReason:runAsNonRoot != true ForbiddenDetail:pod or container "busybox" must set securityContext.runAsNonRoot=true})
+({Allowed:false ForbiddenReason:seccompProfile ForbiddenDetail:pod or container "busybox" must set securityContext.seccompProfile.type to "RuntimeDefault" or "Localhost"})
+
+
+pass: 0, fail: 1, warn: 0, error: 0, skip: 2
+```
+
+## Closing
+
+Pod Security Admission is a major improvement over PodSecurityPolicy and couldn't be simpler to activate and begin using in your clusters. If you have no Pod security today, we absolutely recommend you plug that gap and starting with PSA can be a great way. As we hope we've made clear, there are many places where PSA and Kyverno can be used together and it doesn't have to be an either or decision. Use PSA where it makes sense and provides the necessary level of coverage, and use Kyverno to pick up the slack.
diff --git a/content/en/docs/Installation/customization.md b/content/en/docs/Installation/customization.md
@@ -273,7 +273,7 @@ The following flags can be used to control the advanced behavior of the various
 14. `enableConfigMapCaching` (ABR): enables the ConfigMap caching feature. Defaults to `true`.
 15. `enablePolicyException` (ABR): set to `true` to enable the [PolicyException capability](/docs/writing-policies/exceptions/). Default is `false`.
 16. `enableTracing` (ABCR): set to enable exposing traces. Default is `false`.
-17. `exceptionNamespace` (ABR): set to the name of a Namespace where [PolicyExceptions](/docs/writing-policies/exceptions/) will only be permitted. PolicyExceptions created in any other Namespace will throw a warning. If not set, PolicyExceptions from all namespaces will be considered. Implies the `enablePolicyException` flag is set to `true`.
+17. `exceptionNamespace` (ABR): set to the name of a Namespace where [PolicyExceptions](/docs/writing-policies/exceptions/) will only be permitted. PolicyExceptions created in any other Namespace will throw a warning. If not set, PolicyExceptions from all Namespaces will be considered. Implies the `enablePolicyException` flag is set to `true`. Neither wildcards nor multiple Namespaces are currently accepted.
 18. `forceFailurePolicyIgnore` (A): set to force Failure Policy to `Ignore`. Default is `false`.
 19. `genWorkers` (B): the number of workers for processing generate policies concurrently. Default is `10`.
 20. `imagePullSecrets` (ABR): specifies secret resource names for image registry access credentials. Only a single value accepted currently.
@@ -300,7 +300,7 @@ The following flags can be used to control the advanced behavior of the various
 41. `reportsChunkSize` (R): maximum number of results in generated reports before splitting occurs if there are more results to be stored. Default is `1000`.
 42. `serverIP` (AC): like the `kubeconfig` flag, used when running Kyverno outside of the cluster which it serves.
 43. `servicePort` (AC): port used by the Kyverno Service resource and for webhook configurations. Default is `443`.
-44. `skipResourceFilters` (R): obey the ConfigMap's resourceFilters when performing background report scans. Default is `true`.
+44. `skipResourceFilters` (R): defines whether to obey the ConfigMap's resourceFilters when performing background report scans. Default is `true`. When set to `true`, anything defined in the resourceFilters will not be excluded in background reports. Ex., when set to `true` if the resourceFilters contain the `[*/*,kube-system,*]` entry then background scan reports will be produced for anything in the `kube-system` Namespace. Set this value to `false` to obey resourceFilters in background scan reports. Ex., when set to `false` if the resourceFilters contain the `[*/*,kube-system,*]` entry then background scan reports will NOT be produced for anything in the `kube-system` Namespace.
 45. `skip_headers` (ABCR): if true, avoid header prefixes in the log messages.
 46. `skip_log_headers` (ABCR): if true, avoid headers when opening log files (no effect when -logtostderr=true).
 47. `stderrthreshold` (ABCR): logs at or above this threshold go to stderr when writing to files and stderr (no effect when -logtostderr=true or -alsologtostderr=false). Default is `2`.