1.8.0 updates (#625)

* add docs on using Kyverno CLI in a CI process

Signed-off-by: Chip Zoller <chipzoller@gmail.com>

* typos

Signed-off-by: Chip Zoller <chipzoller@gmail.com>

* instructions for manual CLI binary installation

Signed-off-by: Chip Zoller <chipzoller@gmail.com>

* clarify auto-gen

Signed-off-by: Chip Zoller <chipzoller@gmail.com>

* add applyRules

Signed-off-by: Chip Zoller <chipzoller@gmail.com>

* add missing path

Signed-off-by: Chip Zoller <chipzoller@gmail.com>

* clarify context variable ordering

Signed-off-by: Chip Zoller <chipzoller@gmail.com>

* clarify EKS troubleshooting note

Signed-off-by: Chip Zoller <chipzoller@gmail.com>

* remove conflicting statement

Signed-off-by: Chip Zoller <chipzoller@gmail.com>

* expand info on --imagePullSecrets

Signed-off-by: Chip Zoller <chipzoller@gmail.com>

* add/update docs on ClusterRole aggregation changes

Signed-off-by: Chip Zoller <chipzoller@gmail.com>

* remove under construction notice

Signed-off-by: Chip Zoller <chipzoller@gmail.com>

* add installation note about dedicated namespace

Signed-off-by: Chip Zoller <chipzoller@gmail.com>

* cover updates to failurePolicy

Signed-off-by: Chip Zoller <chipzoller@gmail.com>

* add broken link checker

Signed-off-by: Chip Zoller <chipzoller@gmail.com>

* add note on precondition short circuiting

Signed-off-by: Chip Zoller <chipzoller@gmail.com>

* newline per comment

Signed-off-by: Chip Zoller <chipzoller@gmail.com>

* use env

Signed-off-by: Chip Zoller <chipzoller@gmail.com>

* add note on supported install

Signed-off-by: Chip Zoller <chipzoller@gmail.com>

* Add CLI test note

Signed-off-by: Chip Zoller <chipzoller@gmail.com>

* update homepage to incubating

Signed-off-by: Chip Zoller <chipzoller@gmail.com>

* add info on extended API calls

Signed-off-by: Chip Zoller <chipzoller@gmail.com>

* remove space

Signed-off-by: Chip Zoller <chipzoller@gmail.com>

* add example of using operator in validate

Signed-off-by: Chip Zoller <chipzoller@gmail.com>

* fix negation anchor description

Signed-off-by: Chip Zoller <chipzoller@gmail.com>

* add JMESPath random() filter docs

Signed-off-by: Chip Zoller <chipzoller@gmail.com>

* add and update flags

Signed-off-by: Chip Zoller <chipzoller@gmail.com>

* add resources[] and generate to test

Signed-off-by: Chip Zoller <chipzoller@gmail.com>

Signed-off-by: Chip Zoller <chipzoller@gmail.com>

content/en/_index.md

@@ -63,7 +63,7 @@ security. The Kyverno CLI can be used to test policies and validate resources as
 
 {{< blocks/lead color="dark" >}}
 
-## Kyverno is a CNCF Sandbox Project { class="text-center mb-4" }
+## Kyverno is a CNCF Incubating Project { class="text-center mb-4" }
 
 <a href="https://www.cncf.io" target="blank">
 	<img class="cncf-logo img-fluid" src="/images/logo_cloudnative.png" alt="Cloud Native Computing Foundation logo">

content/en/docs/Installation/_index.md

@@ -27,7 +27,7 @@ As of v1.7.0, Kyverno follows the same support policy as the Kubernetes project
 
 ## Install Kyverno using Helm
 
-Kyverno can be deployed via a Helm chart--the recommended method for a production install--which is accessible either through the Kyverno repo or on [ArtifactHub](https://artifacthub.io/packages/helm/kyverno/kyverno).
+Kyverno can be deployed via a Helm chart--the only supported method for a production install--which is accessible either through the Kyverno repo or on [ArtifactHub](https://artifacthub.io/packages/helm/kyverno/kyverno).
 
 In order to install Kyverno with Helm, first add the Kyverno Helm repository.
 
@@ -116,7 +116,7 @@ helm install kyverno kyverno/kyverno -n kyverno --create-namespace --devel
 
 ## Install Kyverno using YAMLs
 
-Kyverno can also be installed using a single installation manifest, however for production installation the Helm chart is the recommended method.
+Kyverno can also be installed using a single installation manifest, however for production installation the Helm chart is the only supported method.
 
 This manifest path will always point to the latest `main` branch and is not guaranteed to be stable.
 
@@ -434,13 +434,13 @@ The following flags are used to control the behavior of Kyverno and must be set
 
 1. `excludeGroupRole`: excludeGroupRole role expected string with comma-separated group role. It will exclude all the group role from the user request. Default we are using `system:serviceaccounts:kube-system,system:nodes,system:kube-scheduler`.
 2. `excludeUsername`: excludeUsername expected string with comma-separated kubernetes username. In generate request if user enable `Synchronize` in generate policy then only kyverno can update/delete generated resource but admin can exclude specific username who have access of delete/update generated resource.
-3. `resourceFilters`: Kubernetes resources in the format "[kind,namespace,name]" where the policy is not evaluated by the admission webhook. For example --filterKind "[Deployment, kyverno, kyverno]" --filterKind "[Deployment, kyverno, kyverno],[Events, *, *]".
-4. `generateSuccessEvents`: specifies whether (true/false) to generate success events. Default is set to "false".
+3. `generateSuccessEvents`: specifies whether (true/false) to generate success events. Default is set to "false".
+4. `resourceFilters`: Kubernetes resources in the format "[kind,namespace,name]" where the policy is not evaluated by the admission webhook. For example --filterKind "[Deployment, kyverno, kyverno]" --filterKind "[Deployment, kyverno, kyverno],[Events, *, *]".
 5. `webhooks`: specifies the Namespace or object exclusion to configure in the webhooks managed by Kyverno.
 
 ### Container Flags
 
-The following flags can also be used to control the advanced behavior of Kyverno and must be set on the main `kyverno` container in the form of arguments.
+The following flags can also be used to control the advanced behavior of Kyverno and must be set on the main `kyverno` container in the form of arguments. Unless otherwise stated, all container flags should be prefaced with two dashes (ex., `--autogenInternals`).
 
 1. `-v`: Sets the verbosity level of Kyverno log output. Takes an integer from 1 to 6 with 6 being the most verbose. Level 4 shows variable substitution messages.
 2. `profile`: setting this flag to 'true' will enable profiling.
@@ -458,6 +458,10 @@ The following flags can also be used to control the advanced behavior of Kyverno
 14. `clientRateLimitBurst`: configure the maximum burst for throttling. Uses the client default if zero. Example: `50`
 15. `webhookTimeout`: specifies the timeout for webhooks. After the timeout passes, the webhook call will be ignored or the API call will fail based on the failure policy. The timeout value must be between 1 and 30 seconds, defaults to 10s.
 16. `autogenInternals`: New in Kyverno 1.7.0, this flag activates the (currently beta) [auto-generate](/docs/writing-policies/autogen/) rule calculation to not write to the `.spec` field of Kyverno policies. This is under construction and the behavior will change in the future. Set to `false` by default. Set to `true` to activate this ability.
+17. `maxQueuedEvents`: defines the upper limit of events that are queued internally. Value is an integer.
+18. `maxReportChangeRequests`: defines the number of RCRs that can be created in a given Namespace. When this threshold is reached, no further RCRs will be created and Kyverno will begin to start the clean-up process. This flag should be used when there is high churn rate in a cluster leading to PolicyReport exhaustion and excessive memory growth. Value is an integer.
+19. `splitPolicyReport`: splits ClusterPolicyReports and PolicyReports into individual reports per policy rather than a single entity per cluster and per Namespace. Useful when having Namespaces with many resources which apply to policies. Value is boolean.
+20. `kubeconfig`: specifies the Kubeconfig file to be used when overriding the API server to which Kyverno should communicate.
 
 ### Policy Report access
 
@@ -542,7 +546,7 @@ metadata:
 
 ## Upgrading Kyverno
 
-Upgrading Kyverno is as simple as applying the new YAML manifest, or using Helm depending on how it was installed.
+Upgrading Kyverno is as simple as applying the new YAML manifest, or using Helm depending on how it was installed. You cannot upgrade Kyverno by bumping the image tag on the Deployment as this will not affect the CRDs and other resources necessary for Kyverno's operation.
 
 ### Upgrade Kyverno with YAML manifest
 

content/en/docs/Kyverno CLI/_index.md

@@ -707,6 +707,10 @@ In each test, there are four desired results which can be tested for. If the act
 
 For help with the `test` command, pass the `-h` flag for extensive output including usage, flags, and sample manifests.
 
+{{% alert title="Note" color="info" %}}
+The Kyverno CLI via the `test` command does not embed the Kubernetes control plane components and therefore is not able to perform the types of initial mutations subjected to a resource as part of an in-cluster creation flow. Take care to ensure the manifests you test account for these modifications.
+{{% /alert %}}
+
 #### Test File Structures
 
 The test declaration file format of `kyverno-test.yaml` must be of the following format. In order to quickly generate a sample manifest which you can populate with your specified inputs, use either the `--manifest-mutate` or `--manifest-validate` command and output the result to a `kyverno-test.yaml` file.
@@ -719,25 +723,20 @@ policies:
 resources:
   - <path/to/resource.yaml>
   - <path/to/resource.yaml>
-# optional file for declaring variables. see below for example.
-variables: variables.yaml
-# optional file for declaring admission request information (roles, cluster roles and subjects). see below for example.
-userinfo: user_info.yaml
+variables: variables.yaml # optional file for declaring variables. see below for example.
+userinfo: user_info.yaml # optional file for declaring admission request information (roles, cluster roles and subjects). see below for example.
 results:
 - policy: <name>
   rule: <name>
   resource: <name>
-  # when testing for a resource in a specific Namespace
-  namespace: <name>
-  # when testing a mutate rule supply patchedResource
-  patchedResource: <file_name.yaml>
+  resources: # optional. One of either `resource` or `resources[]` must be specified. Use `resources[]` when a number of different resources should all share the same test result.
+  - <name_1>
+  - <name_2>
+  namespace: <name> # when testing for a resource in a specific Namespace
+  patchedResource: <file_name.yaml> # when testing a mutate rule this field is required.
+  generatedResource: <file_name.yaml> # when testing a generate rule this field is required.
   kind: <kind>
   result: pass
-- policy: <name>
-  rule: <name>
-  resource: <name>
-  kind: <kind>
-  result: fail
 ```
 
 The test declaration consists of the following parts:

content/en/docs/Writing policies/external-data-sources.md

@@ -310,6 +310,62 @@ The output of this will be `apps/v1`. Older versions of Kubernetes (prior to 1.1
 
 {{% /alert %}}
 
+Kyverno can also fetch data from other API locations such as `/version` and [aggregated APIs](https://kubernetes.io/docs/concepts/extend-kubernetes/api-extension/apiserver-aggregation/).
+
+For example, fetching from `/version` might return something similar to what is shown below.
+
+```sh
+$ kubectl get --raw /version
+{
+  "major": "1",
+  "minor": "23",
+  "gitVersion": "v1.23.8+k3s1",
+  "gitCommit": "53f2d4e7d80c09a7db1858e3f4e7ddfa13256c45",
+  "gitTreeState": "clean",
+  "buildDate": "2022-06-27T21:48:01Z",
+  "goVersion": "go1.17.5",
+  "compiler": "gc",
+  "platform": "linux/amd64"
+}
+```
+
+Fetching from an aggregated API, for example the `metrics.k8s.io` group, can be done with `/apis/metrics.k8s.io/<api_version>/<resource_type>` as shown below.
+
+```sh
+$ kubectl get --raw /apis/metrics.k8s.io/v1beta1/nodes | jq
+{
+  "kind": "NodeMetricsList",
+  "apiVersion": "metrics.k8s.io/v1beta1",
+  "metadata": {},
+  "items": [
+    {
+      "metadata": {
+        "name": "k3d-kyv180rc1-server-0",
+        "creationTimestamp": "2022-09-11T13:37:39Z",
+        "labels": {
+          "beta.kubernetes.io/arch": "amd64",
+          "beta.kubernetes.io/instance-type": "k3s",
+          "beta.kubernetes.io/os": "linux",
+          "egress.k3s.io/cluster": "true",
+          "kubernetes.io/arch": "amd64",
+          "kubernetes.io/hostname": "k3d-kyv180rc1-server-0",
+          "kubernetes.io/os": "linux",
+          "node-role.kubernetes.io/control-plane": "true",
+          "node-role.kubernetes.io/master": "true",
+          "node.kubernetes.io/instance-type": "k3s"
+        }
+      },
+      "timestamp": "2022-09-11T13:37:24Z",
+      "window": "10.059s",
+      "usage": {
+        "cpu": "298952967n",
+        "memory": "1311340Ki"
+      }
+    }
+  ]
+}
+```
+
 ### Handling collections
 
 The API server response for a `HTTP GET` on a URL path that requests collections of resources will be an object with a list of items (resources).

content/en/docs/Writing policies/generate.md

@@ -116,7 +116,7 @@ spec:
       # namespace for the generated resource
       namespace: "{{request.object.metadata.name}}"
       # propagate changes from the upstream resource
-      synchronize : true
+      synchronize: true
       clone:
         namespace: default
         name: config-template

content/en/docs/Writing policies/jmespath.md

@@ -1394,6 +1394,56 @@ metadata:
 </p>
 </details>
 
+### Random
+
+<details><summary>Expand</summary>
+<p>
+
+The `random()` filter is used to generate a random sequence of string data based upon the input pattern, expressed as regex. The input it takes is a combination of the composition of the pattern and the length of each pattern. This filter is useful in a variety of ways including generating unique resource names. Some other use cases include creating Pod hashes, auth tokens, license keys, GUIDs, and more.
+
+For example, `random('[0-9a-z]{5}')` will produce a string output of exactly 5 characters long composed of numbers in the collection `0-9` and lower-case letters in the collection `a-z`. The output might be `"91t6f"`. More complex random output can be created by chaining multiple pattern and length combinations together. For example, to create a faux license key you could use the expression `random('[A-Z0-9]{8}-[A-Z0-9]{4}-[A-Z0-9]{4}-[A-Z0-9]{4}-[A-Z0-9]{12}')` which may generate the output `"K284DW7Y-7LMT-XHR3-ZZ53-36366O8JVDG9"`.
+
+| Input 1            | Output   |
+|--------------------|----------|
+| String             | String   |
+
+<br>
+
+**Example:** This policy uses `random()` to mutate a new Secret to add a label with key `randomoutput` and the value of which is `random-` followed by 6 random characters composed of lower-case letters `a-z` and numbers `0-9`.
+
+```yaml
+apiVersion: kyverno.io/v1
+kind: ClusterPolicy
+metadata:
+  name: ver-test
+spec:
+  rules:
+  - name: test-ver-ver
+    match:
+      any:
+      - resources:
+          kinds:
+          - Secret
+    preconditions:
+      all:
+      - key: "{{request.operation}}"
+        operator: In
+        value:
+        - CREATE
+    context:
+    - name: randomtest
+      variable:
+        jmesPath: random('[a-z0-9]{6}')
+    mutate:
+      patchStrategicMerge:
+        metadata:
+          labels:
+            randomoutput: random-{{randomtest}}
+```
+
+</p>
+</details>
+
 ### Regex_match
 
 <details><summary>Expand</summary>

content/en/docs/Writing policies/validate.md

@@ -230,6 +230,29 @@ The `!-` operator provides an easier way of validating the value in question fal
 There is no operator for `equals` as providing a field value in the pattern requires equality to the value.
 {{% /alert %}}
 
+An example of using an operator in a pattern style validation rule is shown below.
+
+```yaml
+apiVersion: kyverno.io/v1
+kind: ClusterPolicy
+metadata:
+  name: validate
+spec:
+  validationFailureAction: enforce
+  rules:
+    - name: validate-replica-count
+      match:
+        any:
+        - resources:
+            kinds:
+            - Deployment
+      validate:
+        message: "Replica count for a Deployment must be greater than or equal to 2."
+        pattern:
+          spec:
+            replicas: ">=2"
+```
+
 ### Anchors
 
 Anchors allow conditional processing (i.e. "if-then-else") and other logical checks in validation patterns. The following types of anchors are supported:
@@ -239,7 +262,7 @@ Anchors allow conditional processing (i.e. "if-then-else") and other logical che
 | Conditional | ()  | If tag with the given value (including child elements) is specified, then peer elements will be processed. <br/>e.g. If image has tag latest then imagePullPolicy cannot be IfNotPresent. <br/>&nbsp;&nbsp;&nbsp;&nbsp;(image): "*:latest" <br>&nbsp;&nbsp;&nbsp;&nbsp;imagePullPolicy: "!IfNotPresent"<br/> |
 | Equality    | =() | If tag is specified, then processing continues. For tags with scalar values, the value must match. For tags with child elements, the child element is further evaluated as a validation pattern.  <br/>e.g. If hostPath is defined then the path cannot be /var/lib<br/>&nbsp;&nbsp;&nbsp;&nbsp;=(hostPath):<br/>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;path: "!/var/lib"<br/> |
 | Existence   | ^() | Works on the list/array type only. If at least one element in the list satisfies the pattern. In contrast, a conditional anchor would validate that all elements in the list match the pattern. <br/>e.g. At least one container with image nginx:latest must exist. <br/>&nbsp;&nbsp;&nbsp;&nbsp;^(containers):<br/>&nbsp;&nbsp;&nbsp;&nbsp;- image: nginx:latest<br/> |
-| Negation    | X() | The tag cannot be specified. The value of the tag is not evaluated (use exclamation point to negate a value). The value should ideally be set to `null`. <br/>e.g. Hostpath tag cannot be defined.<br/>&nbsp;&nbsp;&nbsp;&nbsp;X(hostPath):<br/> |
+| Negation    | X() | The tag cannot be specified. The value of the tag is not evaluated (use exclamation point to negate a value). The value should ideally be set to `"null"` (quotes surrounding null). <br/>e.g. Hostpath tag cannot be defined.<br/>&nbsp;&nbsp;&nbsp;&nbsp;X(hostPath): "null"<br/> |
 | Global      | <() | The content of this condition, if false, will cause the entire rule to be skipped. Valid for both validate and strategic merge patches. |
 
 #### Anchors and child elements: Conditional and Equality