From 52e0e292ed0cab569afc58b742505ad1ba8fddc9 Mon Sep 17 00:00:00 2001 From: Vincent Hou Date: Thu, 2 Apr 2020 14:00:52 -0400 Subject: [PATCH 01/19] Explain how to configure the ConfigMaps with operator CRs --- docs/install/knative-with-operators.md | 4 + .../operator/configuring-serving-cr.md | 90 +++++++++++++++++++ 2 files changed, 94 insertions(+) create mode 100644 docs/install/operator/configuring-serving-cr.md diff --git a/docs/install/knative-with-operators.md b/docs/install/knative-with-operators.md index 6334e3f4ff3..4c857c2f40f 100644 --- a/docs/install/knative-with-operators.md +++ b/docs/install/knative-with-operators.md @@ -260,3 +260,7 @@ for the source: ``` ko delete -f config/ ``` + +## What's next + +- [Configure Knative Serving using Operator](./operator/configuring-serving-cr.md) diff --git a/docs/install/operator/configuring-serving-cr.md b/docs/install/operator/configuring-serving-cr.md new file mode 100644 index 00000000000..2b305d8ae2c --- /dev/null +++ b/docs/install/operator/configuring-serving-cr.md @@ -0,0 +1,90 @@ +--- +title: "Configuring the Serving Operator Custom Resource" +weight: 10 +type: "docs" +aliases: +- /docs/operator/configuring-serving-cr/ +--- + +The Knative Serving operator can be configured with these options: + +- [All the ConfigMaps](#all-the-configmaps) + +__NOTE:__ Kubernetes spec level policies cannot be configured using the Knative operators. + +## All the ConfigMaps + +All the ConfigMaps can be configured in Knative Serving with the custom resource. The values in the custom resource will +overwrite the existing values in ConfigMaps. In the latest release of Knative Serving, there are multiple ConfigMaps, +e.g. `config-autoscaler`, `config-default`, `config-deployment`, etc. All the ConfigMaps are named with the prefix +`config-`. and in the format of `config-`. A field named `config` is defined under the section `spec` to specify +all the ConfigMaps. Under the section `spec.config`, use the name after the hyphen(`-`) sign, ``, as the field to +specify all the key-value pairs, which are exactly the same as we have in the section `data` for each ConfigMap. + +As in the example of how to [setup a custom domain](https://knative.dev/development/serving/using-a-custom-domain/), you can see the content of the ConfigMap +`config-domain` is: + +``` +apiVersion: v1 +kind: ConfigMap +metadata: + name: config-domain + namespace: knative-serving +data: + example.org: | + selector: + app: prod + example.com: "" +``` + +To specify the ConfigMap `config-domain`, you can change the content of the operator CR into: + +``` +apiVersion: operator.knative.dev/v1alpha1 +kind: KnativeServing +metadata: + name: knative-serving + namespace: knative-serving +spec: + config: + domain: + example.org: | + selector: + app: prod + example.com: "" +``` + +Next, save the CR into a file named `operator-cr.yaml`, and run the command: + +``` +kubectl apply -f operator-cr.yaml +``` + +If you want to change another ConfigMap, e.g. `config-autoscaler`, by specifying `stable-window` to `60s`. Continue to +change your operator CR into: + +``` +apiVersion: operator.knative.dev/v1alpha1 +kind: KnativeServing +metadata: + name: knative-serving + namespace: knative-serving +spec: + config: + domain: + example.org: | + selector: + app: prod + example.com: "" + autoscaler: + stable-window: "60s" +``` + +Then, save the content in a file named `operator-cr.yaml`, and run the command: + +``` +kubectl apply -f operator-cr.yaml +``` + +All the ConfigMaps should be defined under the same namespace as the operator CR. You can use the operator CR as the +unique entry point to edit all of them. From a698cbaa6ec96aedc3216a9df67bb14c4335702a Mon Sep 17 00:00:00 2001 From: Vincent Hou Date: Thu, 2 Apr 2020 14:19:08 -0400 Subject: [PATCH 02/19] Describe how to configure private repository and private secrets --- .../operator/configuring-serving-cr.md | 160 ++++++++++++++++-- 1 file changed, 150 insertions(+), 10 deletions(-) diff --git a/docs/install/operator/configuring-serving-cr.md b/docs/install/operator/configuring-serving-cr.md index 2b305d8ae2c..382ba9eddc0 100644 --- a/docs/install/operator/configuring-serving-cr.md +++ b/docs/install/operator/configuring-serving-cr.md @@ -1,14 +1,5 @@ ---- -title: "Configuring the Serving Operator Custom Resource" -weight: 10 -type: "docs" -aliases: -- /docs/operator/configuring-serving-cr/ ---- - -The Knative Serving operator can be configured with these options: - - [All the ConfigMaps](#all-the-configmaps) +- [Private repository and private secret](#private-repository-and-private-secrets) __NOTE:__ Kubernetes spec level policies cannot be configured using the Knative operators. @@ -88,3 +79,152 @@ kubectl apply -f operator-cr.yaml All the ConfigMaps should be defined under the same namespace as the operator CR. You can use the operator CR as the unique entry point to edit all of them. + +## Private repository and private secrets + +As in the latest release of Knative Serving, there are six `Deployment` resources: `activator`, `autoscaler`, `controller`, +`webhook`, `autoscaler-hpa` & `networking-istio`, under the apiVersion `apps/v1`, and one image: `queue-proxy`, under the +apiVersion `caching.internal.knative.dev/v1alpha1`. The images can be downloaded from the links specified in the `spec.image` +section for each of the resources. Knative Serving Operator provides us a way to download the images from private +repositories for Knative deployments and image(s). + +Under the section `spec` of the operator CR, you can create a section of `registry`, containing all the fields to define +the information about the private registry: + +- `default`: this field expects a string value, used to define image reference template for all Knative images. The format +is in `example-registry.io/custom/path/${NAME}:{CUSTOM-TAG}`. Since all your private images can be saved in the same +repository with the same tag, the only difference is the image name. `${NAME}` should be kept as it is, because this is +a pre-defined container variable in operator. If you name the images after the names of the `Deployment` resources: +`activator`, `autoscaler`, `controller`, `webhook`, `autoscaler-hpa` & `networking-istio`, for all `Deployment` resources, +and name the image after `queue-proxy`, for the `Image` resource, you do not need to do any further configuration in the +next section `override`, because operator can automatically replace `${NAME}` with the corresponding name of the `Deployment` +resource. + +- `override`: this field expects a map of key-value pairs, with container name or image name as the key, and the full image +location as the value. We usually need to configure this section, when we do not have a common format for all the image +links. This field is used alternatively with the previous field `default`. + +- `imagePullSecrets`: this field is used to define a list of secrets to be used when pulling the knative images. The secret +must be created in the same namespace as the Knative Serving deployments. You do not need to define any secret here if +your image is publicly available. Configuration of this field is equivalent to the configuration of [deploying images +from a private container registry](https://knative.dev/development/serving/deploying/private-registry/). + +This `registry` section is used to specify the links of the custom Knative images, and the appropriate credentials to access +them. We will use examples to illustrate how we define all the fields, regarding your custom image links and private secrets. + +### Download images in a predefined format without secrets: + +This example shows how you can define custom image links that can be defined in the CR using the simplified format +`docker.io/knative-images/${NAME}:{CUSTOM-TAG}`. + +In the example below: + +- the custom tag `v0.13.0` is used for all images +- all image links are accessible without using secrets +- images are defined in the accepted format `docker.io/knative-images/${NAME}:{CUSTOM-TAG}` + +First, you need to make sure your images are saved in the following link: + +- Image of `activator`: `docker.io/knative-images/activator:v0.13.0`. +- Image of `autoscaler`: `docker.io/knative-images/autoscaler:v0.13.0`. +- Image of `controller`: `docker.io/knative-images/controller:v0.13.0`. +- Image of `webhook`: `docker.io/knative-images/webhook:v0.13.0`. +- Image of `autoscaler-hpa`: `docker.io/knative-images/autoscaler-hpa:v0.13.0`. +- Image of `networking-istio`: `docker.io/knative-images/networking-istio:v0.13.0`. +- Cache image `queue-proxy`: `docker.io/knative-images/queue-proxy:v0.13.0`. + +Then, you need to define your operator CR with following content: + +``` +apiVersion: operator.knative.dev/v1alpha1 +kind: KnativeServing +metadata: + name: knative-serving + namespace: knative-serving +spec: + registry: + default: docker.io/knative-images/${NAME}:v0.13.0 +``` + +Replace `{CUSTOM-TAG}` with the custom tag `v0.13.0`. `${NAME}` needs to map the same name of each `Deployment` resource. +The field `default` is used to define the image format for all `Deployment` and `Image` resources, make sure you want to +replace the images for all `Deployment` and `Image` resources in Knative Serving with your own images, by specifying +the field `default`. + +### Download images individually without secrets: + +If your custom image links are not defined in a uniform format by default, you will need to individually include each +link in the CR. + +For example, to define the list of images: + +- Image of `activator`: `docker.io/knative-images-repo1/activator:v0.13.0`. +- Image of `autoscaler`: `docker.io/knative-images-repo2/autoscaler:v0.13.0`. +- Image of `controller`: `docker.io/knative-images-repo3/controller:v0.13.0`. +- Image of `webhook`: `docker.io/knative-images-repo4/webhook:v0.13.0`. +- Image of `autoscaler-hpa`: `docker.io/knative-images-repo5/autoscaler-hpa:v0.13.0`. +- Image of `networking-istio`: `docker.io/knative-images-repo6/prefix-networking-istio:v0.13.0`. +- Cache image `queue-proxy`: `docker.io/knative-images-repo7/queue-proxy-suffix:v0.13.0`. + +The operator CR should be modified to include the full list: + +``` +apiVersion: operator.knative.dev/v1alpha1 +kind: KnativeServing +metadata: + name: knative-serving + namespace: knative-serving +spec: + registry: + override: + activator: docker.io/knative-images-repo1/activator:v0.13.0 + autoscaler: docker.io/knative-images-repo2/autoscaler:v0.13.0 + controller: docker.io/knative-images-repo3/controller:v0.13.0 + webhook: docker.io/knative-images-repo4/webhook:v0.13.0 + autoscaler-hpa: docker.io/knative-images-repo5/autoscaler-hpa:v0.13.0 + networking-istio: docker.io/knative-images-repo6/prefix-networking-istio:v0.13.0 + queue-proxy: docker.io/knative-images-repo7/queue-proxy-suffix:v0.13.0 +``` + +### Download images with secrets: + +If you use the default or override attributes to define image links, and the image links require private secrets for +access, you must append the `imagePullSecrets` attribute. + +This example uses a secret named `regcred`. You must create your own private secrets if these are required: + +- [From existing docker credentials](https://kubernetes.io/docs/tasks/configure-pod-container/pull-image-private-registry/#registry-secret-existing-credentials) +- [From command line for docker credentials](https://kubernetes.io/docs/tasks/configure-pod-container/pull-image-private-registry/#create-a-secret-by-providing-credentials-on-the-command-line) +- [Create your own secret](https://kubernetes.io/docs/concepts/configuration/secret/#creating-your-own-secrets) + +After you create this secret, edit your operator CR by appending the content below ...: + +``` +apiVersion: operator.knative.dev/v1alpha1 +kind: KnativeServing +metadata: + name: knative-serving + namespace: knative-serving +spec: + registry: + ... + imagePullSecrets: + - name: regcred +``` + +The field `imagePullSecrets` expects a list of secrets. You can add multiple secrets to access the images as below: + +``` +apiVersion: operator.knative.dev/v1alpha1 +kind: KnativeServing +metadata: + name: knative-serving + namespace: knative-serving +spec: + registry: + ... + imagePullSecrets: + - name: regcred + - name: regcred-2 + ... +``` From a9a2a919cb9dee83fa42033296bb2d85af8fd83e Mon Sep 17 00:00:00 2001 From: Vincent Hou Date: Thu, 2 Apr 2020 14:21:05 -0400 Subject: [PATCH 03/19] Describe how to configure SSL certificate for controller --- .../operator/configuring-serving-cr.md | 43 +++++++++++++++++++ 1 file changed, 43 insertions(+) diff --git a/docs/install/operator/configuring-serving-cr.md b/docs/install/operator/configuring-serving-cr.md index 382ba9eddc0..b23070f4c24 100644 --- a/docs/install/operator/configuring-serving-cr.md +++ b/docs/install/operator/configuring-serving-cr.md @@ -1,5 +1,16 @@ +--- +title: "Configuring the Serving Operator Custom Resource" +weight: 10 +type: "docs" +aliases: +- /docs/operator/configuring-serving-cr/ +--- + +The Knative Serving operator can be configured with these options: + - [All the ConfigMaps](#all-the-configmaps) - [Private repository and private secret](#private-repository-and-private-secrets) +- [SSL certificate for controller](#ssl-certificate-for-controller) __NOTE:__ Kubernetes spec level policies cannot be configured using the Knative operators. @@ -228,3 +239,35 @@ spec: - name: regcred-2 ... ``` + +## SSL certificate for controller + +Knative Serving needs to access the container registry, based on the feature [enabling tag to digest resolution](https://knative.dev/development/serving/tag-resolution/). The +Serving Operator CR allows you to specify either a custom ConfigMap or a Secret as a self-signed certificate for the +deployment called `controller`. It enables the `controller` to trust registries with self-signed certificates. + +Under the section `spec` of the operator CR, you can create a section of `controller-custom-certs` to contain all the +fields to define the certificate: + +- `name`: this field is used to specify the name of the ConfigMap or the Secret. +- `type`: the value for this field can be either ConfigMap or Secret, indicating the type for the name. + +This section `controller-custom-certs` is used to access the user's images in private repositories with the appropriate +certificate. + +If you create a configMap named `testCertas` as the certificate, you need to change your CR into: + +``` +apiVersion: operator.knative.dev/v1alpha1 +kind: KnativeServing +metadata: + name: knative-serving + namespace: knative-serving +spec: + controller-custom-certs: + name: testCert + type: ConfigMap +``` + +It will make sure this custom certificate is mounted as a volume to the containers launched by the `Deployment` resource +`controller`, and the environment variable is `SSL_CERT_DIR` set correctly. From c2d765bc3f6b3c0a1257e6ac1b225ff45673cb4d Mon Sep 17 00:00:00 2001 From: Vincent Hou Date: Thu, 2 Apr 2020 14:22:17 -0400 Subject: [PATCH 04/19] Describe how to configure ingress gateway --- .../operator/configuring-serving-cr.md | 48 +++++++++++++++++++ 1 file changed, 48 insertions(+) diff --git a/docs/install/operator/configuring-serving-cr.md b/docs/install/operator/configuring-serving-cr.md index b23070f4c24..2f8ec0f98cf 100644 --- a/docs/install/operator/configuring-serving-cr.md +++ b/docs/install/operator/configuring-serving-cr.md @@ -11,6 +11,7 @@ The Knative Serving operator can be configured with these options: - [All the ConfigMaps](#all-the-configmaps) - [Private repository and private secret](#private-repository-and-private-secrets) - [SSL certificate for controller](#ssl-certificate-for-controller) +- [Knative ingress gateway](#configuration-of-knative-ingress-gateway) __NOTE:__ Kubernetes spec level policies cannot be configured using the Knative operators. @@ -271,3 +272,50 @@ spec: It will make sure this custom certificate is mounted as a volume to the containers launched by the `Deployment` resource `controller`, and the environment variable is `SSL_CERT_DIR` set correctly. + +## Configuration of Knative ingress gateway + +To set up custom ingress gateway, follow “**Step 1: Create Gateway Service and Deployment Instance**” [here](https://knative.dev/development/serving/setting-up-custom-ingress-gateway/). + +**Step 2: Update the Knative gateway** + +We use the field `knative-ingress-gateway` to override the knative-ingress-gateway. We only support the field `selector` +to define the selector for ingress-gateway. + +Instead of updating the gateway directly, we modify the operator CR as below: + +``` +apiVersion: operator.knative.dev/v1alpha1 +kind: KnativeServing +metadata: + name: knative-serving + namespace: knative-serving +spec: + registry: + knative-ingress-gateway: + selector: + custom: ingressgateway +``` + +**Step 3: Update Gateway ConfigMap** + +As we explained, all ConfigMaps can be edited as editing the the operator CR. For this example, take the following content +as your operator CR: + +``` +apiVersion: operator.knative.dev/v1alpha1 +kind: KnativeServing +metadata: + name: knative-serving + namespace: knative-serving +spec: + registry: + knative-ingress-gateway: + selector: + custom: ingressgateway + config: + istio: + gateway.knative-serving.knative-ingress-gateway: "custom-ingressgateway.istio-system.svc.cluster.local" +``` + +The key in `spec.config.istio` is in the format of `gateway.{{gateway_namespace}}.{{gateway_name}}`. From f03bb59ff38b599f9708d2a1cd4bd8a73c106cb1 Mon Sep 17 00:00:00 2001 From: Vincent Hou Date: Thu, 2 Apr 2020 14:23:38 -0400 Subject: [PATCH 05/19] Describe how to configure the local gateway --- .../operator/configuring-serving-cr.md | 55 +++++++++++++++++++ 1 file changed, 55 insertions(+) diff --git a/docs/install/operator/configuring-serving-cr.md b/docs/install/operator/configuring-serving-cr.md index 2f8ec0f98cf..dce949c2229 100644 --- a/docs/install/operator/configuring-serving-cr.md +++ b/docs/install/operator/configuring-serving-cr.md @@ -12,6 +12,7 @@ The Knative Serving operator can be configured with these options: - [Private repository and private secret](#private-repository-and-private-secrets) - [SSL certificate for controller](#ssl-certificate-for-controller) - [Knative ingress gateway](#configuration-of-knative-ingress-gateway) +- [Cluster local gateway](#configuration-of-cluster-local-gateway) __NOTE:__ Kubernetes spec level policies cannot be configured using the Knative operators. @@ -319,3 +320,57 @@ spec: ``` The key in `spec.config.istio` is in the format of `gateway.{{gateway_namespace}}.{{gateway_name}}`. + +## Configuration of cluster local gateway: + +We use the field `cluster-local-gateway` to override the the gateway cluster-local-gateway. We only support the field +`selector` to define the selector for the local gateway. + +**Default local gateway name**: + +Go through the guide [here](https://knative.dev/development/install/installing-istio/#updating-your-install-to-use-cluster-local-gateway) to use local cluster gateway. + +After following the above step, your service and deployment for the local gateway are both named `cluster-local-gateway`. +You only need to configure the operator CR as below: + +``` +apiVersion: operator.knative.dev/v1alpha1 +kind: KnativeServing +metadata: + name: knative-serving + namespace: knative-serving +spec: + registry: + cluster-local-gateway: + selector: + istio: cluster-local-gateway +``` + +You can even skip the above change, since there is a gateway called `cluster-local-gateway`, which has +`istio: cluster-local-gateway` as the default selector. If the operator CR does not define the section +cluster-local-gateway, the default `istio: cluster-local-gateway` of the gateway cluster-local-gateway will be chosen. + +**Non-default local gateway name**: + +If you create custom service and deployment for local gateway with a name other than `cluster-local-gateway`, you need +to update gateway configmap `config-istio` under the Knative Serving namespace, and change the selector for the gateway +cluster-local-gateway. + +If you name both of the service and deployment after `custom-local-gateway` in the namespace `istio-system`, with the +label `custom: custom-local-gateway`, the operator CR should be like: + +``` +apiVersion: operator.knative.dev/v1alpha1 +kind: KnativeServing +metadata: + name: knative-serving + namespace: knative-serving +spec: + registry: + cluster-local-gateway: + selector: + custom: custom-local-gateway + config: + istio: + local-gateway.knative-serving.cluster-local-gateway: "custom-local-gateway.istio-system.svc.cluster.local" +``` From 1c2914cf2cf3677001b2c938301bf2427a93c62e Mon Sep 17 00:00:00 2001 From: Vincent Hou Date: Thu, 2 Apr 2020 14:24:10 -0400 Subject: [PATCH 06/19] Describe how to configure HA --- .../operator/configuring-serving-cr.md | 21 +++++++++++++++++++ 1 file changed, 21 insertions(+) diff --git a/docs/install/operator/configuring-serving-cr.md b/docs/install/operator/configuring-serving-cr.md index dce949c2229..8ce8159b1a6 100644 --- a/docs/install/operator/configuring-serving-cr.md +++ b/docs/install/operator/configuring-serving-cr.md @@ -13,6 +13,7 @@ The Knative Serving operator can be configured with these options: - [SSL certificate for controller](#ssl-certificate-for-controller) - [Knative ingress gateway](#configuration-of-knative-ingress-gateway) - [Cluster local gateway](#configuration-of-cluster-local-gateway) +- [High availability](#high-availability) __NOTE:__ Kubernetes spec level policies cannot be configured using the Knative operators. @@ -374,3 +375,23 @@ spec: istio: local-gateway.knative-serving.cluster-local-gateway: "custom-local-gateway.istio-system.svc.cluster.local" ``` + +## High availability: + +Knative Serving Operator CR provides the capability to configure the high availability for the following `Deployment` +resources: `controller`, `autoscaler-hpa` and `networking-istio`. A field named `high-availability` is defined under the +section `spec` to configure the high availability control plane. In the latest release of Knative Serving Operator, you can +specify the number of replicas that HA parts of the control plane will be scaled to. + +If you want to specify 3 as the number of replicas, that your `Deployment` resources can scale up to, change your CR into: + +``` +apiVersion: operator.knative.dev/v1alpha1 +kind: KnativeServing +metadata: + name: knative-serving + namespace: knative-serving +spec: + high-availability: + replicas: 3 +``` From 469d957e212c826dcdc19ab0408b60ce5819140a Mon Sep 17 00:00:00 2001 From: Vincent Hou Date: Thu, 2 Apr 2020 14:25:18 -0400 Subject: [PATCH 07/19] Add instruction on eventing operator CR --- docs/install/knative-with-operators.md | 1 + .../operator/configuring-eventing-cr.md | 120 ++++++++++++++++++ 2 files changed, 121 insertions(+) create mode 100644 docs/install/operator/configuring-eventing-cr.md diff --git a/docs/install/knative-with-operators.md b/docs/install/knative-with-operators.md index 4c857c2f40f..dfb16ac1d1d 100644 --- a/docs/install/knative-with-operators.md +++ b/docs/install/knative-with-operators.md @@ -264,3 +264,4 @@ ko delete -f config/ ## What's next - [Configure Knative Serving using Operator](./operator/configuring-serving-cr.md) +- [Configure Knative Eventing using Operator](./operator/configuring-eventing-cr.md) diff --git a/docs/install/operator/configuring-eventing-cr.md b/docs/install/operator/configuring-eventing-cr.md new file mode 100644 index 00000000000..b4d62a821b2 --- /dev/null +++ b/docs/install/operator/configuring-eventing-cr.md @@ -0,0 +1,120 @@ +--- +title: "Configuring the Eventing Operator Custom Resource" +weight: 10 +type: "docs" +aliases: +- /docs/operator/configuring-eventing-cr/ +--- + +Knative Eventing can only be configured using a private repository and private secret. + +The Knative Eventing operator CR is configured similarly to the Knative Serving operator CR. For more information, +see the documentation on “[Private repository and private secret](configuring-serving-cr.md#private-repository-and-private-secrets)” in Serving operator for detailed instruction. +The difference is that Knative Eventing Operator only allows us to customize the images for all `deployment` resources: +`eventing-controller`, `eventing-webhook`, `imc-controller`, `imc-dispatcher`, and `broker-controller`. You need to +either use these names as your names of the images in the repository, or to map your image links on a one-on-one basis. + +## Download images in a predefined format without secrets: + +This example shows how you can define custom image links that can be defined in the CR using the simplified format +`docker.io/knative-images/${NAME}:{CUSTOM-TAG}`. + +In the example below: + +- the custom tag `v0.13.0` is used for all images +- all image links are accessible without using secrets +- images are defined in the accepted format `docker.io/knative-images/${NAME}:{CUSTOM-TAG}` + +First, you need to make sure your images are saved in the following link: + +- Image of `eventing-controller`: `docker.io/knative-images/eventing-controller:v0.13.0`. +- Image of `eventing-webhook`: `docker.io/knative-images/eventing-webhook:v0.13.0`. +- Image of `imc-controller`: `docker.io/knative-images/imc-controller:v0.13.0`. +- Image of `imc-dispatcher`: `docker.io/knative-images/imc-dispatcher:v0.13.0`. +- Image of `broker-controller`: `docker.io/knative-images/broker-controller:v0.13.0`. + +Then, you need to define your operator CR with following content: + +``` +apiVersion: operator.knative.dev/v1alpha1 +kind: KnativeEventing +metadata: + name: knative-eventing + namespace: knative-eventing +spec: + registry: + default: docker.io/knative-images/${NAME}:v0.13.0 +``` + +Replace `{CUSTOM-TAG}` with the custom tag `v0.13.0`. `${NAME}` needs to map the same name of each `Deployment` resource. +The field `default` is used to define the image format for all `Deployment` resources, make sure you want to replace the +images for all `Deployment` resources in Knative Eventing with your own images, by specifying this field `default`. + +## Download images from different repositories without secrets: + +If your custom image links are not defined in a uniform format by default, you will need to individually include each +link in the CR. + +For example, to define the list of images: + +- Image of `eventing-controller`: `docker.io/knative-images-repo1/eventing-controller:v0.13.0`. +- Image of `eventing-webhook`: `docker.io/knative-images-repo2/eventing-webhook:v0.13.0`. +- Image of `imc-controller`: `docker.io/knative-images-repo3/imc-controller:v0.13.0`. +- Image of `imc-dispatcher`: `docker.io/knative-images-repo4/imc-dispatcher:v0.13.0`. +- Image of `broker-controller`: `docker.io/knative-images-repo5/broker-controller:v0.13.0`. + +The operator CR should be modified to include the full list: + +``` +apiVersion: operator.knative.dev/v1alpha1 +kind: KnativeServing +metadata: + name: knative-serving + namespace: knative-serving +spec: + registry: + override: + eventing-controller: docker.io/knative-images-repo1/eventing-controller:v0.13.0 + eventing-webhook: docker.io/knative-images-repo2/eventing-webhook:v0.13.0 + imc-controller: docker.io/knative-images-repo3/imc-controller:v0.13.0 + imc-dispatcher: docker.io/knative-images-repo4/imc-dispatcher:v0.13.0 + broker-controller: docker.io/knative-images-repo5/broker-controller:v0.13.0 +``` + +## Download images with secrets: + +If you use the default or override attributes to define image links, and the image links require private secrets for +access, you must append the `imagePullSecrets` attribute. + +This example uses a secret named `regcred`. You must create your own private secrets if these are required. After you +create this secret, edit your operator CR by appending the content below ...: + +``` +apiVersion: operator.knative.dev/v1alpha1 +kind: KnativeEventing +metadata: + name: knative-eventing + namespace: knative-eventing +spec: + registry: + ... + imagePullSecrets: + - name: regcred +``` + +The field `imagePullSecrets` expects a list of secrets. You can add multiple secrets to access the images as below: + +``` +apiVersion: operator.knative.dev/v1alpha1 +kind: KnativeEventing +metadata: + name: knative-eventing + namespace: knative-eventing +spec: + registry: + ... + imagePullSecrets: + - name: regcred + - name: regcred-2 + ... +``` From 1bcb7715cf603c775fad53e37850f6b41b7b55b3 Mon Sep 17 00:00:00 2001 From: Vincent Hou Date: Fri, 3 Apr 2020 16:07:19 -0400 Subject: [PATCH 08/19] Change the description on `default` and `override` fields --- .../operator/configuring-serving-cr.md | 19 +++++++++++-------- 1 file changed, 11 insertions(+), 8 deletions(-) diff --git a/docs/install/operator/configuring-serving-cr.md b/docs/install/operator/configuring-serving-cr.md index 8ce8159b1a6..9f84610939f 100644 --- a/docs/install/operator/configuring-serving-cr.md +++ b/docs/install/operator/configuring-serving-cr.md @@ -108,15 +108,18 @@ the information about the private registry: - `default`: this field expects a string value, used to define image reference template for all Knative images. The format is in `example-registry.io/custom/path/${NAME}:{CUSTOM-TAG}`. Since all your private images can be saved in the same repository with the same tag, the only difference is the image name. `${NAME}` should be kept as it is, because this is -a pre-defined container variable in operator. If you name the images after the names of the `Deployment` resources: +a pre-defined container variable in operator. If you name the images after the container names within all `Deployment` resources: `activator`, `autoscaler`, `controller`, `webhook`, `autoscaler-hpa` & `networking-istio`, for all `Deployment` resources, and name the image after `queue-proxy`, for the `Image` resource, you do not need to do any further configuration in the -next section `override`, because operator can automatically replace `${NAME}` with the corresponding name of the `Deployment` -resource. +next section `override`, because operator can automatically replace `${NAME}` with the corresponding container name. For +serving-operator, there is only one container defined in each `Deployment` resource. The container shares the same as +its parent deployment. - `override`: this field expects a map of key-value pairs, with container name or image name as the key, and the full image location as the value. We usually need to configure this section, when we do not have a common format for all the image -links. This field is used alternatively with the previous field `default`. +links. This field is used alternatively with the previous field `default`. If the image for a certain container or `Image` +resource is specified in both this field and the `default` field, this `override` field takes precedence. This field can +also be used as the supplement of the `default` field, if the image link can not match the predefined format. - `imagePullSecrets`: this field is used to define a list of secrets to be used when pulling the knative images. The secret must be created in the same namespace as the Knative Serving deployments. You do not need to define any secret here if @@ -160,10 +163,10 @@ spec: default: docker.io/knative-images/${NAME}:v0.13.0 ``` -Replace `{CUSTOM-TAG}` with the custom tag `v0.13.0`. `${NAME}` needs to map the same name of each `Deployment` resource. -The field `default` is used to define the image format for all `Deployment` and `Image` resources, make sure you want to -replace the images for all `Deployment` and `Image` resources in Knative Serving with your own images, by specifying -the field `default`. +Replace `{CUSTOM-TAG}` with the custom tag `v0.13.0`. `${NAME}` needs to map the same name of each container or each +`Image` resource. The field `default` is used to define the image format for all containers and the `Image` resource. +Make sure you want to replace the images for all containers and `Image` resources in Knative Serving with your own +images, by specifying the field `default`. ### Download images individually without secrets: From fce20f7ecf662f9c7992e0337af5b56953385440 Mon Sep 17 00:00:00 2001 From: Vincent Hou Date: Fri, 3 Apr 2020 16:49:49 -0400 Subject: [PATCH 09/19] Change the instruction on image overriding for Knative Eventing operator --- .../operator/configuring-eventing-cr.md | 102 ++++++++++++++---- 1 file changed, 79 insertions(+), 23 deletions(-) diff --git a/docs/install/operator/configuring-eventing-cr.md b/docs/install/operator/configuring-eventing-cr.md index b4d62a821b2..4296ae5112c 100644 --- a/docs/install/operator/configuring-eventing-cr.md +++ b/docs/install/operator/configuring-eventing-cr.md @@ -6,15 +6,35 @@ aliases: - /docs/operator/configuring-eventing-cr/ --- -Knative Eventing can only be configured using a private repository and private secret. +The Knative Eventing operator can be configured with these options: -The Knative Eventing operator CR is configured similarly to the Knative Serving operator CR. For more information, +- [Private repository and private secret](#private-repository-and-private-secrets) + +__NOTE:__ Kubernetes spec level policies cannot be configured using the Knative operators. + +## Private repository and private secrets + +The Knative Eventing operator CR is configured the same way as the Knative Serving operator CR. For more information, see the documentation on “[Private repository and private secret](configuring-serving-cr.md#private-repository-and-private-secrets)” in Serving operator for detailed instruction. -The difference is that Knative Eventing Operator only allows us to customize the images for all `deployment` resources: -`eventing-controller`, `eventing-webhook`, `imc-controller`, `imc-dispatcher`, and `broker-controller`. You need to -either use these names as your names of the images in the repository, or to map your image links on a one-on-one basis. -## Download images in a predefined format without secrets: +Knative Eventing also specifies only one container, within one `Deployment` resource. However, the container does not use +the same name as its parent `Deployment`, which means the container name in Knative Eventing is not the unique identifier +as in Knative Serving. Here is the list of containers within each `Deployment` resource: + +- Container in Deployment `eventing-controller`: eventing-controller +- Container in Deployment `eventing-webhook`: eventing-webhook +- Container in Deployment `broker-controller`: eventing-controller +- Container in Deployment `imc-controller`: controller +- Container in Deployment `imc-dispatcher`: dispatcher + +The `default` field can still be used to replace the images in a predefined format. However, if the container name is not +a unique identifier, e.g. `eventing-controller`, you need to use the `override` field to replace it, by specifying +`deployment/container` as the unique key. + +Some images are defined via environment variable in Knative Eventing. They can be replaced by taking advantage of the +`override` field. + +## Download images in predefined format without secrets: This example shows how you can define custom image links that can be defined in the CR using the simplified format `docker.io/knative-images/${NAME}:{CUSTOM-TAG}`. @@ -27,11 +47,11 @@ In the example below: First, you need to make sure your images are saved in the following link: -- Image of `eventing-controller`: `docker.io/knative-images/eventing-controller:v0.13.0`. +- Image of `eventing-controller` in the deployment `eventing-controller`: `docker.io/knative-images/eventing-controller:v0.13.0`. - Image of `eventing-webhook`: `docker.io/knative-images/eventing-webhook:v0.13.0`. -- Image of `imc-controller`: `docker.io/knative-images/imc-controller:v0.13.0`. -- Image of `imc-dispatcher`: `docker.io/knative-images/imc-dispatcher:v0.13.0`. -- Image of `broker-controller`: `docker.io/knative-images/broker-controller:v0.13.0`. +- Image of `controller`: `docker.io/knative-images/controller:v0.13.0`. +- Image of `dispatcher`: `docker.io/knative-images/dispatcher:v0.13.0`. +- Image of `eventing-controller` in the deployment `broker-controller`: `docker.io/knative-images/broker-eventing-controller:v0.13.0`. Then, you need to define your operator CR with following content: @@ -46,9 +66,23 @@ spec: default: docker.io/knative-images/${NAME}:v0.13.0 ``` -Replace `{CUSTOM-TAG}` with the custom tag `v0.13.0`. `${NAME}` needs to map the same name of each `Deployment` resource. -The field `default` is used to define the image format for all `Deployment` resources, make sure you want to replace the -images for all `Deployment` resources in Knative Eventing with your own images, by specifying this field `default`. +Replace `{CUSTOM-TAG}` with the custom tag `v0.13.0`. `${NAME}` needs to map the container name in each `Deployment` resource. +The field `default` is used to define the image format for all containers, except the container `eventing-controller` in +the deployment `broker-controller`. To replace the image for this container, you need to take advatage of the `override` +field to specify individually, by using `broker-controller/eventing-controller` as the key`. Change your CR into the following: + +``` +apiVersion: operator.knative.dev/v1alpha1 +kind: KnativeEventing +metadata: + name: knative-eventing + namespace: knative-eventing +spec: + registry: + default: docker.io/knative-images/${NAME}:v0.13.0 + override: + broker-controller/eventing-controller: docker.io/knative-images-repo1/broker-eventing-controller:v0.13.0 +``` ## Download images from different repositories without secrets: @@ -57,11 +91,12 @@ link in the CR. For example, to define the list of images: -- Image of `eventing-controller`: `docker.io/knative-images-repo1/eventing-controller:v0.13.0`. -- Image of `eventing-webhook`: `docker.io/knative-images-repo2/eventing-webhook:v0.13.0`. -- Image of `imc-controller`: `docker.io/knative-images-repo3/imc-controller:v0.13.0`. -- Image of `imc-dispatcher`: `docker.io/knative-images-repo4/imc-dispatcher:v0.13.0`. -- Image of `broker-controller`: `docker.io/knative-images-repo5/broker-controller:v0.13.0`. +- Image of `eventing-controller` in the deployment `eventing-controller`: `docker.io/knative-images/eventing-controller:v0.13.0`. +- Image of `eventing-webhook`: `docker.io/knative-images/eventing-webhook:v0.13.0`. +- Image of `controller`: `docker.io/knative-images/controller:v0.13.0`. +- Image of `dispatcher`: `docker.io/knative-images/dispatcher:v0.13.0`. +- Image of `eventing-controller` in the deployment `broker-controller`: `docker.io/knative-images/broker-eventing-controller:v0.13.0`. + The operator CR should be modified to include the full list: @@ -74,11 +109,32 @@ metadata: spec: registry: override: - eventing-controller: docker.io/knative-images-repo1/eventing-controller:v0.13.0 - eventing-webhook: docker.io/knative-images-repo2/eventing-webhook:v0.13.0 - imc-controller: docker.io/knative-images-repo3/imc-controller:v0.13.0 - imc-dispatcher: docker.io/knative-images-repo4/imc-dispatcher:v0.13.0 - broker-controller: docker.io/knative-images-repo5/broker-controller:v0.13.0 + eventing-controller/eventing-controller: docker.io/knative-images-repo1/eventing-controller:v0.13.0 + eventing-webhook/eventing-webhook: docker.io/knative-images-repo2/eventing-webhook:v0.13.0 + imc-controller/controller: docker.io/knative-images-repo3/imc-controller:v0.13.0 + imc-dispatcher/dispatcher: docker.io/knative-images-repo4/imc-dispatcher:v0.13.0 + broker-controller/eventing-controller: docker.io/knative-images-repo5/broker-eventing-controller:v0.13.0 +``` + +If you would like to replace the image defined by environment variable, e.g. the envorinment variable `DISPATCHER_IMAGE` +in the container `controller` of the deployment `imc-controller`, you need to adjust your CR into the following, if the +target image is `docker.io/knative-images-repo5/DISPATCHER_IMAGE:v0.13.0`: + +``` +apiVersion: operator.knative.dev/v1alpha1 +kind: KnativeServing +metadata: + name: knative-serving + namespace: knative-serving +spec: + registry: + override: + eventing-controller/eventing-controller: docker.io/knative-images-repo1/eventing-controller:v0.13.0 + eventing-webhook/eventing-webhook: docker.io/knative-images-repo2/eventing-webhook:v0.13.0 + imc-controller/controller: docker.io/knative-images-repo3/imc-controller:v0.13.0 + imc-dispatcher/dispatcher: docker.io/knative-images-repo4/imc-dispatcher:v0.13.0 + broker-controller/eventing-controller: docker.io/knative-images-repo5/broker-eventing-controller:v0.13.0 + DISPATCHER_IMAGE: docker.io/knative-images-repo5/DISPATCHER_IMAGE:v0.13.0 ``` ## Download images with secrets: From c55107ea3dbe258948eb6e49653c66fee9b2f68a Mon Sep 17 00:00:00 2001 From: Vincent Hou Date: Thu, 9 Apr 2020 16:47:03 -0400 Subject: [PATCH 10/19] Describe how to manage the container resources --- .../operator/configuring-serving-cr.md | 87 +++++++++++++++++++ 1 file changed, 87 insertions(+) diff --git a/docs/install/operator/configuring-serving-cr.md b/docs/install/operator/configuring-serving-cr.md index 9f84610939f..e1188762425 100644 --- a/docs/install/operator/configuring-serving-cr.md +++ b/docs/install/operator/configuring-serving-cr.md @@ -14,6 +14,7 @@ The Knative Serving operator can be configured with these options: - [Knative ingress gateway](#configuration-of-knative-ingress-gateway) - [Cluster local gateway](#configuration-of-cluster-local-gateway) - [High availability](#high-availability) +- [Managing Resources for Containers](#managing-resources-for-containers) __NOTE:__ Kubernetes spec level policies cannot be configured using the Knative operators. @@ -398,3 +399,89 @@ spec: high-availability: replicas: 3 ``` + +## Managing resources for containers + +Knative Serving Operator CR allows you to configure the resources for the containers defined in the `Deployment` resource. +Each `Deployment` resource has only one container with the same deployment name. The following containers can be configured +in terms of the required and maximum resources to be allocated: `activator`, `autoscaler`, `controller`, `webhook`, `autoscaler-hpa`, +`networking-istio` and `queue-proxy`. The field called `resources` holds the stanza of a list of containers for resource +configuration. The field `require` defines how much resource the container requires. The field `limit` defines the maximum +resource the container can use. Within each field, each container can be configured individually with the following +parameters: `memory`, `cpu`, `storage` and `ephemeral-storage`. The value in `require` should always be smaller than the +corresponding value in `limit`. The field `container` is used to specify the container name. + +Suppose you would like to configure the container called `activator` with the required resources: + +``` +cpu: 30m +memory: 40Mi +storage: 1Gi +ephemeral-storage: 2Gi +``` + +, and the limit resources: + +``` +cpu: 300m +memory: 4000Mi +storage: 2Gi +ephemeral-storage: 4Gi +``` + +Here is how you should write your CR: + +``` +apiVersion: operator.knative.dev/v1alpha1 +kind: KnativeServing +metadata: + name: knative-serving + namespace: knative-serving +spec: + resources: + - container: activator + requests: + cpu: 30m + memory: 40Mi + storage: 1Gi + ephemeral-storage: 2Gi + limits: + cpu: 300m + memory: 400Mi + storage: 2Gi + ephemeral-storage: 4Gi +``` + +If you would like to add another container `autoscaler` with the same configuration, you need to change your CR as below: + +``` +apiVersion: operator.knative.dev/v1alpha1 +kind: KnativeServing +metadata: + name: knative-serving + namespace: knative-serving +spec: + resources: + - container: activator + requests: + cpu: 30m + memory: 40Mi + storage: 1Gi + ephemeral-storage: 2Gi + limits: + cpu: 300m + memory: 400Mi + storage: 2Gi + ephemeral-storage: 4Gi + - container: autoscaler + requests: + cpu: 30m + memory: 40Mi + storage: 1Gi + ephemeral-storage: 2Gi + limits: + cpu: 300m + memory: 400Mi + storage: 2Gi + ephemeral-storage: 4Gi +``` From cacb5f976c333f37b6b225da9abcc0875971f26b Mon Sep 17 00:00:00 2001 From: Vincent Hou Date: Thu, 16 Apr 2020 13:42:53 -0400 Subject: [PATCH 11/19] Add the description on defaultBrokerClass --- .../operator/configuring-eventing-cr.md | 24 ++++++++++++++++--- 1 file changed, 21 insertions(+), 3 deletions(-) diff --git a/docs/install/operator/configuring-eventing-cr.md b/docs/install/operator/configuring-eventing-cr.md index 4296ae5112c..8a0d7d5ce80 100644 --- a/docs/install/operator/configuring-eventing-cr.md +++ b/docs/install/operator/configuring-eventing-cr.md @@ -9,6 +9,7 @@ aliases: The Knative Eventing operator can be configured with these options: - [Private repository and private secret](#private-repository-and-private-secrets) +- [Configuring default broker class](#configuring-default-broker-class) __NOTE:__ Kubernetes spec level policies cannot be configured using the Knative operators. @@ -34,7 +35,7 @@ a unique identifier, e.g. `eventing-controller`, you need to use the `override` Some images are defined via environment variable in Knative Eventing. They can be replaced by taking advantage of the `override` field. -## Download images in predefined format without secrets: +### Download images in predefined format without secrets: This example shows how you can define custom image links that can be defined in the CR using the simplified format `docker.io/knative-images/${NAME}:{CUSTOM-TAG}`. @@ -84,7 +85,7 @@ spec: broker-controller/eventing-controller: docker.io/knative-images-repo1/broker-eventing-controller:v0.13.0 ``` -## Download images from different repositories without secrets: +### Download images from different repositories without secrets: If your custom image links are not defined in a uniform format by default, you will need to individually include each link in the CR. @@ -137,7 +138,7 @@ spec: DISPATCHER_IMAGE: docker.io/knative-images-repo5/DISPATCHER_IMAGE:v0.13.0 ``` -## Download images with secrets: +### Download images with secrets: If you use the default or override attributes to define image links, and the image links require private secrets for access, you must append the `imagePullSecrets` attribute. @@ -174,3 +175,20 @@ spec: - name: regcred-2 ... ``` + +## Configuring default broker class + +Knative Eventing support two types of default broker classes: `ChannelBasedBroker` and `MTChannelBasedBroker`. It is can +be specified with the field called `defaultBrokerClass`. If this field is left empty, `ChannelBasedBroker` will be taken +as the broker class. If we would like to specify the default broker class to `MTChannelBasedBroker`, the Eventing +Operator CR should be: + +``` +apiVersion: operator.knative.dev/v1alpha1 +kind: KnativeEventing +metadata: + name: knative-eventing + namespace: knative-eventing +spec: + defaultBrokerClass: MTChannelBasedBroker +``` From 78b477272e2de0b74ffe1fa007ac9c8b4a113038 Mon Sep 17 00:00:00 2001 From: Vincent Hou Date: Thu, 16 Apr 2020 13:50:08 -0400 Subject: [PATCH 12/19] Add HA configuration to HorizontalPodAutoscaler --- docs/install/operator/configuring-serving-cr.md | 10 ++++++---- 1 file changed, 6 insertions(+), 4 deletions(-) diff --git a/docs/install/operator/configuring-serving-cr.md b/docs/install/operator/configuring-serving-cr.md index e1188762425..89cf870e8db 100644 --- a/docs/install/operator/configuring-serving-cr.md +++ b/docs/install/operator/configuring-serving-cr.md @@ -383,11 +383,13 @@ spec: ## High availability: Knative Serving Operator CR provides the capability to configure the high availability for the following `Deployment` -resources: `controller`, `autoscaler-hpa` and `networking-istio`. A field named `high-availability` is defined under the -section `spec` to configure the high availability control plane. In the latest release of Knative Serving Operator, you can -specify the number of replicas that HA parts of the control plane will be scaled to. +resources: `controller`, `autoscaler-hpa` & `networking-istio`, and all the `HorizontalPodAutoscaler` resources. A field +named `high-availability` is defined under the section `spec` to configure the high availability control plane. In the +latest release of Knative Serving Operator, you can specify the minimum number of replicas for the `Deployment` and +`HorizontalPodAutoscaler` resources. -If you want to specify 3 as the number of replicas, that your `Deployment` resources can scale up to, change your CR into: +If you want to specify 3 as the minimum number of replicas for three of the `Deployment` resources and the `HorizontalPodAutoscaler` +resources, change your CR into: ``` apiVersion: operator.knative.dev/v1alpha1 From de9c2784b43864dc89687515a3cdb7ad3983924b Mon Sep 17 00:00:00 2001 From: Vincent Date: Fri, 17 Apr 2020 15:18:38 -0400 Subject: [PATCH 13/19] Update docs/install/operator/configuring-eventing-cr.md Co-Authored-By: Evan Anderson Update docs/install/operator/configuring-eventing-cr.md Co-Authored-By: Evan Anderson Update docs/install/operator/configuring-eventing-cr.md Co-Authored-By: Evan Anderson Update docs/install/operator/configuring-serving-cr.md Co-Authored-By: Evan Anderson Update docs/install/operator/configuring-serving-cr.md Co-Authored-By: Evan Anderson Update docs/install/operator/configuring-serving-cr.md Co-Authored-By: Evan Anderson Update docs/install/operator/configuring-serving-cr.md Co-Authored-By: Evan Anderson Update docs/install/operator/configuring-eventing-cr.md Co-Authored-By: Evan Anderson Update docs/install/operator/configuring-eventing-cr.md Co-Authored-By: Evan Anderson Update docs/install/operator/configuring-eventing-cr.md Co-Authored-By: Evan Anderson Update docs/install/operator/configuring-eventing-cr.md Co-Authored-By: Evan Anderson Update docs/install/operator/configuring-serving-cr.md Co-Authored-By: Evan Anderson Update docs/install/operator/configuring-serving-cr.md Co-Authored-By: Evan Anderson Update docs/install/operator/configuring-serving-cr.md Co-Authored-By: Evan Anderson Update docs/install/operator/configuring-serving-cr.md Co-Authored-By: Evan Anderson Update docs/install/operator/configuring-serving-cr.md Co-Authored-By: Evan Anderson Update docs/install/operator/configuring-serving-cr.md Co-Authored-By: Evan Anderson Update docs/install/operator/configuring-serving-cr.md Co-Authored-By: Evan Anderson Update docs/install/operator/configuring-serving-cr.md Co-Authored-By: Evan Anderson Update docs/install/operator/configuring-serving-cr.md Co-Authored-By: Matt Moore Update docs/install/operator/configuring-serving-cr.md Co-Authored-By: Evan Anderson Update docs/install/operator/configuring-serving-cr.md Co-Authored-By: Evan Anderson Update docs/install/operator/configuring-serving-cr.md Co-Authored-By: Evan Anderson Update docs/install/operator/configuring-serving-cr.md Co-Authored-By: Evan Anderson Update docs/install/operator/configuring-serving-cr.md Co-Authored-By: Evan Anderson Update docs/install/operator/configuring-serving-cr.md Co-Authored-By: Evan Anderson Update docs/install/operator/configuring-serving-cr.md Co-Authored-By: Evan Anderson Update docs/install/operator/configuring-serving-cr.md Co-Authored-By: Evan Anderson Update docs/install/operator/configuring-serving-cr.md Co-Authored-By: Evan Anderson Update docs/install/operator/configuring-serving-cr.md Co-Authored-By: Evan Anderson Update docs/install/operator/configuring-serving-cr.md Co-Authored-By: Evan Anderson Update docs/install/operator/configuring-serving-cr.md Co-Authored-By: Evan Anderson Update docs/install/operator/configuring-serving-cr.md Co-Authored-By: Evan Anderson Update docs/install/operator/configuring-serving-cr.md Co-Authored-By: Evan Anderson Update docs/install/operator/configuring-serving-cr.md Co-Authored-By: Evan Anderson Update docs/install/operator/configuring-serving-cr.md Co-Authored-By: Evan Anderson Update docs/install/operator/configuring-serving-cr.md Co-Authored-By: Evan Anderson Update docs/install/operator/configuring-serving-cr.md Co-Authored-By: Evan Anderson Update docs/install/operator/configuring-serving-cr.md Co-Authored-By: Evan Anderson Update docs/install/operator/configuring-serving-cr.md Co-Authored-By: Evan Anderson Update docs/install/operator/configuring-serving-cr.md Co-Authored-By: Evan Anderson Update docs/install/operator/configuring-serving-cr.md Co-Authored-By: Evan Anderson Update docs/install/operator/configuring-serving-cr.md Co-Authored-By: Evan Anderson Update docs/install/operator/configuring-serving-cr.md Co-Authored-By: Evan Anderson Update docs/install/operator/configuring-serving-cr.md Co-Authored-By: Evan Anderson Update docs/install/operator/configuring-serving-cr.md Co-Authored-By: Evan Anderson Update docs/install/operator/configuring-serving-cr.md Co-Authored-By: Evan Anderson --- .../operator/configuring-eventing-cr.md | 48 ++-- .../operator/configuring-serving-cr.md | 259 +++++++----------- 2 files changed, 120 insertions(+), 187 deletions(-) diff --git a/docs/install/operator/configuring-eventing-cr.md b/docs/install/operator/configuring-eventing-cr.md index 8a0d7d5ce80..670e30a9fb1 100644 --- a/docs/install/operator/configuring-eventing-cr.md +++ b/docs/install/operator/configuring-eventing-cr.md @@ -18,15 +18,17 @@ __NOTE:__ Kubernetes spec level policies cannot be configured using the Knative The Knative Eventing operator CR is configured the same way as the Knative Serving operator CR. For more information, see the documentation on “[Private repository and private secret](configuring-serving-cr.md#private-repository-and-private-secrets)” in Serving operator for detailed instruction. -Knative Eventing also specifies only one container, within one `Deployment` resource. However, the container does not use +Knative Eventing also specifies only one container within one `Deployment` resource. However, the container does not use the same name as its parent `Deployment`, which means the container name in Knative Eventing is not the unique identifier as in Knative Serving. Here is the list of containers within each `Deployment` resource: -- Container in Deployment `eventing-controller`: eventing-controller -- Container in Deployment `eventing-webhook`: eventing-webhook -- Container in Deployment `broker-controller`: eventing-controller -- Container in Deployment `imc-controller`: controller -- Container in Deployment `imc-dispatcher`: dispatcher +| Component | Deployment name | Container name | +|-----------|-----------------|----------------| +| Core eventing | `eventing-controller` | `eventing-controller` | +| Core eventing | `eventing-webhook` | `eventing-webhook` | +| Eventing Broker | `broker-controller` | `eventing-controller` | +| In-Memory Channel | `imc-controller` | `controller` | +| In-Memory Channel | `imc-dispatcher` | `dispatcher` | The `default` field can still be used to replace the images in a predefined format. However, if the container name is not a unique identifier, e.g. `eventing-controller`, you need to use the `override` field to replace it, by specifying @@ -48,11 +50,13 @@ In the example below: First, you need to make sure your images are saved in the following link: -- Image of `eventing-controller` in the deployment `eventing-controller`: `docker.io/knative-images/eventing-controller:v0.13.0`. -- Image of `eventing-webhook`: `docker.io/knative-images/eventing-webhook:v0.13.0`. -- Image of `controller`: `docker.io/knative-images/controller:v0.13.0`. -- Image of `dispatcher`: `docker.io/knative-images/dispatcher:v0.13.0`. -- Image of `eventing-controller` in the deployment `broker-controller`: `docker.io/knative-images/broker-eventing-controller:v0.13.0`. +| Deployment | Container | Docker image | +|----|----|----| +| `eventing-controller` | `eventing-controller` | `docker.io/knative-images/eventing-controller:v0.13.0` | +| | `eventing-webhook` | `docker.io/knative-images/eventing-webhook:v0.13.0` | +| `broker-controller` | `eventing-controller` | `docker.io/knative-images/broker-eventing-controller:v0.13.0` | +| | `controller` | `docker.io/knative-images/controller:v0.13.0` | +| | `dispatcher` | `docker.io/knative-images/dispatcher:v0.13.0` | Then, you need to define your operator CR with following content: @@ -92,11 +96,13 @@ link in the CR. For example, to define the list of images: -- Image of `eventing-controller` in the deployment `eventing-controller`: `docker.io/knative-images/eventing-controller:v0.13.0`. -- Image of `eventing-webhook`: `docker.io/knative-images/eventing-webhook:v0.13.0`. -- Image of `controller`: `docker.io/knative-images/controller:v0.13.0`. -- Image of `dispatcher`: `docker.io/knative-images/dispatcher:v0.13.0`. -- Image of `eventing-controller` in the deployment `broker-controller`: `docker.io/knative-images/broker-eventing-controller:v0.13.0`. +| Deployment | Container | Docker Image | +|----|----|----| +| `eventing-controller` | `eventing-controller` | `docker.io/knative-images/eventing-controller:v0.13.0` | +| | `eventing-webhook` | `docker.io/knative-images/eventing-webhook:v0.13.0` | +| | `controller` | `docker.io/knative-images/controller:v0.13.0` | +| | `dispatcher` | `docker.io/knative-images/dispatcher:v0.13.0` | +| `broker-controller` | `eventing-controller` | `docker.io/knative-images/broker-eventing-controller:v0.13.0` | The operator CR should be modified to include the full list: @@ -140,11 +146,11 @@ spec: ### Download images with secrets: -If you use the default or override attributes to define image links, and the image links require private secrets for +If your image repository requires private secrets for access, you must append the `imagePullSecrets` attribute. This example uses a secret named `regcred`. You must create your own private secrets if these are required. After you -create this secret, edit your operator CR by appending the content below ...: +create this secret, edit your operator CR by appending the content below: ``` apiVersion: operator.knative.dev/v1alpha1 @@ -178,10 +184,8 @@ spec: ## Configuring default broker class -Knative Eventing support two types of default broker classes: `ChannelBasedBroker` and `MTChannelBasedBroker`. It is can -be specified with the field called `defaultBrokerClass`. If this field is left empty, `ChannelBasedBroker` will be taken -as the broker class. If we would like to specify the default broker class to `MTChannelBasedBroker`, the Eventing -Operator CR should be: +Knative Eventing allows you to define a default broker class when the user does not specify one. The operator ships with two broker classes: `ChannelBasedBroker` and `MTChannelBasedBroker`. The field `defaultBrokerClass` indicates which class to use; if empty, the `ChannelBasedBroker` will be used. +Here is an example specifying `MTChannelBasedBroker` as the default: ``` apiVersion: operator.knative.dev/v1alpha1 diff --git a/docs/install/operator/configuring-serving-cr.md b/docs/install/operator/configuring-serving-cr.md index 89cf870e8db..6ef80d83de3 100644 --- a/docs/install/operator/configuring-serving-cr.md +++ b/docs/install/operator/configuring-serving-cr.md @@ -20,14 +20,11 @@ __NOTE:__ Kubernetes spec level policies cannot be configured using the Knative ## All the ConfigMaps -All the ConfigMaps can be configured in Knative Serving with the custom resource. The values in the custom resource will -overwrite the existing values in ConfigMaps. In the latest release of Knative Serving, there are multiple ConfigMaps, -e.g. `config-autoscaler`, `config-default`, `config-deployment`, etc. All the ConfigMaps are named with the prefix -`config-`. and in the format of `config-`. A field named `config` is defined under the section `spec` to specify -all the ConfigMaps. Under the section `spec.config`, use the name after the hyphen(`-`) sign, ``, as the field to -specify all the key-value pairs, which are exactly the same as we have in the section `data` for each ConfigMap. - -As in the example of how to [setup a custom domain](https://knative.dev/development/serving/using-a-custom-domain/), you can see the content of the ConfigMap +Because the operator manages the Knative Serving installation, it will overwrite any updates to the `ConfigMaps` which are used to configure Knative Serving. +The `KnativeServing` custom resource allows you to set values for these ConfigMaps via the operator. Knative Serving has multiple ConfigMaps named with the prefix +`config-`. The `spec.config` in `KnativeServing` has one entry `` for each ConfigMap named `config-`, with a value which will be used for the ConfigMap's `data`. + +In the [setup a custom domain example](https://knative.dev/development/serving/using-a-custom-domain/), you can see the content of the ConfigMap `config-domain` is: ``` @@ -43,7 +40,7 @@ data: example.com: "" ``` -To specify the ConfigMap `config-domain`, you can change the content of the operator CR into: +Using the operator, specify the ConfigMap `config-domain` using the operator CR: ``` apiVersion: operator.knative.dev/v1alpha1 @@ -66,8 +63,7 @@ Next, save the CR into a file named `operator-cr.yaml`, and run the command: kubectl apply -f operator-cr.yaml ``` -If you want to change another ConfigMap, e.g. `config-autoscaler`, by specifying `stable-window` to `60s`. Continue to -change your operator CR into: +You can apply values to multiple ConfigMaps. This example sets `stable-window` to 60s in `config-autoscaler` as well as specifying `config-domain`: ``` apiVersion: operator.knative.dev/v1alpha1 @@ -86,49 +82,27 @@ spec: stable-window: "60s" ``` -Then, save the content in a file named `operator-cr.yaml`, and run the command: - -``` -kubectl apply -f operator-cr.yaml -``` All the ConfigMaps should be defined under the same namespace as the operator CR. You can use the operator CR as the unique entry point to edit all of them. ## Private repository and private secrets -As in the latest release of Knative Serving, there are six `Deployment` resources: `activator`, `autoscaler`, `controller`, -`webhook`, `autoscaler-hpa` & `networking-istio`, under the apiVersion `apps/v1`, and one image: `queue-proxy`, under the -apiVersion `caching.internal.knative.dev/v1alpha1`. The images can be downloaded from the links specified in the `spec.image` -section for each of the resources. Knative Serving Operator provides us a way to download the images from private -repositories for Knative deployments and image(s). - -Under the section `spec` of the operator CR, you can create a section of `registry`, containing all the fields to define -the information about the private registry: - -- `default`: this field expects a string value, used to define image reference template for all Knative images. The format -is in `example-registry.io/custom/path/${NAME}:{CUSTOM-TAG}`. Since all your private images can be saved in the same -repository with the same tag, the only difference is the image name. `${NAME}` should be kept as it is, because this is -a pre-defined container variable in operator. If you name the images after the container names within all `Deployment` resources: -`activator`, `autoscaler`, `controller`, `webhook`, `autoscaler-hpa` & `networking-istio`, for all `Deployment` resources, -and name the image after `queue-proxy`, for the `Image` resource, you do not need to do any further configuration in the -next section `override`, because operator can automatically replace `${NAME}` with the corresponding container name. For -serving-operator, there is only one container defined in each `Deployment` resource. The container shares the same as -its parent deployment. - -- `override`: this field expects a map of key-value pairs, with container name or image name as the key, and the full image -location as the value. We usually need to configure this section, when we do not have a common format for all the image -links. This field is used alternatively with the previous field `default`. If the image for a certain container or `Image` -resource is specified in both this field and the `default` field, this `override` field takes precedence. This field can -also be used as the supplement of the `default` field, if the image link can not match the predefined format. - -- `imagePullSecrets`: this field is used to define a list of secrets to be used when pulling the knative images. The secret -must be created in the same namespace as the Knative Serving deployments. You do not need to define any secret here if -your image is publicly available. Configuration of this field is equivalent to the configuration of [deploying images -from a private container registry](https://knative.dev/development/serving/deploying/private-registry/). - -This `registry` section is used to specify the links of the custom Knative images, and the appropriate credentials to access -them. We will use examples to illustrate how we define all the fields, regarding your custom image links and private secrets. + +You can use the `spec.registry` section of the operator CR to change the image references to point to a private registry or [specify imagePullSecrets](https://kubernetes.io/docs/concepts/containers/images/#specifying-imagepullsecrets-on-a-pod): + +- `default`: this field defines a image reference template for all Knative images. The format +is `example-registry.io/custom/path/${NAME}:{CUSTOM-TAG}`. If you use the same tag for all your images, the only difference is the image name. `${NAME}` is +a pre-defined variable in the operator corresponding to the container name. If you name the images in your private repo to align with the container names ( +`activator`, `autoscaler`, `controller`, `webhook`, `autoscaler-hpa`, `networking-istio`, and `queue-proxy`), the `default` argument should be sufficient. + +- `override`: a map from container name to the full registry +location. This section is only needed when the registry images do not match the common naming format. For containers whose name matches a key, the value is used in preference to the image name calculated by `default`. If a container's name does not match a key in `override`, the template in `default` is used. + +- `imagePullSecrets`: a list of Secret names used when pulling Knative container images. The Secrets +must be created in the same namespace as the Knative Serving Deployments. See [deploying images +from a private container registry](https://knative.dev/development/serving/deploying/private-registry/) for configuration details. + ### Download images in a predefined format without secrets: @@ -139,17 +113,19 @@ In the example below: - the custom tag `v0.13.0` is used for all images - all image links are accessible without using secrets -- images are defined in the accepted format `docker.io/knative-images/${NAME}:{CUSTOM-TAG}` +- images are pushed as `docker.io/knative-images/${NAME}:{CUSTOM-TAG}` -First, you need to make sure your images are saved in the following link: +First, you need to make sure your images pushed to the following image tags: -- Image of `activator`: `docker.io/knative-images/activator:v0.13.0`. -- Image of `autoscaler`: `docker.io/knative-images/autoscaler:v0.13.0`. -- Image of `controller`: `docker.io/knative-images/controller:v0.13.0`. -- Image of `webhook`: `docker.io/knative-images/webhook:v0.13.0`. -- Image of `autoscaler-hpa`: `docker.io/knative-images/autoscaler-hpa:v0.13.0`. -- Image of `networking-istio`: `docker.io/knative-images/networking-istio:v0.13.0`. -- Cache image `queue-proxy`: `docker.io/knative-images/queue-proxy:v0.13.0`. +| Container | Docker Image | +|----|----| +|`activator` | `docker.io/knative-images/activator:v0.13.0` | +| `autoscaler` | `docker.io/knative-images/autoscaler:v0.13.0` | +| `controller` | `docker.io/knative-images/controller:v0.13.0` | +| `webhook` | `docker.io/knative-images/webhook:v0.13.0` | +| `autoscaler-hpa` | `docker.io/knative-images/autoscaler-hpa:v0.13.0` | +| `networking-istio` | `docker.io/knative-images/networking-istio:v0.13.0` | +| `queue-proxy` | `docker.io/knative-images/queue-proxy:v0.13.0` | Then, you need to define your operator CR with following content: @@ -164,25 +140,23 @@ spec: default: docker.io/knative-images/${NAME}:v0.13.0 ``` -Replace `{CUSTOM-TAG}` with the custom tag `v0.13.0`. `${NAME}` needs to map the same name of each container or each -`Image` resource. The field `default` is used to define the image format for all containers and the `Image` resource. -Make sure you want to replace the images for all containers and `Image` resources in Knative Serving with your own -images, by specifying the field `default`. ### Download images individually without secrets: If your custom image links are not defined in a uniform format by default, you will need to individually include each link in the CR. -For example, to define the list of images: +For example, to given the following images: -- Image of `activator`: `docker.io/knative-images-repo1/activator:v0.13.0`. -- Image of `autoscaler`: `docker.io/knative-images-repo2/autoscaler:v0.13.0`. -- Image of `controller`: `docker.io/knative-images-repo3/controller:v0.13.0`. -- Image of `webhook`: `docker.io/knative-images-repo4/webhook:v0.13.0`. -- Image of `autoscaler-hpa`: `docker.io/knative-images-repo5/autoscaler-hpa:v0.13.0`. -- Image of `networking-istio`: `docker.io/knative-images-repo6/prefix-networking-istio:v0.13.0`. -- Cache image `queue-proxy`: `docker.io/knative-images-repo7/queue-proxy-suffix:v0.13.0`. +| Container | Docker Image | +|----|----| +| `activator` | `docker.io/knative-images-repo1/activator:v0.13.0` | +| `autoscaler` | `docker.io/knative-images-repo2/autoscaler:v0.13.0` | +| `controller` | `docker.io/knative-images-repo3/controller:v0.13.0` | +| `webhook` | `docker.io/knative-images-repo4/webhook:v0.13.0` | +| `autoscaler-hpa` | `docker.io/knative-images-repo5/autoscaler-hpa:v0.13.0` | +| `networking-istio` | `docker.io/knative-images-repo6/prefix-networking-istio:v0.13.0` | +| `queue-proxy` | `docker.io/knative-images-repo7/queue-proxy-suffix:v0.13.0` | The operator CR should be modified to include the full list: @@ -206,8 +180,8 @@ spec: ### Download images with secrets: -If you use the default or override attributes to define image links, and the image links require private secrets for -access, you must append the `imagePullSecrets` attribute. +If your image repository requires private secrets for +access, include the `imagePullSecrets` attribute. This example uses a secret named `regcred`. You must create your own private secrets if these are required: @@ -215,7 +189,7 @@ This example uses a secret named `regcred`. You must create your own private sec - [From command line for docker credentials](https://kubernetes.io/docs/tasks/configure-pod-container/pull-image-private-registry/#create-a-secret-by-providing-credentials-on-the-command-line) - [Create your own secret](https://kubernetes.io/docs/concepts/configuration/secret/#creating-your-own-secrets) -After you create this secret, edit your operator CR by appending the content below ...: +After you create this secret, edit your operator CR by appending the content below: ``` apiVersion: operator.knative.dev/v1alpha1 @@ -249,20 +223,16 @@ spec: ## SSL certificate for controller -Knative Serving needs to access the container registry, based on the feature [enabling tag to digest resolution](https://knative.dev/development/serving/tag-resolution/). The -Serving Operator CR allows you to specify either a custom ConfigMap or a Secret as a self-signed certificate for the -deployment called `controller`. It enables the `controller` to trust registries with self-signed certificates. +To [enable tag to digest resolution](https://knative.dev/development/serving/tag-resolution/), the Knative Serving controller needs to access the container registry. +To allow the controller to trust a self-signed registry cert, you can use the Operator to specify the certificate using a ConfigMap or Secret. -Under the section `spec` of the operator CR, you can create a section of `controller-custom-certs` to contain all the -fields to define the certificate: +Specify the following fields in `spec.controller-custom-certs` to select a custom registry certificate: -- `name`: this field is used to specify the name of the ConfigMap or the Secret. -- `type`: the value for this field can be either ConfigMap or Secret, indicating the type for the name. +- `name`: the name of the ConfigMap or Secret. +- `type`: either the string "ConfigMap" or "Secret". -This section `controller-custom-certs` is used to access the user's images in private repositories with the appropriate -certificate. -If you create a configMap named `testCertas` as the certificate, you need to change your CR into: +If you create a ConfigMap named `testCert` containing the certificate, change your CR: ``` apiVersion: operator.knative.dev/v1alpha1 @@ -276,19 +246,14 @@ spec: type: ConfigMap ``` -It will make sure this custom certificate is mounted as a volume to the containers launched by the `Deployment` resource -`controller`, and the environment variable is `SSL_CERT_DIR` set correctly. ## Configuration of Knative ingress gateway -To set up custom ingress gateway, follow “**Step 1: Create Gateway Service and Deployment Instance**” [here](https://knative.dev/development/serving/setting-up-custom-ingress-gateway/). +To set up custom ingress gateway, follow [**Step 1: Create Gateway Service and Deployment Instance**](https://knative.dev/development/serving/setting-up-custom-ingress-gateway/). **Step 2: Update the Knative gateway** -We use the field `knative-ingress-gateway` to override the knative-ingress-gateway. We only support the field `selector` -to define the selector for ingress-gateway. - -Instead of updating the gateway directly, we modify the operator CR as below: +Update `spec.knative-ingress-gateway` to select the labels of the new ingress gateway: ``` apiVersion: operator.knative.dev/v1alpha1 @@ -297,16 +262,14 @@ metadata: name: knative-serving namespace: knative-serving spec: - registry: - knative-ingress-gateway: - selector: - custom: ingressgateway + knative-ingress-gateway: + selector: + custom: ingressgateway ``` **Step 3: Update Gateway ConfigMap** -As we explained, all ConfigMaps can be edited as editing the the operator CR. For this example, take the following content -as your operator CR: +Additionally, you will need to update the Istio ConfigMap: ``` apiVersion: operator.knative.dev/v1alpha1 @@ -315,28 +278,25 @@ metadata: name: knative-serving namespace: knative-serving spec: - registry: - knative-ingress-gateway: - selector: - custom: ingressgateway - config: - istio: - gateway.knative-serving.knative-ingress-gateway: "custom-ingressgateway.istio-system.svc.cluster.local" + knative-ingress-gateway: + selector: + custom: ingressgateway + config: + istio: + gateway.knative-serving.knative-ingress-gateway: "custom-ingressgateway.istio-system.svc.cluster.local" ``` The key in `spec.config.istio` is in the format of `gateway.{{gateway_namespace}}.{{gateway_name}}`. ## Configuration of cluster local gateway: -We use the field `cluster-local-gateway` to override the the gateway cluster-local-gateway. We only support the field -`selector` to define the selector for the local gateway. +Update `spec.cluster-local-gateway` to select the labels of the new cluster-local ingress gateway: **Default local gateway name**: Go through the guide [here](https://knative.dev/development/install/installing-istio/#updating-your-install-to-use-cluster-local-gateway) to use local cluster gateway. -After following the above step, your service and deployment for the local gateway are both named `cluster-local-gateway`. -You only need to configure the operator CR as below: +Configure the operator CR to match the labels on the cluster-local-gateway: ``` apiVersion: operator.knative.dev/v1alpha1 @@ -345,10 +305,9 @@ metadata: name: knative-serving namespace: knative-serving spec: - registry: - cluster-local-gateway: - selector: - istio: cluster-local-gateway + cluster-local-gateway: + selector: + istio: cluster-local-gateway ``` You can even skip the above change, since there is a gateway called `cluster-local-gateway`, which has @@ -357,12 +316,11 @@ cluster-local-gateway, the default `istio: cluster-local-gateway` of the gateway **Non-default local gateway name**: -If you create custom service and deployment for local gateway with a name other than `cluster-local-gateway`, you need -to update gateway configmap `config-istio` under the Knative Serving namespace, and change the selector for the gateway -cluster-local-gateway. +If you create custom local gateway with a name other than `cluster-local-gateway`, update `config.istio` and the +`cluster-local-gateway` selector: -If you name both of the service and deployment after `custom-local-gateway` in the namespace `istio-system`, with the -label `custom: custom-local-gateway`, the operator CR should be like: +This example shows a service and deployment `custom-local-gateway` in the namespace `istio-system`, with the +label `custom: custom-local-gw`: ``` apiVersion: operator.knative.dev/v1alpha1 @@ -371,22 +329,17 @@ metadata: name: knative-serving namespace: knative-serving spec: - registry: - cluster-local-gateway: - selector: - custom: custom-local-gateway - config: - istio: - local-gateway.knative-serving.cluster-local-gateway: "custom-local-gateway.istio-system.svc.cluster.local" + cluster-local-gateway: + selector: + custom: custom-local-gateway + config: + istio: + local-gateway.knative-serving.cluster-local-gateway: "custom-local-gateway.istio-system.svc.cluster.local" ``` ## High availability: -Knative Serving Operator CR provides the capability to configure the high availability for the following `Deployment` -resources: `controller`, `autoscaler-hpa` & `networking-istio`, and all the `HorizontalPodAutoscaler` resources. A field -named `high-availability` is defined under the section `spec` to configure the high availability control plane. In the -latest release of Knative Serving Operator, you can specify the minimum number of replicas for the `Deployment` and -`HorizontalPodAutoscaler` resources. +By default, Knative Serving runs a single instance of each controller. The `spec.high-availability` field allows you to configure the number of replicas for the following master-elected controllers: `controller`, `autoscaler-hpa`, `networking-istio`. This field also configures the `HorizontalPodAutoscaler` resources for the data plane (`activator`): If you want to specify 3 as the minimum number of replicas for three of the `Deployment` resources and the `HorizontalPodAutoscaler` resources, change your CR into: @@ -404,36 +357,15 @@ spec: ## Managing resources for containers -Knative Serving Operator CR allows you to configure the resources for the containers defined in the `Deployment` resource. -Each `Deployment` resource has only one container with the same deployment name. The following containers can be configured -in terms of the required and maximum resources to be allocated: `activator`, `autoscaler`, `controller`, `webhook`, `autoscaler-hpa`, -`networking-istio` and `queue-proxy`. The field called `resources` holds the stanza of a list of containers for resource -configuration. The field `require` defines how much resource the container requires. The field `limit` defines the maximum -resource the container can use. Within each field, each container can be configured individually with the following -parameters: `memory`, `cpu`, `storage` and `ephemeral-storage`. The value in `require` should always be smaller than the -corresponding value in `limit`. The field `container` is used to specify the container name. - -Suppose you would like to configure the container called `activator` with the required resources: - -``` -cpu: 30m -memory: 40Mi -storage: 1Gi -ephemeral-storage: 2Gi -``` - -, and the limit resources: +The operator custom resource allows you allows you to configure resource requests and limits for the Knative system containers. +Requests and limits may be configured for the following containers: `activator`, `autoscaler`, `controller`, `webhook`, `autoscaler-hpa`, +`networking-istio` and `queue-proxy`. -``` -cpu: 300m -memory: 4000Mi -storage: 2Gi -ephemeral-storage: 4Gi -``` +To override resource settings for a specific container, create an entry in the `spec.resources` list with the container name and the [Kubernetes resource settings](https://kubernetes.io/docs/concepts/configuration/manage-compute-resources-container/#resource-requests-and-limits-of-pod-and-container). -Here is how you should write your CR: +For example, the following KnativeServing resource configures the `activator` to request 0.3 CPU and 100MB of RAM, and sets hard limits of 1 CPU, 250MB RAM, and 4GB of local storage: -``` +\``` apiVersion: operator.knative.dev/v1alpha1 kind: KnativeServing metadata: @@ -442,17 +374,14 @@ metadata: spec: resources: - container: activator - requests: - cpu: 30m - memory: 40Mi - storage: 1Gi - ephemeral-storage: 2Gi - limits: - cpu: 300m - memory: 400Mi - storage: 2Gi - ephemeral-storage: 4Gi -``` + requests: + cpu: 300m + memory: 100Mi + limits: + cpu: 1000m + memory: 250Mi + ephemeral-storage: 4Gi +\``` If you would like to add another container `autoscaler` with the same configuration, you need to change your CR as below: From 1314955599910022991c5551f6c4a2cc41e37331 Mon Sep 17 00:00:00 2001 From: Vincent Hou Date: Fri, 17 Apr 2020 16:35:43 -0400 Subject: [PATCH 14/19] Change the content based on the non-suggestion comments --- .../operator/configuring-eventing-cr.md | 32 +++++---------- .../operator/configuring-serving-cr.md | 39 ++++--------------- 2 files changed, 17 insertions(+), 54 deletions(-) diff --git a/docs/install/operator/configuring-eventing-cr.md b/docs/install/operator/configuring-eventing-cr.md index 670e30a9fb1..610097bb07f 100644 --- a/docs/install/operator/configuring-eventing-cr.md +++ b/docs/install/operator/configuring-eventing-cr.md @@ -1,6 +1,6 @@ --- title: "Configuring the Eventing Operator Custom Resource" -weight: 10 +weight: 60 type: "docs" aliases: - /docs/operator/configuring-eventing-cr/ @@ -35,7 +35,7 @@ a unique identifier, e.g. `eventing-controller`, you need to use the `override` `deployment/container` as the unique key. Some images are defined via environment variable in Knative Eventing. They can be replaced by taking advantage of the -`override` field. +`override` field. As Knative does not have a consistent way to specify container images, we have a known issue [here](https://github.com/knative-sandbox/operator/issues/22). ### Download images in predefined format without secrets: @@ -48,7 +48,7 @@ In the example below: - all image links are accessible without using secrets - images are defined in the accepted format `docker.io/knative-images/${NAME}:{CUSTOM-TAG}` -First, you need to make sure your images are saved in the following link: +First, you need to make sure your images are pushed to the following image tags: | Deployment | Container | Docker image | |----|----|----| @@ -69,25 +69,14 @@ metadata: spec: registry: default: docker.io/knative-images/${NAME}:v0.13.0 + override: + broker-controller/eventing-controller: docker.io/knative-images-repo1/broker-eventing-controller:v0.13.0 ``` -Replace `{CUSTOM-TAG}` with the custom tag `v0.13.0`. `${NAME}` needs to map the container name in each `Deployment` resource. +As indicated, you replace `{CUSTOM-TAG}` with the custom tag `v0.13.0`. `${NAME}` maps to the container name in each `Deployment` resource. The field `default` is used to define the image format for all containers, except the container `eventing-controller` in the deployment `broker-controller`. To replace the image for this container, you need to take advatage of the `override` -field to specify individually, by using `broker-controller/eventing-controller` as the key`. Change your CR into the following: - -``` -apiVersion: operator.knative.dev/v1alpha1 -kind: KnativeEventing -metadata: - name: knative-eventing - namespace: knative-eventing -spec: - registry: - default: docker.io/knative-images/${NAME}:v0.13.0 - override: - broker-controller/eventing-controller: docker.io/knative-images-repo1/broker-eventing-controller:v0.13.0 -``` +field to specify individually, by using `broker-controller/eventing-controller` as the key`. ### Download images from different repositories without secrets: @@ -146,11 +135,10 @@ spec: ### Download images with secrets: -If your image repository requires private secrets for -access, you must append the `imagePullSecrets` attribute. +If your image repository requires private secrets for access, you must append the `imagePullSecrets` attribute. -This example uses a secret named `regcred`. You must create your own private secrets if these are required. After you -create this secret, edit your operator CR by appending the content below: +This example uses a secret named `regcred`. Refer to [this guide](https://kubernetes.io/docs/concepts/containers/images/#specifying-imagepullsecrets-on-a-pod) to create your own private secrets. +After you create the secret, edit your operator CR by appending the content below: ``` apiVersion: operator.knative.dev/v1alpha1 diff --git a/docs/install/operator/configuring-serving-cr.md b/docs/install/operator/configuring-serving-cr.md index 6ef80d83de3..db7c742330c 100644 --- a/docs/install/operator/configuring-serving-cr.md +++ b/docs/install/operator/configuring-serving-cr.md @@ -1,6 +1,6 @@ --- title: "Configuring the Serving Operator Custom Resource" -weight: 10 +weight: 20 type: "docs" aliases: - /docs/operator/configuring-serving-cr/ @@ -57,12 +57,6 @@ spec: example.com: "" ``` -Next, save the CR into a file named `operator-cr.yaml`, and run the command: - -``` -kubectl apply -f operator-cr.yaml -``` - You can apply values to multiple ConfigMaps. This example sets `stable-window` to 60s in `config-autoscaler` as well as specifying `config-domain`: ``` @@ -82,8 +76,7 @@ spec: stable-window: "60s" ``` - -All the ConfigMaps should be defined under the same namespace as the operator CR. You can use the operator CR as the +All the ConfigMaps are created in the same namespace as the operator CR. You can use the operator CR as the unique entry point to edit all of them. ## Private repository and private secrets @@ -292,29 +285,12 @@ The key in `spec.config.istio` is in the format of `gateway.{{gateway_namespace} Update `spec.cluster-local-gateway` to select the labels of the new cluster-local ingress gateway: -**Default local gateway name**: - -Go through the guide [here](https://knative.dev/development/install/installing-istio/#updating-your-install-to-use-cluster-local-gateway) to use local cluster gateway. - -Configure the operator CR to match the labels on the cluster-local-gateway: - -``` -apiVersion: operator.knative.dev/v1alpha1 -kind: KnativeServing -metadata: - name: knative-serving - namespace: knative-serving -spec: - cluster-local-gateway: - selector: - istio: cluster-local-gateway -``` +### Default local gateway name: -You can even skip the above change, since there is a gateway called `cluster-local-gateway`, which has -`istio: cluster-local-gateway` as the default selector. If the operator CR does not define the section -cluster-local-gateway, the default `istio: cluster-local-gateway` of the gateway cluster-local-gateway will be chosen. +Go through the guide [here](https://knative.dev/development/install/installing-istio/#updating-your-install-to-use-cluster-local-gateway) to use local cluster gateway, +if you use the default gateway called `cluster-local-gateway`. -**Non-default local gateway name**: +### Non-default local gateway name: If you create custom local gateway with a name other than `cluster-local-gateway`, update `config.istio` and the `cluster-local-gateway` selector: @@ -341,8 +317,7 @@ spec: By default, Knative Serving runs a single instance of each controller. The `spec.high-availability` field allows you to configure the number of replicas for the following master-elected controllers: `controller`, `autoscaler-hpa`, `networking-istio`. This field also configures the `HorizontalPodAutoscaler` resources for the data plane (`activator`): -If you want to specify 3 as the minimum number of replicas for three of the `Deployment` resources and the `HorizontalPodAutoscaler` -resources, change your CR into: +The following configuration specifies a replica count of 3 for the controllers and a minimum of 3 activators (which may scale higher if needed): ``` apiVersion: operator.knative.dev/v1alpha1 From 729828f74bfeb928cd0484c3f1c06b6e7ec732ac Mon Sep 17 00:00:00 2001 From: Vincent Date: Fri, 17 Apr 2020 17:52:34 -0400 Subject: [PATCH 15/19] Update docs/install/operator/configuring-serving-cr.md Co-Authored-By: Evan Anderson --- docs/install/operator/configuring-serving-cr.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/install/operator/configuring-serving-cr.md b/docs/install/operator/configuring-serving-cr.md index db7c742330c..89fb43a1783 100644 --- a/docs/install/operator/configuring-serving-cr.md +++ b/docs/install/operator/configuring-serving-cr.md @@ -164,7 +164,7 @@ spec: override: activator: docker.io/knative-images-repo1/activator:v0.13.0 autoscaler: docker.io/knative-images-repo2/autoscaler:v0.13.0 - controller: docker.io/knative-images-repo3/controller:v0.13.0 + controller: docker.io/knative-images-repo3/controller:v0.13.0 webhook: docker.io/knative-images-repo4/webhook:v0.13.0 autoscaler-hpa: docker.io/knative-images-repo5/autoscaler-hpa:v0.13.0 networking-istio: docker.io/knative-images-repo6/prefix-networking-istio:v0.13.0 From 7f3b11f7927651423c5ab34f7e6f94ca4485be4f Mon Sep 17 00:00:00 2001 From: Vincent Date: Fri, 17 Apr 2020 17:52:42 -0400 Subject: [PATCH 16/19] Update docs/install/operator/configuring-serving-cr.md Co-Authored-By: Evan Anderson --- docs/install/operator/configuring-serving-cr.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/install/operator/configuring-serving-cr.md b/docs/install/operator/configuring-serving-cr.md index 89fb43a1783..32aa552bb34 100644 --- a/docs/install/operator/configuring-serving-cr.md +++ b/docs/install/operator/configuring-serving-cr.md @@ -260,7 +260,7 @@ spec: custom: ingressgateway ``` -**Step 3: Update Gateway ConfigMap** +### Step 3: Update Gateway ConfigMap Additionally, you will need to update the Istio ConfigMap: From bde33594559a25e88923754b34c8d55245f40929 Mon Sep 17 00:00:00 2001 From: Vincent Date: Fri, 17 Apr 2020 17:53:29 -0400 Subject: [PATCH 17/19] Update docs/install/operator/configuring-serving-cr.md Co-Authored-By: Evan Anderson --- .../operator/configuring-serving-cr.md | 34 ++++++++----------- 1 file changed, 14 insertions(+), 20 deletions(-) diff --git a/docs/install/operator/configuring-serving-cr.md b/docs/install/operator/configuring-serving-cr.md index 32aa552bb34..4b6a9912d3d 100644 --- a/docs/install/operator/configuring-serving-cr.md +++ b/docs/install/operator/configuring-serving-cr.md @@ -369,25 +369,19 @@ metadata: spec: resources: - container: activator - requests: - cpu: 30m - memory: 40Mi - storage: 1Gi - ephemeral-storage: 2Gi - limits: - cpu: 300m - memory: 400Mi - storage: 2Gi - ephemeral-storage: 4Gi + requests: + cpu: 300m + memory: 100Mi + limits: + cpu: 1000m + memory: 250Mi + ephemeral-storage: 4Gi - container: autoscaler - requests: - cpu: 30m - memory: 40Mi - storage: 1Gi - ephemeral-storage: 2Gi - limits: - cpu: 300m - memory: 400Mi - storage: 2Gi - ephemeral-storage: 4Gi + requests: + cpu: 300m + memory: 100Mi + limits: + cpu: 1000m + memory: 250Mi + ephemeral-storage: 4Gi ``` From 1928a2785bffe11e1969b1d2a88a984ae54be28c Mon Sep 17 00:00:00 2001 From: Vincent Date: Fri, 17 Apr 2020 17:54:18 -0400 Subject: [PATCH 18/19] Update docs/install/operator/configuring-serving-cr.md Co-Authored-By: Evan Anderson --- docs/install/operator/configuring-serving-cr.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/install/operator/configuring-serving-cr.md b/docs/install/operator/configuring-serving-cr.md index 4b6a9912d3d..123a98b4ce4 100644 --- a/docs/install/operator/configuring-serving-cr.md +++ b/docs/install/operator/configuring-serving-cr.md @@ -244,7 +244,7 @@ spec: To set up custom ingress gateway, follow [**Step 1: Create Gateway Service and Deployment Instance**](https://knative.dev/development/serving/setting-up-custom-ingress-gateway/). -**Step 2: Update the Knative gateway** +### Step 2: Update the Knative gateway Update `spec.knative-ingress-gateway` to select the labels of the new ingress gateway: From d82ae8b3ca971c79606c20996dbfe0ef89ca566c Mon Sep 17 00:00:00 2001 From: Vincent Hou Date: Fri, 17 Apr 2020 17:57:34 -0400 Subject: [PATCH 19/19] Fix based on the comments --- .../operator/configuring-serving-cr.md | 22 +++++++++---------- 1 file changed, 10 insertions(+), 12 deletions(-) diff --git a/docs/install/operator/configuring-serving-cr.md b/docs/install/operator/configuring-serving-cr.md index 123a98b4ce4..e9cadd82665 100644 --- a/docs/install/operator/configuring-serving-cr.md +++ b/docs/install/operator/configuring-serving-cr.md @@ -8,17 +8,15 @@ aliases: The Knative Serving operator can be configured with these options: -- [All the ConfigMaps](#all-the-configmaps) +- [Service Configuration by ConfigMap](#service-configuration-by-configMap) - [Private repository and private secret](#private-repository-and-private-secrets) - [SSL certificate for controller](#ssl-certificate-for-controller) - [Knative ingress gateway](#configuration-of-knative-ingress-gateway) - [Cluster local gateway](#configuration-of-cluster-local-gateway) - [High availability](#high-availability) -- [Managing Resources for Containers](#managing-resources-for-containers) +- [System Resource Settings](#system-resource-settings) -__NOTE:__ Kubernetes spec level policies cannot be configured using the Knative operators. - -## All the ConfigMaps +## Service Configuration by ConfigMap Because the operator manages the Knative Serving installation, it will overwrite any updates to the `ConfigMaps` which are used to configure Knative Serving. The `KnativeServing` custom resource allows you to set values for these ConfigMaps via the operator. Knative Serving has multiple ConfigMaps named with the prefix @@ -281,7 +279,7 @@ spec: The key in `spec.config.istio` is in the format of `gateway.{{gateway_namespace}}.{{gateway_name}}`. -## Configuration of cluster local gateway: +## Configuration of cluster local gateway Update `spec.cluster-local-gateway` to select the labels of the new cluster-local ingress gateway: @@ -313,7 +311,7 @@ spec: local-gateway.knative-serving.cluster-local-gateway: "custom-local-gateway.istio-system.svc.cluster.local" ``` -## High availability: +## High availability By default, Knative Serving runs a single instance of each controller. The `spec.high-availability` field allows you to configure the number of replicas for the following master-elected controllers: `controller`, `autoscaler-hpa`, `networking-istio`. This field also configures the `HorizontalPodAutoscaler` resources for the data plane (`activator`): @@ -330,17 +328,17 @@ spec: replicas: 3 ``` -## Managing resources for containers +## System Resource Settings -The operator custom resource allows you allows you to configure resource requests and limits for the Knative system containers. -Requests and limits may be configured for the following containers: `activator`, `autoscaler`, `controller`, `webhook`, `autoscaler-hpa`, +The operator custom resource allows you allows you to configure system resources for the Knative system containers. +Requests and limits can be configured for the following containers: `activator`, `autoscaler`, `controller`, `webhook`, `autoscaler-hpa`, `networking-istio` and `queue-proxy`. To override resource settings for a specific container, create an entry in the `spec.resources` list with the container name and the [Kubernetes resource settings](https://kubernetes.io/docs/concepts/configuration/manage-compute-resources-container/#resource-requests-and-limits-of-pod-and-container). For example, the following KnativeServing resource configures the `activator` to request 0.3 CPU and 100MB of RAM, and sets hard limits of 1 CPU, 250MB RAM, and 4GB of local storage: -\``` +``` apiVersion: operator.knative.dev/v1alpha1 kind: KnativeServing metadata: @@ -356,7 +354,7 @@ spec: cpu: 1000m memory: 250Mi ephemeral-storage: 4Gi -\``` +``` If you would like to add another container `autoscaler` with the same configuration, you need to change your CR as below: