openshift · vikram-redhat · Mar 17, 2021 · Mar 17, 2021
diff --git a/_topic_map.yml b/_topic_map.yml
@@ -2451,6 +2451,8 @@ Topics:
     File: upgrading-ossm
   - Name: Customizing the installation
     File: customizing-installation-ossm
+  - Name: Performance and scalability
+    File: ossm-performance-scalability 
   - Name: Deploying applications on Service Mesh
     File: prepare-to-deploy-applications-ossm
   - Name: Data visualization and observability

diff --git a/modules/ossm-control-plane-deploy.adoc b/modules/ossm-control-plane-deploy.adoc
@@ -15,6 +15,11 @@ You can deploy a basic installation of the {ProductShortName} control plane by u
 
 To get started, deploy the basic installation with the following steps. You can customize the `ServiceMeshControlPlane` resource later.
 
+[NOTE]
+====
+The control plane should be installed in a separate namespace from the data plane. The `istio-system` namespace is used as an example throughout the {ProductShortName} documentation, but you can deploy your control plane in any namespace as long as it is separate from your applications.
+====
+
 [id="ossm-control-plane-deploy-operatorhub_{context}"]
 == Deploying the control plane from the web console
 

diff --git a/modules/ossm-cr-istio-global-1x.adoc b/modules/ossm-cr-istio-global-1x.adoc
@@ -75,7 +75,7 @@ These parameters are specific to the proxy subset of global parameters.
 |===
 |Type |Parameter |Description |Values |Default value
 
-|Requests
+|`requests`
 |`cpu`
 |The amount of CPU resources requested for Envoy proxy.
 |CPU resources, specified in cores or millicores (for example, 200m, 0.5, 1) based on your environment’s configuration.

diff --git a/modules/ossm-cr-istio-global.adoc b/modules/ossm-cr-istio-global.adoc
@@ -75,7 +75,7 @@ These parameters are specific to the proxy subset of global parameters.
 |===
 |Type |Parameter |Description |Values |Default value
 
-|Requests
+|`requests`
 |`cpu`
 |The amount of CPU resources requested for Envoy proxy.
 |CPU resources, specified in cores or millicores (for example, 200m, 0.5, 1) based on your environment’s configuration.
@@ -87,7 +87,7 @@ These parameters are specific to the proxy subset of global parameters.
 |Available memory in bytes(for example, 200Ki, 50Mi, 5Gi) based on your environment’s configuration.
 |`128Mi`
 
-|Limits
+|`limits`
 |`cpu`
 |The maximum amount of CPU resources requested for Envoy proxy.
 |CPU resources, specified in cores or millicores (for example, 200m, 0.5, 1) based on your environment’s configuration.

diff --git a/modules/ossm-load-test-results.adoc b/modules/ossm-load-test-results.adoc
@@ -0,0 +1,75 @@
+////
+This module included in the following assemblies:
+- /v2x/ossm-performance-scalability.adoc
+////
+
+[id="ossm-load-test-results_{context}"]
+= Load test results
+
+The upstream Istio community load tests mesh consists of *1000* services and *2000* sidecars with 70,000 mesh-wide requests per second.
+Running the tests using Istio 1.6.8, generated the following results:
+
+* The Envoy proxy uses *0.5 vCPU* and *50 MB memory* per 1000 requests per second going through the proxy.
+* Istiod uses *1 vCPU* and *1.5 GB* of memory.
+* The Envoy proxy adds *3.12 ms* to the 90th percentile latency.
+* The legacy `istio-telemetry` service (disabled by default in Service Mesh 2.0) uses *0.6 vCPU* per 1000 mesh-wide requests per second for deployments that use Mixer.
+// TODO The Envoy numbers goes down in 1.9, check for the latest data with next version of Istio.
+The data plane components, the Envoy proxies, handle data flowing through the system. The control plane component, Istiod, configures the data plane. The data plane and control plane have distinct performance concerns.
+
+== Control plane performance
+
+Istiod configures sidecar proxies based on user authored configuration files and the current state of the system.
+In a Kubernetes environment, Custom Resource Definitions (CRDs) and deployments constitute the configuration and state of the system.
+The Istio configuration objects like gateways and virtual services, provide the user-authored configuration.
+To produce the configuration for the proxies, Istiod processes the combined configuration and system state from the Kubernetes environment and the user-authored configuration.
+
+The control plane supports thousands of services, spread across thousands of pods with a similar number of user authored virtual services and other configuration objects.
+Istiod's CPU and memory requirements scale with the number of configurations and possible system states.
+The CPU consumption scales with the following factors:
+
+* The rate of deployment changes.
+* The rate of configuration changes.
+* The number of proxies connecting to Istiod.
+
+However this part is inherently horizontally scalable.
+
+//Do we support namespace isolation?  When namespace isolation is enabled, a single Istiod instance can support 1000 services, 2000 sidecars with 1 vCPU and 1.5 GB of memory.
+//You can increase the number of Istiod instances to reduce the amount of time it takes for the configuration to reach all proxies.
+
+== Data plane performance
+
+Data plane performance depends on many factors, for example:
+
+* Number of client connections
+* Target request rate
+* Request size and response size
+* Number of proxy worker threads
+* Protocol
+* CPU cores
+* Number and types of proxy filters, specifically telemetry v2 related filters.
+
+The latency, throughput, and the proxies' CPU and memory consumption are measured as a function of these factors.
+
+=== CPU and memory consumption
+
+Since the sidecar proxy performs additional work on the data path, it consumes CPU and memory. As of Istio 1.1, a proxy consumes about 0.6 vCPU per 1000 requests per second.
+//TODO As of Istio 1.7, a proxy consumes about 0.5 vCPU per 1000 requests per second.
+
+The memory consumption of the proxy depends on the total configuration state the proxy holds.
+A large number of listeners, clusters, and routes can increase memory usage.
+//Istio 1.1 introduced namespace isolation to limit the scope of the configuration sent to a proxy. In a large namespace, the proxy consumes approximately 50 MB of memory.
+
+Since the proxy normally doesn't buffer the data passing through, request rate doesn't affect the memory consumption.
+
+=== Additional latency
+
+Since Istio injects a sidecar proxy on the data path, latency is an important consideration. Istio adds an authentication filter, a telemetry filter, and a metadata exchange filter to the proxy.
+Every additional filter adds to the path length inside the proxy and affects latency.
+
+The Envoy proxy collects raw telemetry data after a response is sent to the client.
+The time spent collecting raw telemetry for a request does not contribute to the total time taken to complete that request.
+However, since the worker is busy handling the request, the worker won't start handling the next request immediately.
+This process adds to the queue wait time of the next request and affects average and tail latencies.
+The actual tail latency depends on the traffic pattern.
+
+Inside the mesh, a request traverses the client-side proxy and then the server-side proxy. In the default configuration of Istio 1.6.8 (that is, Istio with telemetry v2), the two proxies add about 3.12 ms and 3.13 ms to the 90th and 99th percentile latency, respectively, over the baseline data plane latency.
diff --git a/modules/ossm-recommended-resources.adoc b/modules/ossm-recommended-resources.adoc
@@ -0,0 +1,40 @@
+////
+This module included in the following assemblies:
+- /v2x/ossm-performance-scalability.adoc
+////
+
+[id="ossm-recommended-resources_{context}"]
+= Recommended compute resources
+
+By default, `spec.proxy` has the settings `cpu:10m` and  `memory:128M`.  If you are using Pilot, `spec.runtime.components.pilot` has the same default values.
+The settings in the following example are based on 1,000 services and 1,000 requests per second. You can change the values for `cpu` and `memory` as necessary based on your environment.
+
+.Example version 2.0 ServiceMeshControlPlane
+[source,yaml]
+----
+apiVersion: maistra.io/v2
+kind: ServiceMeshControlPlane
+metadata:
+  name: basic
+  namespace: istio-system
+spec:
+  version: v2.0
+  proxy:
+    runtime:
+      container:
+        resources:
+          requests:
+            cpu: 600m
+            memory: 50Mi
+          limits: {}
+
+  runtime:
+    components:
+      pilot:
+        container:
+          resources:
+            requests:
+              cpu: 1000m
+              memory: 1.6Gi
+            limits: {}
+----
diff --git a/service_mesh/v1x/customizing-installation-ossm.adoc b/service_mesh/v1x/customizing-installation-ossm.adoc
@@ -21,4 +21,4 @@ include::modules/ossm-updating-smcp.adoc[leveloffset=+1]
 
 * xref:../../service_mesh/v1x/prepare-to-deploy-applications-ossm.adoc#deploying-applications-ossm-v1x[Prepare to deploy applications] on {ProductName}.
 
-// point to the reference topic
+// point to the reference topic
diff --git a/service_mesh/v2x/customizing-installation-ossm.adoc b/service_mesh/v2x/customizing-installation-ossm.adoc
@@ -4,7 +4,7 @@ include::modules/ossm-document-attributes.adoc[]
 :context: customizing-installation-ossm-v2x
 toc::[]
 
-After your default `ServiceMeshControlPlane` resource is deployed, you can configure the resource to suit your environment.
+After your default `ServiceMeshControlPlane` resource is deployed, you must configure the resource to suit your environment.  Note that the default Jaeger deployment must be changed, as the default `allinone` deployment does not supply persistent memory.
 
 == Resources for configuring your `ServiceMeshControlPlane` resource
 

diff --git a/service_mesh/v2x/installing-ossm.adoc b/service_mesh/v2x/installing-ossm.adoc
@@ -51,6 +51,11 @@ include::modules/ossm-update-app-sidecar.adoc[leveloffset=+2]
 
 == Next steps
 
+[NOTE]
+====
+The default `ServiceMeshControlPlane` settings are not intended for production use; they are designed to install successfully on a default OpenShift installation, which is an extremely resource limited environment.   After you have verified a successful SMCP installation, you should modify the settings defined within the SMCP to suit your environment.
+====
+
 * xref:../../service_mesh/v2x/customizing-installation-ossm.adoc#customize-installation-ossm-v2x[Customize the {ProductName} installation].
 
 * xref:../../service_mesh/v2x/prepare-to-deploy-applications-ossm.adoc#deploying-applications-ossm[Prepare to deploy applications] on {ProductName}.
diff --git a/service_mesh/v2x/ossm-extensions.adoc b/service_mesh/v2x/ossm-extensions.adoc
@@ -3,6 +3,8 @@
 include::modules/ossm-document-attributes.adoc[]
 :context: ossm-extensions
 
+toc::[]
+
 You can use WebAssembly extensions to add new features directly into the {ProductName} proxies, allowing you to move even more common functionality out of your applications, and implement them in a single language that compiles to WebAssembly bytecode.
 
 == WebAssembly extensions

diff --git a/service_mesh/v2x/ossm-performance-scalability.adoc b/service_mesh/v2x/ossm-performance-scalability.adoc
@@ -0,0 +1,15 @@
+[id="ossm-performance-scalability"]
+= Performance and scalability
+:context: performance-scalability
+
+toc::[]
+
+The default `ServiceMeshControlPlane` settings are not intended for production use; they are designed to install successfully on a default OpenShift installation, which is an extremely resource limited environment.   After you have verified a successful SMCP installation, you should modify the settings defined within the SMCP to suit your environment.  This section provides guidance for configuring {ProductName} for production use.
+
+See xref:../../service_mesh/v2x/ossm-custom-resources.adoc#ossm-custom-resources[Custom resources] for more information about the configurable fields in the `ServiceMeshControlPlane` resource.
+
+// The following include statements pull in the module files that comprise the assembly.
+
+include::modules/ossm-recommended-resources.adoc[leveloffset=+1]
+
+include::modules/ossm-load-test-results.adoc[leveloffset=+1]
diff --git a/service_mesh/v2x/prepare-to-deploy-applications-ossm.adoc b/service_mesh/v2x/prepare-to-deploy-applications-ossm.adoc
@@ -7,6 +7,11 @@ toc::[]
 
 When you deploy an application into the {ProductShortName}, there are several differences between the behavior of applications in the upstream community version of Istio and the behavior of applications within a {ProductName} installation.
 
+[NOTE]
+====
+Do not deploy applications within the {ProductShortName} control plane namespace. The control plane (service mesh Operators, Istiod, and `ServiceMeshControlPlane`) and data plane (applications and Envoy proxy) must be in separate namespaces.
+====
+
 == Prerequisites
 
 * Review xref:../../service_mesh/v2x/ossm-vs-community.adoc#ossm-vs-community[{ProductName} and Istio differences]
Original file line number	Diff line number	Diff line change
Expand Up		@@ -21,4 +21,4 @@ include::modules/ossm-updating-smcp.adoc[leveloffset=+1]

		* xref:../../service_mesh/v1x/prepare-to-deploy-applications-ossm.adoc#deploying-applications-ossm-v1x[Prepare to deploy applications] on {ProductName}.

		// point to the reference topic
		// point to the reference topic