diff --git a/_topic_map.yml b/_topic_map.yml index 468e49171934..650af3516c9d 100644 --- a/_topic_map.yml +++ b/_topic_map.yml @@ -3211,6 +3211,8 @@ Topics: File: serverless-autoscaling-scale-bounds - Name: Concurrency File: serverless-autoscaling-concurrency + - Name: Autoscaling dashboard + File: serverless-autoscaling-dashboard # # Knative Eventing - Name: Knative Eventing diff --git a/images/serverless-autoscaling-dashboard-activator-request-concurrency.png b/images/serverless-autoscaling-dashboard-activator-request-concurrency.png new file mode 100644 index 000000000000..d58e827b060d Binary files /dev/null and b/images/serverless-autoscaling-dashboard-activator-request-concurrency.png differ diff --git a/images/serverless-autoscaling-dashboard-activator-request-count.png b/images/serverless-autoscaling-dashboard-activator-request-count.png new file mode 100644 index 000000000000..fa4f46db2379 Binary files /dev/null and b/images/serverless-autoscaling-dashboard-activator-request-count.png differ diff --git a/images/serverless-autoscaling-dashboard-activator-response-time.png b/images/serverless-autoscaling-dashboard-activator-response-time.png new file mode 100644 index 000000000000..d8e7a5c3b229 Binary files /dev/null and b/images/serverless-autoscaling-dashboard-activator-response-time.png differ diff --git a/images/serverless-autoscaling-dashboard-observed-concurrency.png b/images/serverless-autoscaling-dashboard-observed-concurrency.png new file mode 100644 index 000000000000..381984313bc1 Binary files /dev/null and b/images/serverless-autoscaling-dashboard-observed-concurrency.png differ diff --git a/images/serverless-autoscaling-dashboard-observed-rps.png b/images/serverless-autoscaling-dashboard-observed-rps.png new file mode 100644 index 000000000000..96e155513a39 Binary files /dev/null and b/images/serverless-autoscaling-dashboard-observed-rps.png differ diff --git a/images/serverless-autoscaling-dashboard-panic-mode.png b/images/serverless-autoscaling-dashboard-panic-mode.png new file mode 100644 index 000000000000..8580590e806a Binary files /dev/null and b/images/serverless-autoscaling-dashboard-panic-mode.png differ diff --git a/images/serverless-autoscaling-dashboard-pods.png b/images/serverless-autoscaling-dashboard-pods.png new file mode 100644 index 000000000000..b89353348fab Binary files /dev/null and b/images/serverless-autoscaling-dashboard-pods.png differ diff --git a/images/serverless-autoscaling-dashboard-scrape-time.png b/images/serverless-autoscaling-dashboard-scrape-time.png new file mode 100644 index 000000000000..7486278f272a Binary files /dev/null and b/images/serverless-autoscaling-dashboard-scrape-time.png differ diff --git a/modules/serverless-autoscaling-dashboard-activator-metrics.adoc b/modules/serverless-autoscaling-dashboard-activator-metrics.adoc new file mode 100644 index 000000000000..e6a2d497a558 --- /dev/null +++ b/modules/serverless-autoscaling-dashboard-activator-metrics.adoc @@ -0,0 +1,8 @@ +[id="serverless-autoscaling-dashboard-activator-metrics_{context}"] += Activator metrics + +The *Activator* graphs *Request Concurrency*, *Request Count by Response Code (last minute)*, and *Response Time (last minute)* show the timeline of requests going through the activator until the activator is removed from the request path. These graphs can be used, for example, to evaluate whether response count and the returned HTTP codes match expectations. + +image::serverless-autoscaling-dashboard-activator-request-concurrency.png[Activator: Request Concurrency] +image::serverless-autoscaling-dashboard-activator-request-count.png[Activator: Request Count by Response Code (last minute)] +image::serverless-autoscaling-dashboard-activator-response-time.png[Activator: Response Time (last minute)] diff --git a/modules/serverless-autoscaling-dashboard-navigating.adoc b/modules/serverless-autoscaling-dashboard-navigating.adoc new file mode 100644 index 000000000000..7f70d1da6a59 --- /dev/null +++ b/modules/serverless-autoscaling-dashboard-navigating.adoc @@ -0,0 +1,8 @@ +[id="serverless-autoscaling-dashboard-navigating_{context}"] += Navigating to the autoscaling dashboard + +.Procedure + +. In the {product-title} Web UI, go to the *Monitoring -> Dashboards* page. +. In the *Dashboard* field, select the *Knative Serving - Scaling Debugging* dashboard. +. Use the *Namespace*, *Configuration*, and *Revision* fields to specify the workload you want to examine. diff --git a/modules/serverless-autoscaling-dashboard-observed-concurrency.adoc b/modules/serverless-autoscaling-dashboard-observed-concurrency.adoc new file mode 100644 index 000000000000..3ed05ad2a42d --- /dev/null +++ b/modules/serverless-autoscaling-dashboard-observed-concurrency.adoc @@ -0,0 +1,13 @@ +[id="serverless-autoscaling-dashboard-observed-concurrency_{context}"] += Observed concurrency + +The *Observed Concurrency* graph shows the timeline of a set of concurrency-related metrics, including: + +* request concurrency +* panic concurrency +* target concurrency +* excess burst capacity + +Note that `ExcessBurstCapacity` is a negative number, `-200` by default, that increases when a bursty load appears. It is equal to the difference between spare capacity and the configured target burst capacity. If `ExcessBurstCapacity` is negative, then the activator is threaded in the request path by the `PodAutoscaler` controller. + +image::serverless-autoscaling-dashboard-observed-concurrency.png[] diff --git a/modules/serverless-autoscaling-dashboard-observed-rps.adoc b/modules/serverless-autoscaling-dashboard-observed-rps.adoc new file mode 100644 index 000000000000..deaaf93198af --- /dev/null +++ b/modules/serverless-autoscaling-dashboard-observed-rps.adoc @@ -0,0 +1,30 @@ +[id="serverless-autoscaling-dashboard-observed-rps_{context}"] += Requests per second + +For requests-per-second (RPS) services, an additional *Observed RPS* dashboard is available, which visualizes different types of requests per second: + +* `stable_requests_per_second` +* `panic_requests_per_second` +* `target_requests_per_second` + +image::serverless-autoscaling-dashboard-observed-rps.png[Observed RPS] + +Note that the *Observed RPS* dashboard is visible only if the requests annotations are set, for example: + +[source,yaml] +---- +apiVersion: serving.knative.dev/v1 +kind: Service +metadata: + name: helloworld-go + namespace: default +spec: + template: + metadata: + annotations: + autoscaling.knative.dev/target: "10" + autoscaling.knative.dev/metric: "rps" + spec: + containers: + - image: docker.io/openshift/hello-openshift +---- diff --git a/modules/serverless-autoscaling-dashboard-panic-mode.adoc b/modules/serverless-autoscaling-dashboard-panic-mode.adoc new file mode 100644 index 000000000000..fdca049c6ffd --- /dev/null +++ b/modules/serverless-autoscaling-dashboard-panic-mode.adoc @@ -0,0 +1,6 @@ +[id="serverless-autoscaling-dashboard-panic-mode_{context}"] += Panic mode + +The *Panic Mode* graph shows the timeline of times when the Knative service faces a bursty load, which causes the autoscaler to quickly adapt the Service pod number. + +image::serverless-autoscaling-dashboard-panic-mode.png[Panic mode] diff --git a/modules/serverless-autoscaling-dashboard-pod-information.adoc b/modules/serverless-autoscaling-dashboard-pod-information.adoc new file mode 100644 index 000000000000..edbc6d66f65a --- /dev/null +++ b/modules/serverless-autoscaling-dashboard-pod-information.adoc @@ -0,0 +1,6 @@ +[id="serverless-autoscaling-dashboard-pod-information_{context}"] += Pod information + +The top of the *Knative Serving - Scaling Debugging* dashboard shows the counts of the requested pods, as well as of the pods in various stages of deployment. The *Revision Pod Counts (Timeline)* graph shows the same data visualized on the timeline. This information might be useful for general assessment of autoscaling by checking for problems with pod allocation. + +image::serverless-autoscaling-dashboard-pods.png[Pod information] diff --git a/modules/serverless-autoscaling-dashboard-scrape-time.adoc b/modules/serverless-autoscaling-dashboard-scrape-time.adoc new file mode 100644 index 000000000000..9b367a01e284 --- /dev/null +++ b/modules/serverless-autoscaling-dashboard-scrape-time.adoc @@ -0,0 +1,6 @@ +[id="serverless-autoscaling-dashboard-scrape-time_{context}"] += Scrape time + +The *Scrape Time* graph shows the timeline of scrape times for each revision. Since autoscaling makes scaling decisions based on the metrics coming from service pods, high scrape times might cause delays in autoscaling when workload changes. + +image::serverless-autoscaling-dashboard-scrape-time.png[Scrape time] diff --git a/serverless/autoscaling/images b/serverless/autoscaling/images new file mode 120000 index 000000000000..5fa6987088da --- /dev/null +++ b/serverless/autoscaling/images @@ -0,0 +1 @@ +../../images \ No newline at end of file diff --git a/serverless/autoscaling/serverless-autoscaling-dashboard.adoc b/serverless/autoscaling/serverless-autoscaling-dashboard.adoc new file mode 100644 index 000000000000..69e706d4628e --- /dev/null +++ b/serverless/autoscaling/serverless-autoscaling-dashboard.adoc @@ -0,0 +1,53 @@ +[id="serverless-autoscaling-dashboard"] += Autoscaling dashboard +include::modules/common-attributes.adoc[] +include::modules/serverless-document-attributes.adoc[] +:context: serverless-autoscaling-dashboard + +toc::[] + +You can use the *Knative Serving - Scaling Debugging* dashboard to examine detailed and visualized data for Knative Serving autoscaling. The dashboard is useful for several purposes: + +* Troubleshooting your autoscaled workloads +* Improving understanding of how autoscaling works +* Determining why an application was autoscaled +* Evaluating resource footprint of an application, such as number of pods + +[IMPORTANT] +==== +Currently, this dashboard only supports the Knative pod autoscaler (KPA). It does not support the horizontal pod autoscaler (HPA). +==== + +The dashboard demonstrations in this section use an {product-title} cluster with the `autoscale-go` sample application installed. The load is generated using the `hey` load generator. + +The sample application has a concurrency limit of 5 requests. When the limit is exceeded, autoscaling requests additional pods for Knative from Kubernetes. + +.Additional resources + +* See link:https://knative.dev/docs/serving/autoscaling/autoscale-go/[detailed information about autoscale-go]. + +include::modules/serverless-autoscaling-dashboard-navigating.adoc[leveloffset=+1] + +.Additional resources + +* See xref:../../serverless/admin_guide/serverless-admin-metrics.adoc#serverless-admin-metrics[detailed information about metrics]. + +include::modules/serverless-autoscaling-dashboard-pod-information.adoc[leveloffset=+1] + +include::modules/serverless-autoscaling-dashboard-observed-concurrency.adoc[leveloffset=+1] + +include::modules/serverless-autoscaling-dashboard-scrape-time.adoc[leveloffset=+1] + +include::modules/serverless-autoscaling-dashboard-panic-mode.adoc[leveloffset=+1] + +.Additional resources + +* See an link:https://knative.dev/docs/serving/autoscaling/kpa-specific/#modes[overview of Knative Pod Autoscaling modes]. + +include::modules/serverless-autoscaling-dashboard-activator-metrics.adoc[leveloffset=+1] + +.Additional resources + +* See xref:../../serverless/admin_guide/serverless-admin-metrics.adoc#serverless-activator-metrics_serverless-admin-metrics[detailed information about activator metrics]. + +include::modules/serverless-autoscaling-dashboard-observed-rps.adoc[leveloffset=+1]