Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions _topic_map.yml
Original file line number Diff line number Diff line change
Expand Up @@ -3211,6 +3211,8 @@ Topics:
File: serverless-autoscaling-scale-bounds
- Name: Concurrency
File: serverless-autoscaling-concurrency
- Name: Autoscaling dashboard
File: serverless-autoscaling-dashboard
#
# Knative Eventing
- Name: Knative Eventing
Expand Down
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added images/serverless-autoscaling-dashboard-pods.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
[id="serverless-autoscaling-dashboard-activator-metrics_{context}"]
= Activator metrics

The *Activator* graphs *Request Concurrency*, *Request Count by Response Code (last minute)*, and *Response Time (last minute)* show the timeline of requests going through the activator until the activator is removed from the request path. These graphs can be used, for example, to evaluate whether response count and the returned HTTP codes match expectations.

image::serverless-autoscaling-dashboard-activator-request-concurrency.png[Activator: Request Concurrency]
image::serverless-autoscaling-dashboard-activator-request-count.png[Activator: Request Count by Response Code (last minute)]
image::serverless-autoscaling-dashboard-activator-response-time.png[Activator: Response Time (last minute)]
8 changes: 8 additions & 0 deletions modules/serverless-autoscaling-dashboard-navigating.adoc
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
[id="serverless-autoscaling-dashboard-navigating_{context}"]
= Navigating to the autoscaling dashboard

.Procedure

. In the {product-title} Web UI, go to the *Monitoring -> Dashboards* page.
. In the *Dashboard* field, select the *Knative Serving - Scaling Debugging* dashboard.
. Use the *Namespace*, *Configuration*, and *Revision* fields to specify the workload you want to examine.
13 changes: 13 additions & 0 deletions modules/serverless-autoscaling-dashboard-observed-concurrency.adoc
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
[id="serverless-autoscaling-dashboard-observed-concurrency_{context}"]
= Observed concurrency

The *Observed Concurrency* graph shows the timeline of a set of concurrency-related metrics, including:

* request concurrency
* panic concurrency
* target concurrency
* excess burst capacity

Note that `ExcessBurstCapacity` is a negative number, `-200` by default, that increases when a bursty load appears. It is equal to the difference between spare capacity and the configured target burst capacity. If `ExcessBurstCapacity` is negative, then the activator is threaded in the request path by the `PodAutoscaler` controller.

image::serverless-autoscaling-dashboard-observed-concurrency.png[]
30 changes: 30 additions & 0 deletions modules/serverless-autoscaling-dashboard-observed-rps.adoc
Original file line number Diff line number Diff line change
@@ -0,0 +1,30 @@
[id="serverless-autoscaling-dashboard-observed-rps_{context}"]
= Requests per second

For requests-per-second (RPS) services, an additional *Observed RPS* dashboard is available, which visualizes different types of requests per second:

* `stable_requests_per_second`
* `panic_requests_per_second`
* `target_requests_per_second`

image::serverless-autoscaling-dashboard-observed-rps.png[Observed RPS]

Note that the *Observed RPS* dashboard is visible only if the requests annotations are set, for example:

[source,yaml]
----
apiVersion: serving.knative.dev/v1
kind: Service
metadata:
name: helloworld-go
namespace: default
spec:
template:
metadata:
annotations:
autoscaling.knative.dev/target: "10"
autoscaling.knative.dev/metric: "rps"
spec:
containers:
- image: docker.io/openshift/hello-openshift
----
6 changes: 6 additions & 0 deletions modules/serverless-autoscaling-dashboard-panic-mode.adoc
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
[id="serverless-autoscaling-dashboard-panic-mode_{context}"]
= Panic mode

The *Panic Mode* graph shows the timeline of times when the Knative service faces a bursty load, which causes the autoscaler to quickly adapt the Service pod number.

image::serverless-autoscaling-dashboard-panic-mode.png[Panic mode]
6 changes: 6 additions & 0 deletions modules/serverless-autoscaling-dashboard-pod-information.adoc
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
[id="serverless-autoscaling-dashboard-pod-information_{context}"]
= Pod information

The top of the *Knative Serving - Scaling Debugging* dashboard shows the counts of the requested pods, as well as of the pods in various stages of deployment. The *Revision Pod Counts (Timeline)* graph shows the same data visualized on the timeline. This information might be useful for general assessment of autoscaling by checking for problems with pod allocation.

image::serverless-autoscaling-dashboard-pods.png[Pod information]
6 changes: 6 additions & 0 deletions modules/serverless-autoscaling-dashboard-scrape-time.adoc
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
[id="serverless-autoscaling-dashboard-scrape-time_{context}"]
= Scrape time

The *Scrape Time* graph shows the timeline of scrape times for each revision. Since autoscaling makes scaling decisions based on the metrics coming from service pods, high scrape times might cause delays in autoscaling when workload changes.

image::serverless-autoscaling-dashboard-scrape-time.png[Scrape time]
1 change: 1 addition & 0 deletions serverless/autoscaling/images
53 changes: 53 additions & 0 deletions serverless/autoscaling/serverless-autoscaling-dashboard.adoc
Original file line number Diff line number Diff line change
@@ -0,0 +1,53 @@
[id="serverless-autoscaling-dashboard"]
= Autoscaling dashboard
include::modules/common-attributes.adoc[]
include::modules/serverless-document-attributes.adoc[]
:context: serverless-autoscaling-dashboard

toc::[]

You can use the *Knative Serving - Scaling Debugging* dashboard to examine detailed and visualized data for Knative Serving autoscaling. The dashboard is useful for several purposes:

* Troubleshooting your autoscaled workloads
* Improving understanding of how autoscaling works
* Determining why an application was autoscaled
* Evaluating resource footprint of an application, such as number of pods

[IMPORTANT]
====
Currently, this dashboard only supports the Knative pod autoscaler (KPA). It does not support the horizontal pod autoscaler (HPA).
====

The dashboard demonstrations in this section use an {product-title} cluster with the `autoscale-go` sample application installed. The load is generated using the `hey` load generator.

The sample application has a concurrency limit of 5 requests. When the limit is exceeded, autoscaling requests additional pods for Knative from Kubernetes.

.Additional resources

* See link:https://knative.dev/docs/serving/autoscaling/autoscale-go/[detailed information about autoscale-go].

include::modules/serverless-autoscaling-dashboard-navigating.adoc[leveloffset=+1]

.Additional resources

* See xref:../../serverless/admin_guide/serverless-admin-metrics.adoc#serverless-admin-metrics[detailed information about metrics].

include::modules/serverless-autoscaling-dashboard-pod-information.adoc[leveloffset=+1]

include::modules/serverless-autoscaling-dashboard-observed-concurrency.adoc[leveloffset=+1]

include::modules/serverless-autoscaling-dashboard-scrape-time.adoc[leveloffset=+1]

include::modules/serverless-autoscaling-dashboard-panic-mode.adoc[leveloffset=+1]

.Additional resources

* See an link:https://knative.dev/docs/serving/autoscaling/kpa-specific/#modes[overview of Knative Pod Autoscaling modes].

include::modules/serverless-autoscaling-dashboard-activator-metrics.adoc[leveloffset=+1]

.Additional resources

* See xref:../../serverless/admin_guide/serverless-admin-metrics.adoc#serverless-activator-metrics_serverless-admin-metrics[detailed information about activator metrics].

include::modules/serverless-autoscaling-dashboard-observed-rps.adoc[leveloffset=+1]