kube-apiserver metrics apiserver_request_latencies_summary NaN after upgrading to 1.6.1 #44329

mikkeloscar · 2017-04-11T11:55:55Z

What keywords did you search in Kubernetes issues before filing this one? (If you have found any duplicates, you should instead reply there.): metrics

BUG REPORT:

Kubernetes version (use kubectl version):

Client Version: version.Info{Major:"1", Minor:"6", GitVersion:"v1.6.0", GitCommit:"fff5156092b56e6bd60fff75aad4dc9de6b6ef37", GitTreeState:"clean", BuildDate:"2017-03-28T16:36:33Z", GoVersion:"go1.7.5", Compiler:"gc", Platform:"linux/amd64"}
Server Version: version.Info{Major:"1", Minor:"6", GitVersion:"v1.6.1+coreos.0", GitCommit:"9212f77ed8c169a0afa02e58dce87913c6387b3e", GitTreeState:"clean", BuildDate:"2017-04-04T00:32:53Z", GoVersion:"go1.7.5", Compiler:"gc", Platform:"linux/amd64"}

Environment:

Cloud provider or hardware configuration: AWS
OS (e.g. from /etc/os-release):

NAME="Container Linux by CoreOS"
ID=coreos
VERSION=1298.7.0
VERSION_ID=1298.7.0
BUILD_ID=2017-03-31-0215
PRETTY_NAME="Container Linux by CoreOS 1298.7.0 (Ladybug)"
ANSI_COLOR="38;5;75"
HOME_URL="https://coreos.com/"
BUG_REPORT_URL="https://github.com/coreos/bugs/issues"

Kernel (e.g. uname -a): Linux ip-172-31-1-171.eu-central-1.compute.internal 4.9.16-coreos-r1 #1 SMP Fri Mar 31 02:07:42 UTC 2017 x86_64 Intel(R) Xeon(R) CPU E5-2670 v2 @ 2.50GHz GenuineIntel GNU/Linux
Install tools: kubernetes-on-aws
Others:

What happened:

Since upgrading our clusters from v1.5.5 to v1.6.1 we no longer get any metrics for List pods percentiles:

apiserver_request_latencies_summary{resource="pods",verb="LIST",quantile="0.5"} NaN
apiserver_request_latencies_summary{resource="pods",verb="LIST",quantile="0.9"} NaN
apiserver_request_latencies_summary{resource="pods",verb="LIST",quantile="0.99"} NaN

and we are listing pods, so I would expect data:

apiserver_request_count{client="kubectl/v1.6.0 (linux/amd64) kubernetes/fff5156",code="200",contentType="application/json",resource="pods",verb="LIST"} 80

Graph showing no data after upgrading to v1.6.1 (different colors are just different kube-apiserver pods).

What you expected to happen:

I expect to get a real value for the latencies summary.

How to reproduce it (as minimally and precisely as possible):

We see it in multiple clusters, but I'm not able to reproduce in minikube for instance, but I'm not sure what could be a relevant difference for this issue to occur.

This is how we run the apiserver:
https://github.com/zalando-incubator/kubernetes-on-aws/blob/dev/cluster/userdata-master.yaml#L288-L311

Anything else we need to know:

I found #44233 (prometheus/prometheus#860) which might be related, but what I don't understand is that it changed after upgrading to v1.6.1, we still do about the same number of calls to the api, so I would expect similar values for the metric.

Fwiw other metrics also has the value NaN, but I didn't look too closely into those.

All metrics.

# HELP admission_quota_controller_adds Total number of adds handled by workqueue: admission_quota_controller # TYPE admission_quota_controller_adds counter admission_quota_controller_adds 16 # HELP admission_quota_controller_depth Current depth of workqueue: admission_quota_controller # TYPE admission_quota_controller_depth gauge admission_quota_controller_depth 0 # HELP admission_quota_controller_queue_latency How long an item stays in workqueueadmission_quota_controller before being requested. # TYPE admission_quota_controller_queue_latency summary admission_quota_controller_queue_latency{quantile="0.5"} NaN admission_quota_controller_queue_latency{quantile="0.9"} NaN admission_quota_controller_queue_latency{quantile="0.99"} NaN admission_quota_controller_queue_latency_sum 176 admission_quota_controller_queue_latency_count 16 # HELP admission_quota_controller_work_duration How long processing an item from workqueueadmission_quota_controller takes. # TYPE admission_quota_controller_work_duration summary admission_quota_controller_work_duration{quantile="0.5"} NaN admission_quota_controller_work_duration{quantile="0.9"} NaN admission_quota_controller_work_duration{quantile="0.99"} NaN admission_quota_controller_work_duration_sum 78708 admission_quota_controller_work_duration_count 16 # HELP apiserver_request_count Counter of apiserver requests broken out for each verb, API resource, client, and HTTP response contentType and code. # TYPE apiserver_request_count counter apiserver_request_count{client="Apache-HttpClient/4.5.2 (Java/1.8.0_121)",code="200",contentType="application/json",resource="pods",verb="GET"} 22 apiserver_request_count{client="Apache-HttpClient/4.5.2 (Java/1.8.0_121)",code="200",contentType="text/plain",resource="pods",verb="GET"} 25 apiserver_request_count{client="Apache-HttpClient/4.5.2 (Java/1.8.0_121)",code="201",contentType="application/json",resource="pods",verb="POST"} 1 apiserver_request_count{client="Browser",code="200",contentType="application/json",resource="services",verb="CONNECT"} 8 apiserver_request_count{client="Browser",code="200",contentType="application/octet-stream",resource="services",verb="CONNECT"} 3 apiserver_request_count{client="Browser",code="200",contentType="application/x-javascript",resource="services",verb="CONNECT"} 2 apiserver_request_count{client="Browser",code="200",contentType="image/png",resource="services",verb="CONNECT"} 1 apiserver_request_count{client="Browser",code="200",contentType="text/css; charset=utf-8",resource="services",verb="CONNECT"} 1 apiserver_request_count{client="Browser",code="200",contentType="text/html; charset=utf-8",resource="services",verb="CONNECT"} 1 apiserver_request_count{client="Browser",code="301",contentType="",resource="services",verb="CONNECT"} 1 apiserver_request_count{client="Go-http-client/1.1",code="200",contentType="application/json",resource="ingresses",verb="LIST"} 87143 apiserver_request_count{client="Go-http-client/1.1",code="200",contentType="application/json",resource="services",verb="GET"} 2.7014e+06 apiserver_request_count{client="HTTPie/0.9.9",code="200",contentType="application/json",resource="pods",verb="WATCH"} 303 apiserver_request_count{client="clm/v1.6.1 (linux/amd64) kubernetes/$Format",code="200",contentType="application/json",resource="nodes",verb="LIST"} 25 apiserver_request_count{client="clm/v1.6.1 (linux/amd64) kubernetes/$Format",code="200",contentType="application/json",resource="nodes",verb="WATCH"} 195 apiserver_request_count{client="clm/v1.6.1 (linux/amd64) kubernetes/$Format",code="200",contentType="application/json",resource="pods",verb="WATCH"} 10 apiserver_request_count{client="dashboard/v0.0.0 (linux/amd64) kubernetes/$Format",code="200",contentType="application/json",resource="services",verb="PROXY"} 36 apiserver_request_count{client="dashboard/v0.0.0 (linux/amd64) kubernetes/$Format",code="200",contentType="application/json",resource="services",verb="proxy"} 36 apiserver_request_count{client="dashboard/v0.0.0 (linux/amd64) kubernetes/$Format",code="200",contentType="application/vnd.kubernetes.protobuf",resource="configmaps",verb="LIST"} 1 apiserver_request_count{client="dashboard/v0.0.0 (linux/amd64) kubernetes/$Format",code="200",contentType="application/vnd.kubernetes.protobuf",resource="daemonsets",verb="LIST"} 2 apiserver_request_count{client="dashboard/v0.0.0 (linux/amd64) kubernetes/$Format",code="200",contentType="application/vnd.kubernetes.protobuf",resource="deployments",verb="GET"} 16 apiserver_request_count{client="dashboard/v0.0.0 (linux/amd64) kubernetes/$Format",code="200",contentType="application/vnd.kubernetes.protobuf",resource="deployments",verb="LIST"} 2 apiserver_request_count{client="dashboard/v0.0.0 (linux/amd64) kubernetes/$Format",code="200",contentType="application/vnd.kubernetes.protobuf",resource="events",verb="LIST"} 16 apiserver_request_count{client="dashboard/v0.0.0 (linux/amd64) kubernetes/$Format",code="200",contentType="application/vnd.kubernetes.protobuf",resource="horizontalpodautoscalers",verb="LIST"} 5 apiserver_request_count{client="dashboard/v0.0.0 (linux/amd64) kubernetes/$Format",code="200",contentType="application/vnd.kubernetes.protobuf",resource="jobs",verb="LIST"} 3 apiserver_request_count{client="dashboard/v0.0.0 (linux/amd64) kubernetes/$Format",code="200",contentType="application/vnd.kubernetes.protobuf",resource="namespaces",verb="WATCH"} 2 apiserver_request_count{client="dashboard/v0.0.0 (linux/amd64) kubernetes/$Format",code="200",contentType="application/vnd.kubernetes.protobuf",resource="pods",verb="GET"} 3 apiserver_request_count{client="dashboard/v0.0.0 (linux/amd64) kubernetes/$Format",code="200",contentType="application/vnd.kubernetes.protobuf",resource="pods",verb="WATCH"} 20 apiserver_request_count{client="dashboard/v0.0.0 (linux/amd64) kubernetes/$Format",code="200",contentType="application/vnd.kubernetes.protobuf",resource="replicasets",verb="GET"} 5 apiserver_request_count{client="dashboard/v0.0.0 (linux/amd64) kubernetes/$Format",code="200",contentType="application/vnd.kubernetes.protobuf",resource="replicasets",verb="LIST"} 10 apiserver_request_count{client="dashboard/v0.0.0 (linux/amd64) kubernetes/$Format",code="200",contentType="application/vnd.kubernetes.protobuf",resource="replicationcontrollers",verb="LIST"} 2 apiserver_request_count{client="dashboard/v0.0.0 (linux/amd64) kubernetes/$Format",code="200",contentType="application/vnd.kubernetes.protobuf",resource="services",verb="LIST"} 3 apiserver_request_count{client="dashboard/v0.0.0 (linux/amd64) kubernetes/$Format",code="200",contentType="application/vnd.kubernetes.protobuf",resource="statefulsets",verb="LIST"} 2 apiserver_request_count{client="dashboard/v0.0.0 (linux/amd64) kubernetes/$Format",code="200",contentType="application/vnd.kubernetes.protobuf",resource="thirdpartyresources",verb="LIST"} 2 apiserver_request_count{client="dashboard/v0.0.0 (linux/amd64) kubernetes/$Format",code="200",contentType="text/plain",resource="pods",verb="GET"} 1 apiserver_request_count{client="heapster/v0.0.0 (linux/amd64) kubernetes/$Format",code="200",contentType="application/vnd.kubernetes.protobuf",resource="namespaces",verb="WATCH"} 2 apiserver_request_count{client="heapster/v0.0.0 (linux/amd64) kubernetes/$Format",code="200",contentType="application/vnd.kubernetes.protobuf",resource="nodes",verb="WATCH"} 6 apiserver_request_count{client="heapster/v0.0.0 (linux/amd64) kubernetes/$Format",code="200",contentType="application/vnd.kubernetes.protobuf",resource="pods",verb="WATCH"} 2 apiserver_request_count{client="heapster/v0.0.0 (linux/amd64) kubernetes/$Format",code="200",contentType="application/vnd.kubernetes.protobuf;stream=watch",resource="namespaces",verb="WATCH"} 151 apiserver_request_count{client="heapster/v0.0.0 (linux/amd64) kubernetes/$Format",code="200",contentType="application/vnd.kubernetes.protobuf;stream=watch",resource="nodes",verb="WATCH"} 440 apiserver_request_count{client="heapster/v0.0.0 (linux/amd64) kubernetes/$Format",code="200",contentType="application/vnd.kubernetes.protobuf;stream=watch",resource="pods",verb="WATCH"} 144 apiserver_request_count{client="heapster/v1.4.0 (linux/amd64) kubernetes/$Format",code="200",contentType="application/json",resource="namespaces",verb="WATCH"} 1 apiserver_request_count{client="heapster/v1.4.0 (linux/amd64) kubernetes/$Format",code="200",contentType="application/json",resource="namespaces",verb="WATCHLIST"} 5 apiserver_request_count{client="heapster/v1.4.0 (linux/amd64) kubernetes/$Format",code="200",contentType="application/json",resource="nodes",verb="WATCH"} 3 apiserver_request_count{client="heapster/v1.4.0 (linux/amd64) kubernetes/$Format",code="200",contentType="application/json",resource="nodes",verb="WATCHLIST"} 13 apiserver_request_count{client="heapster/v1.4.0 (linux/amd64) kubernetes/$Format",code="200",contentType="application/json",resource="pods",verb="WATCH"} 1 apiserver_request_count{client="heapster/v1.4.0 (linux/amd64) kubernetes/$Format",code="200",contentType="application/json",resource="pods",verb="WATCHLIST"} 4 apiserver_request_count{client="hyperkube/v1.5.5+coreos.0 (linux/amd64) kubernetes/e2cd771",code="200",contentType="application/vnd.kubernetes.protobuf",resource="services",verb="LIST"} 4 apiserver_request_count{client="hyperkube/v1.5.5+coreos.0 (linux/amd64) kubernetes/e2cd771",code="200",contentType="application/vnd.kubernetes.protobuf",resource="services",verb="WATCH"} 3 apiserver_request_count{client="hyperkube/v1.5.5+coreos.0 (linux/amd64) kubernetes/e2cd771",code="200",contentType="application/vnd.kubernetes.protobuf;stream=watch",resource="endpoints",verb="WATCHLIST"} 15 apiserver_request_count{client="hyperkube/v1.5.5+coreos.0 (linux/amd64) kubernetes/e2cd771",code="200",contentType="application/vnd.kubernetes.protobuf;stream=watch",resource="services",verb="WATCHLIST"} 37 apiserver_request_count{client="hyperkube/v1.6.1+coreos.0 (linux/amd64) kubernetes/9212f77",code="200",contentType="application/vnd.kubernetes.protobuf",resource="clusterrolebindings",verb="GET"} 6 apiserver_request_count{client="hyperkube/v1.6.1+coreos.0 (linux/amd64) kubernetes/9212f77",code="200",contentType="application/vnd.kubernetes.protobuf",resource="clusterrolebindings",verb="LIST"} 1 apiserver_request_count{client="hyperkube/v1.6.1+coreos.0 (linux/amd64) kubernetes/9212f77",code="200",contentType="application/vnd.kubernetes.protobuf",resource="clusterrolebindings",verb="PUT"} 6 apiserver_request_count{client="hyperkube/v1.6.1+coreos.0 (linux/amd64) kubernetes/9212f77",code="200",contentType="application/vnd.kubernetes.protobuf",resource="clusterroles",verb="GET"} 9 apiserver_request_count{client="hyperkube/v1.6.1+coreos.0 (linux/amd64) kubernetes/9212f77",code="200",contentType="application/vnd.kubernetes.protobuf",resource="clusterroles",verb="LIST"} 1 apiserver_request_count{client="hyperkube/v1.6.1+coreos.0 (linux/amd64) kubernetes/9212f77",code="200",contentType="application/vnd.kubernetes.protobuf",resource="clusterroles",verb="PUT"} 9 apiserver_request_count{client="hyperkube/v1.6.1+coreos.0 (linux/amd64) kubernetes/9212f77",code="200",contentType="application/vnd.kubernetes.protobuf",resource="endpoints",verb="GET"} 6854 apiserver_request_count{client="hyperkube/v1.6.1+coreos.0 (linux/amd64) kubernetes/9212f77",code="200",contentType="application/vnd.kubernetes.protobuf",resource="endpoints",verb="LIST"} 1 apiserver_request_count{client="hyperkube/v1.6.1+coreos.0 (linux/amd64) kubernetes/9212f77",code="200",contentType="application/vnd.kubernetes.protobuf",resource="endpoints",verb="PUT"} 6823 apiserver_request_count{client="hyperkube/v1.6.1+coreos.0 (linux/amd64) kubernetes/9212f77",code="200",contentType="application/vnd.kubernetes.protobuf",resource="endpoints",verb="WATCH"} 3 apiserver_request_count{client="hyperkube/v1.6.1+coreos.0 (linux/amd64) kubernetes/9212f77",code="200",contentType="application/vnd.kubernetes.protobuf",resource="limitranges",verb="LIST"} 7 apiserver_request_count{client="hyperkube/v1.6.1+coreos.0 (linux/amd64) kubernetes/9212f77",code="200",contentType="application/vnd.kubernetes.protobuf",resource="namespaces",verb="GET"} 9161 apiserver_request_count{client="hyperkube/v1.6.1+coreos.0 (linux/amd64) kubernetes/9212f77",code="200",contentType="application/vnd.kubernetes.protobuf",resource="namespaces",verb="LIST"} 1 apiserver_request_count{client="hyperkube/v1.6.1+coreos.0 (linux/amd64) kubernetes/9212f77",code="200",contentType="application/vnd.kubernetes.protobuf",resource="nodes",verb="GET"} 4 apiserver_request_count{client="hyperkube/v1.6.1+coreos.0 (linux/amd64) kubernetes/9212f77",code="200",contentType="application/vnd.kubernetes.protobuf",resource="poddisruptionbudgets",verb="LIST"} 99 apiserver_request_count{client="hyperkube/v1.6.1+coreos.0 (linux/amd64) kubernetes/9212f77",code="200",contentType="application/vnd.kubernetes.protobuf",resource="podsecuritypolicies",verb="LIST"} 1 apiserver_request_count{client="hyperkube/v1.6.1+coreos.0 (linux/amd64) kubernetes/9212f77",code="200",contentType="application/vnd.kubernetes.protobuf",resource="resourcequotas",verb="LIST"} 10 apiserver_request_count{client="hyperkube/v1.6.1+coreos.0 (linux/amd64) kubernetes/9212f77",code="200",contentType="application/vnd.kubernetes.protobuf",resource="secrets",verb="LIST"} 1 apiserver_request_count{client="hyperkube/v1.6.1+coreos.0 (linux/amd64) kubernetes/9212f77",code="200",contentType="application/vnd.kubernetes.protobuf",resource="serviceaccounts",verb="LIST"} 1 apiserver_request_count{client="hyperkube/v1.6.1+coreos.0 (linux/amd64) kubernetes/9212f77",code="200",contentType="application/vnd.kubernetes.protobuf",resource="services",verb="GET"} 6854 apiserver_request_count{client="hyperkube/v1.6.1+coreos.0 (linux/amd64) kubernetes/9212f77",code="200",contentType="application/vnd.kubernetes.protobuf",resource="services",verb="LIST"} 9 apiserver_request_count{client="hyperkube/v1.6.1+coreos.0 (linux/amd64) kubernetes/9212f77",code="200",contentType="application/vnd.kubernetes.protobuf",resource="services",verb="WATCH"} 770 apiserver_request_count{client="hyperkube/v1.6.1+coreos.0 (linux/amd64) kubernetes/9212f77",code="200",contentType="application/vnd.kubernetes.protobuf",resource="storageclasses",verb="LIST"} 1 apiserver_request_count{client="hyperkube/v1.6.1+coreos.0 (linux/amd64) kubernetes/9212f77",code="200",contentType="application/vnd.kubernetes.protobuf",resource="thirdpartyresources",verb="LIST"} 6879 apiserver_request_count{client="hyperkube/v1.6.1+coreos.0 (linux/amd64) kubernetes/9212f77",code="200",contentType="application/vnd.kubernetes.protobuf;stream=watch",resource="endpoints",verb="WATCH"} 616 apiserver_request_count{client="hyperkube/v1.6.1+coreos.0 (linux/amd64) kubernetes/9212f77",code="200",contentType="application/vnd.kubernetes.protobuf;stream=watch",resource="limitranges",verb="WATCH"} 154 apiserver_request_count{client="hyperkube/v1.6.1+coreos.0 (linux/amd64) kubernetes/9212f77",code="200",contentType="application/vnd.kubernetes.protobuf;stream=watch",resource="namespaces",verb="WATCH"} 155 apiserver_request_count{client="hyperkube/v1.6.1+coreos.0 (linux/amd64) kubernetes/9212f77",code="200",contentType="application/vnd.kubernetes.protobuf;stream=watch",resource="podsecuritypolicies",verb="WATCH"} 151 apiserver_request_count{client="hyperkube/v1.6.1+coreos.0 (linux/amd64) kubernetes/9212f77",code="200",contentType="application/vnd.kubernetes.protobuf;stream=watch",resource="resourcequotas",verb="WATCH"} 153 apiserver_request_count{client="hyperkube/v1.6.1+coreos.0 (linux/amd64) kubernetes/9212f77",code="200",contentType="application/vnd.kubernetes.protobuf;stream=watch",resource="secrets",verb="WATCH"} 151 apiserver_request_count{client="hyperkube/v1.6.1+coreos.0 (linux/amd64) kubernetes/9212f77",code="200",contentType="application/vnd.kubernetes.protobuf;stream=watch",resource="serviceaccounts",verb="WATCH"} 154 apiserver_request_count{client="hyperkube/v1.6.1+coreos.0 (linux/amd64) kubernetes/9212f77",code="200",contentType="application/vnd.kubernetes.protobuf;stream=watch",resource="services",verb="WATCH"} 1038 apiserver_request_count{client="hyperkube/v1.6.1+coreos.0 (linux/amd64) kubernetes/9212f77",code="200",contentType="application/vnd.kubernetes.protobuf;stream=watch",resource="storageclasses",verb="WATCH"} 149 apiserver_request_count{client="hyperkube/v1.6.1+coreos.0 (linux/amd64) kubernetes/9212f77",code="201",contentType="application/vnd.kubernetes.protobuf",resource="clusterrolebindings",verb="POST"} 24 apiserver_request_count{client="hyperkube/v1.6.1+coreos.0 (linux/amd64) kubernetes/9212f77",code="201",contentType="application/vnd.kubernetes.protobuf",resource="clusterroles",verb="POST"} 30 apiserver_request_count{client="hyperkube/v1.6.1+coreos.0 (linux/amd64) kubernetes/9212f77",code="201",contentType="application/vnd.kubernetes.protobuf",resource="events",verb="POST"} 4 apiserver_request_count{client="hyperkube/v1.6.1+coreos.0 (linux/amd64) kubernetes/9212f77",code="201",contentType="application/vnd.kubernetes.protobuf",resource="namespaces",verb="POST"} 1 apiserver_request_count{client="hyperkube/v1.6.1+coreos.0 (linux/amd64) kubernetes/9212f77",code="201",contentType="application/vnd.kubernetes.protobuf",resource="rolebindings",verb="POST"} 3 apiserver_request_count{client="hyperkube/v1.6.1+coreos.0 (linux/amd64) kubernetes/9212f77",code="201",contentType="application/vnd.kubernetes.protobuf",resource="roles",verb="POST"} 4 apiserver_request_count{client="hyperkube/v1.6.1+coreos.0 (linux/amd64) kubernetes/9212f77",code="404",contentType="application/vnd.kubernetes.protobuf",resource="clusterrolebindings",verb="GET"} 24 apiserver_request_count{client="hyperkube/v1.6.1+coreos.0 (linux/amd64) kubernetes/9212f77",code="404",contentType="application/vnd.kubernetes.protobuf",resource="clusterroles",verb="GET"} 30 apiserver_request_count{client="hyperkube/v1.6.1+coreos.0 (linux/amd64) kubernetes/9212f77",code="404",contentType="application/vnd.kubernetes.protobuf",resource="namespaces",verb="GET"} 1 apiserver_request_count{client="hyperkube/v1.6.1+coreos.0 (linux/amd64) kubernetes/9212f77",code="404",contentType="application/vnd.kubernetes.protobuf",resource="rolebindings",verb="GET"} 3 apiserver_request_count{client="hyperkube/v1.6.1+coreos.0 (linux/amd64) kubernetes/9212f77",code="404",contentType="application/vnd.kubernetes.protobuf",resource="roles",verb="GET"} 4 apiserver_request_count{client="hyperkube/v1.6.1+coreos.0 (linux/amd64) kubernetes/9212f77/leader-election",code="200",contentType="application/vnd.kubernetes.protobuf",resource="endpoints",verb="GET"} 42344 apiserver_request_count{client="hyperkube/v1.6.1+coreos.0 (linux/amd64) kubernetes/9212f77/leader-election",code="200",contentType="application/vnd.kubernetes.protobuf",resource="nodes",verb="LIST"} 1 apiserver_request_count{client="hyperkube/v1.6.1+coreos.0 (linux/amd64) kubernetes/9212f77/leader-election",code="200",contentType="application/vnd.kubernetes.protobuf",resource="persistentvolumeclaims",verb="LIST"} 1 apiserver_request_count{client="hyperkube/v1.6.1+coreos.0 (linux/amd64) kubernetes/9212f77/leader-election",code="200",contentType="application/vnd.kubernetes.protobuf",resource="persistentvolumes",verb="LIST"} 1 apiserver_request_count{client="hyperkube/v1.6.1+coreos.0 (linux/amd64) kubernetes/9212f77/leader-election",code="200",contentType="application/vnd.kubernetes.protobuf",resource="pods",verb="LIST"} 2 apiserver_request_count{client="hyperkube/v1.6.1+coreos.0 (linux/amd64) kubernetes/9212f77/leader-election",code="200",contentType="application/vnd.kubernetes.protobuf",resource="replicasets",verb="LIST"} 1 apiserver_request_count{client="hyperkube/v1.6.1+coreos.0 (linux/amd64) kubernetes/9212f77/leader-election",code="200",contentType="application/vnd.kubernetes.protobuf",resource="replicationcontrollers",verb="LIST"} 1 apiserver_request_count{client="hyperkube/v1.6.1+coreos.0 (linux/amd64) kubernetes/9212f77/leader-election",code="200",contentType="application/vnd.kubernetes.protobuf",resource="services",verb="LIST"} 1 apiserver_request_count{client="hyperkube/v1.6.1+coreos.0 (linux/amd64) kubernetes/9212f77/leader-election",code="200",contentType="application/vnd.kubernetes.protobuf",resource="statefulsets",verb="LIST"} 1 apiserver_request_count{client="hyperkube/v1.6.1+coreos.0 (linux/amd64) kubernetes/9212f77/leader-election",code="200",contentType="application/vnd.kubernetes.protobuf;stream=watch",resource="nodes",verb="WATCH"} 155 apiserver_request_count{client="hyperkube/v1.6.1+coreos.0 (linux/amd64) kubernetes/9212f77/leader-election",code="200",contentType="application/vnd.kubernetes.protobuf;stream=watch",resource="persistentvolumeclaims",verb="WATCH"} 150 apiserver_request_count{client="hyperkube/v1.6.1+coreos.0 (linux/amd64) kubernetes/9212f77/leader-election",code="200",contentType="application/vnd.kubernetes.protobuf;stream=watch",resource="persistentvolumes",verb="WATCH"} 152 apiserver_request_count{client="hyperkube/v1.6.1+coreos.0 (linux/amd64) kubernetes/9212f77/leader-election",code="200",contentType="application/vnd.kubernetes.protobuf;stream=watch",resource="pods",verb="WATCH"} 311 apiserver_request_count{client="hyperkube/v1.6.1+coreos.0 (linux/amd64) kubernetes/9212f77/leader-election",code="200",contentType="application/vnd.kubernetes.protobuf;stream=watch",resource="replicasets",verb="WATCH"} 153 apiserver_request_count{client="hyperkube/v1.6.1+coreos.0 (linux/amd64) kubernetes/9212f77/leader-election",code="200",contentType="application/vnd.kubernetes.protobuf;stream=watch",resource="replicationcontrollers",verb="WATCH"} 150 apiserver_request_count{client="hyperkube/v1.6.1+coreos.0 (linux/amd64) kubernetes/9212f77/leader-election",code="200",contentType="application/vnd.kubernetes.protobuf;stream=watch",resource="services",verb="WATCH"} 154 apiserver_request_count{client="hyperkube/v1.6.1+coreos.0 (linux/amd64) kubernetes/9212f77/leader-election",code="200",contentType="application/vnd.kubernetes.protobuf;stream=watch",resource="statefulsets",verb="WATCH"} 153 apiserver_request_count{client="kube-dns/v0.0.0 (linux/amd64) kubernetes/$Format",code="200",contentType="application/vnd.kubernetes.protobuf",resource="endpoints",verb="WATCH"} 2 apiserver_request_count{client="kube-dns/v0.0.0 (linux/amd64) kubernetes/$Format",code="200",contentType="application/vnd.kubernetes.protobuf",resource="services",verb="WATCH"} 2 apiserver_request_count{client="kube-dns/v0.0.0 (linux/amd64) kubernetes/$Format",code="200",contentType="application/vnd.kubernetes.protobuf;stream=watch",resource="endpoints",verb="WATCHLIST"} 298 apiserver_request_count{client="kube-dns/v0.0.0 (linux/amd64) kubernetes/$Format",code="200",contentType="application/vnd.kubernetes.protobuf;stream=watch",resource="services",verb="WATCHLIST"} 300 apiserver_request_count{client="kube-dns/v1.6.0 (linux/amd64) kubernetes/3872cb9",code="200",contentType="application/json",resource="configmaps",verb="LIST"} 1 apiserver_request_count{client="kube-dns/v1.6.0 (linux/amd64) kubernetes/3872cb9",code="200",contentType="application/json",resource="configmaps",verb="WATCHLIST"} 4 apiserver_request_count{client="kube-dns/v1.6.0 (linux/amd64) kubernetes/3872cb9",code="200",contentType="application/json",resource="endpoints",verb="WATCH"} 1 apiserver_request_count{client="kube-dns/v1.6.0 (linux/amd64) kubernetes/3872cb9",code="200",contentType="application/json",resource="endpoints",verb="WATCHLIST"} 3 apiserver_request_count{client="kube-dns/v1.6.0 (linux/amd64) kubernetes/3872cb9",code="200",contentType="application/json",resource="services",verb="GET"} 1 apiserver_request_count{client="kube-dns/v1.6.0 (linux/amd64) kubernetes/3872cb9",code="200",contentType="application/json",resource="services",verb="WATCH"} 1 apiserver_request_count{client="kube-dns/v1.6.0 (linux/amd64) kubernetes/3872cb9",code="200",contentType="application/json",resource="services",verb="WATCHLIST"} 3 apiserver_request_count{client="kube-dns/v1.6.0 (linux/amd64) kubernetes/3872cb9",code="404",contentType="application/json",resource="configmaps",verb="GET"} 1 apiserver_request_count{client="kube-ingress-aws-controller",code="200",contentType="application/json",resource="ingresses",verb="LIST"} 1316 apiserver_request_count{client="kube2iam/v1.4.6+$Format:%h$ (linux/amd64) kubernetes/$Format",code="200",contentType="application/json",resource="pods",verb="WATCH"} 21 apiserver_request_count{client="kube2iam/v1.4.6+$Format:%h$ (linux/amd64) kubernetes/$Format",code="200",contentType="application/json",resource="pods",verb="WATCHLIST"} 483 apiserver_request_count{client="kubectl-v1.6.0/v1.6.0 (linux/amd64) kubernetes/fff5156",code="200",contentType="application/json",resource="cronjobs",verb="LIST"} 1 apiserver_request_count{client="kubectl-v1.6.0/v1.6.0 (linux/amd64) kubernetes/fff5156",code="200",contentType="application/json",resource="deployments",verb="LIST"} 15 apiserver_request_count{client="kubectl-v1.6.0/v1.6.0 (linux/amd64) kubernetes/fff5156",code="200",contentType="application/json",resource="nodes",verb="WATCH"} 4 apiserver_request_count{client="kubectl-v1.6.0/v1.6.0 (linux/amd64) kubernetes/fff5156",code="200",contentType="application/json",resource="pods",verb="GET"} 1 apiserver_request_count{client="kubectl-v1.6.0/v1.6.0 (linux/amd64) kubernetes/fff5156",code="200",contentType="application/json",resource="pods",verb="WATCH"} 30 apiserver_request_count{client="kubectl-v1.6.0/v1.6.0 (linux/amd64) kubernetes/fff5156",code="200",contentType="application/json",resource="secrets",verb="GET"} 1 apiserver_request_count{client="kubectl-v1.6.0/v1.6.0 (linux/amd64) kubernetes/fff5156",code="200",contentType="application/json",resource="services",verb="CONNECT"} 1 apiserver_request_count{client="kubectl-v1.6.0/v1.6.0 (linux/amd64) kubernetes/fff5156",code="200",contentType="text/plain",resource="pods",verb="GET"} 1 apiserver_request_count{client="kubectl/v0.0.0 (darwin/amd64) kubernetes/$Format",code="200",contentType="application/json",resource="pods",verb="WATCH"} 61 apiserver_request_count{client="kubectl/v1.5.1 (darwin/amd64) kubernetes/82450d0",code="200",contentType="application/json",resource="pods",verb="WATCH"} 1 apiserver_request_count{client="kubectl/v1.5.5+coreos.0 (linux/amd64) kubernetes/e2cd771",code="200",contentType="application/json",resource="daemonsets",verb="GET"} 36 apiserver_request_count{client="kubectl/v1.5.5+coreos.0 (linux/amd64) kubernetes/e2cd771",code="200",contentType="application/json",resource="nodes",verb="GET"} 3 apiserver_request_count{client="kubectl/v1.5.5+coreos.0 (linux/amd64) kubernetes/e2cd771",code="200",contentType="application/json",resource="pods",verb="WATCH"} 3 apiserver_request_count{client="kubectl/v1.5.5+coreos.0 (linux/amd64) kubernetes/e2cd771",code="404",contentType="application/json",resource="nodes",verb="GET"} 1 apiserver_request_count{client="kubectl/v1.6.0 (linux/amd64) kubernetes/fff5156",code="200",contentType="application/json",resource="configmaps",verb="GET"} 2 apiserver_request_count{client="kubectl/v1.6.0 (linux/amd64) kubernetes/fff5156",code="200",contentType="application/json",resource="configmaps",verb="PATCH"} 2 apiserver_request_count{client="kubectl/v1.6.0 (linux/amd64) kubernetes/fff5156",code="200",contentType="application/json",resource="cronjobs",verb="DELETE"} 1 apiserver_request_count{client="kubectl/v1.6.0 (linux/amd64) kubernetes/fff5156",code="200",contentType="application/json",resource="cronjobs",verb="LIST"} 1 apiserver_request_count{client="kubectl/v1.6.0 (linux/amd64) kubernetes/fff5156",code="200",contentType="application/json",resource="daemonsets",verb="GET"} 64 apiserver_request_count{client="kubectl/v1.6.0 (linux/amd64) kubernetes/fff5156",code="200",contentType="application/json",resource="daemonsets",verb="PATCH"} 4 apiserver_request_count{client="kubectl/v1.6.0 (linux/amd64) kubernetes/fff5156",code="200",contentType="application/json",resource="deployments",verb="GET"} 12 apiserver_request_count{client="kubectl/v1.6.0 (linux/amd64) kubernetes/fff5156",code="200",contentType="application/json",resource="deployments",verb="LIST"} 4 apiserver_request_count{client="kubectl/v1.6.0 (linux/amd64) kubernetes/fff5156",code="200",contentType="application/json",resource="deployments",verb="PATCH"} 12 apiserver_request_count{client="kubectl/v1.6.0 (linux/amd64) kubernetes/fff5156",code="200",contentType="application/json",resource="events",verb="LIST"} 2 apiserver_request_count{client="kubectl/v1.6.0 (linux/amd64) kubernetes/fff5156",code="200",contentType="application/json",resource="ingresses",verb="GET"} 2 apiserver_request_count{client="kubectl/v1.6.0 (linux/amd64) kubernetes/fff5156",code="200",contentType="application/json",resource="jobs",verb="GET"} 11 apiserver_request_count{client="kubectl/v1.6.0 (linux/amd64) kubernetes/fff5156",code="200",contentType="application/json",resource="limitranges",verb="GET"} 2 apiserver_request_count{client="kubectl/v1.6.0 (linux/amd64) kubernetes/fff5156",code="200",contentType="application/json",resource="limitranges",verb="PATCH"} 2 apiserver_request_count{client="kubectl/v1.6.0 (linux/amd64) kubernetes/fff5156",code="200",contentType="application/json",resource="namespaces",verb="GET"} 3 apiserver_request_count{client="kubectl/v1.6.0 (linux/amd64) kubernetes/fff5156",code="200",contentType="application/json",resource="namespaces",verb="WATCH"} 7 apiserver_request_count{client="kubectl/v1.6.0 (linux/amd64) kubernetes/fff5156",code="200",contentType="application/json",resource="nodes",verb="GET"} 10 apiserver_request_count{client="kubectl/v1.6.0 (linux/amd64) kubernetes/fff5156",code="200",contentType="application/json",resource="nodes",verb="LIST"} 79 apiserver_request_count{client="kubectl/v1.6.0 (linux/amd64) kubernetes/fff5156",code="200",contentType="application/json",resource="nodes",verb="PUT"} 5 apiserver_request_count{client="kubectl/v1.6.0 (linux/amd64) kubernetes/fff5156",code="200",contentType="application/json",resource="nodes",verb="WATCH"} 501 apiserver_request_count{client="kubectl/v1.6.0 (linux/amd64) kubernetes/fff5156",code="200",contentType="application/json",resource="platformcredentialssets",verb="GET"} 1 apiserver_request_count{client="kubectl/v1.6.0 (linux/amd64) kubernetes/fff5156",code="200",contentType="application/json",resource="platformcredentialssets",verb="LIST"} 1 apiserver_request_count{client="kubectl/v1.6.0 (linux/amd64) kubernetes/fff5156",code="200",contentType="application/json",resource="pods",verb="GET"} 594 apiserver_request_count{client="kubectl/v1.6.0 (linux/amd64) kubernetes/fff5156",code="200",contentType="application/json",resource="pods",verb="LIST"} 80 apiserver_request_count{client="kubectl/v1.6.0 (linux/amd64) kubernetes/fff5156",code="200",contentType="application/json",resource="pods",verb="WATCH"} 701 apiserver_request_count{client="kubectl/v1.6.0 (linux/amd64) kubernetes/fff5156",code="200",contentType="application/json",resource="podsecuritypolicies",verb="GET"} 2 apiserver_request_count{client="kubectl/v1.6.0 (linux/amd64) kubernetes/fff5156",code="200",contentType="application/json",resource="podsecuritypolicies",verb="PATCH"} 2 apiserver_request_count{client="kubectl/v1.6.0 (linux/amd64) kubernetes/fff5156",code="200",contentType="application/json",resource="replicasets",verb="GET"} 104 apiserver_request_count{client="kubectl/v1.6.0 (linux/amd64) kubernetes/fff5156",code="200",contentType="application/json",resource="secrets",verb="GET"} 1 apiserver_request_count{client="kubectl/v1.6.0 (linux/amd64) kubernetes/fff5156",code="200",contentType="application/json",resource="secrets",verb="PATCH"} 1 apiserver_request_count{client="kubectl/v1.6.0 (linux/amd64) kubernetes/fff5156",code="200",contentType="application/json",resource="services",verb="GET"} 2 apiserver_request_count{client="kubectl/v1.6.0 (linux/amd64) kubernetes/fff5156",code="200",contentType="application/json",resource="services",verb="PATCH"} 2 apiserver_request_count{client="kubectl/v1.6.0 (linux/amd64) kubernetes/fff5156",code="200",contentType="application/json",resource="statefulsets",verb="GET"} 18 apiserver_request_count{client="kubectl/v1.6.0 (linux/amd64) kubernetes/fff5156",code="200",contentType="application/json",resource="storageclasses",verb="GET"} 1 apiserver_request_count{client="kubectl/v1.6.0 (linux/amd64) kubernetes/fff5156",code="200",contentType="application/json",resource="storageclasses",verb="PATCH"} 1 apiserver_request_count{client="kubectl/v1.6.0 (linux/amd64) kubernetes/fff5156",code="200",contentType="application/json",resource="thirdpartyresources",verb="GET"} 1 apiserver_request_count{client="kubectl/v1.6.0 (linux/amd64) kubernetes/fff5156",code="200",contentType="application/json",resource="thirdpartyresources",verb="PATCH"} 1 apiserver_request_count{client="kubectl/v1.6.0 (linux/amd64) kubernetes/fff5156",code="200",contentType="text/plain",resource="pods",verb="GET"} 12 apiserver_request_count{client="kubectl/v1.6.0 (linux/amd64) kubernetes/fff5156",code="201",contentType="application/json",resource="cronjobs",verb="POST"} 2 apiserver_request_count{client="kubectl/v1.6.0 (linux/amd64) kubernetes/fff5156",code="201",contentType="application/json",resource="deployments",verb="POST"} 1 apiserver_request_count{client="kubectl/v1.6.0 (linux/amd64) kubernetes/fff5156",code="201",contentType="application/json",resource="pods",verb="POST"} 85 apiserver_request_count{client="kubectl/v1.6.0 (linux/amd64) kubernetes/fff5156",code="400",contentType="application/json",resource="pods",verb="GET"} 2 apiserver_request_count{client="kubectl/v1.6.0 (linux/amd64) kubernetes/fff5156",code="404",contentType="application/json",resource="cronjobs",verb="GET"} 2 apiserver_request_count{client="kubectl/v1.6.0 (linux/amd64) kubernetes/fff5156",code="404",contentType="application/json",resource="deployments",verb="GET"} 1 apiserver_request_count{client="kubectl/v1.6.0 (linux/amd64) kubernetes/fff5156",code="404",contentType="application/json",resource="pods",verb="GET"} 55 apiserver_request_count{client="kubectl/v1.6.1+coreos.0 (linux/amd64) kubernetes/9212f77",code="200",contentType="application/json",resource="daemonsets",verb="GET"} 12 apiserver_request_count{client="kubectl/v1.6.1+coreos.0 (linux/amd64) kubernetes/9212f77",code="200",contentType="application/json",resource="jobs",verb="GET"} 1 apiserver_request_count{client="kubectl/v1.6.1+coreos.0 (linux/amd64) kubernetes/9212f77",code="200",contentType="application/json",resource="nodes",verb="GET"} 2 apiserver_request_count{client="kubectl/v1.6.1+coreos.0 (linux/amd64) kubernetes/9212f77",code="200",contentType="application/json",resource="nodes",verb="PUT"} 1 apiserver_request_count{client="kubectl/v1.6.1+coreos.0 (linux/amd64) kubernetes/9212f77",code="200",contentType="application/json",resource="pods",verb="GET"} 215 apiserver_request_count{client="kubectl/v1.6.1+coreos.0 (linux/amd64) kubernetes/9212f77",code="200",contentType="application/json",resource="pods",verb="WATCH"} 1 apiserver_request_count{client="kubectl/v1.6.1+coreos.0 (linux/amd64) kubernetes/9212f77",code="200",contentType="application/json",resource="replicasets",verb="GET"} 26 apiserver_request_count{client="kubectl/v1.6.1+coreos.0 (linux/amd64) kubernetes/9212f77",code="201",contentType="application/json",resource="pods",verb="POST"} 14 apiserver_request_count{client="kubectl/v1.6.1+coreos.0 (linux/amd64) kubernetes/9212f77",code="404",contentType="application/json",resource="pods",verb="GET"} 14 apiserver_request_count{client="kubelet/v1.5.5+coreos.0 (linux/amd64) kubernetes/e2cd771",code="200",contentType="application/vnd.kubernetes.protobuf",resource="configmaps",verb="GET"} 53 apiserver_request_count{client="kubelet/v1.5.5+coreos.0 (linux/amd64) kubernetes/e2cd771",code="200",contentType="application/vnd.kubernetes.protobuf",resource="events",verb="PATCH"} 16 apiserver_request_count{client="kubelet/v1.5.5+coreos.0 (linux/amd64) kubernetes/e2cd771",code="200",contentType="application/vnd.kubernetes.protobuf",resource="nodes",verb="GET"} 5 apiserver_request_count{client="kubelet/v1.5.5+coreos.0 (linux/amd64) kubernetes/e2cd771",code="200",contentType="application/vnd.kubernetes.protobuf",resource="nodes",verb="LIST"} 54 apiserver_request_count{client="kubelet/v1.5.5+coreos.0 (linux/amd64) kubernetes/e2cd771",code="200",contentType="application/vnd.kubernetes.protobuf",resource="nodes",verb="PUT"} 639 apiserver_request_count{client="kubelet/v1.5.5+coreos.0 (linux/amd64) kubernetes/e2cd771",code="200",contentType="application/vnd.kubernetes.protobuf",resource="nodes",verb="WATCH"} 588 apiserver_request_count{client="kubelet/v1.5.5+coreos.0 (linux/amd64) kubernetes/e2cd771",code="200",contentType="application/vnd.kubernetes.protobuf",resource="persistentvolumeclaims",verb="GET"} 120 apiserver_request_count{client="kubelet/v1.5.5+coreos.0 (linux/amd64) kubernetes/e2cd771",code="200",contentType="application/vnd.kubernetes.protobuf",resource="persistentvolumes",verb="GET"} 115 apiserver_request_count{client="kubelet/v1.5.5+coreos.0 (linux/amd64) kubernetes/e2cd771",code="200",contentType="application/vnd.kubernetes.protobuf",resource="pods",verb="DELETE"} 76 apiserver_request_count{client="kubelet/v1.5.5+coreos.0 (linux/amd64) kubernetes/e2cd771",code="200",contentType="application/vnd.kubernetes.protobuf",resource="pods",verb="GET"} 207 apiserver_request_count{client="kubelet/v1.5.5+coreos.0 (linux/amd64) kubernetes/e2cd771",code="200",contentType="application/vnd.kubernetes.protobuf",resource="pods",verb="LIST"} 2 apiserver_request_count{client="kubelet/v1.5.5+coreos.0 (linux/amd64) kubernetes/e2cd771",code="200",contentType="application/vnd.kubernetes.protobuf",resource="pods",verb="PUT"} 197 apiserver_request_count{client="kubelet/v1.5.5+coreos.0 (linux/amd64) kubernetes/e2cd771",code="200",contentType="application/vnd.kubernetes.protobuf",resource="pods",verb="WATCH"} 3 apiserver_request_count{client="kubelet/v1.5.5+coreos.0 (linux/amd64) kubernetes/e2cd771",code="200",contentType="application/vnd.kubernetes.protobuf",resource="secrets",verb="GET"} 2574 apiserver_request_count{client="kubelet/v1.5.5+coreos.0 (linux/amd64) kubernetes/e2cd771",code="200",contentType="application/vnd.kubernetes.protobuf",resource="services",verb="LIST"} 2 apiserver_request_count{client="kubelet/v1.5.5+coreos.0 (linux/amd64) kubernetes/e2cd771",code="200",contentType="application/vnd.kubernetes.protobuf",resource="services",verb="WATCH"} 5 apiserver_request_count{client="kubelet/v1.5.5+coreos.0 (linux/amd64) kubernetes/e2cd771",code="200",contentType="application/vnd.kubernetes.protobuf;stream=watch",resource="nodes",verb="WATCHLIST"} 18 apiserver_request_count{client="kubelet/v1.5.5+coreos.0 (linux/amd64) kubernetes/e2cd771",code="200",contentType="application/vnd.kubernetes.protobuf;stream=watch",resource="pods",verb="WATCHLIST"} 20 apiserver_request_count{client="kubelet/v1.5.5+coreos.0 (linux/amd64) kubernetes/e2cd771",code="200",contentType="application/vnd.kubernetes.protobuf;stream=watch",resource="services",verb="WATCHLIST"} 32 apiserver_request_count{client="kubelet/v1.5.5+coreos.0 (linux/amd64) kubernetes/e2cd771",code="201",contentType="application/vnd.kubernetes.protobuf",resource="events",verb="POST"} 200 apiserver_request_count{client="kubelet/v1.5.5+coreos.0 (linux/amd64) kubernetes/e2cd771",code="404",contentType="application/vnd.kubernetes.protobuf",resource="configmaps",verb="GET"} 208 apiserver_request_count{client="kubelet/v1.5.5+coreos.0 (linux/amd64) kubernetes/e2cd771",code="404",contentType="application/vnd.kubernetes.protobuf",resource="events",verb="PATCH"} 2 apiserver_request_count{client="kubelet/v1.5.5+coreos.0 (linux/amd64) kubernetes/e2cd771",code="404",contentType="application/vnd.kubernetes.protobuf",resource="pods",verb="DELETE"} 1 apiserver_request_count{client="kubelet/v1.5.5+coreos.0 (linux/amd64) kubernetes/e2cd771",code="404",contentType="application/vnd.kubernetes.protobuf",resource="pods",verb="GET"} 89 apiserver_request_count{client="kubelet/v1.5.5+coreos.0 (linux/amd64) kubernetes/e2cd771",code="409",contentType="application/vnd.kubernetes.protobuf",resource="pods",verb="PUT"} 2 apiserver_request_count{client="kubelet/v1.6.1+coreos.0 (linux/amd64) kubernetes/9212f77",code="200",contentType="application/vnd.kubernetes.protobuf",resource="configmaps",verb="GET"} 2255 apiserver_request_count{client="kubelet/v1.6.1+coreos.0 (linux/amd64) kubernetes/9212f77",code="200",contentType="application/vnd.kubernetes.protobuf",resource="events",verb="PATCH"} 214 apiserver_request_count{client="kubelet/v1.6.1+coreos.0 (linux/amd64) kubernetes/9212f77",code="200",contentType="application/vnd.kubernetes.protobuf",resource="nodes",verb="GET"} 34083 apiserver_request_count{client="kubelet/v1.6.1+coreos.0 (linux/amd64) kubernetes/9212f77",code="200",contentType="application/vnd.kubernetes.protobuf",resource="nodes",verb="LIST"} 1 apiserver_request_count{client="kubelet/v1.6.1+coreos.0 (linux/amd64) kubernetes/9212f77",code="200",contentType="application/vnd.kubernetes.protobuf",resource="nodes",verb="PATCH"} 34042 apiserver_request_count{client="kubelet/v1.6.1+coreos.0 (linux/amd64) kubernetes/9212f77",code="200",contentType="application/vnd.kubernetes.protobuf",resource="nodes",verb="WATCH"} 6 apiserver_request_count{client="kubelet/v1.6.1+coreos.0 (linux/amd64) kubernetes/9212f77",code="200",contentType="application/vnd.kubernetes.protobuf",resource="persistentvolumeclaims",verb="GET"} 3668 apiserver_request_count{client="kubelet/v1.6.1+coreos.0 (linux/amd64) kubernetes/9212f77",code="200",contentType="application/vnd.kubernetes.protobuf",resource="persistentvolumes",verb="GET"} 3674 apiserver_request_count{client="kubelet/v1.6.1+coreos.0 (linux/amd64) kubernetes/9212f77",code="200",contentType="application/vnd.kubernetes.protobuf",resource="pods",verb="DELETE"} 85 apiserver_request_count{client="kubelet/v1.6.1+coreos.0 (linux/amd64) kubernetes/9212f77",code="200",contentType="application/vnd.kubernetes.protobuf",resource="pods",verb="GET"} 1519 apiserver_request_count{client="kubelet/v1.6.1+coreos.0 (linux/amd64) kubernetes/9212f77",code="200",contentType="application/vnd.kubernetes.protobuf",resource="pods",verb="LIST"} 1 apiserver_request_count{client="kubelet/v1.6.1+coreos.0 (linux/amd64) kubernetes/9212f77",code="200",contentType="application/vnd.kubernetes.protobuf",resource="pods",verb="PUT"} 1517 apiserver_request_count{client="kubelet/v1.6.1+coreos.0 (linux/amd64) kubernetes/9212f77",code="200",contentType="application/vnd.kubernetes.protobuf",resource="pods",verb="WATCH"} 10 apiserver_request_count{client="kubelet/v1.6.1+coreos.0 (linux/amd64) kubernetes/9212f77",code="200",contentType="application/vnd.kubernetes.protobuf",resource="secrets",verb="GET"} 121153 apiserver_request_count{client="kubelet/v1.6.1+coreos.0 (linux/amd64) kubernetes/9212f77",code="200",contentType="application/vnd.kubernetes.protobuf",resource="services",verb="LIST"} 1 apiserver_request_count{client="kubelet/v1.6.1+coreos.0 (linux/amd64) kubernetes/9212f77",code="200",contentType="application/vnd.kubernetes.protobuf",resource="services",verb="WATCH"} 8 apiserver_request_count{client="kubelet/v1.6.1+coreos.0 (linux/amd64) kubernetes/9212f77",code="200",contentType="application/vnd.kubernetes.protobuf;stream=watch",resource="nodes",verb="WATCH"} 751 apiserver_request_count{client="kubelet/v1.6.1+coreos.0 (linux/amd64) kubernetes/9212f77",code="200",contentType="application/vnd.kubernetes.protobuf;stream=watch",resource="pods",verb="WATCH"} 979 apiserver_request_count{client="kubelet/v1.6.1+coreos.0 (linux/amd64) kubernetes/9212f77",code="200",contentType="application/vnd.kubernetes.protobuf;stream=watch",resource="services",verb="WATCH"} 1076 apiserver_request_count{client="kubelet/v1.6.1+coreos.0 (linux/amd64) kubernetes/9212f77",code="201",contentType="application/vnd.kubernetes.protobuf",resource="events",verb="POST"} 2617 apiserver_request_count{client="kubelet/v1.6.1+coreos.0 (linux/amd64) kubernetes/9212f77",code="201",contentType="application/vnd.kubernetes.protobuf",resource="nodes",verb="POST"} 8 apiserver_request_count{client="kubelet/v1.6.1+coreos.0 (linux/amd64) kubernetes/9212f77",code="201",contentType="application/vnd.kubernetes.protobuf",resource="pods",verb="POST"} 10 apiserver_request_count{client="kubelet/v1.6.1+coreos.0 (linux/amd64) kubernetes/9212f77",code="404",contentType="application/vnd.kubernetes.protobuf",resource="configmaps",verb="GET"} 12409 apiserver_request_count{client="kubelet/v1.6.1+coreos.0 (linux/amd64) kubernetes/9212f77",code="404",contentType="application/vnd.kubernetes.protobuf",resource="events",verb="PATCH"} 5 apiserver_request_count{client="kubelet/v1.6.1+coreos.0 (linux/amd64) kubernetes/9212f77",code="404",contentType="application/vnd.kubernetes.protobuf",resource="pods",verb="DELETE"} 11 apiserver_request_count{client="kubelet/v1.6.1+coreos.0 (linux/amd64) kubernetes/9212f77",code="404",contentType="application/vnd.kubernetes.protobuf",resource="pods",verb="GET"} 270 apiserver_request_count{client="kubelet/v1.6.1+coreos.0 (linux/amd64) kubernetes/9212f77",code="409",contentType="application/vnd.kubernetes.protobuf",resource="pods",verb="PUT"} 17 apiserver_request_count{client="mate/v0.0.0 (linux/amd64) kubernetes/$Format",code="200",contentType="application/json",resource="ingresses",verb="LIST"} 1081 apiserver_request_count{client="mate/v0.0.0 (linux/amd64) kubernetes/$Format",code="200",contentType="application/json",resource="ingresses",verb="WATCHLIST"} 26 apiserver_request_count{client="mate/v0.0.0 (linux/amd64) kubernetes/$Format",code="200",contentType="application/json",resource="services",verb="WATCH"} 1081 apiserver_request_count{client="mate/v0.0.0 (linux/amd64) kubernetes/$Format",code="200",contentType="application/json",resource="services",verb="WATCHLIST"} 23 apiserver_request_count{client="pod_nanny/v0.0.0 (linux/amd64) kubernetes/$Format",code="200",contentType="application/json",resource="deployments",verb="GET"} 2 apiserver_request_count{client="pod_nanny/v0.0.0 (linux/amd64) kubernetes/$Format",code="200",contentType="application/json",resource="deployments",verb="PUT"} 2 apiserver_request_count{client="pod_nanny/v0.0.0 (linux/amd64) kubernetes/$Format",code="200",contentType="application/json",resource="nodes",verb="WATCH"} 1 apiserver_request_count{client="pod_nanny/v0.0.0 (linux/amd64) kubernetes/$Format",code="200",contentType="application/json",resource="nodes",verb="WATCHLIST"} 4 apiserver_request_count{client="pod_nanny/v0.0.0 (linux/amd64) kubernetes/$Format",code="200",contentType="application/json",resource="pods",verb="GET"} 3174 apiserver_request_count{client="python-requests/2.12.4",code="200",contentType="application/json",resource="deployments",verb="DELETE"} 3 apiserver_request_count{client="python-requests/2.12.4",code="200",contentType="application/json",resource="deployments",verb="GET"} 1 apiserver_request_count{client="python-requests/2.12.4",code="200",contentType="application/json",resource="deployments",verb="PATCH"} 3 apiserver_request_count{client="python-requests/2.12.4",code="200",contentType="application/json",resource="ingresses",verb="GET"} 1 apiserver_request_count{client="python-requests/2.12.4",code="200",contentType="application/json",resource="ingresses",verb="PATCH"} 1 apiserver_request_count{client="python-requests/2.12.4",code="200",contentType="application/json",resource="platformcredentialssets",verb="GET"} 2 apiserver_request_count{client="python-requests/2.12.4",code="200",contentType="application/json",resource="platformcredentialssets",verb="PUT"} 2 apiserver_request_count{client="python-requests/2.12.4",code="200",contentType="application/json",resource="services",verb="GET"} 3 apiserver_request_count{client="python-requests/2.12.4",code="200",contentType="application/json",resource="services",verb="PATCH"} 3 apiserver_request_count{client="python-requests/2.12.4",code="200",contentType="application/json",resource="thirdpartyresources",verb="LIST"} 7 apiserver_request_count{client="python-requests/2.12.4",code="201",contentType="application/json",resource="deployments",verb="POST"} 3 apiserver_request_count{client="python-requests/2.12.4",code="404",contentType="application/json",resource="deployments",verb="DELETE"} 1 apiserver_request_count{client="python-requests/2.12.4",code="404",contentType="application/json",resource="deployments",verb="GET"} 3 apiserver_request_count{client="python-requests/2.12.5",code="200",contentType="application/json",resource="nodes",verb="LIST"} 1 apiserver_request_count{client="python-requests/2.12.5",code="200",contentType="application/json",resource="nodes",verb="WATCH"} 599 apiserver_request_count{client="python-requests/2.12.5",code="200",contentType="application/json",resource="pods",verb="LIST"} 1 apiserver_request_count{client="python-requests/2.12.5",code="200",contentType="application/json",resource="pods",verb="WATCH"} 599 apiserver_request_count{client="python-requests/2.13.0",code="200",contentType="application/json",resource="daemonsets",verb="LIST"} 3646 apiserver_request_count{client="python-requests/2.13.0",code="200",contentType="application/json",resource="deployments",verb="LIST"} 987 apiserver_request_count{client="python-requests/2.13.0",code="200",contentType="application/json",resource="endpoints",verb="LIST"} 76977 apiserver_request_count{client="python-requests/2.13.0",code="200",contentType="application/json",resource="jobs",verb="DELETE"} 181 apiserver_request_count{client="python-requests/2.13.0",code="200",contentType="application/json",resource="jobs",verb="LIST"} 8 apiserver_request_count{client="python-requests/2.13.0",code="200",contentType="application/json",resource="namespaces",verb="LIST"} 2 apiserver_request_count{client="python-requests/2.13.0",code="200",contentType="application/json",resource="namespaces",verb="WATCH"} 508 apiserver_request_count{client="python-requests/2.13.0",code="200",contentType="application/json",resource="nodes",verb="LIST"} 35 apiserver_request_count{client="python-requests/2.13.0",code="200",contentType="application/json",resource="nodes",verb="WATCH"} 12137 apiserver_request_count{client="python-requests/2.13.0",code="200",contentType="application/json",resource="pods",verb="DELETE"} 156 apiserver_request_count{client="python-requests/2.13.0",code="200",contentType="application/json",resource="pods",verb="LIST"} 47 apiserver_request_count{client="python-requests/2.13.0",code="200",contentType="application/json",resource="pods",verb="WATCH"} 17650 apiserver_request_count{client="python-requests/2.13.0",code="200",contentType="application/json",resource="replicasets",verb="LIST"} 3041 apiserver_request_count{client="python-requests/2.13.0",code="200",contentType="application/json",resource="services",verb="CONNECT"} 6482 apiserver_request_count{client="python-requests/2.13.0",code="200",contentType="application/json",resource="services",verb="LIST"} 3416 apiserver_request_count{client="python-requests/2.13.0",code="200",contentType="application/json",resource="statefulsets",verb="LIST"} 3425 apiserver_request_count{client="python-requests/2.13.0",code="200",contentType="text/html; charset=utf-8",resource="services",verb="CONNECT"} 1 apiserver_request_count{client="python-requests/2.13.0",code="301",contentType="",resource="services",verb="CONNECT"} 1 apiserver_request_count{client="python-requests/2.13.0",code="503",contentType="application/json",resource="services",verb="CONNECT"} 32 apiserver_request_count{client="rescheduler/v0.0.0 (linux/amd64) kubernetes/$Format",code="200",contentType="application/vnd.kubernetes.protobuf",resource="nodes",verb="LIST"} 1 apiserver_request_count{client="rescheduler/v0.0.0 (linux/amd64) kubernetes/$Format",code="200",contentType="application/vnd.kubernetes.protobuf",resource="pods",verb="LIST"} 3 apiserver_request_count{client="rescheduler/v0.0.0 (linux/amd64) kubernetes/$Format",code="200",contentType="application/vnd.kubernetes.protobuf;stream=watch",resource="nodes",verb="WATCH"} 152 apiserver_request_count{client="rescheduler/v0.0.0 (linux/amd64) kubernetes/$Format",code="200",contentType="application/vnd.kubernetes.protobuf;stream=watch",resource="pods",verb="WATCH"} 469 apiserver_request_count{client="secretary/v0.0.0 (linux/amd64) kubernetes/$Format",code="200",contentType="application/json",resource="namespaces",verb="WATCH"} 5 apiserver_request_count{client="secretary/v0.0.0 (linux/amd64) kubernetes/$Format",code="200",contentType="application/json",resource="secrets",verb="PUT"} 15 apiserver_request_count{client="secretary/v0.0.0 (linux/amd64) kubernetes/$Format",code="200",contentType="application/json",resource="serviceaccounts",verb="GET"} 15 apiserver_request_count{client="secretary/v1.6.1 (linux/amd64) kubernetes/$Format",code="200",contentType="application/json",resource="namespaces",verb="WATCH"} 111 apiserver_request_count{client="secretary/v1.6.1 (linux/amd64) kubernetes/$Format",code="200",contentType="application/json",resource="secrets",verb="PUT"} 333 apiserver_request_count{client="secretary/v1.6.1 (linux/amd64) kubernetes/$Format",code="200",contentType="application/json",resource="serviceaccounts",verb="GET"} 333 apiserver_request_count{client="unirest-java/1.3.11",code="200",contentType="application/json",resource="platformcredentialssets",verb="LIST"} 566 apiserver_request_count{client="unirest-java/1.3.11",code="200",contentType="application/json",resource="secrets",verb="GET"} 7888 apiserver_request_count{client="unirest-java/1.3.11",code="200",contentType="application/json",resource="secrets",verb="PUT"} 7961 # HELP apiserver_request_latencies Response latency distribution in microseconds for each verb, resource and client. # TYPE apiserver_request_latencies histogram apiserver_request_latencies_bucket{resource="clusterrolebindings",verb="GET",le="125000"} 30 apiserver_request_latencies_bucket{resource="clusterrolebindings",verb="GET",le="250000"} 30 apiserver_request_latencies_bucket{resource="clusterrolebindings",verb="GET",le="500000"} 30 apiserver_request_latencies_bucket{resource="clusterrolebindings",verb="GET",le="1e+06"} 30 apiserver_request_latencies_bucket{resource="clusterrolebindings",verb="GET",le="2e+06"} 30 apiserver_request_latencies_bucket{resource="clusterrolebindings",verb="GET",le="4e+06"} 30 apiserver_request_latencies_bucket{resource="clusterrolebindings",verb="GET",le="8e+06"} 30 apiserver_request_latencies_bucket{resource="clusterrolebindings",verb="GET",le="+Inf"} 30 apiserver_request_latencies_sum{resource="clusterrolebindings",verb="GET"} 37213 apiserver_request_latencies_count{resource="clusterrolebindings",verb="GET"} 30 apiserver_request_latencies_bucket{resource="clusterrolebindings",verb="LIST",le="125000"} 1 apiserver_request_latencies_bucket{resource="clusterrolebindings",verb="LIST",le="250000"} 1 apiserver_request_latencies_bucket{resource="clusterrolebindings",verb="LIST",le="500000"} 1 apiserver_request_latencies_bucket{resource="clusterrolebindings",verb="LIST",le="1e+06"} 1 apiserver_request_latencies_bucket{resource="clusterrolebindings",verb="LIST",le="2e+06"} 1 apiserver_request_latencies_bucket{resource="clusterrolebindings",verb="LIST",le="4e+06"} 1 apiserver_request_latencies_bucket{resource="clusterrolebindings",verb="LIST",le="8e+06"} 1 apiserver_request_latencies_bucket{resource="clusterrolebindings",verb="LIST",le="+Inf"} 1 apiserver_request_latencies_sum{resource="clusterrolebindings",verb="LIST"} 1546 apiserver_request_latencies_count{resource="clusterrolebindings",verb="LIST"} 1 apiserver_request_latencies_bucket{resource="clusterrolebindings",verb="POST",le="125000"} 24 apiserver_request_latencies_bucket{resource="clusterrolebindings",verb="POST",le="250000"} 24 apiserver_request_latencies_bucket{resource="clusterrolebindings",verb="POST",le="500000"} 24 apiserver_request_latencies_bucket{resource="clusterrolebindings",verb="POST",le="1e+06"} 24 apiserver_request_latencies_bucket{resource="clusterrolebindings",verb="POST",le="2e+06"} 24 apiserver_request_latencies_bucket{resource="clusterrolebindings",verb="POST",le="4e+06"} 24 apiserver_request_latencies_bucket{resource="clusterrolebindings",verb="POST",le="8e+06"} 24 apiserver_request_latencies_bucket{resource="clusterrolebindings",verb="POST",le="+Inf"} 24 apiserver_request_latencies_sum{resource="clusterrolebindings",verb="POST"} 206842 apiserver_request_latencies_count{resource="clusterrolebindings",verb="POST"} 24 apiserver_request_latencies_bucket{resource="clusterrolebindings",verb="PUT",le="125000"} 6 apiserver_request_latencies_bucket{resource="clusterrolebindings",verb="PUT",le="250000"} 6 apiserver_request_latencies_bucket{resource="clusterrolebindings",verb="PUT",le="500000"} 6 apiserver_request_latencies_bucket{resource="clusterrolebindings",verb="PUT",le="1e+06"} 6 apiserver_request_latencies_bucket{resource="clusterrolebindings",verb="PUT",le="2e+06"} 6 apiserver_request_latencies_bucket{resource="clusterrolebindings",verb="PUT",le="4e+06"} 6 apiserver_request_latencies_bucket{resource="clusterrolebindings",verb="PUT",le="8e+06"} 6 apiserver_request_latencies_bucket{resource="clusterrolebindings",verb="PUT",le="+Inf"} 6 apiserver_request_latencies_sum{resource="clusterrolebindings",verb="PUT"} 94392 apiserver_request_latencies_count{resource="clusterrolebindings",verb="PUT"} 6 apiserver_request_latencies_bucket{resource="clusterroles",verb="GET",le="125000"} 39 apiserver_request_latencies_bucket{resource="clusterroles",verb="GET",le="250000"} 39 apiserver_request_latencies_bucket{resource="clusterroles",verb="GET",le="500000"} 39 apiserver_request_latencies_bucket{resource="clusterroles",verb="GET",le="1e+06"} 39 apiserver_request_latencies_bucket{resource="clusterroles",verb="GET",le="2e+06"} 39 apiserver_request_latencies_bucket{resource="clusterroles",verb="GET",le="4e+06"} 39 apiserver_request_latencies_bucket{resource="clusterroles",verb="GET",le="8e+06"} 39 apiserver_request_latencies_bucket{resource="clusterroles",verb="GET",le="+Inf"} 39 apiserver_request_latencies_sum{resource="clusterroles",verb="GET"} 266494 apiserver_request_latencies_count{resource="clusterroles",verb="GET"} 39 apiserver_request_latencies_bucket{resource="clusterroles",verb="LIST",le="125000"} 1 apiserver_request_latencies_bucket{resource="clusterroles",verb="LIST",le="250000"} 1 apiserver_request_latencies_bucket{resource="clusterroles",verb="LIST",le="500000"} 1 apiserver_request_latencies_bucket{resource="clusterroles",verb="LIST",le="1e+06"} 1 apiserver_request_latencies_bucket{resource="clusterroles",verb="LIST",le="2e+06"} 1 apiserver_request_latencies_bucket{resource="clusterroles",verb="LIST",le="4e+06"} 1 apiserver_request_latencies_bucket{resource="clusterroles",verb="LIST",le="8e+06"} 1 apiserver_request_latencies_bucket{resource="clusterroles",verb="LIST",le="+Inf"} 1 apiserver_request_latencies_sum{resource="clusterroles",verb="LIST"} 3845 apiserver_request_latencies_count{resource="clusterroles",verb="LIST"} 1 apiserver_request_latencies_bucket{resource="clusterroles",verb="POST",le="125000"} 30 apiserver_request_latencies_bucket{resource="clusterroles",verb="POST",le="250000"} 30 apiserver_request_latencies_bucket{resource="clusterroles",verb="POST",le="500000"} 30 apiserver_request_latencies_bucket{resource="clusterroles",verb="POST",le="1e+06"} 30 apiserver_request_latencies_bucket{resource="clusterroles",verb="POST",le="2e+06"} 30 apiserver_request_latencies_bucket{resource="clusterroles",verb="POST",le="4e+06"} 30 apiserver_request_latencies_bucket{resource="clusterroles",verb="POST",le="8e+06"} 30 apiserver_request_latencies_bucket{resource="clusterroles",verb="POST",le="+Inf"} 30 apiserver_request_latencies_sum{resource="clusterroles",verb="POST"} 208038 apiserver_request_latencies_count{resource="clusterroles",verb="POST"} 30 apiserver_request_latencies_bucket{resource="clusterroles",verb="PUT",le="125000"} 7 apiserver_request_latencies_bucket{resource="clusterroles",verb="PUT",le="250000"} 8 apiserver_request_latencies_bucket{resource="clusterroles",verb="PUT",le="500000"} 9 apiserver_request_latencies_bucket{resource="clusterroles",verb="PUT",le="1e+06"} 9 apiserver_request_latencies_bucket{resource="clusterroles",verb="PUT",le="2e+06"} 9 apiserver_request_latencies_bucket{resource="clusterroles",verb="PUT",le="4e+06"} 9 apiserver_request_latencies_bucket{resource="clusterroles",verb="PUT",le="8e+06"} 9 apiserver_request_latencies_bucket{resource="clusterroles",verb="PUT",le="+Inf"} 9 apiserver_request_latencies_sum{resource="clusterroles",verb="PUT"} 657777 apiserver_request_latencies_count{resource="clusterroles",verb="PUT"} 9 apiserver_request_latencies_bucket{resource="configmaps",verb="GET",le="125000"} 13999 apiserver_request_latencies_bucket{resource="configmaps",verb="GET",le="250000"} 14449 apiserver_request_latencies_bucket{resource="configmaps",verb="GET",le="500000"} 14774 apiserver_request_latencies_bucket{resource="configmaps",verb="GET",le="1e+06"} 14910 apiserver_request_latencies_bucket{resource="configmaps",verb="GET",le="2e+06"} 14928 apiserver_request_latencies_bucket{resource="configmaps",verb="GET",le="4e+06"} 14928 apiserver_request_latencies_bucket{resource="configmaps",verb="GET",le="8e+06"} 14928 apiserver_request_latencies_bucket{resource="configmaps",verb="GET",le="+Inf"} 14928 apiserver_request_latencies_sum{resource="configmaps",verb="GET"} 5.10270682e+08 apiserver_request_latencies_count{resource="configmaps",verb="GET"} 14928 apiserver_request_latencies_bucket{resource="configmaps",verb="LIST",le="125000"} 2 apiserver_request_latencies_bucket{resource="configmaps",verb="LIST",le="250000"} 2 apiserver_request_latencies_bucket{resource="configmaps",verb="LIST",le="500000"} 2 apiserver_request_latencies_bucket{resource="configmaps",verb="LIST",le="1e+06"} 2 apiserver_request_latencies_bucket{resource="configmaps",verb="LIST",le="2e+06"} 2 apiserver_request_latencies_bucket{resource="configmaps",verb="LIST",le="4e+06"} 2 apiserver_request_latencies_bucket{resource="configmaps",verb="LIST",le="8e+06"} 2 apiserver_request_latencies_bucket{resource="configmaps",verb="LIST",le="+Inf"} 2 apiserver_request_latencies_sum{resource="configmaps",verb="LIST"} 1947 apiserver_request_latencies_count{resource="configmaps",verb="LIST"} 2 apiserver_request_latencies_bucket{resource="configmaps",verb="PATCH",le="125000"} 2 apiserver_request_latencies_bucket{resource="configmaps",verb="PATCH",le="250000"} 2 apiserver_request_latencies_bucket{resource="configmaps",verb="PATCH",le="500000"} 2 apiserver_request_latencies_bucket{resource="configmaps",verb="PATCH",le="1e+06"} 2 apiserver_request_latencies_bucket{resource="configmaps",verb="PATCH",le="2e+06"} 2 apiserver_request_latencies_bucket{resource="configmaps",verb="PATCH",le="4e+06"} 2 apiserver_request_latencies_bucket{resource="configmaps",verb="PATCH",le="8e+06"} 2 apiserver_request_latencies_bucket{resource="configmaps",verb="PATCH",le="+Inf"} 2 apiserver_request_latencies_sum{resource="configmaps",verb="PATCH"} 140776 apiserver_request_latencies_count{resource="configmaps",verb="PATCH"} 2 apiserver_request_latencies_bucket{resource="configmaps",verb="WATCHLIST",le="125000"} 0 apiserver_request_latencies_bucket{resource="configmaps",verb="WATCHLIST",le="250000"} 0 apiserver_request_latencies_bucket{resource="configmaps",verb="WATCHLIST",le="500000"} 0 apiserver_request_latencies_bucket{resource="configmaps",verb="WATCHLIST",le="1e+06"} 0 apiserver_request_latencies_bucket{resource="configmaps",verb="WATCHLIST",le="2e+06"} 0 apiserver_request_latencies_bucket{resource="configmaps",verb="WATCHLIST",le="4e+06"} 0 apiserver_request_latencies_bucket{resource="configmaps",verb="WATCHLIST",le="8e+06"} 0 apiserver_request_latencies_bucket{resource="configmaps",verb="WATCHLIST",le="+Inf"} 4 apiserver_request_latencies_sum{resource="configmaps",verb="WATCHLIST"} 1.178198112e+09 apiserver_request_latencies_count{resource="configmaps",verb="WATCHLIST"} 4 apiserver_request_latencies_bucket{resource="cronjobs",verb="DELETE",le="125000"} 1 apiserver_request_latencies_bucket{resource="cronjobs",verb="DELETE",le="250000"} 1 apiserver_request_latencies_bucket{resource="cronjobs",verb="DELETE",le="500000"} 1 apiserver_request_latencies_bucket{resource="cronjobs",verb="DELETE",le="1e+06"} 1 apiserver_request_latencies_bucket{resource="cronjobs",verb="DELETE",le="2e+06"} 1 apiserver_request_latencies_bucket{resource="cronjobs",verb="DELETE",le="4e+06"} 1 apiserver_request_latencies_bucket{resource="cronjobs",verb="DELETE",le="8e+06"} 1 apiserver_request_latencies_bucket{resource="cronjobs",verb="DELETE",le="+Inf"} 1 apiserver_request_latencies_sum{resource="cronjobs",verb="DELETE"} 105882 apiserver_request_latencies_count{resource="cronjobs",verb="DELETE"} 1 apiserver_request_latencies_bucket{resource="cronjobs",verb="GET",le="125000"} 2 apiserver_request_latencies_bucket{resource="cronjobs",verb="GET",le="250000"} 2 apiserver_request_latencies_bucket{resource="cronjobs",verb="GET",le="500000"} 2 apiserver_request_latencies_bucket{resource="cronjobs",verb="GET",le="1e+06"} 2 apiserver_request_latencies_bucket{resource="cronjobs",verb="GET",le="2e+06"} 2 apiserver_request_latencies_bucket{resource="cronjobs",verb="GET",le="4e+06"} 2 apiserver_request_latencies_bucket{resource="cronjobs",verb="GET",le="8e+06"} 2 apiserver_request_latencies_bucket{resource="cronjobs",verb="GET",le="+Inf"} 2 apiserver_request_latencies_sum{resource="cronjobs",verb="GET"} 19176 apiserver_request_latencies_count{resource="cronjobs",verb="GET"} 2 apiserver_request_latencies_bucket{resource="cronjobs",verb="LIST",le="125000"} 2 apiserver_request_latencies_bucket{resource="cronjobs",verb="LIST",le="250000"} 2 apiserver_request_latencies_bucket{resource="cronjobs",verb="LIST",le="500000"} 2 apiserver_request_latencies_bucket{resource="cronjobs",verb="LIST",le="1e+06"} 2 apiserver_request_latencies_bucket{resource="cronjobs",verb="LIST",le="2e+06"} 2 apiserver_request_latencies_bucket{resource="cronjobs",verb="LIST",le="4e+06"} 2 apiserver_request_latencies_bucket{resource="cronjobs",verb="LIST",le="8e+06"} 2 apiserver_request_latencies_bucket{resource="cronjobs",verb="LIST",le="+Inf"} 2 apiserver_request_latencies_sum{resource="cronjobs",verb="LIST"} 92955 apiserver_request_latencies_count{resource="cronjobs",verb="LIST"} 2 apiserver_request_latencies_bucket{resource="cronjobs",verb="POST",le="125000"} 2 apiserver_request_latencies_bucket{resource="cronjobs",verb="POST",le="250000"} 2 apiserver_request_latencies_bucket{resource="cronjobs",verb="POST",le="500000"} 2 apiserver_request_latencies_bucket{resource="cronjobs",verb="POST",le="1e+06"} 2 apiserver_request_latencies_bucket{resource="cronjobs",verb="POST",le="2e+06"} 2 apiserver_request_latencies_bucket{resource="cronjobs",verb="POST",le="4e+06"} 2 apiserver_request_latencies_bucket{resource="cronjobs",verb="POST",le="8e+06"} 2 apiserver_request_latencies_bucket{resource="cronjobs",verb="POST",le="+Inf"} 2 apiserver_request_latencies_sum{resource="cronjobs",verb="POST"} 8842 apiserver_request_latencies_count{resource="cronjobs",verb="POST"} 2 apiserver_request_latencies_bucket{resource="daemonsets",verb="GET",le="125000"} 101 apiserver_request_latencies_bucket{resource="daemonsets",verb="GET",le="250000"} 108 apiserver_request_latencies_bucket{resource="daemonsets",verb="GET",le="500000"} 112 apiserver_request_latencies_bucket{resource="daemonsets",verb="GET",le="1e+06"} 112 apiserver_request_latencies_bucket{resource="daemonsets",verb="GET",le="2e+06"} 112 apiserver_request_latencies_bucket{resource="daemonsets",verb="GET",le="4e+06"} 112 apiserver_request_latencies_bucket{resource="daemonsets",verb="GET",le="8e+06"} 112 apiserver_request_latencies_bucket{resource="daemonsets",verb="GET",le="+Inf"} 112 apiserver_request_latencies_sum{resource="daemonsets",verb="GET"} 4.6151e+06 apiserver_request_latencies_count{resource="daemonsets",verb="GET"} 112 apiserver_request_latencies_bucket{resource="daemonsets",verb="LIST",le="125000"} 3572 apiserver_request_latencies_bucket{resource="daemonsets",verb="LIST",le="250000"} 3618 apiserver_request_latencies_bucket{resource="daemonsets",verb="LIST",le="500000"} 3639 apiserver_request_latencies_bucket{resource="daemonsets",verb="LIST",le="1e+06"} 3648 apiserver_request_latencies_bucket{resource="daemonsets",verb="LIST",le="2e+06"} 3648 apiserver_request_latencies_bucket{resource="daemonsets",verb="LIST",le="4e+06"} 3648 apiserver_request_latencies_bucket{resource="daemonsets",verb="LIST",le="8e+06"} 3648 apiserver_request_latencies_bucket{resource="daemonsets",verb="LIST",le="+Inf"} 3648 apiserver_request_latencies_sum{resource="daemonsets",verb="LIST"} 6.3406995e+07 apiserver_request_latencies_count{resource="daemonsets",verb="LIST"} 3648 apiserver_request_latencies_bucket{resource="daemonsets",verb="PATCH",le="125000"} 2 apiserver_request_latencies_bucket{resource="daemonsets",verb="PATCH",le="250000"} 2 apiserver_request_latencies_bucket{resource="daemonsets",verb="PATCH",le="500000"} 4 apiserver_request_latencies_bucket{resource="daemonsets",verb="PATCH",le="1e+06"} 4 apiserver_request_latencies_bucket{resource="daemonsets",verb="PATCH",le="2e+06"} 4 apiserver_request_latencies_bucket{resource="daemonsets",verb="PATCH",le="4e+06"} 4 apiserver_request_latencies_bucket{resource="daemonsets",verb="PATCH",le="8e+06"} 4 apiserver_request_latencies_bucket{resource="daemonsets",verb="PATCH",le="+Inf"} 4 apiserver_request_latencies_sum{resource="daemonsets",verb="PATCH"} 745497 apiserver_request_latencies_count{resource="daemonsets",verb="PATCH"} 4 apiserver_request_latencies_bucket{resource="deployments",verb="DELETE",le="125000"} 4 apiserver_request_latencies_bucket{resource="deployments",verb="DELETE",le="250000"} 4 apiserver_request_latencies_bucket{resource="deployments",verb="DELETE",le="500000"} 4 apiserver_request_latencies_bucket{resource="deployments",verb="DELETE",le="1e+06"} 4 apiserver_request_latencies_bucket{resource="deployments",verb="DELETE",le="2e+06"} 4 apiserver_request_latencies_bucket{resource="deployments",verb="DELETE",le="4e+06"} 4 apiserver_request_latencies_bucket{resource="deployments",verb="DELETE",le="8e+06"} 4 apiserver_request_latencies_bucket{resource="deployments",verb="DELETE",le="+Inf"} 4 apiserver_request_latencies_sum{resource="deployments",verb="DELETE"} 36392 apiserver_request_latencies_count{resource="deployments",verb="DELETE"} 4 apiserver_request_latencies_bucket{resource="deployments",verb="GET",le="125000"} 35 apiserver_request_latencies_bucket{resource="deployments",verb="GET",le="250000"} 35 apiserver_request_latencies_bucket{resource="deployments",verb="GET",le="500000"} 35 apiserver_request_latencies_bucket{resource="deployments",verb="GET",le="1e+06"} 35 apiserver_request_latencies_bucket{resource="deployments",verb="GET",le="2e+06"} 35 apiserver_request_latencies_bucket{resource="deployments",verb="GET",le="4e+06"} 35 apiserver_request_latencies_bucket{resource="deployments",verb="GET",le="8e+06"} 35 apiserver_request_latencies_bucket{resource="deployments",verb="GET",le="+Inf"} 35 apiserver_request_latencies_sum{resource="deployments",verb="GET"} 458918 apiserver_request_latencies_count{resource="deployments",verb="GET"} 35 apiserver_request_latencies_bucket{resource="deployments",verb="LIST",le="125000"} 730 apiserver_request_latencies_bucket{resource="deployments",verb="LIST",le="250000"} 851 apiserver_request_latencies_bucket{resource="deployments",verb="LIST",le="500000"} 978 apiserver_request_latencies_bucket{resource="deployments",verb="LIST",le="1e+06"} 999 apiserver_request_latencies_bucket{resource="deployments",verb="LIST",le="2e+06"} 1007 apiserver_request_latencies_bucket{resource="deployments",verb="LIST",le="4e+06"} 1008 apiserver_request_latencies_bucket{resource="deployments",verb="LIST",le="8e+06"} 1008 apiserver_request_latencies_bucket{resource="deployments",verb="LIST",le="+Inf"} 1008 apiserver_request_latencies_sum{resource="deployments",verb="LIST"} 1.12914081e+08 apiserver_request_latencies_count{resource="deployments",verb="LIST"} 1008 apiserver_request_latencies_bucket{resource="deployments",verb="PATCH",le="125000"} 15 apiserver_request_latencies_bucket{resource="deployments",verb="PATCH",le="250000"} 15 apiserver_request_latencies_bucket{resource="deployments",verb="PATCH",le="500000"} 15 apiserver_request_latencies_bucket{resource="deployments",verb="PATCH",le="1e+06"} 15 apiserver_request_latencies_bucket{resource="deployments",verb="PATCH",le="2e+06"} 15 apiserver_request_latencies_bucket{resource="deployments",verb="PATCH",le="4e+06"} 15 apiserver_request_latencies_bucket{resource="deployments",verb="PATCH",le="8e+06"} 15 apiserver_request_latencies_bucket{resource="deployments",verb="PATCH",le="+Inf"} 15 apiserver_request_latencies_sum{resource="deployments",verb="PATCH"} 382446 apiserver_request_latencies_count{resource="deployments",verb="PATCH"} 15 apiserver_request_latencies_bucket{resource="deployments",verb="POST",le="125000"} 4 apiserver_request_latencies_bucket{resource="deployments",verb="POST",le="250000"} 4 apiserver_request_latencies_bucket{resource="deployments",verb="POST",le="500000"} 4 apiserver_request_latencies_bucket{resource="deployments",verb="POST",le="1e+06"} 4 apiserver_request_latencies_bucket{resource="deployments",verb="POST",le="2e+06"} 4 apiserver_request_latencies_bucket{resource="deployments",verb="POST",le="4e+06"} 4 apiserver_request_latencies_bucket{resource="deployments",verb="POST",le="8e+06"} 4 apiserver_request_latencies_bucket{resource="deployments",verb="POST",le="+Inf"} 4 apiserver_request_latencies_sum{resource="deployments",verb="POST"} 109828 apiserver_request_latencies_count{resource="deployments",verb="POST"} 4 apiserver_request_latencies_bucket{resource="deployments",verb="PUT",le="125000"} 2 apiserver_request_latencies_bucket{resource="deployments",verb="PUT",le="250000"} 2 apiserver_request_latencies_bucket{resource="deployments",verb="PUT",le="500000"} 2 apiserver_request_latencies_bucket{resource="deployments",verb="PUT",le="1e+06"} 2 apiserver_request_latencies_bucket{resource="deployments",verb="PUT",le="2e+06"} 2 apiserver_request_latencies_bucket{resource="deployments",verb="PUT",le="4e+06"} 2 apiserver_request_latencies_bucket{resource="deployments",verb="PUT",le="8e+06"} 2 apiserver_request_latencies_bucket{resource="deployments",verb="PUT",le="+Inf"} 2 apiserver_request_latencies_sum{resource="deployments",verb="PUT"} 14939 apiserver_request_latencies_count{resource="deployments",verb="PUT"} 2 apiserver_request_latencies_bucket{resource="endpoints",verb="GET",le="125000"} 47132 apiserver_request_latencies_bucket{resource="endpoints",verb="GET",le="250000"} 48271 apiserver_request_latencies_bucket{resource="endpoints",verb="GET",le="500000"} 49006 apiserver_request_latencies_bucket{resource="endpoints",verb="GET",le="1e+06"} 49179 apiserver_request_latencies_bucket{resource="endpoints",verb="GET",le="2e+06"} 49197 apiserver_request_latencies_bucket{resource="endpoints",verb="GET",le="4e+06"} 49198 apiserver_request_latencies_bucket{resource="endpoints",verb="GET",le="8e+06"} 49198 apiserver_request_latencies_bucket{resource="endpoints",verb="GET",le="+Inf"} 49198 apiserver_request_latencies_sum{resource="endpoints",verb="GET"} 1.165225393e+09 apiserver_request_latencies_count{resource="endpoints",verb="GET"} 49198 apiserver_request_latencies_bucket{resource="endpoints",verb="LIST",le="125000"} 48676 apiserver_request_latencies_bucket{resource="endpoints",verb="LIST",le="250000"} 65859 apiserver_request_latencies_bucket{resource="endpoints",verb="LIST",le="500000"} 75036 apiserver_request_latencies_bucket{resource="endpoints",verb="LIST",le="1e+06"} 76878 apiserver_request_latencies_bucket{resource="endpoints",verb="LIST",le="2e+06"} 76976 apiserver_request_latencies_bucket{resource="endpoints",verb="LIST",le="4e+06"} 76978 apiserver_request_latencies_bucket{resource="endpoints",verb="LIST",le="8e+06"} 76978 apiserver_request_latencies_bucket{resource="endpoints",verb="LIST",le="+Inf"} 76978 apiserver_request_latencies_sum{resource="endpoints",verb="LIST"} 1.0344581295e+10 apiserver_request_latencies_count{resource="endpoints",verb="LIST"} 76978 apiserver_request_latencies_bucket{resource="endpoints",verb="PUT",le="125000"} 6643 apiserver_request_latencies_bucket{resource="endpoints",verb="PUT",le="250000"} 6740 apiserver_request_latencies_bucket{resource="endpoints",verb="PUT",le="500000"} 6818 apiserver_request_latencies_bucket{resource="endpoints",verb="PUT",le="1e+06"} 6823 apiserver_request_latencies_bucket{resource="endpoints",verb="PUT",le="2e+06"} 6823 apiserver_request_latencies_bucket{resource="endpoints",verb="PUT",le="4e+06"} 6823 apiserver_request_latencies_bucket{resource="endpoints",verb="PUT",le="8e+06"} 6823 apiserver_request_latencies_bucket{resource="endpoints",verb="PUT",le="+Inf"} 6823 apiserver_request_latencies_sum{resource="endpoints",verb="PUT"} 1.5347824e+08 apiserver_request_latencies_count{resource="endpoints",verb="PUT"} 6823 apiserver_request_latencies_bucket{resource="endpoints",verb="WATCH",le="125000"} 6 apiserver_request_latencies_bucket{resource="endpoints",verb="WATCH",le="250000"} 6 apiserver_request_latencies_bucket{resource="endpoints",verb="WATCH",le="500000"} 6 apiserver_request_latencies_bucket{resource="endpoints",verb="WATCH",le="1e+06"} 6 apiserver_request_latencies_bucket{resource="endpoints",verb="WATCH",le="2e+06"} 6 apiserver_request_latencies_bucket{resource="endpoints",verb="WATCH",le="4e+06"} 6 apiserver_request_latencies_bucket{resource="endpoints",verb="WATCH",le="8e+06"} 6 apiserver_request_latencies_bucket{resource="endpoints",verb="WATCH",le="+Inf"} 622 apiserver_request_latencies_sum{resource="endpoints",verb="WATCH"} 2.75517142166e+11 apiserver_request_latencies_count{resource="endpoints",verb="WATCH"} 622 apiserver_request_latencies_bucket{resource="endpoints",verb="WATCHLIST",le="125000"} 0 apiserver_request_latencies_bucket{resource="endpoints",verb="WATCHLIST",le="250000"} 0 apiserver_request_latencies_bucket{resource="endpoints",verb="WATCHLIST",le="500000"} 0 apiserver_request_latencies_bucket{resource="endpoints",verb="WATCHLIST",le="1e+06"} 0 apiserver_request_latencies_bucket{resource="endpoints",verb="WATCHLIST",le="2e+06"} 0 apiserver_request_latencies_bucket{resource="endpoints",verb="WATCHLIST",le="4e+06"} 0 apiserver_request_latencies_bucket{resource="endpoints",verb="WATCHLIST",le="8e+06"} 0 apiserver_request_latencies_bucket{resource="endpoints",verb="WATCHLIST",le="+Inf"} 316 apiserver_request_latencies_sum{resource="endpoints",verb="WATCHLIST"} 1.40073125175e+11 apiserver_request_latencies_count{resource="endpoints",verb="WATCHLIST"} 316 apiserver_request_latencies_bucket{resource="events",verb="LIST",le="125000"} 15 apiserver_request_latencies_bucket{resource="events",verb="LIST",le="250000"} 16 apiserver_request_latencies_bucket{resource="events",verb="LIST",le="500000"} 17 apiserver_request_latencies_bucket{resource="events",verb="LIST",le="1e+06"} 17 apiserver_request_latencies_bucket{resource="events",verb="LIST",le="2e+06"} 17 apiserver_request_latencies_bucket{resource="events",verb="LIST",le="4e+06"} 18 apiserver_request_latencies_bucket{resource="events",verb="LIST",le="8e+06"} 18 apiserver_request_latencies_bucket{resource="events",verb="LIST",le="+Inf"} 18 apiserver_request_latencies_sum{resource="events",verb="LIST"} 3.634932e+06 apiserver_request_latencies_count{resource="events",verb="LIST"} 18 apiserver_request_latencies_bucket{resource="events",verb="PATCH",le="125000"} 224 apiserver_request_latencies_bucket{resource="events",verb="PATCH",le="250000"} 233 apiserver_request_latencies_bucket{resource="events",verb="PATCH",le="500000"} 236 apiserver_request_latencies_bucket{resource="events",verb="PATCH",le="1e+06"} 237 apiserver_request_latencies_bucket{resource="events",verb="PATCH",le="2e+06"} 237 apiserver_request_latencies_bucket{resource="events",verb="PATCH",le="4e+06"} 237 apiserver_request_latencies_bucket{resource="events",verb="PATCH",le="8e+06"} 237 apiserver_request_latencies_bucket{resource="events",verb="PATCH",le="+Inf"} 237 apiserver_request_latencies_sum{resource="events",verb="PATCH"} 9.610014e+06 apiserver_request_latencies_count{resource="events",verb="PATCH"} 237 apiserver_request_latencies_bucket{resource="events",verb="POST",le="125000"} 2696 apiserver_request_latencies_bucket{resource="events",verb="POST",le="250000"} 2768 apiserver_request_latencies_bucket{resource="events",verb="POST",le="500000"} 2809 apiserver_request_latencies_bucket{resource="events",verb="POST",le="1e+06"} 2818 apiserver_request_latencies_bucket{resource="events",verb="POST",le="2e+06"} 2821 apiserver_request_latencies_bucket{resource="events",verb="POST",le="4e+06"} 2821 apiserver_request_latencies_bucket{resource="events",verb="POST",le="8e+06"} 2821 apiserver_request_latencies_bucket{resource="events",verb="POST",le="+Inf"} 2821 apiserver_request_latencies_sum{resource="events",verb="POST"} 8.1423503e+07 apiserver_request_latencies_count{resource="events",verb="POST"} 2821 apiserver_request_latencies_bucket{resource="horizontalpodautoscalers",verb="LIST",le="125000"} 5 apiserver_request_latencies_bucket{resource="horizontalpodautoscalers",verb="LIST",le="250000"} 5 apiserver_request_latencies_bucket{resource="horizontalpodautoscalers",verb="LIST",le="500000"} 5 apiserver_request_latencies_bucket{resource="horizontalpodautoscalers",verb="LIST",le="1e+06"} 5 apiserver_request_latencies_bucket{resource="horizontalpodautoscalers",verb="LIST",le="2e+06"} 5 apiserver_request_latencies_bucket{resource="horizontalpodautoscalers",verb="LIST",le="4e+06"} 5 apiserver_request_latencies_bucket{resource="horizontalpodautoscalers",verb="LIST",le="8e+06"} 5 apiserver_request_latencies_bucket{resource="horizontalpodautoscalers",verb="LIST",le="+Inf"} 5 apiserver_request_latencies_sum{resource="horizontalpodautoscalers",verb="LIST"} 25424 apiserver_request_latencies_count{resource="horizontalpodautoscalers",verb="LIST"} 5 apiserver_request_latencies_bucket{resource="ingresses",verb="GET",le="125000"} 3 apiserver_request_latencies_bucket{resource="ingresses",verb="GET",le="250000"} 3 apiserver_request_latencies_bucket{resource="ingresses",verb="GET",le="500000"} 3 apiserver_request_latencies_bucket{resource="ingresses",verb="GET",le="1e+06"} 3 apiserver_request_latencies_bucket{resource="ingresses",verb="GET",le="2e+06"} 3 apiserver_request_latencies_bucket{resource="ingresses",verb="GET",le="4e+06"} 3 apiserver_request_latencies_bucket{resource="ingresses",verb="GET",le="8e+06"} 3 apiserver_request_latencies_bucket{resource="ingresses",verb="GET",le="+Inf"} 3 apiserver_request_latencies_sum{resource="ingresses",verb="GET"} 96659 apiserver_request_latencies_count{resource="ingresses",verb="GET"} 3 apiserver_request_latencies_bucket{resource="ingresses",verb="LIST",le="125000"} 84065 apiserver_request_latencies_bucket{resource="ingresses",verb="LIST",le="250000"} 86940 apiserver_request_latencies_bucket{resource="ingresses",verb="LIST",le="500000"} 88870 apiserver_request_latencies_bucket{resource="ingresses",verb="LIST",le="1e+06"} 89465 apiserver_request_latencies_bucket{resource="ingresses",verb="LIST",le="2e+06"} 89532 apiserver_request_latencies_bucket{resource="ingresses",verb="LIST",le="4e+06"} 89540 apiserver_request_latencies_bucket{resource="ingresses",verb="LIST",le="8e+06"} 89540 apiserver_request_latencies_bucket{resource="ingresses",verb="LIST",le="+Inf"} 89540 apiserver_request_latencies_sum{resource="ingresses",verb="LIST"} 3.266173711e+09 apiserver_request_latencies_count{resource="ingresses",verb="LIST"} 89540 apiserver_request_latencies_bucket{resource="ingresses",verb="PATCH",le="125000"} 1 apiserver_request_latencies_bucket{resource="ingresses",verb="PATCH",le="250000"} 1 apiserver_request_latencies_bucket{resource="ingresses",verb="PATCH",le="500000"} 1 apiserver_request_latencies_bucket{resource="ingresses",verb="PATCH",le="1e+06"} 1 apiserver_request_latencies_bucket{resource="ingresses",verb="PATCH",le="2e+06"} 1 apiserver_request_latencies_bucket{resource="ingresses",verb="PATCH",le="4e+06"} 1 apiserver_request_latencies_bucket{resource="ingresses",verb="PATCH",le="8e+06"} 1 apiserver_request_latencies_bucket{resource="ingresses",verb="PATCH",le="+Inf"} 1 apiserver_request_latencies_sum{resource="ingresses",verb="PATCH"} 4068 apiserver_request_latencies_count{resource="ingresses",verb="PATCH"} 1 apiserver_request_latencies_bucket{resource="ingresses",verb="WATCHLIST",le="125000"} 0 apiserver_request_latencies_bucket{resource="ingresses",verb="WATCHLIST",le="250000"} 0 apiserver_request_latencies_bucket{resource="ingresses",verb="WATCHLIST",le="500000"} 0 apiserver_request_latencies_bucket{resource="ingresses",verb="WATCHLIST",le="1e+06"} 0 apiserver_request_latencies_bucket{resource="ingresses",verb="WATCHLIST",le="2e+06"} 0 apiserver_request_latencies_bucket{resource="ingresses",verb="WATCHLIST",le="4e+06"} 0 apiserver_request_latencies_bucket{resource="ingresses",verb="WATCHLIST",le="8e+06"} 0 apiserver_request_latencies_bucket{resource="ingresses",verb="WATCHLIST",le="+Inf"} 26 apiserver_request_latencies_sum{resource="ingresses",verb="WATCHLIST"} 6.6265207664e+10 apiserver_request_latencies_count{resource="ingresses",verb="WATCHLIST"} 26 apiserver_request_latencies_bucket{resource="jobs",verb="DELETE",le="125000"} 171 apiserver_request_latencies_bucket{resource="jobs",verb="DELETE",le="250000"} 178 apiserver_request_latencies_bucket{resource="jobs",verb="DELETE",le="500000"} 179 apiserver_request_latencies_bucket{resource="jobs",verb="DELETE",le="1e+06"} 181 apiserver_request_latencies_bucket{resource="jobs",verb="DELETE",le="2e+06"} 181 apiserver_request_latencies_bucket{resource="jobs",verb="DELETE",le="4e+06"} 181 apiserver_request_latencies_bucket{resource="jobs",verb="DELETE",le="8e+06"} 181 apiserver_request_latencies_bucket{resource="jobs",verb="DELETE",le="+Inf"} 181 apiserver_request_latencies_sum{resource="jobs",verb="DELETE"} 5.697304e+06 apiserver_request_latencies_count{resource="jobs",verb="DELETE"} 181 apiserver_request_latencies_bucket{resource="jobs",verb="GET",le="125000"} 12 apiserver_request_latencies_bucket{resource="jobs",verb="GET",le="250000"} 12 apiserver_request_latencies_bucket{resource="jobs",verb="GET",le="500000"} 12 apiserver_request_latencies_bucket{resource="jobs",verb="GET",le="1e+06"} 12 apiserver_request_latencies_bucket{resource="jobs",verb="GET",le="2e+06"} 12 apiserver_request_latencies_bucket{resource="jobs",verb="GET",le="4e+06"} 12 apiserver_request_latencies_bucket{resource="jobs",verb="GET",le="8e+06"} 12 apiserver_request_latencies_bucket{resource="jobs",verb="GET",le="+Inf"} 12 apiserver_request_latencies_sum{resource="jobs",verb="GET"} 56190 apiserver_request_latencies_count{resource="jobs",verb="GET"} 12 apiserver_request_latencies_bucket{resource="jobs",verb="LIST",le="125000"} 9 apiserver_request_latencies_bucket{resource="jobs",verb="LIST",le="250000"} 10 apiserver_request_latencies_bucket{resource="jobs",verb="LIST",le="500000"} 10 apiserver_request_latencies_bucket{resource="jobs",verb="LIST",le="1e+06"} 10 apiserver_request_latencies_bucket{resource="jobs",verb="LIST",le="2e+06"} 11 apiserver_request_latencies_bucket{resource="jobs",verb="LIST",le="4e+06"} 11 apiserver_request_latencies_bucket{resource="jobs",verb="LIST",le="8e+06"} 11 apiserver_request_latencies_bucket{resource="jobs",verb="LIST",le="+Inf"} 11 apiserver_request_latencies_sum{resource="jobs",verb="LIST"} 1.518463e+06 apiserver_request_latencies_count{resource="jobs",verb="LIST"} 11 apiserver_request_latencies_bucket{resource="limitranges",verb="GET",le="125000"} 2 apiserver_request_latencies_bucket{resource="limitranges",verb="GET",le="250000"} 2 apiserver_request_latencies_bucket{resource="limitranges",verb="GET",le="500000"} 2 apiserver_request_latencies_bucket{resource="limitranges",verb="GET",le="1e+06"} 2 apiserver_request_latencies_bucket{resource="limitranges",verb="GET",le="2e+06"} 2 apiserver_request_latencies_bucket{resource="limitranges",verb="GET",le="4e+06"} 2 apiserver_request_latencies_bucket{resource="limitranges",verb="GET",le="8e+06"} 2 apiserver_request_latencies_bucket{resource="limitranges",verb="GET",le="+Inf"} 2 apiserver_request_latencies_sum{resource="limitranges",verb="GET"} 3016 apiserver_request_latencies_count{resource="limitranges",verb="GET"} 2 apiserver_request_latencies_bucket{resource="limitranges",verb="LIST",le="125000"} 7 apiserver_request_latencies_bucket{resource="limitranges",verb="LIST",le="250000"} 7 apiserver_request_latencies_bucket{resource="limitranges",verb="LIST",le="500000"} 7 apiserver_request_latencies_bucket{resource="limitranges",verb="LIST",le="1e+06"} 7 apiserver_request_latencies_bucket{resource="limitranges",verb="LIST",le="2e+06"} 7 apiserver_request_latencies_bucket{resource="limitranges",verb="LIST",le="4e+06"} 7 apiserver_request_latencies_bucket{resource="limitranges",verb="LIST",le="8e+06"} 7 apiserver_request_latencies_bucket{resource="limitranges",verb="LIST",le="+Inf"} 7 apiserver_request_latencies_sum{resource="limitranges",verb="LIST"} 159548 apiserver_request_latencies_count{resource="limitranges",verb="LIST"} 7 apiserver_request_latencies_bucket{resource="limitranges",verb="PATCH",le="125000"} 2 apiserver_request_latencies_bucket{resource="limitranges",verb="PATCH",le="250000"} 2 apiserver_request_latencies_bucket{resource="limitranges",verb="PATCH",le="500000"} 2 apiserver_request_latencies_bucket{resource="limitranges",verb="PATCH",le="1e+06"} 2 apiserver_request_latencies_bucket{resource="limitranges",verb="PATCH",le="2e+06"} 2 apiserver_request_latencies_bucket{resource="limitranges",verb="PATCH",le="4e+06"} 2 apiserver_request_latencies_bucket{resource="limitranges",verb="PATCH",le="8e+06"} 2 apiserver_request_latencies_bucket{resource="limitranges",verb="PATCH",le="+Inf"} 2 apiserver_request_latencies_sum{resource="limitranges",verb="PATCH"} 3588 apiserver_request_latencies_count{resource="limitranges",verb="PATCH"} 2 apiserver_request_latencies_bucket{resource="limitranges",verb="WATCH",le="125000"} 0 apiserver_request_latencies_bucket{resource="limitranges",verb="WATCH",le="250000"} 0 apiserver_request_latencies_bucket{resource="limitranges",verb="WATCH",le="500000"} 0 apiserver_request_latencies_bucket{resource="limitranges",verb="WATCH",le="1e+06"} 0 apiserver_request_latencies_bucket{resource="limitranges",verb="WATCH",le="2e+06"} 0 apiserver_request_latencies_bucket{resource="limitranges",verb="WATCH",le="4e+06"} 0 apiserver_request_latencies_bucket{resource="limitranges",verb="WATCH",le="8e+06"} 0 apiserver_request_latencies_bucket{resource="limitranges",verb="WATCH",le="+Inf"} 154 apiserver_request_latencies_sum{resource="limitranges",verb="WATCH"} 6.9036552855e+10 apiserver_request_latencies_count{resource="limitranges",verb="WATCH"} 154 apiserver_request_latencies_bucket{resource="namespaces",verb="GET",le="125000"} 9008 apiserver_request_latencies_bucket{resource="namespaces",verb="GET",le="250000"} 9117 apiserver_request_latencies_bucket{resource="namespaces",verb="GET",le="500000"} 9161 apiserver_request_latencies_bucket{resource="namespaces",verb="GET",le="1e+06"} 9165 apiserver_request_latencies_bucket{resource="namespaces",verb="GET",le="2e+06"} 9165 apiserver_request_latencies_bucket{resource="namespaces",verb="GET",le="4e+06"} 9165 apiserver_request_latencies_bucket{resource="namespaces",verb="GET",le="8e+06"} 9165 apiserver_request_latencies_bucket{resource="namespaces",verb="GET",le="+Inf"} 9165 apiserver_request_latencies_sum{resource="namespaces",verb="GET"} 1.41950381e+08 apiserver_request_latencies_count{resource="namespaces",verb="GET"} 9165 apiserver_request_latencies_bucket{resource="namespaces",verb="LIST",le="125000"} 3 apiserver_request_latencies_bucket{resource="namespaces",verb="LIST",le="250000"} 3 apiserver_request_latencies_bucket{resource="namespaces",verb="LIST",le="500000"} 3 apiserver_request_latencies_bucket{resource="namespaces",verb="LIST",le="1e+06"} 3 apiserver_request_latencies_bucket{resource="namespaces",verb="LIST",le="2e+06"} 3 apiserver_request_latencies_bucket{resource="namespaces",verb="LIST",le="4e+06"} 3 apiserver_request_latencies_bucket{resource="namespaces",verb="LIST",le="8e+06"} 3 apiserver_request_latencies_bucket{resource="namespaces",verb="LIST",le="+Inf"} 3 apiserver_request_latencies_sum{resource="namespaces",verb="LIST"} 24493 apiserver_request_latencies_count{resource="namespaces",verb="LIST"} 3 apiserver_request_latencies_bucket{resource="namespaces",verb="POST",le="125000"} 1 apiserver_request_latencies_bucket{resource="namespaces",verb="POST",le="250000"} 1 apiserver_request_latencies_bucket{resource="namespaces",verb="POST",le="500000"} 1 apiserver_request_latencies_bucket{resource="namespaces",verb="POST",le="1e+06"} 1 apiserver_request_latencies_bucket{resource="namespaces",verb="POST",le="2e+06"} 1 apiserver_request_latencies_bucket{resource="namespaces",verb="POST",le="4e+06"} 1 apiserver_request_latencies_bucket{resource="namespaces",verb="POST",le="8e+06"} 1 apiserver_request_latencies_bucket{resource="namespaces",verb="POST",le="+Inf"} 1 apiserver_request_latencies_sum{resource="namespaces",verb="POST"} 98086 apiserver_request_latencies_count{resource="namespaces",verb="POST"} 1 apiserver_request_latencies_bucket{resource="namespaces",verb="WATCH",le="125000"} 626 apiserver_request_latencies_bucket{resource="namespaces",verb="WATCH",le="250000"} 631 apiserver_request_latencies_bucket{resource="namespaces",verb="WATCH",le="500000"} 636 apiserver_request_latencies_bucket{resource="namespaces",verb="WATCH",le="1e+06"} 636 apiserver_request_latencies_bucket{resource="namespaces",verb="WATCH",le="2e+06"} 637 apiserver_request_latencies_bucket{resource="namespaces",verb="WATCH",le="4e+06"} 637 apiserver_request_latencies_bucket{resource="namespaces",verb="WATCH",le="8e+06"} 637 apiserver_request_latencies_bucket{resource="namespaces",verb="WATCH",le="+Inf"} 942 apiserver_request_latencies_sum{resource="namespaces",verb="WATCH"} 1.35214363507e+11 apiserver_request_latencies_count{resource="namespaces",verb="WATCH"} 942 apiserver_request_latencies_bucket{resource="namespaces",verb="WATCHLIST",le="125000"} 0 apiserver_request_latencies_bucket{resource="namespaces",verb="WATCHLIST",le="250000"} 0 apiserver_request_latencies_bucket{resource="namespaces",verb="WATCHLIST",le="500000"} 0 apiserver_request_latencies_bucket{resource="namespaces",verb="WATCHLIST",le="1e+06"} 0 apiserver_request_latencies_bucket{resource="namespaces",verb="WATCHLIST",le="2e+06"} 0 apiserver_request_latencies_bucket{resource="namespaces",verb="WATCHLIST",le="4e+06"} 0 apiserver_request_latencies_bucket{resource="namespaces",verb="WATCHLIST",le="8e+06"} 0 apiserver_request_latencies_bucket{resource="namespaces",verb="WATCHLIST",le="+Inf"} 5 apiserver_request_latencies_sum{resource="namespaces",verb="WATCHLIST"} 1.770343085e+09 apiserver_request_latencies_count{resource="namespaces",verb="WATCHLIST"} 5 apiserver_request_latencies_bucket{resource="nodes",verb="GET",le="125000"} 34082 apiserver_request_latencies_bucket{resource="nodes",verb="GET",le="250000"} 34102 apiserver_request_latencies_bucket{resource="nodes",verb="GET",le="500000"} 34106 apiserver_request_latencies_bucket{resource="nodes",verb="GET",le="1e+06"} 34107 apiserver_request_latencies_bucket{resource="nodes",verb="GET",le="2e+06"} 34108 apiserver_request_latencies_bucket{resource="nodes",verb="GET",le="4e+06"} 34108 apiserver_request_latencies_bucket{resource="nodes",verb="GET",le="8e+06"} 34108 apiserver_request_latencies_bucket{resource="nodes",verb="GET",le="+Inf"} 34108 apiserver_request_latencies_sum{resource="nodes",verb="GET"} 8.6578745e+07 apiserver_request_latencies_count{resource="nodes",verb="GET"} 34108 apiserver_request_latencies_bucket{resource="nodes",verb="LIST",le="125000"} 187 apiserver_request_latencies_bucket{resource="nodes",verb="LIST",le="250000"} 197 apiserver_request_latencies_bucket{resource="nodes",verb="LIST",le="500000"} 197 apiserver_request_latencies_bucket{resource="nodes",verb="LIST",le="1e+06"} 197 apiserver_request_latencies_bucket{resource="nodes",verb="LIST",le="2e+06"} 197 apiserver_request_latencies_bucket{resource="nodes",verb="LIST",le="4e+06"} 197 apiserver_request_latencies_bucket{resource="nodes",verb="LIST",le="8e+06"} 197 apiserver_request_latencies_bucket{resource="nodes",verb="LIST",le="+Inf"} 197 apiserver_request_latencies_sum{resource="nodes",verb="LIST"} 6.570098e+06 apiserver_request_latencies_count{resource="nodes",verb="LIST"} 197 apiserver_request_latencies_bucket{resource="nodes",verb="PATCH",le="125000"} 31108 apiserver_request_latencies_bucket{resource="nodes",verb="PATCH",le="250000"} 32466 apiserver_request_latencies_bucket{resource="nodes",verb="PATCH",le="500000"} 33520 apiserver_request_latencies_bucket{resource="nodes",verb="PATCH",le="1e+06"} 33968 apiserver_request_latencies_bucket{resource="nodes",verb="PATCH",le="2e+06"} 34036 apiserver_request_latencies_bucket{resource="nodes",verb="PATCH",le="4e+06"} 34042 apiserver_request_latencies_bucket{resource="nodes",verb="PATCH",le="8e+06"} 34042 apiserver_request_latencies_bucket{resource="nodes",verb="PATCH",le="+Inf"} 34042 apiserver_request_latencies_sum{resource="nodes",verb="PATCH"} 1.871017595e+09 apiserver_request_latencies_count{resource="nodes",verb="PATCH"} 34042 apiserver_request_latencies_bucket{resource="nodes",verb="POST",le="125000"} 8 apiserver_request_latencies_bucket{resource="nodes",verb="POST",le="250000"} 8 apiserver_request_latencies_bucket{resource="nodes",verb="POST",le="500000"} 8 apiserver_request_latencies_bucket{resource="nodes",verb="POST",le="1e+06"} 8 apiserver_request_latencies_bucket{resource="nodes",verb="POST",le="2e+06"} 8 apiserver_request_latencies_bucket{resource="nodes",verb="POST",le="4e+06"} 8 apiserver_request_latencies_bucket{resource="nodes",verb="POST",le="8e+06"} 8 apiserver_request_latencies_bucket{resource="nodes",verb="POST",le="+Inf"} 8 apiserver_request_latencies_sum{resource="nodes",verb="POST"} 59161 apiserver_request_latencies_count{resource="nodes",verb="POST"} 8 apiserver_request_latencies_bucket{resource="nodes",verb="PUT",le="125000"} 576 apiserver_request_latencies_bucket{resource="nodes",verb="PUT",le="250000"} 612 apiserver_request_latencies_bucket{resource="nodes",verb="PUT",le="500000"} 636 apiserver_request_latencies_bucket{resource="nodes",verb="PUT",le="1e+06"} 644 apiserver_request_latencies_bucket{resource="nodes",verb="PUT",le="2e+06"} 645 apiserver_request_latencies_bucket{resource="nodes",verb="PUT",le="4e+06"} 645 apiserver_request_latencies_bucket{resource="nodes",verb="PUT",le="8e+06"} 645 apiserver_request_latencies_bucket{resource="nodes",verb="PUT",le="+Inf"} 645 apiserver_request_latencies_sum{resource="nodes",verb="PUT"} 3.8600184e+07 apiserver_request_latencies_count{resource="nodes",verb="PUT"} 645 apiserver_request_latencies_bucket{resource="nodes",verb="WATCH",le="125000"} 13191 apiserver_request_latencies_bucket{resource="nodes",verb="WATCH",le="250000"} 13644 apiserver_request_latencies_bucket{resource="nodes",verb="WATCH",le="500000"} 13919 apiserver_request_latencies_bucket{resource="nodes",verb="WATCH",le="1e+06"} 14018 apiserver_request_latencies_bucket{resource="nodes",verb="WATCH",le="2e+06"} 14040 apiserver_request_latencies_bucket{resource="nodes",verb="WATCH",le="4e+06"} 14043 apiserver_request_latencies_bucket{resource="nodes",verb="WATCH",le="8e+06"} 14043 apiserver_request_latencies_bucket{resource="nodes",verb="WATCH",le="+Inf"} 15538 apiserver_request_latencies_sum{resource="nodes",verb="WATCH"} 6.7447290162e+11 apiserver_request_latencies_count{resource="nodes",verb="WATCH"} 15538 apiserver_request_latencies_bucket{resource="nodes",verb="WATCHLIST",le="125000"} 0 apiserver_request_latencies_bucket{resource="nodes",verb="WATCHLIST",le="250000"} 0 apiserver_request_latencies_bucket{resource="nodes",verb="WATCHLIST",le="500000"} 0 apiserver_request_latencies_bucket{resource="nodes",verb="WATCHLIST",le="1e+06"} 0 apiserver_request_latencies_bucket{resource="nodes",verb="WATCHLIST",le="2e+06"} 0 apiserver_request_latencies_bucket{resource="nodes",verb="WATCHLIST",le="4e+06"} 0 apiserver_request_latencies_bucket{resource="nodes",verb="WATCHLIST",le="8e+06"} 0 apiserver_request_latencies_bucket{resource="nodes",verb="WATCHLIST",le="+Inf"} 35 apiserver_request_latencies_sum{resource="nodes",verb="WATCHLIST"} 1.3354929587e+10 apiserver_request_latencies_count{resource="nodes",verb="WATCHLIST"} 35 apiserver_request_latencies_bucket{resource="persistentvolumeclaims",verb="GET",le="125000"} 3638 apiserver_request_latencies_bucket{resource="persistentvolumeclaims",verb="GET",le="250000"} 3723 apiserver_request_latencies_bucket{resource="persistentvolumeclaims",verb="GET",le="500000"} 3778 apiserver_request_latencies_bucket{resource="persistentvolumeclaims",verb="GET",le="1e+06"} 3785 apiserver_request_latencies_bucket{resource="persistentvolumeclaims",verb="GET",le="2e+06"} 3788 apiserver_request_latencies_bucket{resource="persistentvolumeclaims",verb="GET",le="4e+06"} 3788 apiserver_request_latencies_bucket{resource="persistentvolumeclaims",verb="GET",le="8e+06"} 3788 apiserver_request_latencies_bucket{resource="persistentvolumeclaims",verb="GET",le="+Inf"} 3788 apiserver_request_latencies_sum{resource="persistentvolumeclaims",verb="GET"} 8.6392202e+07 apiserver_request_latencies_count{resource="persistentvolumeclaims",verb="GET"} 3788 apiserver_request_latencies_bucket{resource="persistentvolumeclaims",verb="LIST",le="125000"} 1 apiserver_request_latencies_bucket{resource="persistentvolumeclaims",verb="LIST",le="250000"} 1 apiserver_request_latencies_bucket{resource="persistentvolumeclaims",verb="LIST",le="500000"} 1 apiserver_request_latencies_bucket{resource="persistentvolumeclaims",verb="LIST",le="1e+06"} 1 apiserver_request_latencies_bucket{resource="persistentvolumeclaims",verb="LIST",le="2e+06"} 1 apiserver_request_latencies_bucket{resource="persistentvolumeclaims",verb="LIST",le="4e+06"} 1 apiserver_request_latencies_bucket{resource="persistentvolumeclaims",verb="LIST",le="8e+06"} 1 apiserver_request_latencies_bucket{resource="persistentvolumeclaims",verb="LIST",le="+Inf"} 1 apiserver_request_latencies_sum{resource="persistentvolumeclaims",verb="LIST"} 288 apiserver_request_latencies_count{resource="persistentvolumeclaims",verb="LIST"} 1 apiserver_request_latencies_bucket{resource="persistentvolumeclaims",verb="WATCH",le="125000"} 0 apiserver_request_latencies_bucket{resource="persistentvolumeclaims",verb="WATCH",le="250000"} 0 apiserver_request_latencies_bucket{resource="persistentvolumeclaims",verb="WATCH",le="500000"} 0 apiserver_request_latencies_bucket{resource="persistentvolumeclaims",verb="WATCH",le="1e+06"} 0 apiserver_request_latencies_bucket{resource="persistentvolumeclaims",verb="WATCH",le="2e+06"} 0 apiserver_request_latencies_bucket{resource="persistentvolumeclaims",verb="WATCH",le="4e+06"} 0 apiserver_request_latencies_bucket{resource="persistentvolumeclaims",verb="WATCH",le="8e+06"} 0 apiserver_request_latencies_bucket{resource="persistentvolumeclaims",verb="WATCH",le="+Inf"} 150 apiserver_request_latencies_sum{resource="persistentvolumeclaims",verb="WATCH"} 6.9312180305e+10 apiserver_request_latencies_count{resource="persistentvolumeclaims",verb="WATCH"} 150 apiserver_request_latencies_bucket{resource="persistentvolumes",verb="GET",le="125000"} 3703 apiserver_request_latencies_bucket{resource="persistentvolumes",verb="GET",le="250000"} 3758 apiserver_request_latencies_bucket{resource="persistentvolumes",verb="GET",le="500000"} 3788 apiserver_request_latencies_bucket{resource="persistentvolumes",verb="GET",le="1e+06"} 3789 apiserver_request_latencies_bucket{resource="persistentvolumes",verb="GET",le="2e+06"} 3789 apiserver_request_latencies_bucket{resource="persistentvolumes",verb="GET",le="4e+06"} 3789 apiserver_request_latencies_bucket{resource="persistentvolumes",verb="GET",le="8e+06"} 3789 apiserver_request_latencies_bucket{resource="persistentvolumes",verb="GET",le="+Inf"} 3789 apiserver_request_latencies_sum{resource="persistentvolumes",verb="GET"} 4.9947084e+07 apiserver_request_latencies_count{resource="persistentvolumes",verb="GET"} 3789 apiserver_request_latencies_bucket{resource="persistentvolumes",verb="LIST",le="125000"} 1 apiserver_request_latencies_bucket{resource="persistentvolumes",verb="LIST",le="250000"} 1 apiserver_request_latencies_bucket{resource="persistentvolumes",verb="LIST",le="500000"} 1 apiserver_request_latencies_bucket{resource="persistentvolumes",verb="LIST",le="1e+06"} 1 apiserver_request_latencies_bucket{resource="persistentvolumes",verb="LIST",le="2e+06"} 1 apiserver_request_latencies_bucket{resource="persistentvolumes",verb="LIST",le="4e+06"} 1 apiserver_request_latencies_bucket{resource="persistentvolumes",verb="LIST",le="8e+06"} 1 apiserver_request_latencies_bucket{resource="persistentvolumes",verb="LIST",le="+Inf"} 1 apiserver_request_latencies_sum{resource="persistentvolumes",verb="LIST"} 268 apiserver_request_latencies_count{resource="persistentvolumes",verb="LIST"} 1 apiserver_request_latencies_bucket{resource="persistentvolumes",verb="WATCH",le="125000"} 0 apiserver_request_latencies_bucket{resource="persistentvolumes",verb="WATCH",le="250000"} 0 apiserver_request_latencies_bucket{resource="persistentvolumes",verb="WATCH",le="500000"} 0 apiserver_request_latencies_bucket{resource="persistentvolumes",verb="WATCH",le="1e+06"} 0 apiserver_request_latencies_bucket{resource="persistentvolumes",verb="WATCH",le="2e+06"} 0 apiserver_request_latencies_bucket{resource="persistentvolumes",verb="WATCH",le="4e+06"} 0 apiserver_request_latencies_bucket{resource="persistentvolumes",verb="WATCH",le="8e+06"} 0 apiserver_request_latencies_bucket{resource="persistentvolumes",verb="WATCH",le="+Inf"} 152 apiserver_request_latencies_sum{resource="persistentvolumes",verb="WATCH"} 6.9265196092e+10 apiserver_request_latencies_count{resource="persistentvolumes",verb="WATCH"} 152 apiserver_request_latencies_bucket{resource="platformcredentialssets",verb="GET",le="125000"} 2 apiserver_request_latencies_bucket{resource="platformcredentialssets",verb="GET",le="250000"} 3 apiserver_request_latencies_bucket{resource="platformcredentialssets",verb="GET",le="500000"} 3 apiserver_request_latencies_bucket{resource="platformcredentialssets",verb="GET",le="1e+06"} 3 apiserver_request_latencies_bucket{resource="platformcredentialssets",verb="GET",le="2e+06"} 3 apiserver_request_latencies_bucket{resource="platformcredentialssets",verb="GET",le="4e+06"} 3 apiserver_request_latencies_bucket{resource="platformcredentialssets",verb="GET",le="8e+06"} 3 apiserver_request_latencies_bucket{resource="platformcredentialssets",verb="GET",le="+Inf"} 3 apiserver_request_latencies_sum{resource="platformcredentialssets",verb="GET"} 138689 apiserver_request_latencies_count{resource="platformcredentialssets",verb="GET"} 3 apiserver_request_latencies_bucket{resource="platformcredentialssets",verb="LIST",le="125000"} 548 apiserver_request_latencies_bucket{resource="platformcredentialssets",verb="LIST",le="250000"} 564 apiserver_request_latencies_bucket{resource="platformcredentialssets",verb="LIST",le="500000"} 567 apiserver_request_latencies_bucket{resource="platformcredentialssets",verb="LIST",le="1e+06"} 567 apiserver_request_latencies_bucket{resource="platformcredentialssets",verb="LIST",le="2e+06"} 567 apiserver_request_latencies_bucket{resource="platformcredentialssets",verb="LIST",le="4e+06"} 567 apiserver_request_latencies_bucket{resource="platformcredentialssets",verb="LIST",le="8e+06"} 567 apiserver_request_latencies_bucket{resource="platformcredentialssets",verb="LIST",le="+Inf"} 567 apiserver_request_latencies_sum{resource="platformcredentialssets",verb="LIST"} 1.0165435e+07 apiserver_request_latencies_count{resource="platformcredentialssets",verb="LIST"} 567 apiserver_request_latencies_bucket{resource="platformcredentialssets",verb="PUT",le="125000"} 2 apiserver_request_latencies_bucket{resource="platformcredentialssets",verb="PUT",le="250000"} 2 apiserver_request_latencies_bucket{resource="platformcredentialssets",verb="PUT",le="500000"} 2 apiserver_request_latencies_bucket{resource="platformcredentialssets",verb="PUT",le="1e+06"} 2 apiserver_request_latencies_bucket{resource="platformcredentialssets",verb="PUT",le="2e+06"} 2 apiserver_request_latencies_bucket{resource="platformcredentialssets",verb="PUT",le="4e+06"} 2 apiserver_request_latencies_bucket{resource="platformcredentialssets",verb="PUT",le="8e+06"} 2 apiserver_request_latencies_bucket{resource="platformcredentialssets",verb="PUT",le="+Inf"} 2 apiserver_request_latencies_sum{resource="platformcredentialssets",verb="PUT"} 11771 apiserver_request_latencies_count{resource="platformcredentialssets",verb="PUT"} 2 apiserver_request_latencies_bucket{resource="poddisruptionbudgets",verb="LIST",le="125000"} 95 apiserver_request_latencies_bucket{resource="poddisruptionbudgets",verb="LIST",le="250000"} 97 apiserver_request_latencies_bucket{resource="poddisruptionbudgets",verb="LIST",le="500000"} 99 apiserver_request_latencies_bucket{resource="poddisruptionbudgets",verb="LIST",le="1e+06"} 99 apiserver_request_latencies_bucket{resource="poddisruptionbudgets",verb="LIST",le="2e+06"} 99 apiserver_request_latencies_bucket{resource="poddisruptionbudgets",verb="LIST",le="4e+06"} 99 apiserver_request_latencies_bucket{resource="poddisruptionbudgets",verb="LIST",le="8e+06"} 99 apiserver_request_latencies_bucket{resource="poddisruptionbudgets",verb="LIST",le="+Inf"} 99 apiserver_request_latencies_sum{resource="poddisruptionbudgets",verb="LIST"} 3.206562e+06 apiserver_request_latencies_count{resource="poddisruptionbudgets",verb="LIST"} 99 apiserver_request_latencies_bucket{resource="pods",verb="DELETE",le="125000"} 254 apiserver_request_latencies_bucket{resource="pods",verb="DELETE",le="250000"} 298 apiserver_request_latencies_bucket{resource="pods",verb="DELETE",le="500000"} 321 apiserver_request_latencies_bucket{resource="pods",verb="DELETE",le="1e+06"} 327 apiserver_request_latencies_bucket{resource="pods",verb="DELETE",le="2e+06"} 329 apiserver_request_latencies_bucket{resource="pods",verb="DELETE",le="4e+06"} 329 apiserver_request_latencies_bucket{resource="pods",verb="DELETE",le="8e+06"} 329 apiserver_request_latencies_bucket{resource="pods",verb="DELETE",le="+Inf"} 329 apiserver_request_latencies_sum{resource="pods",verb="DELETE"} 3.2938e+07 apiserver_request_latencies_count{resource="pods",verb="DELETE"} 329 apiserver_request_latencies_bucket{resource="pods",verb="GET",le="125000"} 5884 apiserver_request_latencies_bucket{resource="pods",verb="GET",le="250000"} 6055 apiserver_request_latencies_bucket{resource="pods",verb="GET",le="500000"} 6171 apiserver_request_latencies_bucket{resource="pods",verb="GET",le="1e+06"} 6191 apiserver_request_latencies_bucket{resource="pods",verb="GET",le="2e+06"} 6196 apiserver_request_latencies_bucket{resource="pods",verb="GET",le="4e+06"} 6196 apiserver_request_latencies_bucket{resource="pods",verb="GET",le="8e+06"} 6196 apiserver_request_latencies_bucket{resource="pods",verb="GET",le="+Inf"} 6204 apiserver_request_latencies_sum{resource="pods",verb="GET"} 1.5191193458e+10 apiserver_request_latencies_count{resource="pods",verb="GET"} 6204 apiserver_request_latencies_bucket{resource="pods",verb="LIST",le="125000"} 87 apiserver_request_latencies_bucket{resource="pods",verb="LIST",le="250000"} 109 apiserver_request_latencies_bucket{resource="pods",verb="LIST",le="500000"} 124 apiserver_request_latencies_bucket{resource="pods",verb="LIST",le="1e+06"} 131 apiserver_request_latencies_bucket{resource="pods",verb="LIST",le="2e+06"} 136 apiserver_request_latencies_bucket{resource="pods",verb="LIST",le="4e+06"} 136 apiserver_request_latencies_bucket{resource="pods",verb="LIST",le="8e+06"} 136 apiserver_request_latencies_bucket{resource="pods",verb="LIST",le="+Inf"} 136 apiserver_request_latencies_sum{resource="pods",verb="LIST"} 2.5924607e+07 apiserver_request_latencies_count{resource="pods",verb="LIST"} 136 apiserver_request_latencies_bucket{resource="pods",verb="POST",le="125000"} 54 apiserver_request_latencies_bucket{resource="pods",verb="POST",le="250000"} 78 apiserver_request_latencies_bucket{resource="pods",verb="POST",le="500000"} 91 apiserver_request_latencies_bucket{resource="pods",verb="POST",le="1e+06"} 104 apiserver_request_latencies_bucket{resource="pods",verb="POST",le="2e+06"} 109 apiserver_request_latencies_bucket{resource="pods",verb="POST",le="4e+06"} 110 apiserver_request_latencies_bucket{resource="pods",verb="POST",le="8e+06"} 110 apiserver_request_latencies_bucket{resource="pods",verb="POST",le="+Inf"} 110 apiserver_request_latencies_sum{resource="pods",verb="POST"} 3.0661325e+07 apiserver_request_latencies_count{resource="pods",verb="POST"} 110 apiserver_request_latencies_bucket{resource="pods",verb="PUT",le="125000"} 1595 apiserver_request_latencies_bucket{resource="pods",verb="PUT",le="250000"} 1671 apiserver_request_latencies_bucket{resource="pods",verb="PUT",le="500000"} 1719 apiserver_request_latencies_bucket{resource="pods",verb="PUT",le="1e+06"} 1732 apiserver_request_latencies_bucket{resource="pods",verb="PUT",le="2e+06"} 1733 apiserver_request_latencies_bucket{resource="pods",verb="PUT",le="4e+06"} 1733 apiserver_request_latencies_bucket{resource="pods",verb="PUT",le="8e+06"} 1733 apiserver_request_latencies_bucket{resource="pods",verb="PUT",le="+Inf"} 1733 apiserver_request_latencies_sum{resource="pods",verb="PUT"} 7.9591485e+07 apiserver_request_latencies_count{resource="pods",verb="PUT"} 1733 apiserver_request_latencies_bucket{resource="pods",verb="WATCH",le="125000"} 11073 apiserver_request_latencies_bucket{resource="pods",verb="WATCH",le="250000"} 15810 apiserver_request_latencies_bucket{resource="pods",verb="WATCH",le="500000"} 17897 apiserver_request_latencies_bucket{resource="pods",verb="WATCH",le="1e+06"} 18578 apiserver_request_latencies_bucket{resource="pods",verb="WATCH",le="2e+06"} 19264 apiserver_request_latencies_bucket{resource="pods",verb="WATCH",le="4e+06"} 19419 apiserver_request_latencies_bucket{resource="pods",verb="WATCH",le="8e+06"} 19421 apiserver_request_latencies_bucket{resource="pods",verb="WATCH",le="+Inf"} 21319 apiserver_request_latencies_sum{resource="pods",verb="WATCH"} 7.49392191107e+11 apiserver_request_latencies_count{resource="pods",verb="WATCH"} 21319 apiserver_request_latencies_bucket{resource="pods",verb="WATCHLIST",le="125000"} 4 apiserver_request_latencies_bucket{resource="pods",verb="WATCHLIST",le="250000"} 4 apiserver_request_latencies_bucket{resource="pods",verb="WATCHLIST",le="500000"} 4 apiserver_request_latencies_bucket{resource="pods",verb="WATCHLIST",le="1e+06"} 4 apiserver_request_latencies_bucket{resource="pods",verb="WATCHLIST",le="2e+06"} 4 apiserver_request_latencies_bucket{resource="pods",verb="WATCHLIST",le="4e+06"} 4 apiserver_request_latencies_bucket{resource="pods",verb="WATCHLIST",le="8e+06"} 4 apiserver_request_latencies_bucket{resource="pods",verb="WATCHLIST",le="+Inf"} 507 apiserver_request_latencies_sum{resource="pods",verb="WATCHLIST"} 2.19963769117e+11 apiserver_request_latencies_count{resource="pods",verb="WATCHLIST"} 507 apiserver_request_latencies_bucket{resource="podsecuritypolicies",verb="GET",le="125000"} 2 apiserver_request_latencies_bucket{resource="podsecuritypolicies",verb="GET",le="250000"} 2 apiserver_request_latencies_bucket{resource="podsecuritypolicies",verb="GET",le="500000"} 2 apiserver_request_latencies_bucket{resource="podsecuritypolicies",verb="GET",le="1e+06"} 2 apiserver_request_latencies_bucket{resource="podsecuritypolicies",verb="GET",le="2e+06"} 2 apiserver_request_latencies_bucket{resource="podsecuritypolicies",verb="GET",le="4e+06"} 2 apiserver_request_latencies_bucket{resource="podsecuritypolicies",verb="GET",le="8e+06"} 2 apiserver_request_latencies_bucket{resource="podsecuritypolicies",verb="GET",le="+Inf"} 2 apiserver_request_latencies_sum{resource="podsecuritypolicies",verb="GET"} 118464 apiserver_request_latencies_count{resource="podsecuritypolicies",verb="GET"} 2 apiserver_request_latencies_bucket{resource="podsecuritypolicies",verb="LIST",le="125000"} 1 apiserver_request_latencies_bucket{resource="podsecuritypolicies",verb="LIST",le="250000"} 1 apiserver_request_latencies_bucket{resource="podsecuritypolicies",verb="LIST",le="500000"} 1 apiserver_request_latencies_bucket{resource="podsecuritypolicies",verb="LIST",le="1e+06"} 1 apiserver_request_latencies_bucket{resource="podsecuritypolicies",verb="LIST",le="2e+06"} 1 apiserver_request_latencies_bucket{resource="podsecuritypolicies",verb="LIST",le="4e+06"} 1 apiserver_request_latencies_bucket{resource="podsecuritypolicies",verb="LIST",le="8e+06"} 1 apiserver_request_latencies_bucket{resource="podsecuritypolicies",verb="LIST",le="+Inf"} 1 apiserver_request_latencies_sum{resource="podsecuritypolicies",verb="LIST"} 266 apiserver_request_latencies_count{resource="podsecuritypolicies",verb="LIST"} 1 apiserver_request_latencies_bucket{resource="podsecuritypolicies",verb="PATCH",le="125000"} 1 apiserver_request_latencies_bucket{resource="podsecuritypolicies",verb="PATCH",le="250000"} 2 apiserver_request_latencies_bucket{resource="podsecuritypolicies",verb="PATCH",le="500000"} 2 apiserver_request_latencies_bucket{resource="podsecuritypolicies",verb="PATCH",le="1e+06"} 2 apiserver_request_latencies_bucket{resource="podsecuritypolicies",verb="PATCH",le="2e+06"} 2 apiserver_request_latencies_bucket{resource="podsecuritypolicies",verb="PATCH",le="4e+06"} 2 apiserver_request_latencies_bucket{resource="podsecuritypolicies",verb="PATCH",le="8e+06"} 2 apiserver_request_latencies_bucket{resource="podsecuritypolicies",verb="PATCH",le="+Inf"} 2 apiserver_request_latencies_sum{resource="podsecuritypolicies",verb="PATCH"} 133715 apiserver_request_latencies_count{resource="podsecuritypolicies",verb="PATCH"} 2 apiserver_request_latencies_bucket{resource="podsecuritypolicies",verb="WATCH",le="125000"} 0 apiserver_request_latencies_bucket{resource="podsecuritypolicies",verb="WATCH",le="250000"} 0 apiserver_request_latencies_bucket{resource="podsecuritypolicies",verb="WATCH",le="500000"} 0 apiserver_request_latencies_bucket{resource="podsecuritypolicies",verb="WATCH",le="1e+06"} 0 apiserver_request_latencies_bucket{resource="podsecuritypolicies",verb="WATCH",le="2e+06"} 0 apiserver_request_latencies_bucket{resource="podsecuritypolicies",verb="WATCH",le="4e+06"} 0 apiserver_request_latencies_bucket{resource="podsecuritypolicies",verb="WATCH",le="8e+06"} 0 apiserver_request_latencies_bucket{resource="podsecuritypolicies",verb="WATCH",le="+Inf"} 151 apiserver_request_latencies_sum{resource="podsecuritypolicies",verb="WATCH"} 6.9172585559e+10 apiserver_request_latencies_count{resource="podsecuritypolicies",verb="WATCH"} 151 apiserver_request_latencies_bucket{resource="replicasets",verb="GET",le="125000"} 133 apiserver_request_latencies_bucket{resource="replicasets",verb="GET",le="250000"} 135 apiserver_request_latencies_bucket{resource="replicasets",verb="GET",le="500000"} 135 apiserver_request_latencies_bucket{resource="replicasets",verb="GET",le="1e+06"} 135 apiserver_request_latencies_bucket{resource="replicasets",verb="GET",le="2e+06"} 135 apiserver_request_latencies_bucket{resource="replicasets",verb="GET",le="4e+06"} 135 apiserver_request_latencies_bucket{resource="replicasets",verb="GET",le="8e+06"} 135 apiserver_request_latencies_bucket{resource="replicasets",verb="GET",le="+Inf"} 135 apiserver_request_latencies_sum{resource="replicasets",verb="GET"} 1.616098e+06 apiserver_request_latencies_count{resource="replicasets",verb="GET"} 135 apiserver_request_latencies_bucket{resource="replicasets",verb="LIST",le="125000"} 1850 apiserver_request_latencies_bucket{resource="replicasets",verb="LIST",le="250000"} 1929 apiserver_request_latencies_bucket{resource="replicasets",verb="LIST",le="500000"} 2489 apiserver_request_latencies_bucket{resource="replicasets",verb="LIST",le="1e+06"} 2868 apiserver_request_latencies_bucket{resource="replicasets",verb="LIST",le="2e+06"} 2930 apiserver_request_latencies_bucket{resource="replicasets",verb="LIST",le="4e+06"} 3052 apiserver_request_latencies_bucket{resource="replicasets",verb="LIST",le="8e+06"} 3052 apiserver_request_latencies_bucket{resource="replicasets",verb="LIST",le="+Inf"} 3052 apiserver_request_latencies_sum{resource="replicasets",verb="LIST"} 9.29912859e+08 apiserver_request_latencies_count{resource="replicasets",verb="LIST"} 3052 apiserver_request_latencies_bucket{resource="replicasets",verb="WATCH",le="125000"} 0 apiserver_request_latencies_bucket{resource="replicasets",verb="WATCH",le="250000"} 0 apiserver_request_latencies_bucket{resource="replicasets",verb="WATCH",le="500000"} 0 apiserver_request_latencies_bucket{resource="replicasets",verb="WATCH",le="1e+06"} 0 apiserver_request_latencies_bucket{resource="replicasets",verb="WATCH",le="2e+06"} 0 apiserver_request_latencies_bucket{resource="replicasets",verb="WATCH",le="4e+06"} 0 apiserver_request_latencies_bucket{resource="replicasets",verb="WATCH",le="8e+06"} 0 apiserver_request_latencies_bucket{resource="replicasets",verb="WATCH",le="+Inf"} 153 apiserver_request_latencies_sum{resource="replicasets",verb="WATCH"} 6.9002435522e+10 apiserver_request_latencies_count{resource="replicasets",verb="WATCH"} 153 apiserver_request_latencies_bucket{resource="replicationcontrollers",verb="LIST",le="125000"} 1 apiserver_request_latencies_bucket{resource="replicationcontrollers",verb="LIST",le="250000"} 1 apiserver_request_latencies_bucket{resource="replicationcontrollers",verb="LIST",le="500000"} 3 apiserver_request_latencies_bucket{resource="replicationcontrollers",verb="LIST",le="1e+06"} 3 apiserver_request_latencies_bucket{resource="replicationcontrollers",verb="LIST",le="2e+06"} 3 apiserver_request_latencies_bucket{resource="replicationcontrollers",verb="LIST",le="4e+06"} 3 apiserver_request_latencies_bucket{resource="replicationcontrollers",verb="LIST",le="8e+06"} 3 apiserver_request_latencies_bucket{resource="replicationcontrollers",verb="LIST",le="+Inf"} 3 apiserver_request_latencies_sum{resource="replicationcontrollers",verb="LIST"} 853253 apiserver_request_latencies_count{resource="replicationcontrollers",verb="LIST"} 3 apiserver_request_latencies_bucket{resource="replicationcontrollers",verb="WATCH",le="125000"} 0 apiserver_request_latencies_bucket{resource="replicationcontrollers",verb="WATCH",le="250000"} 0 apiserver_request_latencies_bucket{resource="replicationcontrollers",verb="WATCH",le="500000"} 0 apiserver_request_latencies_bucket{resource="replicationcontrollers",verb="WATCH",le="1e+06"} 0 apiserver_request_latencies_bucket{resource="replicationcontrollers",verb="WATCH",le="2e+06"} 0 apiserver_request_latencies_bucket{resource="replicationcontrollers",verb="WATCH",le="4e+06"} 0 apiserver_request_latencies_bucket{resource="replicationcontrollers",verb="WATCH",le="8e+06"} 0 apiserver_request_latencies_bucket{resource="replicationcontrollers",verb="WATCH",le="+Inf"} 150 apiserver_request_latencies_sum{resource="replicationcontrollers",verb="WATCH"} 6.8803924319e+10 apiserver_request_latencies_count{resource="replicationcontrollers",verb="WATCH"} 150 apiserver_request_latencies_bucket{resource="resourcequotas",verb="LIST",le="125000"} 10 apiserver_request_latencies_bucket{resource="resourcequotas",verb="LIST",le="250000"} 10 apiserver_request_latencies_bucket{resource="resourcequotas",verb="LIST",le="500000"} 10 apiserver_request_latencies_bucket{resource="resourcequotas",verb="LIST",le="1e+06"} 10 apiserver_request_latencies_bucket{resource="resourcequotas",verb="LIST",le="2e+06"} 10 apiserver_request_latencies_bucket{resource="resourcequotas",verb="LIST",le="4e+06"} 10 apiserver_request_latencies_bucket{resource="resourcequotas",verb="LIST",le="8e+06"} 10 apiserver_request_latencies_bucket{resource="resourcequotas",verb="LIST",le="+Inf"} 10 apiserver_request_latencies_sum{resource="resourcequotas",verb="LIST"} 69519 apiserver_request_latencies_count{resource="resourcequotas",verb="LIST"} 10 apiserver_request_latencies_bucket{resource="resourcequotas",verb="WATCH",le="125000"} 0 apiserver_request_latencies_bucket{resource="resourcequotas",verb="WATCH",le="250000"} 0 apiserver_request_latencies_bucket{resource="resourcequotas",verb="WATCH",le="500000"} 0 apiserver_request_latencies_bucket{resource="resourcequotas",verb="WATCH",le="1e+06"} 0 apiserver_request_latencies_bucket{resource="resourcequotas",verb="WATCH",le="2e+06"} 0 apiserver_request_latencies_bucket{resource="resourcequotas",verb="WATCH",le="4e+06"} 0 apiserver_request_latencies_bucket{resource="resourcequotas",verb="WATCH",le="8e+06"} 0 apiserver_request_latencies_bucket{resource="resourcequotas",verb="WATCH",le="+Inf"} 153 apiserver_request_latencies_sum{resource="resourcequotas",verb="WATCH"} 6.8889954315e+10 apiserver_request_latencies_count{resource="resourcequotas",verb="WATCH"} 153 apiserver_request_latencies_bucket{resource="rolebindings",verb="GET",le="125000"} 3 apiserver_request_latencies_bucket{resource="rolebindings",verb="GET",le="250000"} 3 apiserver_request_latencies_bucket{resource="rolebindings",verb="GET",le="500000"} 3 apiserver_request_latencies_bucket{resource="rolebindings",verb="GET",le="1e+06"} 3 apiserver_request_latencies_bucket{resource="rolebindings",verb="GET",le="2e+06"} 3 apiserver_request_latencies_bucket{resource="rolebindings",verb="GET",le="4e+06"} 3 apiserver_request_latencies_bucket{resource="rolebindings",verb="GET",le="8e+06"} 3 apiserver_request_latencies_bucket{resource="rolebindings",verb="GET",le="+Inf"} 3 apiserver_request_latencies_sum{resource="rolebindings",verb="GET"} 3078 apiserver_request_latencies_count{resource="rolebindings",verb="GET"} 3 apiserver_request_latencies_bucket{resource="rolebindings",verb="POST",le="125000"} 3 apiserver_request_latencies_bucket{resource="rolebindings",verb="POST",le="250000"} 3 apiserver_request_latencies_bucket{resource="rolebindings",verb="POST",le="500000"} 3 apiserver_request_latencies_bucket{resource="rolebindings",verb="POST",le="1e+06"} 3 apiserver_request_latencies_bucket{resource="rolebindings",verb="POST",le="2e+06"} 3 apiserver_request_latencies_bucket{resource="rolebindings",verb="POST",le="4e+06"} 3 apiserver_request_latencies_bucket{resource="rolebindings",verb="POST",le="8e+06"} 3 apiserver_request_latencies_bucket{resource="rolebindings",verb="POST",le="+Inf"} 3 apiserver_request_latencies_sum{resource="rolebindings",verb="POST"} 16918 apiserver_request_latencies_count{resource="rolebindings",verb="POST"} 3 apiserver_request_latencies_bucket{resource="roles",verb="GET",le="125000"} 4 apiserver_request_latencies_bucket{resource="roles",verb="GET",le="250000"} 4 apiserver_request_latencies_bucket{resource="roles",verb="GET",le="500000"} 4 apiserver_request_latencies_bucket{resource="roles",verb="GET",le="1e+06"} 4 apiserver_request_latencies_bucket{resource="roles",verb="GET",le="2e+06"} 4 apiserver_request_latencies_bucket{resource="roles",verb="GET",le="4e+06"} 4 apiserver_request_latencies_bucket{resource="roles",verb="GET",le="8e+06"} 4 apiserver_request_latencies_bucket{resource="roles",verb="GET",le="+Inf"} 4 apiserver_request_latencies_sum{resource="roles",verb="GET"} 10760 apiserver_request_latencies_count{resource="roles",verb="GET"} 4 apiserver_request_latencies_bucket{resource="roles",verb="POST",le="125000"} 4 apiserver_request_latencies_bucket{resource="roles",verb="POST",le="250000"} 4 apiserver_request_latencies_bucket{resource="roles",verb="POST",le="500000"} 4 apiserver_request_latencies_bucket{resource="roles",verb="POST",le="1e+06"} 4 apiserver_request_latencies_bucket{resource="roles",verb="POST",le="2e+06"} 4 apiserver_request_latencies_bucket{resource="roles",verb="POST",le="4e+06"} 4 apiserver_request_latencies_bucket{resource="roles",verb="POST",le="8e+06"} 4 apiserver_request_latencies_bucket{resource="roles",verb="POST",le="+Inf"} 4 apiserver_request_latencies_sum{resource="roles",verb="POST"} 25060 apiserver_request_latencies_count{resource="roles",verb="POST"} 4 apiserver_request_latencies_bucket{resource="secrets",verb="GET",le="125000"} 131324 apiserver_request_latencies_bucket{resource="secrets",verb="GET",le="250000"} 131518 apiserver_request_latencies_bucket{resource="secrets",verb="GET",le="500000"} 131596 apiserver_request_latencies_bucket{resource="secrets",verb="GET",le="1e+06"} 131616 apiserver_request_latencies_bucket{resource="secrets",verb="GET",le="2e+06"} 131617 apiserver_request_latencies_bucket{resource="secrets",verb="GET",le="4e+06"} 131617 apiserver_request_latencies_bucket{resource="secrets",verb="GET",le="8e+06"} 131617 apiserver_request_latencies_bucket{resource="secrets",verb="GET",le="+Inf"} 131617 apiserver_request_latencies_sum{resource="secrets",verb="GET"} 3.57061788e+08 apiserver_request_latencies_count{resource="secrets",verb="GET"} 131617 apiserver_request_latencies_bucket{resource="secrets",verb="LIST",le="125000"} 1 apiserver_request_latencies_bucket{resource="secrets",verb="LIST",le="250000"} 1 apiserver_request_latencies_bucket{resource="secrets",verb="LIST",le="500000"} 1 apiserver_request_latencies_bucket{resource="secrets",verb="LIST",le="1e+06"} 1 apiserver_request_latencies_bucket{resource="secrets",verb="LIST",le="2e+06"} 1 apiserver_request_latencies_bucket{resource="secrets",verb="LIST",le="4e+06"} 1 apiserver_request_latencies_bucket{resource="secrets",verb="LIST",le="8e+06"} 1 apiserver_request_latencies_bucket{resource="secrets",verb="LIST",le="+Inf"} 1 apiserver_request_latencies_sum{resource="secrets",verb="LIST"} 1531 apiserver_request_latencies_count{resource="secrets",verb="LIST"} 1 apiserver_request_latencies_bucket{resource="secrets",verb="PATCH",le="125000"} 1 apiserver_request_latencies_bucket{resource="secrets",verb="PATCH",le="250000"} 1 apiserver_request_latencies_bucket{resource="secrets",verb="PATCH",le="500000"} 1 apiserver_request_latencies_bucket{resource="secrets",verb="PATCH",le="1e+06"} 1 apiserver_request_latencies_bucket{resource="secrets",verb="PATCH",le="2e+06"} 1 apiserver_request_latencies_bucket{resource="secrets",verb="PATCH",le="4e+06"} 1 apiserver_request_latencies_bucket{resource="secrets",verb="PATCH",le="8e+06"} 1 apiserver_request_latencies_bucket{resource="secrets",verb="PATCH",le="+Inf"} 1 apiserver_request_latencies_sum{resource="secrets",verb="PATCH"} 13419 apiserver_request_latencies_count{resource="secrets",verb="PATCH"} 1 apiserver_request_latencies_bucket{resource="secrets",verb="PUT",le="125000"} 8242 apiserver_request_latencies_bucket{resource="secrets",verb="PUT",le="250000"} 8291 apiserver_request_latencies_bucket{resource="secrets",verb="PUT",le="500000"} 8309 apiserver_request_latencies_bucket{resource="secrets",verb="PUT",le="1e+06"} 8309 apiserver_request_latencies_bucket{resource="secrets",verb="PUT",le="2e+06"} 8309 apiserver_request_latencies_bucket{resource="secrets",verb="PUT",le="4e+06"} 8309 apiserver_request_latencies_bucket{resource="secrets",verb="PUT",le="8e+06"} 8309 apiserver_request_latencies_bucket{resource="secrets",verb="PUT",le="+Inf"} 8309 apiserver_request_latencies_sum{resource="secrets",verb="PUT"} 7.5898444e+07 apiserver_request_latencies_count{resource="secrets",verb="PUT"} 8309 apiserver_request_latencies_bucket{resource="secrets",verb="WATCH",le="125000"} 0 apiserver_request_latencies_bucket{resource="secrets",verb="WATCH",le="250000"} 0 apiserver_request_latencies_bucket{resource="secrets",verb="WATCH",le="500000"} 0 apiserver_request_latencies_bucket{resource="secrets",verb="WATCH",le="1e+06"} 0 apiserver_request_latencies_bucket{resource="secrets",verb="WATCH",le="2e+06"} 0 apiserver_request_latencies_bucket{resource="secrets",verb="WATCH",le="4e+06"} 0 apiserver_request_latencies_bucket{resource="secrets",verb="WATCH",le="8e+06"} 0 apiserver_request_latencies_bucket{resource="secrets",verb="WATCH",le="+Inf"} 151 apiserver_request_latencies_sum{resource="secrets",verb="WATCH"} 6.9149720375e+10 apiserver_request_latencies_count{resource="secrets",verb="WATCH"} 151 apiserver_request_latencies_bucket{resource="serviceaccounts",verb="GET",le="125000"} 346 apiserver_request_latencies_bucket{resource="serviceaccounts",verb="GET",le="250000"} 348 apiserver_request_latencies_bucket{resource="serviceaccounts",verb="GET",le="500000"} 348 apiserver_request_latencies_bucket{resource="serviceaccounts",verb="GET",le="1e+06"} 348 apiserver_request_latencies_bucket{resource="serviceaccounts",verb="GET",le="2e+06"} 348 apiserver_request_latencies_bucket{resource="serviceaccounts",verb="GET",le="4e+06"} 348 apiserver_request_latencies_bucket{resource="serviceaccounts",verb="GET",le="8e+06"} 348 apiserver_request_latencies_bucket{resource="serviceaccounts",verb="GET",le="+Inf"} 348 apiserver_request_latencies_sum{resource="serviceaccounts",verb="GET"} 2.592425e+06 apiserver_request_latencies_count{resource="serviceaccounts",verb="GET"} 348 apiserver_request_latencies_bucket{resource="serviceaccounts",verb="LIST",le="125000"} 1 apiserver_request_latencies_bucket{resource="serviceaccounts",verb="LIST",le="250000"} 1 apiserver_request_latencies_bucket{resource="serviceaccounts",verb="LIST",le="500000"} 1 apiserver_request_latencies_bucket{resource="serviceaccounts",verb="LIST",le="1e+06"} 1 apiserver_request_latencies_bucket{resource="serviceaccounts",verb="LIST",le="2e+06"} 1 apiserver_request_latencies_bucket{resource="serviceaccounts",verb="LIST",le="4e+06"} 1 apiserver_request_latencies_bucket{resource="serviceaccounts",verb="LIST",le="8e+06"} 1 apiserver_request_latencies_bucket{resource="serviceaccounts",verb="LIST",le="+Inf"} 1 apiserver_request_latencies_sum{resource="serviceaccounts",verb="LIST"} 333 apiserver_request_latencies_count{resource="serviceaccounts",verb="LIST"} 1 apiserver_request_latencies_bucket{resource="serviceaccounts",verb="WATCH",le="125000"} 0 apiserver_request_latencies_bucket{resource="serviceaccounts",verb="WATCH",le="250000"} 0 apiserver_request_latencies_bucket{resource="serviceaccounts",verb="WATCH",le="500000"} 0 apiserver_request_latencies_bucket{resource="serviceaccounts",verb="WATCH",le="1e+06"} 0 apiserver_request_latencies_bucket{resource="serviceaccounts",verb="WATCH",le="2e+06"} 0 apiserver_request_latencies_bucket{resource="serviceaccounts",verb="WATCH",le="4e+06"} 0 apiserver_request_latencies_bucket{resource="serviceaccounts",verb="WATCH",le="8e+06"} 0 apiserver_request_latencies_bucket{resource="serviceaccounts",verb="WATCH",le="+Inf"} 154 apiserver_request_latencies_sum{resource="serviceaccounts",verb="WATCH"} 6.8774857752e+10 apiserver_request_latencies_count{resource="serviceaccounts",verb="WATCH"} 154 apiserver_request_latencies_bucket{resource="services",verb="CONNECT",le="125000"} 5807 apiserver_request_latencies_bucket{resource="services",verb="CONNECT",le="250000"} 6300 apiserver_request_latencies_bucket{resource="services",verb="CONNECT",le="500000"} 6493 apiserver_request_latencies_bucket{resource="services",verb="CONNECT",le="1e+06"} 6522 apiserver_request_latencies_bucket{resource="services",verb="CONNECT",le="2e+06"} 6533 apiserver_request_latencies_bucket{resource="services",verb="CONNECT",le="4e+06"} 6533 apiserver_request_latencies_bucket{resource="services",verb="CONNECT",le="8e+06"} 6534 apiserver_request_latencies_bucket{resource="services",verb="CONNECT",le="+Inf"} 6534 apiserver_request_latencies_sum{resource="services",verb="CONNECT"} 3.85858382e+08 apiserver_request_latencies_count{resource="services",verb="CONNECT"} 6534 apiserver_request_latencies_bucket{resource="services",verb="GET",le="125000"} 2.688195e+06 apiserver_request_latencies_bucket{resource="services",verb="GET",le="250000"} 2.702092e+06 apiserver_request_latencies_bucket{resource="services",verb="GET",le="500000"} 2.707436e+06 apiserver_request_latencies_bucket{resource="services",verb="GET",le="1e+06"} 2.708128e+06 apiserver_request_latencies_bucket{resource="services",verb="GET",le="2e+06"} 2.708259e+06 apiserver_request_latencies_bucket{resource="services",verb="GET",le="4e+06"} 2.70826e+06 apiserver_request_latencies_bucket{resource="services",verb="GET",le="8e+06"} 2.70826e+06 apiserver_request_latencies_bucket{resource="services",verb="GET",le="+Inf"} 2.70826e+06 apiserver_request_latencies_sum{resource="services",verb="GET"} 2.4446125831e+10 apiserver_request_latencies_count{resource="services",verb="GET"} 2.70826e+06 apiserver_request_latencies_bucket{resource="services",verb="LIST",le="125000"} 3411 apiserver_request_latencies_bucket{resource="services",verb="LIST",le="250000"} 3427 apiserver_request_latencies_bucket{resource="services",verb="LIST",le="500000"} 3432 apiserver_request_latencies_bucket{resource="services",verb="LIST",le="1e+06"} 3436 apiserver_request_latencies_bucket{resource="services",verb="LIST",le="2e+06"} 3436 apiserver_request_latencies_bucket{resource="services",verb="LIST",le="4e+06"} 3436 apiserver_request_latencies_bucket{resource="services",verb="LIST",le="8e+06"} 3436 apiserver_request_latencies_bucket{resource="services",verb="LIST",le="+Inf"} 3436 apiserver_request_latencies_sum{resource="services",verb="LIST"} 3.3720752e+07 apiserver_request_latencies_count{resource="services",verb="LIST"} 3436 apiserver_request_latencies_bucket{resource="services",verb="PATCH",le="125000"} 5 apiserver_request_latencies_bucket{resource="services",verb="PATCH",le="250000"} 5 apiserver_request_latencies_bucket{resource="services",verb="PATCH",le="500000"} 5 apiserver_request_latencies_bucket{resource="services",verb="PATCH",le="1e+06"} 5 apiserver_request_latencies_bucket{resource="services",verb="PATCH",le="2e+06"} 5 apiserver_request_latencies_bucket{resource="services",verb="PATCH",le="4e+06"} 5 apiserver_request_latencies_bucket{resource="services",verb="PATCH",le="8e+06"} 5 apiserver_request_latencies_bucket{resource="services",verb="PATCH",le="+Inf"} 5 apiserver_request_latencies_sum{resource="services",verb="PATCH"} 66210 apiserver_request_latencies_count{resource="services",verb="PATCH"} 5 apiserver_request_latencies_bucket{resource="services",verb="PROXY",le="125000"} 32 apiserver_request_latencies_bucket{resource="services",verb="PROXY",le="250000"} 34 apiserver_request_latencies_bucket{resource="services",verb="PROXY",le="500000"} 35 apiserver_request_latencies_bucket{resource="services",verb="PROXY",le="1e+06"} 35 apiserver_request_latencies_bucket{resource="services",verb="PROXY",le="2e+06"} 36 apiserver_request_latencies_bucket{resource="services",verb="PROXY",le="4e+06"} 36 apiserver_request_latencies_bucket{resource="services",verb="PROXY",le="8e+06"} 36 apiserver_request_latencies_bucket{resource="services",verb="PROXY",le="+Inf"} 36 apiserver_request_latencies_sum{resource="services",verb="PROXY"} 3.057711e+06 apiserver_request_latencies_count{resource="services",verb="PROXY"} 36 apiserver_request_latencies_bucket{resource="services",verb="WATCH",le="125000"} 1787 apiserver_request_latencies_bucket{resource="services",verb="WATCH",le="250000"} 1829 apiserver_request_latencies_bucket{resource="services",verb="WATCH",le="500000"} 1860 apiserver_request_latencies_bucket{resource="services",verb="WATCH",le="1e+06"} 1869 apiserver_request_latencies_bucket{resource="services",verb="WATCH",le="2e+06"} 1870 apiserver_request_latencies_bucket{resource="services",verb="WATCH",le="4e+06"} 1870 apiserver_request_latencies_bucket{resource="services",verb="WATCH",le="8e+06"} 1870 apiserver_request_latencies_bucket{resource="services",verb="WATCH",le="+Inf"} 4138 apiserver_request_latencies_sum{resource="services",verb="WATCH"} 7.46005271353e+11 apiserver_request_latencies_count{resource="services",verb="WATCH"} 4138 apiserver_request_latencies_bucket{resource="services",verb="WATCHLIST",le="125000"} 15 apiserver_request_latencies_bucket{resource="services",verb="WATCHLIST",le="250000"} 15 apiserver_request_latencies_bucket{resource="services",verb="WATCHLIST",le="500000"} 15 apiserver_request_latencies_bucket{resource="services",verb="WATCHLIST",le="1e+06"} 15 apiserver_request_latencies_bucket{resource="services",verb="WATCHLIST",le="2e+06"} 15 apiserver_request_latencies_bucket{resource="services",verb="WATCHLIST",le="4e+06"} 15 apiserver_request_latencies_bucket{resource="services",verb="WATCHLIST",le="8e+06"} 16 apiserver_request_latencies_bucket{resource="services",verb="WATCHLIST",le="+Inf"} 395 apiserver_request_latencies_sum{resource="services",verb="WATCHLIST"} 2.14384246956e+11 apiserver_request_latencies_count{resource="services",verb="WATCHLIST"} 395 apiserver_request_latencies_bucket{resource="services",verb="proxy",le="125000"} 32 apiserver_request_latencies_bucket{resource="services",verb="proxy",le="250000"} 34 apiserver_request_latencies_bucket{resource="services",verb="proxy",le="500000"} 35 apiserver_request_latencies_bucket{resource="services",verb="proxy",le="1e+06"} 35 apiserver_request_latencies_bucket{resource="services",verb="proxy",le="2e+06"} 36 apiserver_request_latencies_bucket{resource="services",verb="proxy",le="4e+06"} 36 apiserver_request_latencies_bucket{resource="services",verb="proxy",le="8e+06"} 36 apiserver_request_latencies_bucket{resource="services",verb="proxy",le="+Inf"} 36 apiserver_request_latencies_sum{resource="services",verb="proxy"} 3.057303e+06 apiserver_request_latencies_count{resource="services",verb="proxy"} 36 apiserver_request_latencies_bucket{resource="statefulsets",verb="GET",le="125000"} 18 apiserver_request_latencies_bucket{resource="statefulsets",verb="GET",le="250000"} 18 apiserver_request_latencies_bucket{resource="statefulsets",verb="GET",le="500000"} 18 apiserver_request_latencies_bucket{resource="statefulsets",verb="GET",le="1e+06"} 18 apiserver_request_latencies_bucket{resource="statefulsets",verb="GET",le="2e+06"} 18 apiserver_request_latencies_bucket{resource="statefulsets",verb="GET",le="4e+06"} 18 apiserver_request_latencies_bucket{resource="statefulsets",verb="GET",le="8e+06"} 18 apiserver_request_latencies_bucket{resource="statefulsets",verb="GET",le="+Inf"} 18 apiserver_request_latencies_sum{resource="statefulsets",verb="GET"} 76840 apiserver_request_latencies_count{resource="statefulsets",verb="GET"} 18 apiserver_request_latencies_bucket{resource="statefulsets",verb="LIST",le="125000"} 3392 apiserver_request_latencies_bucket{resource="statefulsets",verb="LIST",le="250000"} 3415 apiserver_request_latencies_bucket{resource="statefulsets",verb="LIST",le="500000"} 3424 apiserver_request_latencies_bucket{resource="statefulsets",verb="LIST",le="1e+06"} 3428 apiserver_request_latencies_bucket{resource="statefulsets",verb="LIST",le="2e+06"} 3428 apiserver_request_latencies_bucket{resource="statefulsets",verb="LIST",le="4e+06"} 3428 apiserver_request_latencies_bucket{resource="statefulsets",verb="LIST",le="8e+06"} 3428 apiserver_request_latencies_bucket{resource="statefulsets",verb="LIST",le="+Inf"} 3428 apiserver_request_latencies_sum{resource="statefulsets",verb="LIST"} 3.6405017e+07 apiserver_request_latencies_count{resource="statefulsets",verb="LIST"} 3428 apiserver_request_latencies_bucket{resource="statefulsets",verb="WATCH",le="125000"} 0 apiserver_request_latencies_bucket{resource="statefulsets",verb="WATCH",le="250000"} 0 apiserver_request_latencies_bucket{resource="statefulsets",verb="WATCH",le="500000"} 0 apiserver_request_latencies_bucket{resource="statefulsets",verb="WATCH",le="1e+06"} 0 apiserver_request_latencies_bucket{resource="statefulsets",verb="WATCH",le="2e+06"} 0 apiserver_request_latencies_bucket{resource="statefulsets",verb="WATCH",le="4e+06"} 0 apiserver_request_latencies_bucket{resource="statefulsets",verb="WATCH",le="8e+06"} 0 apiserver_request_latencies_bucket{resource="statefulsets",verb="WATCH",le="+Inf"} 153 apiserver_request_latencies_sum{resource="statefulsets",verb="WATCH"} 6.9107273702e+10 apiserver_request_latencies_count{resource="statefulsets",verb="WATCH"} 153 apiserver_request_latencies_bucket{resource="storageclasses",verb="GET",le="125000"} 1 apiserver_request_latencies_bucket{resource="storageclasses",verb="GET",le="250000"} 1 apiserver_request_latencies_bucket{resource="storageclasses",verb="GET",le="500000"} 1 apiserver_request_latencies_bucket{resource="storageclasses",verb="GET",le="1e+06"} 1 apiserver_request_latencies_bucket{resource="storageclasses",verb="GET",le="2e+06"} 1 apiserver_request_latencies_bucket{resource="storageclasses",verb="GET",le="4e+06"} 1 apiserver_request_latencies_bucket{resource="storageclasses",verb="GET",le="8e+06"} 1 apiserver_request_latencies_bucket{resource="storageclasses",verb="GET",le="+Inf"} 1 apiserver_request_latencies_sum{resource="storageclasses",verb="GET"} 69073 apiserver_request_latencies_count{resource="storageclasses",verb="GET"} 1 apiserver_request_latencies_bucket{resource="storageclasses",verb="LIST",le="125000"} 1 apiserver_request_latencies_bucket{resource="storageclasses",verb="LIST",le="250000"} 1 apiserver_request_latencies_bucket{resource="storageclasses",verb="LIST",le="500000"} 1 apiserver_request_latencies_bucket{resource="storageclasses",verb="LIST",le="1e+06"} 1 apiserver_request_latencies_bucket{resource="storageclasses",verb="LIST",le="2e+06"} 1 apiserver_request_latencies_bucket{resource="storageclasses",verb="LIST",le="4e+06"} 1 apiserver_request_latencies_bucket{resource="storageclasses",verb="LIST",le="8e+06"} 1 apiserver_request_latencies_bucket{resource="storageclasses",verb="LIST",le="+Inf"} 1 apiserver_request_latencies_sum{resource="storageclasses",verb="LIST"} 316 apiserver_request_latencies_count{resource="storageclasses",verb="LIST"} 1 apiserver_request_latencies_bucket{resource="storageclasses",verb="PATCH",le="125000"} 1 apiserver_request_latencies_bucket{resource="storageclasses",verb="PATCH",le="250000"} 1 apiserver_request_latencies_bucket{resource="storageclasses",verb="PATCH",le="500000"} 1 apiserver_request_latencies_bucket{resource="storageclasses",verb="PATCH",le="1e+06"} 1 apiserver_request_latencies_bucket{resource="storageclasses",verb="PATCH",le="2e+06"} 1 apiserver_request_latencies_bucket{resource="storageclasses",verb="PATCH",le="4e+06"} 1 apiserver_request_latencies_bucket{resource="storageclasses",verb="PATCH",le="8e+06"} 1 apiserver_request_latencies_bucket{resource="storageclasses",verb="PATCH",le="+Inf"} 1 apiserver_request_latencies_sum{resource="storageclasses",verb="PATCH"} 15753 apiserver_request_latencies_count{resource="storageclasses",verb="PATCH"} 1 apiserver_request_latencies_bucket{resource="storageclasses",verb="WATCH",le="125000"} 0 apiserver_request_latencies_bucket{resource="storageclasses",verb="WATCH",le="250000"} 0 apiserver_request_latencies_bucket{resource="storageclasses",verb="WATCH",le="500000"} 0 apiserver_request_latencies_bucket{resource="storageclasses",verb="WATCH",le="1e+06"} 0 apiserver_request_latencies_bucket{resource="storageclasses",verb="WATCH",le="2e+06"} 0 apiserver_request_latencies_bucket{resource="storageclasses",verb="WATCH",le="4e+06"} 0 apiserver_request_latencies_bucket{resource="storageclasses",verb="WATCH",le="8e+06"} 0 apiserver_request_latencies_bucket{resource="storageclasses",verb="WATCH",le="+Inf"} 149 apiserver_request_latencies_sum{resource="storageclasses",verb="WATCH"} 6.8948799683e+10 apiserver_request_latencies_count{resource="storageclasses",verb="WATCH"} 149 apiserver_request_latencies_bucket{resource="thirdpartyresources",verb="GET",le="125000"} 1 apiserver_request_latencies_bucket{resource="thirdpartyresources",verb="GET",le="250000"} 1 apiserver_request_latencies_bucket{resource="thirdpartyresources",verb="GET",le="500000"} 1 apiserver_request_latencies_bucket{resource="thirdpartyresources",verb="GET",le="1e+06"} 1 apiserver_request_latencies_bucket{resource="thirdpartyresources",verb="GET",le="2e+06"} 1 apiserver_request_latencies_bucket{resource="thirdpartyresources",verb="GET",le="4e+06"} 1 apiserver_request_latencies_bucket{resource="thirdpartyresources",verb="GET",le="8e+06"} 1 apiserver_request_latencies_bucket{resource="thirdpartyresources",verb="GET",le="+Inf"} 1 apiserver_request_latencies_sum{resource="thirdpartyresources",verb="GET"} 34480 apiserver_request_latencies_count{resource="thirdpartyresources",verb="GET"} 1 apiserver_request_latencies_bucket{resource="thirdpartyresources",verb="LIST",le="125000"} 6675 apiserver_request_latencies_bucket{resource="thirdpartyresources",verb="LIST",le="250000"} 6815 apiserver_request_latencies_bucket{resource="thirdpartyresources",verb="LIST",le="500000"} 6882 apiserver_request_latencies_bucket{resource="thirdpartyresources",verb="LIST",le="1e+06"} 6888 apiserver_request_latencies_bucket{resource="thirdpartyresources",verb="LIST",le="2e+06"} 6888 apiserver_request_latencies_bucket{resource="thirdpartyresources",verb="LIST",le="4e+06"} 6888 apiserver_request_latencies_bucket{resource="thirdpartyresources",verb="LIST",le="8e+06"} 6888 apiserver_request_latencies_bucket{resource="thirdpartyresources",verb="LIST",le="+Inf"} 6888 apiserver_request_latencies_sum{resource="thirdpartyresources",verb="LIST"} 1.59744481e+08 apiserver_request_latencies_count{resource="thirdpartyresources",verb="LIST"} 6888 apiserver_request_latencies_bucket{resource="thirdpartyresources",verb="PATCH",le="125000"} 0 apiserver_request_latencies_bucket{resource="thirdpartyresources",verb="PATCH",le="250000"} 1 apiserver_request_latencies_bucket{resource="thirdpartyresources",verb="PATCH",le="500000"} 1 apiserver_request_latencies_bucket{resource="thirdpartyresources",verb="PATCH",le="1e+06"} 1 apiserver_request_latencies_bucket{resource="thirdpartyresources",verb="PATCH",le="2e+06"} 1 apiserver_request_latencies_bucket{resource="thirdpartyresources",verb="PATCH",le="4e+06"} 1 apiserver_request_latencies_bucket{resource="thirdpartyresources",verb="PATCH",le="8e+06"} 1 apiserver_request_latencies_bucket{resource="thirdpartyresources",verb="PATCH",le="+Inf"} 1 apiserver_request_latencies_sum{resource="thirdpartyresources",verb="PATCH"} 130468 apiserver_request_latencies_count{resource="thirdpartyresources",verb="PATCH"} 1 # HELP apiserver_request_latencies_summary Response latency summary in microseconds for each verb and resource. # TYPE apiserver_request_latencies_summary summary apiserver_request_latencies_summary{resource="clusterrolebindings",verb="GET",quantile="0.5"} NaN apiserver_request_latencies_summary{resource="clusterrolebindings",verb="GET",quantile="0.9"} NaN apiserver_request_latencies_summary{resource="clusterrolebindings",verb="GET",quantile="0.99"} NaN apiserver_request_latencies_summary_sum{resource="clusterrolebindings",verb="GET"} 37213 apiserver_request_latencies_summary_count{resource="clusterrolebindings",verb="GET"} 30 apiserver_request_latencies_summary{resource="clusterrolebindings",verb="LIST",quantile="0.5"} NaN apiserver_request_latencies_summary{resource="clusterrolebindings",verb="LIST",quantile="0.9"} NaN apiserver_request_latencies_summary{resource="clusterrolebindings",verb="LIST",quantile="0.99"} NaN apiserver_request_latencies_summary_sum{resource="clusterrolebindings",verb="LIST"} 1546 apiserver_request_latencies_summary_count{resource="clusterrolebindings",verb="LIST"} 1 apiserver_request_latencies_summary{resource="clusterrolebindings",verb="POST",quantile="0.5"} NaN apiserver_request_latencies_summary{resource="clusterrolebindings",verb="POST",quantile="0.9"} NaN apiserver_request_latencies_summary{resource="clusterrolebindings",verb="POST",quantile="0.99"} NaN apiserver_request_latencies_summary_sum{resource="clusterrolebindings",verb="POST"} 206842 apiserver_request_latencies_summary_count{resource="clusterrolebindings",verb="POST"} 24 apiserver_request_latencies_summary{resource="clusterrolebindings",verb="PUT",quantile="0.5"} NaN apiserver_request_latencies_summary{resource="clusterrolebindings",verb="PUT",quantile="0.9"} NaN apiserver_request_latencies_summary{resource="clusterrolebindings",verb="PUT",quantile="0.99"} NaN apiserver_request_latencies_summary_sum{resource="clusterrolebindings",verb="PUT"} 94392 apiserver_request_latencies_summary_count{resource="clusterrolebindings",verb="PUT"} 6 apiserver_request_latencies_summary{resource="clusterroles",verb="GET",quantile="0.5"} NaN apiserver_request_latencies_summary{resource="clusterroles",verb="GET",quantile="0.9"} NaN apiserver_request_latencies_summary{resource="clusterroles",verb="GET",quantile="0.99"} NaN apiserver_request_latencies_summary_sum{resource="clusterroles",verb="GET"} 266494 apiserver_request_latencies_summary_count{resource="clusterroles",verb="GET"} 39 apiserver_request_latencies_summary{resource="clusterroles",verb="LIST",quantile="0.5"} NaN apiserver_request_latencies_summary{resource="clusterroles",verb="LIST",quantile="0.9"} NaN apiserver_request_latencies_summary{resource="clusterroles",verb="LIST",quantile="0.99"} NaN apiserver_request_latencies_summary_sum{resource="clusterroles",verb="LIST"} 3845 apiserver_request_latencies_summary_count{resource="clusterroles",verb="LIST"} 1 apiserver_request_latencies_summary{resource="clusterroles",verb="POST",quantile="0.5"} NaN apiserver_request_latencies_summary{resource="clusterroles",verb="POST",quantile="0.9"} NaN apiserver_request_latencies_summary{resource="clusterroles",verb="POST",quantile="0.99"} NaN apiserver_request_latencies_summary_sum{resource="clusterroles",verb="POST"} 208038 apiserver_request_latencies_summary_count{resource="clusterroles",verb="POST"} 30 apiserver_request_latencies_summary{resource="clusterroles",verb="PUT",quantile="0.5"} NaN apiserver_request_latencies_summary{resource="clusterroles",verb="PUT",quantile="0.9"} NaN apiserver_request_latencies_summary{resource="clusterroles",verb="PUT",quantile="0.99"} NaN apiserver_request_latencies_summary_sum{resource="clusterroles",verb="PUT"} 657777 apiserver_request_latencies_summary_count{resource="clusterroles",verb="PUT"} 9 apiserver_request_latencies_summary{resource="configmaps",verb="GET",quantile="0.5"} 2559 apiserver_request_latencies_summary{resource="configmaps",verb="GET",quantile="0.9"} 100907 apiserver_request_latencies_summary{resource="configmaps",verb="GET",quantile="0.99"} 348670 apiserver_request_latencies_summary_sum{resource="configmaps",verb="GET"} 5.10270682e+08 apiserver_request_latencies_summary_count{resource="configmaps",verb="GET"} 14928 apiserver_request_latencies_summary{resource="configmaps",verb="LIST",quantile="0.5"} NaN apiserver_request_latencies_summary{resource="configmaps",verb="LIST",quantile="0.9"} NaN apiserver_request_latencies_summary{resource="configmaps",verb="LIST",quantile="0.99"} NaN apiserver_request_latencies_summary_sum{resource="configmaps",verb="LIST"} 1947 apiserver_request_latencies_summary_count{resource="configmaps",verb="LIST"} 2 apiserver_request_latencies_summary{resource="configmaps",verb="PATCH",quantile="0.5"} NaN apiserver_request_latencies_summary{resource="configmaps",verb="PATCH",quantile="0.9"} NaN apiserver_request_latencies_summary{resource="configmaps",verb="PATCH",quantile="0.99"} NaN apiserver_request_latencies_summary_sum{resource="configmaps",verb="PATCH"} 140776 apiserver_request_latencies_summary_count{resource="configmaps",verb="PATCH"} 2 apiserver_request_latencies_summary{resource="configmaps",verb="WATCHLIST",quantile="0.5"} NaN apiserver_request_latencies_summary{resource="configmaps",verb="WATCHLIST",quantile="0.9"} NaN apiserver_request_latencies_summary{resource="configmaps",verb="WATCHLIST",quantile="0.99"} NaN apiserver_request_latencies_summary_sum{resource="configmaps",verb="WATCHLIST"} 1.178198112e+09 apiserver_request_latencies_summary_count{resource="configmaps",verb="WATCHLIST"} 4 apiserver_request_latencies_summary{resource="cronjobs",verb="DELETE",quantile="0.5"} NaN apiserver_request_latencies_summary{resource="cronjobs",verb="DELETE",quantile="0.9"} NaN apiserver_request_latencies_summary{resource="cronjobs",verb="DELETE",quantile="0.99"} NaN apiserver_request_latencies_summary_sum{resource="cronjobs",verb="DELETE"} 105882 apiserver_request_latencies_summary_count{resource="cronjobs",verb="DELETE"} 1 apiserver_request_latencies_summary{resource="cronjobs",verb="GET",quantile="0.5"} NaN apiserver_request_latencies_summary{resource="cronjobs",verb="GET",quantile="0.9"} NaN apiserver_request_latencies_summary{resource="cronjobs",verb="GET",quantile="0.99"} NaN apiserver_request_latencies_summary_sum{resource="cronjobs",verb="GET"} 19176 apiserver_request_latencies_summary_count{resource="cronjobs",verb="GET"} 2 apiserver_request_latencies_summary{resource="cronjobs",verb="LIST",quantile="0.5"} NaN apiserver_request_latencies_summary{resource="cronjobs",verb="LIST",quantile="0.9"} NaN apiserver_request_latencies_summary{resource="cronjobs",verb="LIST",quantile="0.99"} NaN apiserver_request_latencies_summary_sum{resource="cronjobs",verb="LIST"} 92955 apiserver_request_latencies_summary_count{resource="cronjobs",verb="LIST"} 2 apiserver_request_latencies_summary{resource="cronjobs",verb="POST",quantile="0.5"} NaN apiserver_request_latencies_summary{resource="cronjobs",verb="POST",quantile="0.9"} NaN apiserver_request_latencies_summary{resource="cronjobs",verb="POST",quantile="0.99"} NaN apiserver_request_latencies_summary_sum{resource="cronjobs",verb="POST"} 8842 apiserver_request_latencies_summary_count{resource="cronjobs",verb="POST"} 2 apiserver_request_latencies_summary{resource="daemonsets",verb="GET",quantile="0.5"} NaN apiserver_request_latencies_summary{resource="daemonsets",verb="GET",quantile="0.9"} NaN apiserver_request_latencies_summary{resource="daemonsets",verb="GET",quantile="0.99"} NaN apiserver_request_latencies_summary_sum{resource="daemonsets",verb="GET"} 4.6151e+06 apiserver_request_latencies_summary_count{resource="daemonsets",verb="GET"} 112 apiserver_request_latencies_summary{resource="daemonsets",verb="LIST",quantile="0.5"} 2789 apiserver_request_latencies_summary{resource="daemonsets",verb="LIST",quantile="0.9"} 20483 apiserver_request_latencies_summary{resource="daemonsets",verb="LIST",quantile="0.99"} 100926 apiserver_request_latencies_summary_sum{resource="daemonsets",verb="LIST"} 6.3406995e+07 apiserver_request_latencies_summary_count{resource="daemonsets",verb="LIST"} 3648 apiserver_request_latencies_summary{resource="daemonsets",verb="PATCH",quantile="0.5"} NaN apiserver_request_latencies_summary{resource="daemonsets",verb="PATCH",quantile="0.9"} NaN apiserver_request_latencies_summary{resource="daemonsets",verb="PATCH",quantile="0.99"} NaN apiserver_request_latencies_summary_sum{resource="daemonsets",verb="PATCH"} 745497 apiserver_request_latencies_summary_count{resource="daemonsets",verb="PATCH"} 4 apiserver_request_latencies_summary{resource="deployments",verb="DELETE",quantile="0.5"} 10257 apiserver_request_latencies_summary{resource="deployments",verb="DELETE",quantile="0.9"} 10257 apiserver_request_latencies_summary{resource="deployments",verb="DELETE",quantile="0.99"} 10257 apiserver_request_latencies_summary_sum{resource="deployments",verb="DELETE"} 36392 apiserver_request_latencies_summary_count{resource="deployments",verb="DELETE"} 4 apiserver_request_latencies_summary{resource="deployments",verb="GET",quantile="0.5"} 1180 apiserver_request_latencies_summary{resource="deployments",verb="GET",quantile="0.9"} 1875 apiserver_request_latencies_summary{resource="deployments",verb="GET",quantile="0.99"} 1875 apiserver_request_latencies_summary_sum{resource="deployments",verb="GET"} 458918 apiserver_request_latencies_summary_count{resource="deployments",verb="GET"} 35 apiserver_request_latencies_summary{resource="deployments",verb="LIST",quantile="0.5"} 15321 apiserver_request_latencies_summary{resource="deployments",verb="LIST",quantile="0.9"} 280420 apiserver_request_latencies_summary{resource="deployments",verb="LIST",quantile="0.99"} 393374 apiserver_request_latencies_summary_sum{resource="deployments",verb="LIST"} 1.12914081e+08 apiserver_request_latencies_summary_count{resource="deployments",verb="LIST"} 1008 apiserver_request_latencies_summary{resource="deployments",verb="PATCH",quantile="0.5"} NaN apiserver_request_latencies_summary{resource="deployments",verb="PATCH",quantile="0.9"} NaN apiserver_request_latencies_summary{resource="deployments",verb="PATCH",quantile="0.99"} NaN apiserver_request_latencies_summary_sum{resource="deployments",verb="PATCH"} 382446 apiserver_request_latencies_summary_count{resource="deployments",verb="PATCH"} 15 apiserver_request_latencies_summary{resource="deployments",verb="POST",quantile="0.5"} 3768 apiserver_request_latencies_summary{resource="deployments",verb="POST",quantile="0.9"} 16772 apiserver_request_latencies_summary{resource="deployments",verb="POST",quantile="0.99"} 16772 apiserver_request_latencies_summary_sum{resource="deployments",verb="POST"} 109828 apiserver_request_latencies_summary_count{resource="deployments",verb="POST"} 4 apiserver_request_latencies_summary{resource="deployments",verb="PUT",quantile="0.5"} NaN apiserver_request_latencies_summary{resource="deployments",verb="PUT",quantile="0.9"} NaN apiserver_request_latencies_summary{resource="deployments",verb="PUT",quantile="0.99"} NaN apiserver_request_latencies_summary_sum{resource="deployments",verb="PUT"} 14939 apiserver_request_latencies_summary_count{resource="deployments",verb="PUT"} 2 apiserver_request_latencies_summary{resource="endpoints",verb="GET",quantile="0.5"} 2118 apiserver_request_latencies_summary{resource="endpoints",verb="GET",quantile="0.9"} 95545 apiserver_request_latencies_summary{resource="endpoints",verb="GET",quantile="0.99"} 391919 apiserver_request_latencies_summary_sum{resource="endpoints",verb="GET"} 1.165225393e+09 apiserver_request_latencies_summary_count{resource="endpoints",verb="GET"} 49198 apiserver_request_latencies_summary{resource="endpoints",verb="LIST",quantile="0.5"} 101159 apiserver_request_latencies_summary{resource="endpoints",verb="LIST",quantile="0.9"} 301494 apiserver_request_latencies_summary{resource="endpoints",verb="LIST",quantile="0.99"} 636992 apiserver_request_latencies_summary_sum{resource="endpoints",verb="LIST"} 1.0344581295e+10 apiserver_request_latencies_summary_count{resource="endpoints",verb="LIST"} 76978 apiserver_request_latencies_summary{resource="endpoints",verb="PUT",quantile="0.5"} 7226 apiserver_request_latencies_summary{resource="endpoints",verb="PUT",quantile="0.9"} 86331 apiserver_request_latencies_summary{resource="endpoints",verb="PUT",quantile="0.99"} 230095 apiserver_request_latencies_summary_sum{resource="endpoints",verb="PUT"} 1.5347824e+08 apiserver_request_latencies_summary_count{resource="endpoints",verb="PUT"} 6823 apiserver_request_latencies_summary{resource="endpoints",verb="WATCH",quantile="0.5"} 4.64002612e+08 apiserver_request_latencies_summary{resource="endpoints",verb="WATCH",quantile="0.9"} 5.70000316e+08 apiserver_request_latencies_summary{resource="endpoints",verb="WATCH",quantile="0.99"} 5.88001708e+08 apiserver_request_latencies_summary_sum{resource="endpoints",verb="WATCH"} 2.75517142166e+11 apiserver_request_latencies_summary_count{resource="endpoints",verb="WATCH"} 622 apiserver_request_latencies_summary{resource="endpoints",verb="WATCHLIST",quantile="0.5"} 4.68001136e+08 apiserver_request_latencies_summary{resource="endpoints",verb="WATCHLIST",quantile="0.9"} 5.59000508e+08 apiserver_request_latencies_summary{resource="endpoints",verb="WATCHLIST",quantile="0.99"} 5.70000335e+08 apiserver_request_latencies_summary_sum{resource="endpoints",verb="WATCHLIST"} 1.40073125175e+11 apiserver_request_latencies_summary_count{resource="endpoints",verb="WATCHLIST"} 316 apiserver_request_latencies_summary{resource="events",verb="LIST",quantile="0.5"} NaN apiserver_request_latencies_summary{resource="events",verb="LIST",quantile="0.9"} NaN apiserver_request_latencies_summary{resource="events",verb="LIST",quantile="0.99"} NaN apiserver_request_latencies_summary_sum{resource="events",verb="LIST"} 3.634932e+06 apiserver_request_latencies_summary_count{resource="events",verb="LIST"} 18 apiserver_request_latencies_summary{resource="events",verb="PATCH",quantile="0.5"} 5661 apiserver_request_latencies_summary{resource="events",verb="PATCH",quantile="0.9"} 27162 apiserver_request_latencies_summary{resource="events",verb="PATCH",quantile="0.99"} 27162 apiserver_request_latencies_summary_sum{resource="events",verb="PATCH"} 9.610014e+06 apiserver_request_latencies_summary_count{resource="events",verb="PATCH"} 237 apiserver_request_latencies_summary{resource="events",verb="POST",quantile="0.5"} 4459 apiserver_request_latencies_summary{resource="events",verb="POST",quantile="0.9"} 57641 apiserver_request_latencies_summary{resource="events",verb="POST",quantile="0.99"} 156154 apiserver_request_latencies_summary_sum{resource="events",verb="POST"} 8.1423503e+07 apiserver_request_latencies_summary_count{resource="events",verb="POST"} 2821 apiserver_request_latencies_summary{resource="horizontalpodautoscalers",verb="LIST",quantile="0.5"} NaN apiserver_request_latencies_summary{resource="horizontalpodautoscalers",verb="LIST",quantile="0.9"} NaN apiserver_request_latencies_summary{resource="horizontalpodautoscalers",verb="LIST",quantile="0.99"} NaN apiserver_request_latencies_summary_sum{resource="horizontalpodautoscalers",verb="LIST"} 25424 apiserver_request_latencies_summary_count{resource="horizontalpodautoscalers",verb="LIST"} 5 apiserver_request_latencies_summary{resource="ingresses",verb="GET",quantile="0.5"} NaN apiserver_request_latencies_summary{resource="ingresses",verb="GET",quantile="0.9"} NaN apiserver_request_latencies_summary{resource="ingresses",verb="GET",quantile="0.99"} NaN apiserver_request_latencies_summary_sum{resource="ingresses",verb="GET"} 96659 apiserver_request_latencies_summary_count{resource="ingresses",verb="GET"} 3 apiserver_request_latencies_summary{resource="ingresses",verb="LIST",quantile="0.5"} 5361 apiserver_request_latencies_summary{resource="ingresses",verb="LIST",quantile="0.9"} 103431 apiserver_request_latencies_summary{resource="ingresses",verb="LIST",quantile="0.99"} 396085 apiserver_request_latencies_summary_sum{resource="ingresses",verb="LIST"} 3.266173711e+09 apiserver_request_latencies_summary_count{resource="ingresses",verb="LIST"} 89540 apiserver_request_latencies_summary{resource="ingresses",verb="PATCH",quantile="0.5"} NaN apiserver_request_latencies_summary{resource="ingresses",verb="PATCH",quantile="0.9"} NaN apiserver_request_latencies_summary{resource="ingresses",verb="PATCH",quantile="0.99"} NaN apiserver_request_latencies_summary_sum{resource="ingresses",verb="PATCH"} 4068 apiserver_request_latencies_summary_count{resource="ingresses",verb="PATCH"} 1 apiserver_request_latencies_summary{resource="ingresses",verb="WATCHLIST",quantile="0.5"} 2.636721623e+09 apiserver_request_latencies_summary{resource="ingresses",verb="WATCHLIST",quantile="0.9"} 2.636721623e+09 apiserver_request_latencies_summary{resource="ingresses",verb="WATCHLIST",quantile="0.99"} 2.636721623e+09 apiserver_request_latencies_summary_sum{resource="ingresses",verb="WATCHLIST"} 6.6265207664e+10 apiserver_request_latencies_summary_count{resource="ingresses",verb="WATCHLIST"} 26 apiserver_request_latencies_summary{resource="jobs",verb="DELETE",quantile="0.5"} NaN apiserver_request_latencies_summary{resource="jobs",verb="DELETE",quantile="0.9"} NaN apiserver_request_latencies_summary{resource="jobs",verb="DELETE",quantile="0.99"} NaN apiserver_request_latencies_summary_sum{resource="jobs",verb="DELETE"} 5.697304e+06 apiserver_request_latencies_summary_count{resource="jobs",verb="DELETE"} 181 apiserver_request_latencies_summary{resource="jobs",verb="GET",quantile="0.5"} NaN apiserver_request_latencies_summary{resource="jobs",verb="GET",quantile="0.9"} NaN apiserver_request_latencies_summary{resource="jobs",verb="GET",quantile="0.99"} NaN apiserver_request_latencies_summary_sum{resource="jobs",verb="GET"} 56190 apiserver_request_latencies_summary_count{resource="jobs",verb="GET"} 12 apiserver_request_latencies_summary{resource="jobs",verb="LIST",quantile="0.5"} NaN apiserver_request_latencies_summary{resource="jobs",verb="LIST",quantile="0.9"} NaN apiserver_request_latencies_summary{resource="jobs",verb="LIST",quantile="0.99"} NaN apiserver_request_latencies_summary_sum{resource="jobs",verb="LIST"} 1.518463e+06 apiserver_request_latencies_summary_count{resource="jobs",verb="LIST"} 11 apiserver_request_latencies_summary{resource="limitranges",verb="GET",quantile="0.5"} NaN apiserver_request_latencies_summary{resource="limitranges",verb="GET",quantile="0.9"} NaN apiserver_request_latencies_summary{resource="limitranges",verb="GET",quantile="0.99"} NaN apiserver_request_latencies_summary_sum{resource="limitranges",verb="GET"} 3016 apiserver_request_latencies_summary_count{resource="limitranges",verb="GET"} 2 apiserver_request_latencies_summary{resource="limitranges",verb="LIST",quantile="0.5"} NaN apiserver_request_latencies_summary{resource="limitranges",verb="LIST",quantile="0.9"} NaN apiserver_request_latencies_summary{resource="limitranges",verb="LIST",quantile="0.99"} NaN apiserver_request_latencies_summary_sum{resource="limitranges",verb="LIST"} 159548 apiserver_request_latencies_summary_count{resource="limitranges",verb="LIST"} 7 apiserver_request_latencies_summary{resource="limitranges",verb="PATCH",quantile="0.5"} NaN apiserver_request_latencies_summary{resource="limitranges",verb="PATCH",quantile="0.9"} NaN apiserver_request_latencies_summary{resource="limitranges",verb="PATCH",quantile="0.99"} NaN apiserver_request_latencies_summary_sum{resource="limitranges",verb="PATCH"} 3588 apiserver_request_latencies_summary_count{resource="limitranges",verb="PATCH"} 2 apiserver_request_latencies_summary{resource="limitranges",verb="WATCH",quantile="0.5"} 4.08000641e+08 apiserver_request_latencies_summary{resource="limitranges",verb="WATCH",quantile="0.9"} 5.29000242e+08 apiserver_request_latencies_summary{resource="limitranges",verb="WATCH",quantile="0.99"} 5.29000242e+08 apiserver_request_latencies_summary_sum{resource="limitranges",verb="WATCH"} 6.9036552855e+10 apiserver_request_latencies_summary_count{resource="limitranges",verb="WATCH"} 154 apiserver_request_latencies_summary{resource="namespaces",verb="GET",quantile="0.5"} 1979 apiserver_request_latencies_summary{resource="namespaces",verb="GET",quantile="0.9"} 73378 apiserver_request_latencies_summary{resource="namespaces",verb="GET",quantile="0.99"} 204904 apiserver_request_latencies_summary_sum{resource="namespaces",verb="GET"} 1.41950381e+08 apiserver_request_latencies_summary_count{resource="namespaces",verb="GET"} 9165 apiserver_request_latencies_summary{resource="namespaces",verb="LIST",quantile="0.5"} NaN apiserver_request_latencies_summary{resource="namespaces",verb="LIST",quantile="0.9"} NaN apiserver_request_latencies_summary{resource="namespaces",verb="LIST",quantile="0.99"} NaN apiserver_request_latencies_summary_sum{resource="namespaces",verb="LIST"} 24493 apiserver_request_latencies_summary_count{resource="namespaces",verb="LIST"} 3 apiserver_request_latencies_summary{resource="namespaces",verb="POST",quantile="0.5"} NaN apiserver_request_latencies_summary{resource="namespaces",verb="POST",quantile="0.9"} NaN apiserver_request_latencies_summary{resource="namespaces",verb="POST",quantile="0.99"} NaN apiserver_request_latencies_summary_sum{resource="namespaces",verb="POST"} 98086 apiserver_request_latencies_summary_count{resource="namespaces",verb="POST"} 1 apiserver_request_latencies_summary{resource="namespaces",verb="WATCH",quantile="0.5"} 13000 apiserver_request_latencies_summary{resource="namespaces",verb="WATCH",quantile="0.9"} 4.64099852e+08 apiserver_request_latencies_summary{resource="namespaces",verb="WATCH",quantile="0.99"} 5.46000321e+08 apiserver_request_latencies_summary_sum{resource="namespaces",verb="WATCH"} 1.35214363507e+11 apiserver_request_latencies_summary_count{resource="namespaces",verb="WATCH"} 942 apiserver_request_latencies_summary{resource="namespaces",verb="WATCHLIST",quantile="0.5"} NaN apiserver_request_latencies_summary{resource="namespaces",verb="WATCHLIST",quantile="0.9"} NaN apiserver_request_latencies_summary{resource="namespaces",verb="WATCHLIST",quantile="0.99"} NaN apiserver_request_latencies_summary_sum{resource="namespaces",verb="WATCHLIST"} 1.770343085e+09 apiserver_request_latencies_summary_count{resource="namespaces",verb="WATCHLIST"} 5 apiserver_request_latencies_summary{resource="nodes",verb="GET",quantile="0.5"} 316 apiserver_request_latencies_summary{resource="nodes",verb="GET",quantile="0.9"} 476 apiserver_request_latencies_summary{resource="nodes",verb="GET",quantile="0.99"} 87917 apiserver_request_latencies_summary_sum{resource="nodes",verb="GET"} 8.6578745e+07 apiserver_request_latencies_summary_count{resource="nodes",verb="GET"} 34108 apiserver_request_latencies_summary{resource="nodes",verb="LIST",quantile="0.5"} NaN apiserver_request_latencies_summary{resource="nodes",verb="LIST",quantile="0.9"} NaN apiserver_request_latencies_summary{resource="nodes",verb="LIST",quantile="0.99"} NaN apiserver_request_latencies_summary_sum{resource="nodes",verb="LIST"} 6.570098e+06 apiserver_request_latencies_summary_count{resource="nodes",verb="LIST"} 197 apiserver_request_latencies_summary{resource="nodes",verb="PATCH",quantile="0.5"} 21959 apiserver_request_latencies_summary{resource="nodes",verb="PATCH",quantile="0.9"} 195052 apiserver_request_latencies_summary{resource="nodes",verb="PATCH",quantile="0.99"} 679667 apiserver_request_latencies_summary_sum{resource="nodes",verb="PATCH"} 1.871017595e+09 apiserver_request_latencies_summary_count{resource="nodes",verb="PATCH"} 34042 apiserver_request_latencies_summary{resource="nodes",verb="POST",quantile="0.5"} NaN apiserver_request_latencies_summary{resource="nodes",verb="POST",quantile="0.9"} NaN apiserver_request_latencies_summary{resource="nodes",verb="POST",quantile="0.99"} NaN apiserver_request_latencies_summary_sum{resource="nodes",verb="POST"} 59161 apiserver_request_latencies_summary_count{resource="nodes",verb="POST"} 8 apiserver_request_latencies_summary{resource="nodes",verb="PUT",quantile="0.5"} NaN apiserver_request_latencies_summary{resource="nodes",verb="PUT",quantile="0.9"} NaN apiserver_request_latencies_summary{resource="nodes",verb="PUT",quantile="0.99"} NaN apiserver_request_latencies_summary_sum{resource="nodes",verb="PUT"} 3.8600184e+07 apiserver_request_latencies_summary_count{resource="nodes",verb="PUT"} 645 apiserver_request_latencies_summary{resource="nodes",verb="WATCH",quantile="0.5"} 22446 apiserver_request_latencies_summary{resource="nodes",verb="WATCH",quantile="0.9"} 3.26000476e+08 apiserver_request_latencies_summary{resource="nodes",verb="WATCH",quantile="0.99"} 5.79114359e+08 apiserver_request_latencies_summary_sum{resource="nodes",verb="WATCH"} 6.7447290162e+11 apiserver_request_latencies_summary_count{resource="nodes",verb="WATCH"} 15538 apiserver_request_latencies_summary{resource="nodes",verb="WATCHLIST",quantile="0.5"} NaN apiserver_request_latencies_summary{resource="nodes",verb="WATCHLIST",quantile="0.9"} NaN apiserver_request_latencies_summary{resource="nodes",verb="WATCHLIST",quantile="0.99"} NaN apiserver_request_latencies_summary_sum{resource="nodes",verb="WATCHLIST"} 1.3354929587e+10 apiserver_request_latencies_summary_count{resource="nodes",verb="WATCHLIST"} 35 apiserver_request_latencies_summary{resource="persistentvolumeclaims",verb="GET",quantile="0.5"} 2264 apiserver_request_latencies_summary{resource="persistentvolumeclaims",verb="GET",quantile="0.9"} 91755 apiserver_request_latencies_summary{resource="persistentvolumeclaims",verb="GET",quantile="0.99"} 295723 apiserver_request_latencies_summary_sum{resource="persistentvolumeclaims",verb="GET"} 8.6392202e+07 apiserver_request_latencies_summary_count{resource="persistentvolumeclaims",verb="GET"} 3788 apiserver_request_latencies_summary{resource="persistentvolumeclaims",verb="LIST",quantile="0.5"} NaN apiserver_request_latencies_summary{resource="persistentvolumeclaims",verb="LIST",quantile="0.9"} NaN apiserver_request_latencies_summary{resource="persistentvolumeclaims",verb="LIST",quantile="0.99"} NaN apiserver_request_latencies_summary_sum{resource="persistentvolumeclaims",verb="LIST"} 288 apiserver_request_latencies_summary_count{resource="persistentvolumeclaims",verb="LIST"} 1 apiserver_request_latencies_summary{resource="persistentvolumeclaims",verb="WATCH",quantile="0.5"} 4.39097054e+08 apiserver_request_latencies_summary{resource="persistentvolumeclaims",verb="WATCH",quantile="0.9"} 5.30000345e+08 apiserver_request_latencies_summary{resource="persistentvolumeclaims",verb="WATCH",quantile="0.99"} 5.30000345e+08 apiserver_request_latencies_summary_sum{resource="persistentvolumeclaims",verb="WATCH"} 6.9312180305e+10 apiserver_request_latencies_summary_count{resource="persistentvolumeclaims",verb="WATCH"} 150 apiserver_request_latencies_summary{resource="persistentvolumes",verb="GET",quantile="0.5"} 1849 apiserver_request_latencies_summary{resource="persistentvolumes",verb="GET",quantile="0.9"} 67847 apiserver_request_latencies_summary{resource="persistentvolumes",verb="GET",quantile="0.99"} 159876 apiserver_request_latencies_summary_sum{resource="persistentvolumes",verb="GET"} 4.9947084e+07 apiserver_request_latencies_summary_count{resource="persistentvolumes",verb="GET"} 3789 apiserver_request_latencies_summary{resource="persistentvolumes",verb="LIST",quantile="0.5"} NaN apiserver_request_latencies_summary{resource="persistentvolumes",verb="LIST",quantile="0.9"} NaN apiserver_request_latencies_summary{resource="persistentvolumes",verb="LIST",quantile="0.99"} NaN apiserver_request_latencies_summary_sum{resource="persistentvolumes",verb="LIST"} 268 apiserver_request_latencies_summary_count{resource="persistentvolumes",verb="LIST"} 1 apiserver_request_latencies_summary{resource="persistentvolumes",verb="WATCH",quantile="0.5"} 4.92000414e+08 apiserver_request_latencies_summary{resource="persistentvolumes",verb="WATCH",quantile="0.9"} 5.80000316e+08 apiserver_request_latencies_summary{resource="persistentvolumes",verb="WATCH",quantile="0.99"} 5.80000316e+08 apiserver_request_latencies_summary_sum{resource="persistentvolumes",verb="WATCH"} 6.9265196092e+10 apiserver_request_latencies_summary_count{resource="persistentvolumes",verb="WATCH"} 152 apiserver_request_latencies_summary{resource="platformcredentialssets",verb="GET",quantile="0.5"} 2114 apiserver_request_latencies_summary{resource="platformcredentialssets",verb="GET",quantile="0.9"} 2114 apiserver_request_latencies_summary{resource="platformcredentialssets",verb="GET",quantile="0.99"} 2114 apiserver_request_latencies_summary_sum{resource="platformcredentialssets",verb="GET"} 138689 apiserver_request_latencies_summary_count{resource="platformcredentialssets",verb="GET"} 3 apiserver_request_latencies_summary{resource="platformcredentialssets",verb="LIST",quantile="0.5"} 3158 apiserver_request_latencies_summary{resource="platformcredentialssets",verb="LIST",quantile="0.9"} 140029 apiserver_request_latencies_summary{resource="platformcredentialssets",verb="LIST",quantile="0.99"} 282654 apiserver_request_latencies_summary_sum{resource="platformcredentialssets",verb="LIST"} 1.0165435e+07 apiserver_request_latencies_summary_count{resource="platformcredentialssets",verb="LIST"} 567 apiserver_request_latencies_summary{resource="platformcredentialssets",verb="PUT",quantile="0.5"} 5926 apiserver_request_latencies_summary{resource="platformcredentialssets",verb="PUT",quantile="0.9"} 5926 apiserver_request_latencies_summary{resource="platformcredentialssets",verb="PUT",quantile="0.99"} 5926 apiserver_request_latencies_summary_sum{resource="platformcredentialssets",verb="PUT"} 11771 apiserver_request_latencies_summary_count{resource="platformcredentialssets",verb="PUT"} 2 apiserver_request_latencies_summary{resource="poddisruptionbudgets",verb="LIST",quantile="0.5"} NaN apiserver_request_latencies_summary{resource="poddisruptionbudgets",verb="LIST",quantile="0.9"} NaN apiserver_request_latencies_summary{resource="poddisruptionbudgets",verb="LIST",quantile="0.99"} NaN apiserver_request_latencies_summary_sum{resource="poddisruptionbudgets",verb="LIST"} 3.206562e+06 apiserver_request_latencies_summary_count{resource="poddisruptionbudgets",verb="LIST"} 99 apiserver_request_latencies_summary{resource="pods",verb="DELETE",quantile="0.5"} 25526 apiserver_request_latencies_summary{resource="pods",verb="DELETE",quantile="0.9"} 25526 apiserver_request_latencies_summary{resource="pods",verb="DELETE",quantile="0.99"} 25526 apiserver_request_latencies_summary_sum{resource="pods",verb="DELETE"} 3.2938e+07 apiserver_request_latencies_summary_count{resource="pods",verb="DELETE"} 329 apiserver_request_latencies_summary{resource="pods",verb="GET",quantile="0.5"} 15084 apiserver_request_latencies_summary{resource="pods",verb="GET",quantile="0.9"} 302509 apiserver_request_latencies_summary{resource="pods",verb="GET",quantile="0.99"} 2.03264806e+09 apiserver_request_latencies_summary_sum{resource="pods",verb="GET"} 1.5191193458e+10 apiserver_request_latencies_summary_count{resource="pods",verb="GET"} 6204 apiserver_request_latencies_summary{resource="pods",verb="LIST",quantile="0.5"} NaN apiserver_request_latencies_summary{resource="pods",verb="LIST",quantile="0.9"} NaN apiserver_request_latencies_summary{resource="pods",verb="LIST",quantile="0.99"} NaN apiserver_request_latencies_summary_sum{resource="pods",verb="LIST"} 2.5924607e+07 apiserver_request_latencies_summary_count{resource="pods",verb="LIST"} 136 apiserver_request_latencies_summary{resource="pods",verb="POST",quantile="0.5"} NaN apiserver_request_latencies_summary{resource="pods",verb="POST",quantile="0.9"} NaN apiserver_request_latencies_summary{resource="pods",verb="POST",quantile="0.99"} NaN apiserver_request_latencies_summary_sum{resource="pods",verb="POST"} 3.0661325e+07 apiserver_request_latencies_summary_count{resource="pods",verb="POST"} 110 apiserver_request_latencies_summary{resource="pods",verb="PUT",quantile="0.5"} 11627 apiserver_request_latencies_summary{resource="pods",verb="PUT",quantile="0.9"} 140255 apiserver_request_latencies_summary{resource="pods",verb="PUT",quantile="0.99"} 431418 apiserver_request_latencies_summary_sum{resource="pods",verb="PUT"} 7.9591485e+07 apiserver_request_latencies_summary_count{resource="pods",verb="PUT"} 1733 apiserver_request_latencies_summary{resource="pods",verb="WATCH",quantile="0.5"} 178658 apiserver_request_latencies_summary{resource="pods",verb="WATCH",quantile="0.9"} 1.473512e+06 apiserver_request_latencies_summary{resource="pods",verb="WATCH",quantile="0.99"} 5.38097152e+08 apiserver_request_latencies_summary_sum{resource="pods",verb="WATCH"} 7.49392191107e+11 apiserver_request_latencies_summary_count{resource="pods",verb="WATCH"} 21319 apiserver_request_latencies_summary{resource="pods",verb="WATCHLIST",quantile="0.5"} 4.49038218e+08 apiserver_request_latencies_summary{resource="pods",verb="WATCHLIST",quantile="0.9"} 5.30002642e+08 apiserver_request_latencies_summary{resource="pods",verb="WATCHLIST",quantile="0.99"} 5.78011208e+08 apiserver_request_latencies_summary_sum{resource="pods",verb="WATCHLIST"} 2.19963769117e+11 apiserver_request_latencies_summary_count{resource="pods",verb="WATCHLIST"} 507 apiserver_request_latencies_summary{resource="podsecuritypolicies",verb="GET",quantile="0.5"} NaN apiserver_request_latencies_summary{resource="podsecuritypolicies",verb="GET",quantile="0.9"} NaN apiserver_request_latencies_summary{resource="podsecuritypolicies",verb="GET",quantile="0.99"} NaN apiserver_request_latencies_summary_sum{resource="podsecuritypolicies",verb="GET"} 118464 apiserver_request_latencies_summary_count{resource="podsecuritypolicies",verb="GET"} 2 apiserver_request_latencies_summary{resource="podsecuritypolicies",verb="LIST",quantile="0.5"} NaN apiserver_request_latencies_summary{resource="podsecuritypolicies",verb="LIST",quantile="0.9"} NaN apiserver_request_latencies_summary{resource="podsecuritypolicies",verb="LIST",quantile="0.99"} NaN apiserver_request_latencies_summary_sum{resource="podsecuritypolicies",verb="LIST"} 266 apiserver_request_latencies_summary_count{resource="podsecuritypolicies",verb="LIST"} 1 apiserver_request_latencies_summary{resource="podsecuritypolicies",verb="PATCH",quantile="0.5"} NaN apiserver_request_latencies_summary{resource="podsecuritypolicies",verb="PATCH",quantile="0.9"} NaN apiserver_request_latencies_summary{resource="podsecuritypolicies",verb="PATCH",quantile="0.99"} NaN apiserver_request_latencies_summary_sum{resource="podsecuritypolicies",verb="PATCH"} 133715 apiserver_request_latencies_summary_count{resource="podsecuritypolicies",verb="PATCH"} 2 apiserver_request_latencies_summary{resource="podsecuritypolicies",verb="WATCH",quantile="0.5"} 4.16096411e+08 apiserver_request_latencies_summary{resource="podsecuritypolicies",verb="WATCH",quantile="0.9"} 5.30102237e+08 apiserver_request_latencies_summary{resource="podsecuritypolicies",verb="WATCH",quantile="0.99"} 5.30102237e+08 apiserver_request_latencies_summary_sum{resource="podsecuritypolicies",verb="WATCH"} 6.9172585559e+10 apiserver_request_latencies_summary_count{resource="podsecuritypolicies",verb="WATCH"} 151 apiserver_request_latencies_summary{resource="replicasets",verb="GET",quantile="0.5"} NaN apiserver_request_latencies_summary{resource="replicasets",verb="GET",quantile="0.9"} NaN apiserver_request_latencies_summary{resource="replicasets",verb="GET",quantile="0.99"} NaN apiserver_request_latencies_summary_sum{resource="replicasets",verb="GET"} 1.616098e+06 apiserver_request_latencies_summary_count{resource="replicasets",verb="GET"} 135 apiserver_request_latencies_summary{resource="replicasets",verb="LIST",quantile="0.5"} 49174 apiserver_request_latencies_summary{resource="replicasets",verb="LIST",quantile="0.9"} 700515 apiserver_request_latencies_summary{resource="replicasets",verb="LIST",quantile="0.99"} 3.169516e+06 apiserver_request_latencies_summary_sum{resource="replicasets",verb="LIST"} 9.29912859e+08 apiserver_request_latencies_summary_count{resource="replicasets",verb="LIST"} 3052 apiserver_request_latencies_summary{resource="replicasets",verb="WATCH",quantile="0.5"} 4.13000378e+08 apiserver_request_latencies_summary{resource="replicasets",verb="WATCH",quantile="0.9"} 5.90015222e+08 apiserver_request_latencies_summary{resource="replicasets",verb="WATCH",quantile="0.99"} 5.90015222e+08 apiserver_request_latencies_summary_sum{resource="replicasets",verb="WATCH"} 6.9002435522e+10 apiserver_request_latencies_summary_count{resource="replicasets",verb="WATCH"} 153 apiserver_request_latencies_summary{resource="replicationcontrollers",verb="LIST",quantile="0.5"} NaN apiserver_request_latencies_summary{resource="replicationcontrollers",verb="LIST",quantile="0.9"} NaN apiserver_request_latencies_summary{resource="replicationcontrollers",verb="LIST",quantile="0.99"} NaN apiserver_request_latencies_summary_sum{resource="replicationcontrollers",verb="LIST"} 853253 apiserver_request_latencies_summary_count{resource="replicationcontrollers",verb="LIST"} 3 apiserver_request_latencies_summary{resource="replicationcontrollers",verb="WATCH",quantile="0.5"} 4.02105428e+08 apiserver_request_latencies_summary{resource="replicationcontrollers",verb="WATCH",quantile="0.9"} 5.3400547e+08 apiserver_request_latencies_summary{resource="replicationcontrollers",verb="WATCH",quantile="0.99"} 5.3400547e+08 apiserver_request_latencies_summary_sum{resource="replicationcontrollers",verb="WATCH"} 6.8803924319e+10 apiserver_request_latencies_summary_count{resource="replicationcontrollers",verb="WATCH"} 150 apiserver_request_latencies_summary{resource="resourcequotas",verb="LIST",quantile="0.5"} 1680 apiserver_request_latencies_summary{resource="resourcequotas",verb="LIST",quantile="0.9"} 1680 apiserver_request_latencies_summary{resource="resourcequotas",verb="LIST",quantile="0.99"} 1680 apiserver_request_latencies_summary_sum{resource="resourcequotas",verb="LIST"} 69519 apiserver_request_latencies_summary_count{resource="resourcequotas",verb="LIST"} 10 apiserver_request_latencies_summary{resource="resourcequotas",verb="WATCH",quantile="0.5"} 4.14007543e+08 apiserver_request_latencies_summary{resource="resourcequotas",verb="WATCH",quantile="0.9"} 5.01009368e+08 apiserver_request_latencies_summary{resource="resourcequotas",verb="WATCH",quantile="0.99"} 5.01009368e+08 apiserver_request_latencies_summary_sum{resource="resourcequotas",verb="WATCH"} 6.8889954315e+10 apiserver_request_latencies_summary_count{resource="resourcequotas",verb="WATCH"} 153 apiserver_request_latencies_summary{resource="rolebindings",verb="GET",quantile="0.5"} NaN apiserver_request_latencies_summary{resource="rolebindings",verb="GET",quantile="0.9"} NaN apiserver_request_latencies_summary{resource="rolebindings",verb="GET",quantile="0.99"} NaN apiserver_request_latencies_summary_sum{resource="rolebindings",verb="GET"} 3078 apiserver_request_latencies_summary_count{resource="rolebindings",verb="GET"} 3 apiserver_request_latencies_summary{resource="rolebindings",verb="POST",quantile="0.5"} NaN apiserver_request_latencies_summary{resource="rolebindings",verb="POST",quantile="0.9"} NaN apiserver_request_latencies_summary{resource="rolebindings",verb="POST",quantile="0.99"} NaN apiserver_request_latencies_summary_sum{resource="rolebindings",verb="POST"} 16918 apiserver_request_latencies_summary_count{resource="rolebindings",verb="POST"} 3 apiserver_request_latencies_summary{resource="roles",verb="GET",quantile="0.5"} NaN apiserver_request_latencies_summary{resource="roles",verb="GET",quantile="0.9"} NaN apiserver_request_latencies_summary{resource="roles",verb="GET",quantile="0.99"} NaN apiserver_request_latencies_summary_sum{resource="roles",verb="GET"} 10760 apiserver_request_latencies_summary_count{resource="roles",verb="GET"} 4 apiserver_request_latencies_summary{resource="roles",verb="POST",quantile="0.5"} NaN apiserver_request_latencies_summary{resource="roles",verb="POST",quantile="0.9"} NaN apiserver_request_latencies_summary{resource="roles",verb="POST",quantile="0.99"} NaN apiserver_request_latencies_summary_sum{resource="roles",verb="POST"} 25060 apiserver_request_latencies_summary_count{resource="roles",verb="POST"} 4 apiserver_request_latencies_summary{resource="secrets",verb="GET",quantile="0.5"} 228 apiserver_request_latencies_summary{resource="secrets",verb="GET",quantile="0.9"} 676 apiserver_request_latencies_summary{resource="secrets",verb="GET",quantile="0.99"} 88423 apiserver_request_latencies_summary_sum{resource="secrets",verb="GET"} 3.57061788e+08 apiserver_request_latencies_summary_count{resource="secrets",verb="GET"} 131617 apiserver_request_latencies_summary{resource="secrets",verb="LIST",quantile="0.5"} NaN apiserver_request_latencies_summary{resource="secrets",verb="LIST",quantile="0.9"} NaN apiserver_request_latencies_summary{resource="secrets",verb="LIST",quantile="0.99"} NaN apiserver_request_latencies_summary_sum{resource="secrets",verb="LIST"} 1531 apiserver_request_latencies_summary_count{resource="secrets",verb="LIST"} 1 apiserver_request_latencies_summary{resource="secrets",verb="PATCH",quantile="0.5"} NaN apiserver_request_latencies_summary{resource="secrets",verb="PATCH",quantile="0.9"} NaN apiserver_request_latencies_summary{resource="secrets",verb="PATCH",quantile="0.99"} NaN apiserver_request_latencies_summary_sum{resource="secrets",verb="PATCH"} 13419 apiserver_request_latencies_summary_count{resource="secrets",verb="PATCH"} 1 apiserver_request_latencies_summary{resource="secrets",verb="PUT",quantile="0.5"} 2381 apiserver_request_latencies_summary{resource="secrets",verb="PUT",quantile="0.9"} 40409 apiserver_request_latencies_summary{resource="secrets",verb="PUT",quantile="0.99"} 116052 apiserver_request_latencies_summary_sum{resource="secrets",verb="PUT"} 7.5898444e+07 apiserver_request_latencies_summary_count{resource="secrets",verb="PUT"} 8309 apiserver_request_latencies_summary{resource="secrets",verb="WATCH",quantile="0.5"} 4.55000207e+08 apiserver_request_latencies_summary{resource="secrets",verb="WATCH",quantile="0.9"} 5.52000283e+08 apiserver_request_latencies_summary{resource="secrets",verb="WATCH",quantile="0.99"} 5.52000283e+08 apiserver_request_latencies_summary_sum{resource="secrets",verb="WATCH"} 6.9149720375e+10 apiserver_request_latencies_summary_count{resource="secrets",verb="WATCH"} 151 apiserver_request_latencies_summary{resource="serviceaccounts",verb="GET",quantile="0.5"} 6451 apiserver_request_latencies_summary{resource="serviceaccounts",verb="GET",quantile="0.9"} 26912 apiserver_request_latencies_summary{resource="serviceaccounts",verb="GET",quantile="0.99"} 58865 apiserver_request_latencies_summary_sum{resource="serviceaccounts",verb="GET"} 2.592425e+06 apiserver_request_latencies_summary_count{resource="serviceaccounts",verb="GET"} 348 apiserver_request_latencies_summary{resource="serviceaccounts",verb="LIST",quantile="0.5"} NaN apiserver_request_latencies_summary{resource="serviceaccounts",verb="LIST",quantile="0.9"} NaN apiserver_request_latencies_summary{resource="serviceaccounts",verb="LIST",quantile="0.99"} NaN apiserver_request_latencies_summary_sum{resource="serviceaccounts",verb="LIST"} 333 apiserver_request_latencies_summary_count{resource="serviceaccounts",verb="LIST"} 1 apiserver_request_latencies_summary{resource="serviceaccounts",verb="WATCH",quantile="0.5"} 4.65000319e+08 apiserver_request_latencies_summary{resource="serviceaccounts",verb="WATCH",quantile="0.9"} 5.78000753e+08 apiserver_request_latencies_summary{resource="serviceaccounts",verb="WATCH",quantile="0.99"} 5.78000753e+08 apiserver_request_latencies_summary_sum{resource="serviceaccounts",verb="WATCH"} 6.8774857752e+10 apiserver_request_latencies_summary_count{resource="serviceaccounts",verb="WATCH"} 154 apiserver_request_latencies_summary{resource="services",verb="CONNECT",quantile="0.5"} 31864 apiserver_request_latencies_summary{resource="services",verb="CONNECT",quantile="0.9"} 202213 apiserver_request_latencies_summary{resource="services",verb="CONNECT",quantile="0.99"} 423823 apiserver_request_latencies_summary_sum{resource="services",verb="CONNECT"} 3.85858382e+08 apiserver_request_latencies_summary_count{resource="services",verb="CONNECT"} 6534 apiserver_request_latencies_summary{resource="services",verb="GET",quantile="0.5"} 1842 apiserver_request_latencies_summary{resource="services",verb="GET",quantile="0.9"} 34003 apiserver_request_latencies_summary{resource="services",verb="GET",quantile="0.99"} 174914 apiserver_request_latencies_summary_sum{resource="services",verb="GET"} 2.4446125831e+10 apiserver_request_latencies_summary_count{resource="services",verb="GET"} 2.70826e+06 apiserver_request_latencies_summary{resource="services",verb="LIST",quantile="0.5"} 2919 apiserver_request_latencies_summary{resource="services",verb="LIST",quantile="0.9"} 35428 apiserver_request_latencies_summary{resource="services",verb="LIST",quantile="0.99"} 93095 apiserver_request_latencies_summary_sum{resource="services",verb="LIST"} 3.3720752e+07 apiserver_request_latencies_summary_count{resource="services",verb="LIST"} 3436 apiserver_request_latencies_summary{resource="services",verb="PATCH",quantile="0.5"} 6660 apiserver_request_latencies_summary{resource="services",verb="PATCH",quantile="0.9"} 13098 apiserver_request_latencies_summary{resource="services",verb="PATCH",quantile="0.99"} 13098 apiserver_request_latencies_summary_sum{resource="services",verb="PATCH"} 66210 apiserver_request_latencies_summary_count{resource="services",verb="PATCH"} 5 apiserver_request_latencies_summary{resource="services",verb="PROXY",quantile="0.5"} NaN apiserver_request_latencies_summary{resource="services",verb="PROXY",quantile="0.9"} NaN apiserver_request_latencies_summary{resource="services",verb="PROXY",quantile="0.99"} NaN apiserver_request_latencies_summary_sum{resource="services",verb="PROXY"} 3.057711e+06 apiserver_request_latencies_summary_count{resource="services",verb="PROXY"} 36 apiserver_request_latencies_summary{resource="services",verb="WATCH",quantile="0.5"} 2.99260944e+08 apiserver_request_latencies_summary{resource="services",verb="WATCH",quantile="0.9"} 3.21000272e+08 apiserver_request_latencies_summary{resource="services",verb="WATCH",quantile="0.99"} 5.6100129e+08 apiserver_request_latencies_summary_sum{resource="services",verb="WATCH"} 7.46005271353e+11 apiserver_request_latencies_summary_count{resource="services",verb="WATCH"} 4138 apiserver_request_latencies_summary{resource="services",verb="WATCHLIST",quantile="0.5"} 4.20001532e+08 apiserver_request_latencies_summary{resource="services",verb="WATCHLIST",quantile="0.9"} 5.72023259e+08 apiserver_request_latencies_summary{resource="services",verb="WATCHLIST",quantile="0.99"} 5.75000296e+08 apiserver_request_latencies_summary_sum{resource="services",verb="WATCHLIST"} 2.14384246956e+11 apiserver_request_latencies_summary_count{resource="services",verb="WATCHLIST"} 395 apiserver_request_latencies_summary{resource="services",verb="proxy",quantile="0.5"} NaN apiserver_request_latencies_summary{resource="services",verb="proxy",quantile="0.9"} NaN apiserver_request_latencies_summary{resource="services",verb="proxy",quantile="0.99"} NaN apiserver_request_latencies_summary_sum{resource="services",verb="proxy"} 3.057303e+06 apiserver_request_latencies_summary_count{resource="services",verb="proxy"} 36 apiserver_request_latencies_summary{resource="statefulsets",verb="GET",quantile="0.5"} NaN apiserver_request_latencies_summary{resource="statefulsets",verb="GET",quantile="0.9"} NaN apiserver_request_latencies_summary{resource="statefulsets",verb="GET",quantile="0.99"} NaN apiserver_request_latencies_summary_sum{resource="statefulsets",verb="GET"} 76840 apiserver_request_latencies_summary_count{resource="statefulsets",verb="GET"} 18 apiserver_request_latencies_summary{resource="statefulsets",verb="LIST",quantile="0.5"} 1874 apiserver_request_latencies_summary{resource="statefulsets",verb="LIST",quantile="0.9"} 11993 apiserver_request_latencies_summary{resource="statefulsets",verb="LIST",quantile="0.99"} 100288 apiserver_request_latencies_summary_sum{resource="statefulsets",verb="LIST"} 3.6405017e+07 apiserver_request_latencies_summary_count{resource="statefulsets",verb="LIST"} 3428 apiserver_request_latencies_summary{resource="statefulsets",verb="WATCH",quantile="0.5"} 3.65000606e+08 apiserver_request_latencies_summary{resource="statefulsets",verb="WATCH",quantile="0.9"} 5.24075773e+08 apiserver_request_latencies_summary{resource="statefulsets",verb="WATCH",quantile="0.99"} 5.24075773e+08 apiserver_request_latencies_summary_sum{resource="statefulsets",verb="WATCH"} 6.9107273702e+10 apiserver_request_latencies_summary_count{resource="statefulsets",verb="WATCH"} 153 apiserver_request_latencies_summary{resource="storageclasses",verb="GET",quantile="0.5"} NaN apiserver_request_latencies_summary{resource="storageclasses",verb="GET",quantile="0.9"} NaN apiserver_request_latencies_summary{resource="storageclasses",verb="GET",quantile="0.99"} NaN apiserver_request_latencies_summary_sum{resource="storageclasses",verb="GET"} 69073 apiserver_request_latencies_summary_count{resource="storageclasses",verb="GET"} 1 apiserver_request_latencies_summary{resource="storageclasses",verb="LIST",quantile="0.5"} NaN apiserver_request_latencies_summary{resource="storageclasses",verb="LIST",quantile="0.9"} NaN apiserver_request_latencies_summary{resource="storageclasses",verb="LIST",quantile="0.99"} NaN apiserver_request_latencies_summary_sum{resource="storageclasses",verb="LIST"} 316 apiserver_request_latencies_summary_count{resource="storageclasses",verb="LIST"} 1 apiserver_request_latencies_summary{resource="storageclasses",verb="PATCH",quantile="0.5"} NaN apiserver_request_latencies_summary{resource="storageclasses",verb="PATCH",quantile="0.9"} NaN apiserver_request_latencies_summary{resource="storageclasses",verb="PATCH",quantile="0.99"} NaN apiserver_request_latencies_summary_sum{resource="storageclasses",verb="PATCH"} 15753 apiserver_request_latencies_summary_count{resource="storageclasses",verb="PATCH"} 1 apiserver_request_latencies_summary{resource="storageclasses",verb="WATCH",quantile="0.5"} 4.25589578e+08 apiserver_request_latencies_summary{resource="storageclasses",verb="WATCH",quantile="0.9"} 5.63000356e+08 apiserver_request_latencies_summary{resource="storageclasses",verb="WATCH",quantile="0.99"} 5.63000356e+08 apiserver_request_latencies_summary_sum{resource="storageclasses",verb="WATCH"} 6.8948799683e+10 apiserver_request_latencies_summary_count{resource="storageclasses",verb="WATCH"} 149 apiserver_request_latencies_summary{resource="thirdpartyresources",verb="GET",quantile="0.5"} NaN apiserver_request_latencies_summary{resource="thirdpartyresources",verb="GET",quantile="0.9"} NaN apiserver_request_latencies_summary{resource="thirdpartyresources",verb="GET",quantile="0.99"} NaN apiserver_request_latencies_summary_sum{resource="thirdpartyresources",verb="GET"} 34480 apiserver_request_latencies_summary_count{resource="thirdpartyresources",verb="GET"} 1 apiserver_request_latencies_summary{resource="thirdpartyresources",verb="LIST",quantile="0.5"} 2161 apiserver_request_latencies_summary{resource="thirdpartyresources",verb="LIST",quantile="0.9"} 91926 apiserver_request_latencies_summary{resource="thirdpartyresources",verb="LIST",quantile="0.99"} 274245 apiserver_request_latencies_summary_sum{resource="thirdpartyresources",verb="LIST"} 1.59744481e+08 apiserver_request_latencies_summary_count{resource="thirdpartyresources",verb="LIST"} 6888 apiserver_request_latencies_summary{resource="thirdpartyresources",verb="PATCH",quantile="0.5"} NaN apiserver_request_latencies_summary{resource="thirdpartyresources",verb="PATCH",quantile="0.9"} NaN apiserver_request_latencies_summary{resource="thirdpartyresources",verb="PATCH",quantile="0.99"} NaN apiserver_request_latencies_summary_sum{resource="thirdpartyresources",verb="PATCH"} 130468 apiserver_request_latencies_summary_count{resource="thirdpartyresources",verb="PATCH"} 1 # HELP authenticated_user_requests Counter of authenticated requests broken out by username. # TYPE authenticated_user_requests counter authenticated_user_requests{username="kubelet"} 203340 authenticated_user_requests{username="other"} 93227 authenticated_user_requests{username="system:serviceaccount:kube-system:default"} 2.915006e+06 # HELP etcd_helper_cache_entry_count Counter of etcd helper cache entries. This can be different from etcd_helper_cache_miss_count because two concurrent threads can miss the cache and generate the same entry twice. # TYPE etcd_helper_cache_entry_count counter etcd_helper_cache_entry_count 179866 # HELP etcd_helper_cache_hit_count Counter of etcd helper cache hits. # TYPE etcd_helper_cache_hit_count counter etcd_helper_cache_hit_count 5.645214e+06 # HELP etcd_helper_cache_miss_count Counter of etcd helper cache miss. # TYPE etcd_helper_cache_miss_count counter etcd_helper_cache_miss_count 179866 # HELP etcd_request_cache_add_latencies_summary Latency in microseconds of adding an object to etcd cache # TYPE etcd_request_cache_add_latencies_summary summary etcd_request_cache_add_latencies_summary{quantile="0.5"} 16 etcd_request_cache_add_latencies_summary{quantile="0.9"} 37 etcd_request_cache_add_latencies_summary{quantile="0.99"} 230 etcd_request_cache_add_latencies_summary_sum 2.8603437e+07 etcd_request_cache_add_latencies_summary_count 179866 # HELP etcd_request_cache_get_latencies_summary Latency in microseconds of getting an object from etcd cache # TYPE etcd_request_cache_get_latencies_summary summary etcd_request_cache_get_latencies_summary{quantile="0.5"} 12 etcd_request_cache_get_latencies_summary{quantile="0.9"} 20 etcd_request_cache_get_latencies_summary{quantile="0.99"} 173 etcd_request_cache_get_latencies_summary_sum 6.48834905e+08 etcd_request_cache_get_latencies_summary_count 5.840501e+06 # HELP etcd_request_latencies_summary Etcd request latency summary in microseconds for each operation and object type. # TYPE etcd_request_latencies_summary summary etcd_request_latencies_summary{operation="compareAndSwap",type="*api.Endpoints",quantile="0.5"} 3737 etcd_request_latencies_summary{operation="compareAndSwap",type="*api.Endpoints",quantile="0.9"} 16793 etcd_request_latencies_summary{operation="compareAndSwap",type="*api.Endpoints",quantile="0.99"} 93497 etcd_request_latencies_summary_sum{operation="compareAndSwap",type="*api.Endpoints"} 9.2942938e+07 etcd_request_latencies_summary_count{operation="compareAndSwap",type="*api.Endpoints"} 6823 etcd_request_latencies_summary{operation="compareAndSwap",type="*api.Event",quantile="0.5"} NaN etcd_request_latencies_summary{operation="compareAndSwap",type="*api.Event",quantile="0.9"} NaN etcd_request_latencies_summary{operation="compareAndSwap",type="*api.Event",quantile="0.99"} NaN etcd_request_latencies_summary_sum{operation="compareAndSwap",type="*api.Event"} 4.249507e+06 etcd_request_latencies_summary_count{operation="compareAndSwap",type="*api.Event"} 230 etcd_request_latencies_summary{operation="compareAndSwap",type="*api.Node",quantile="0.5"} 8526 etcd_request_latencies_summary{operation="compareAndSwap",type="*api.Node",quantile="0.9"} 101979 etcd_request_latencies_summary{operation="compareAndSwap",type="*api.Node",quantile="0.99"} 300515 etcd_request_latencies_summary_sum{operation="compareAndSwap",type="*api.Node"} 9.5665247e+08 etcd_request_latencies_summary_count{operation="compareAndSwap",type="*api.Node"} 34679 etcd_request_latencies_summary{operation="compareAndSwap",type="*api.Pod",quantile="0.5"} 17458 etcd_request_latencies_summary{operation="compareAndSwap",type="*api.Pod",quantile="0.9"} 17458 etcd_request_latencies_summary{operation="compareAndSwap",type="*api.Pod",quantile="0.99"} 17458 etcd_request_latencies_summary_sum{operation="compareAndSwap",type="*api.Pod"} 5.2436431e+07 etcd_request_latencies_summary_count{operation="compareAndSwap",type="*api.Pod"} 1856 etcd_request_latencies_summary{operation="compareAndSwap",type="*api.Secret",quantile="0.5"} 6366 etcd_request_latencies_summary{operation="compareAndSwap",type="*api.Secret",quantile="0.9"} 6366 etcd_request_latencies_summary{operation="compareAndSwap",type="*api.Secret",quantile="0.99"} 6366 etcd_request_latencies_summary_sum{operation="compareAndSwap",type="*api.Secret"} 1.405841e+06 etcd_request_latencies_summary_count{operation="compareAndSwap",type="*api.Secret"} 185 etcd_request_latencies_summary{operation="compareAndSwap",type="*extensions.DaemonSet",quantile="0.5"} NaN etcd_request_latencies_summary{operation="compareAndSwap",type="*extensions.DaemonSet",quantile="0.9"} NaN etcd_request_latencies_summary{operation="compareAndSwap",type="*extensions.DaemonSet",quantile="0.99"} NaN etcd_request_latencies_summary_sum{operation="compareAndSwap",type="*extensions.DaemonSet"} 183741 etcd_request_latencies_summary_count{operation="compareAndSwap",type="*extensions.DaemonSet"} 4 etcd_request_latencies_summary{operation="compareAndSwap",type="*extensions.Deployment",quantile="0.5"} NaN etcd_request_latencies_summary{operation="compareAndSwap",type="*extensions.Deployment",quantile="0.9"} NaN etcd_request_latencies_summary{operation="compareAndSwap",type="*extensions.Deployment",quantile="0.99"} NaN etcd_request_latencies_summary_sum{operation="compareAndSwap",type="*extensions.Deployment"} 180535 etcd_request_latencies_summary_count{operation="compareAndSwap",type="*extensions.Deployment"} 13 etcd_request_latencies_summary{operation="compareAndSwap",type="*extensions.ThirdPartyResourceData",quantile="0.5"} NaN etcd_request_latencies_summary{operation="compareAndSwap",type="*extensions.ThirdPartyResourceData",quantile="0.9"} NaN etcd_request_latencies_summary{operation="compareAndSwap",type="*extensions.ThirdPartyResourceData",quantile="0.99"} NaN etcd_request_latencies_summary_sum{operation="compareAndSwap",type="*extensions.ThirdPartyResourceData"} 7639 etcd_request_latencies_summary_count{operation="compareAndSwap",type="*extensions.ThirdPartyResourceData"} 2 etcd_request_latencies_summary{operation="compareAndSwap",type="*rbac.ClusterRole",quantile="0.5"} NaN etcd_request_latencies_summary{operation="compareAndSwap",type="*rbac.ClusterRole",quantile="0.9"} NaN etcd_request_latencies_summary{operation="compareAndSwap",type="*rbac.ClusterRole",quantile="0.99"} NaN etcd_request_latencies_summary_sum{operation="compareAndSwap",type="*rbac.ClusterRole"} 435202 etcd_request_latencies_summary_count{operation="compareAndSwap",type="*rbac.ClusterRole"} 9 etcd_request_latencies_summary{operation="compareAndSwap",type="*rbac.ClusterRoleBinding",quantile="0.5"} NaN etcd_request_latencies_summary{operation="compareAndSwap",type="*rbac.ClusterRoleBinding",quantile="0.9"} NaN etcd_request_latencies_summary{operation="compareAndSwap",type="*rbac.ClusterRoleBinding",quantile="0.99"} NaN etcd_request_latencies_summary_sum{operation="compareAndSwap",type="*rbac.ClusterRoleBinding"} 24286 etcd_request_latencies_summary_count{operation="compareAndSwap",type="*rbac.ClusterRoleBinding"} 6 etcd_request_latencies_summary{operation="compareAndSwap",type="*storage.StorageClass",quantile="0.5"} NaN etcd_request_latencies_summary{operation="compareAndSwap",type="*storage.StorageClass",quantile="0.9"} NaN etcd_request_latencies_summary{operation="compareAndSwap",type="*storage.StorageClass",quantile="0.99"} NaN etcd_request_latencies_summary_sum{operation="compareAndSwap",type="*storage.StorageClass"} 4100 etcd_request_latencies_summary_count{operation="compareAndSwap",type="*storage.StorageClass"} 1 etcd_request_latencies_summary{operation="create",type="*api.Event",quantile="0.5"} 3210 etcd_request_latencies_summary{operation="create",type="*api.Event",quantile="0.9"} 3698 etcd_request_latencies_summary{operation="create",type="*api.Event",quantile="0.99"} 3826 etcd_request_latencies_summary_sum{operation="create",type="*api.Event"} 7.3673064e+07 etcd_request_latencies_summary_count{operation="create",type="*api.Event"} 2821 etcd_request_latencies_summary{operation="create",type="*api.Namespace",quantile="0.5"} NaN etcd_request_latencies_summary{operation="create",type="*api.Namespace",quantile="0.9"} NaN etcd_request_latencies_summary{operation="create",type="*api.Namespace",quantile="0.99"} NaN etcd_request_latencies_summary_sum{operation="create",type="*api.Namespace"} 97615 etcd_request_latencies_summary_count{operation="create",type="*api.Namespace"} 1 etcd_request_latencies_summary{operation="create",type="*api.Node",quantile="0.5"} NaN etcd_request_latencies_summary{operation="create",type="*api.Node",quantile="0.9"} NaN etcd_request_latencies_summary{operation="create",type="*api.Node",quantile="0.99"} NaN etcd_request_latencies_summary_sum{operation="create",type="*api.Node"} 51890 etcd_request_latencies_summary_count{operation="create",type="*api.Node"} 8 etcd_request_latencies_summary{operation="create",type="*api.Pod",quantile="0.5"} NaN etcd_request_latencies_summary{operation="create",type="*api.Pod",quantile="0.9"} NaN etcd_request_latencies_summary{operation="create",type="*api.Pod",quantile="0.99"} NaN etcd_request_latencies_summary_sum{operation="create",type="*api.Pod"} 258408 etcd_request_latencies_summary_count{operation="create",type="*api.Pod"} 11 etcd_request_latencies_summary{operation="create",type="*batch.CronJob",quantile="0.5"} NaN etcd_request_latencies_summary{operation="create",type="*batch.CronJob",quantile="0.9"} NaN etcd_request_latencies_summary{operation="create",type="*batch.CronJob",quantile="0.99"} NaN etcd_request_latencies_summary_sum{operation="create",type="*batch.CronJob"} 7305 etcd_request_latencies_summary_count{operation="create",type="*batch.CronJob"} 2 etcd_request_latencies_summary{operation="create",type="*extensions.Deployment",quantile="0.5"} NaN etcd_request_latencies_summary{operation="create",type="*extensions.Deployment",quantile="0.9"} NaN etcd_request_latencies_summary{operation="create",type="*extensions.Deployment",quantile="0.99"} NaN etcd_request_latencies_summary_sum{operation="create",type="*extensions.Deployment"} 105658 etcd_request_latencies_summary_count{operation="create",type="*extensions.Deployment"} 4 etcd_request_latencies_summary{operation="create",type="*rbac.ClusterRole",quantile="0.5"} NaN etcd_request_latencies_summary{operation="create",type="*rbac.ClusterRole",quantile="0.9"} NaN etcd_request_latencies_summary{operation="create",type="*rbac.ClusterRole",quantile="0.99"} NaN etcd_request_latencies_summary_sum{operation="create",type="*rbac.ClusterRole"} 145696 etcd_request_latencies_summary_count{operation="create",type="*rbac.ClusterRole"} 30 etcd_request_latencies_summary{operation="create",type="*rbac.ClusterRoleBinding",quantile="0.5"} NaN etcd_request_latencies_summary{operation="create",type="*rbac.ClusterRoleBinding",quantile="0.9"} NaN etcd_request_latencies_summary{operation="create",type="*rbac.ClusterRoleBinding",quantile="0.99"} NaN etcd_request_latencies_summary_sum{operation="create",type="*rbac.ClusterRoleBinding"} 197904 etcd_request_latencies_summary_count{operation="create",type="*rbac.ClusterRoleBinding"} 24 etcd_request_latencies_summary{operation="create",type="*rbac.Role",quantile="0.5"} NaN etcd_request_latencies_summary{operation="create",type="*rbac.Role",quantile="0.9"} NaN etcd_request_latencies_summary{operation="create",type="*rbac.Role",quantile="0.99"} NaN etcd_request_latencies_summary_sum{operation="create",type="*rbac.Role"} 14999 etcd_request_latencies_summary_count{operation="create",type="*rbac.Role"} 4 etcd_request_latencies_summary{operation="create",type="*rbac.RoleBinding",quantile="0.5"} NaN etcd_request_latencies_summary{operation="create",type="*rbac.RoleBinding",quantile="0.9"} NaN etcd_request_latencies_summary{operation="create",type="*rbac.RoleBinding",quantile="0.99"} NaN etcd_request_latencies_summary_sum{operation="create",type="*rbac.RoleBinding"} 15543 etcd_request_latencies_summary_count{operation="create",type="*rbac.RoleBinding"} 3 etcd_request_latencies_summary{operation="delete",type="*api.Pod",quantile="0.5"} NaN etcd_request_latencies_summary{operation="delete",type="*api.Pod",quantile="0.9"} NaN etcd_request_latencies_summary{operation="delete",type="*api.Pod",quantile="0.99"} NaN etcd_request_latencies_summary_sum{operation="delete",type="*api.Pod"} 9.020814e+06 etcd_request_latencies_summary_count{operation="delete",type="*api.Pod"} 351 etcd_request_latencies_summary{operation="delete",type="*batch.CronJob",quantile="0.5"} NaN etcd_request_latencies_summary{operation="delete",type="*batch.CronJob",quantile="0.9"} NaN etcd_request_latencies_summary{operation="delete",type="*batch.CronJob",quantile="0.99"} NaN etcd_request_latencies_summary_sum{operation="delete",type="*batch.CronJob"} 11427 etcd_request_latencies_summary_count{operation="delete",type="*batch.CronJob"} 1 etcd_request_latencies_summary{operation="delete",type="*batch.Job",quantile="0.5"} NaN etcd_request_latencies_summary{operation="delete",type="*batch.Job",quantile="0.9"} NaN etcd_request_latencies_summary{operation="delete",type="*batch.Job",quantile="0.99"} NaN etcd_request_latencies_summary_sum{operation="delete",type="*batch.Job"} 2.646038e+06 etcd_request_latencies_summary_count{operation="delete",type="*batch.Job"} 181 etcd_request_latencies_summary{operation="get",type="*[]api.Endpoints",quantile="0.5"} 101842 etcd_request_latencies_summary{operation="get",type="*[]api.Endpoints",quantile="0.9"} 296613 etcd_request_latencies_summary{operation="get",type="*[]api.Endpoints",quantile="0.99"} 504078 etcd_request_latencies_summary_sum{operation="get",type="*[]api.Endpoints"} 1.0171874913e+10 etcd_request_latencies_summary_count{operation="get",type="*[]api.Endpoints"} 73936 etcd_request_latencies_summary{operation="get",type="*[]api.Node",quantile="0.5"} 15324 etcd_request_latencies_summary{operation="get",type="*[]api.Node",quantile="0.9"} 90819 etcd_request_latencies_summary{operation="get",type="*[]api.Node",quantile="0.99"} 108254 etcd_request_latencies_summary_sum{operation="get",type="*[]api.Node"} 2.9509571e+08 etcd_request_latencies_summary_count{operation="get",type="*[]api.Node"} 6094 etcd_request_latencies_summary{operation="get",type="*api.ConfigMap",quantile="0.5"} 1862 etcd_request_latencies_summary{operation="get",type="*api.ConfigMap",quantile="0.9"} 100945 etcd_request_latencies_summary{operation="get",type="*api.ConfigMap",quantile="0.99"} 252514 etcd_request_latencies_summary_sum{operation="get",type="*api.ConfigMap"} 5.03630274e+08 etcd_request_latencies_summary_count{operation="get",type="*api.ConfigMap"} 14930 etcd_request_latencies_summary{operation="get",type="*api.Endpoints",quantile="0.5"} 1642 etcd_request_latencies_summary{operation="get",type="*api.Endpoints",quantile="0.9"} 77291 etcd_request_latencies_summary{operation="get",type="*api.Endpoints",quantile="0.99"} 324222 etcd_request_latencies_summary_sum{operation="get",type="*api.Endpoints"} 1.254797659e+09 etcd_request_latencies_summary_count{operation="get",type="*api.Endpoints"} 62591 etcd_request_latencies_summary{operation="get",type="*api.Event",quantile="0.5"} NaN etcd_request_latencies_summary{operation="get",type="*api.Event",quantile="0.9"} NaN etcd_request_latencies_summary{operation="get",type="*api.Event",quantile="0.99"} NaN etcd_request_latencies_summary_sum{operation="get",type="*api.Event"} 4.338097e+06 etcd_request_latencies_summary_count{operation="get",type="*api.Event"} 237 etcd_request_latencies_summary{operation="get",type="*api.LimitRange",quantile="0.5"} NaN etcd_request_latencies_summary{operation="get",type="*api.LimitRange",quantile="0.9"} NaN etcd_request_latencies_summary{operation="get",type="*api.LimitRange",quantile="0.99"} NaN etcd_request_latencies_summary_sum{operation="get",type="*api.LimitRange"} 4060 etcd_request_latencies_summary_count{operation="get",type="*api.LimitRange"} 4 etcd_request_latencies_summary{operation="get",type="*api.Namespace",quantile="0.5"} 1453 etcd_request_latencies_summary{operation="get",type="*api.Namespace",quantile="0.9"} 12632 etcd_request_latencies_summary{operation="get",type="*api.Namespace",quantile="0.99"} 95610 etcd_request_latencies_summary_sum{operation="get",type="*api.Namespace"} 1.35662085e+08 etcd_request_latencies_summary_count{operation="get",type="*api.Namespace"} 9165 etcd_request_latencies_summary{operation="get",type="*api.Node",quantile="0.5"} 2474 etcd_request_latencies_summary{operation="get",type="*api.Node",quantile="0.9"} 101447 etcd_request_latencies_summary{operation="get",type="*api.Node",quantile="0.99"} 323953 etcd_request_latencies_summary_sum{operation="get",type="*api.Node"} 6.40095822e+08 etcd_request_latencies_summary_count{operation="get",type="*api.Node"} 34775 etcd_request_latencies_summary{operation="get",type="*api.PersistentVolume",quantile="0.5"} 1467 etcd_request_latencies_summary{operation="get",type="*api.PersistentVolume",quantile="0.9"} 18691 etcd_request_latencies_summary{operation="get",type="*api.PersistentVolume",quantile="0.99"} 75247 etcd_request_latencies_summary_sum{operation="get",type="*api.PersistentVolume"} 4.681344e+07 etcd_request_latencies_summary_count{operation="get",type="*api.PersistentVolume"} 3789 etcd_request_latencies_summary{operation="get",type="*api.PersistentVolumeClaim",quantile="0.5"} 1634 etcd_request_latencies_summary{operation="get",type="*api.PersistentVolumeClaim",quantile="0.9"} 55595 etcd_request_latencies_summary{operation="get",type="*api.PersistentVolumeClaim",quantile="0.99"} 91630 etcd_request_latencies_summary_sum{operation="get",type="*api.PersistentVolumeClaim"} 8.2420278e+07 etcd_request_latencies_summary_count{operation="get",type="*api.PersistentVolumeClaim"} 3788 etcd_request_latencies_summary{operation="get",type="*api.Pod",quantile="0.5"} 11246 etcd_request_latencies_summary{operation="get",type="*api.Pod",quantile="0.9"} 91441 etcd_request_latencies_summary{operation="get",type="*api.Pod",quantile="0.99"} 91441 etcd_request_latencies_summary_sum{operation="get",type="*api.Pod"} 2.29711616e+08 etcd_request_latencies_summary_count{operation="get",type="*api.Pod"} 9238 etcd_request_latencies_summary{operation="get",type="*api.RangeAllocation",quantile="0.5"} 1454 etcd_request_latencies_summary{operation="get",type="*api.RangeAllocation",quantile="0.9"} 3704 etcd_request_latencies_summary{operation="get",type="*api.RangeAllocation",quantile="0.99"} 5479 etcd_request_latencies_summary_sum{operation="get",type="*api.RangeAllocation"} 2.3108008e+07 etcd_request_latencies_summary_count{operation="get",type="*api.RangeAllocation"} 1544 etcd_request_latencies_summary{operation="get",type="*api.Secret",quantile="0.5"} 1262 etcd_request_latencies_summary{operation="get",type="*api.Secret",quantile="0.9"} 5571 etcd_request_latencies_summary{operation="get",type="*api.Secret",quantile="0.99"} 96028 etcd_request_latencies_summary_sum{operation="get",type="*api.Secret"} 2.25644585e+08 etcd_request_latencies_summary_count{operation="get",type="*api.Secret"} 21763 etcd_request_latencies_summary{operation="get",type="*api.Service",quantile="0.5"} 1510 etcd_request_latencies_summary{operation="get",type="*api.Service",quantile="0.9"} 25757 etcd_request_latencies_summary{operation="get",type="*api.Service",quantile="0.99"} 161221 etcd_request_latencies_summary_sum{operation="get",type="*api.Service"} 2.2974712136e+10 etcd_request_latencies_summary_count{operation="get",type="*api.Service"} 2.70827e+06 etcd_request_latencies_summary{operation="get",type="*api.ServiceAccount",quantile="0.5"} 15731 etcd_request_latencies_summary{operation="get",type="*api.ServiceAccount",quantile="0.9"} 24099 etcd_request_latencies_summary{operation="get",type="*api.ServiceAccount",quantile="0.99"} 24099 etcd_request_latencies_summary_sum{operation="get",type="*api.ServiceAccount"} 2.346068e+06 etcd_request_latencies_summary_count{operation="get",type="*api.ServiceAccount"} 348 etcd_request_latencies_summary{operation="get",type="*apps.StatefulSet",quantile="0.5"} NaN etcd_request_latencies_summary{operation="get",type="*apps.StatefulSet",quantile="0.9"} NaN etcd_request_latencies_summary{operation="get",type="*apps.StatefulSet",quantile="0.99"} NaN etcd_request_latencies_summary_sum{operation="get",type="*apps.StatefulSet"} 67959 etcd_request_latencies_summary_count{operation="get",type="*apps.StatefulSet"} 18 etcd_request_latencies_summary{operation="get",type="*batch.CronJob",quantile="0.5"} NaN etcd_request_latencies_summary{operation="get",type="*batch.CronJob",quantile="0.9"} NaN etcd_request_latencies_summary{operation="get",type="*batch.CronJob",quantile="0.99"} NaN etcd_request_latencies_summary_sum{operation="get",type="*batch.CronJob"} 112704 etcd_request_latencies_summary_count{operation="get",type="*batch.CronJob"} 4 etcd_request_latencies_summary{operation="get",type="*batch.Job",quantile="0.5"} NaN etcd_request_latencies_summary{operation="get",type="*batch.Job",quantile="0.9"} NaN etcd_request_latencies_summary{operation="get",type="*batch.Job",quantile="0.99"} NaN etcd_request_latencies_summary_sum{operation="get",type="*batch.Job"} 2.753511e+06 etcd_request_latencies_summary_count{operation="get",type="*batch.Job"} 374 etcd_request_latencies_summary{operation="get",type="*extensions.DaemonSet",quantile="0.5"} NaN etcd_request_latencies_summary{operation="get",type="*extensions.DaemonSet",quantile="0.9"} NaN etcd_request_latencies_summary{operation="get",type="*extensions.DaemonSet",quantile="0.99"} NaN etcd_request_latencies_summary_sum{operation="get",type="*extensions.DaemonSet"} 4.117896e+06 etcd_request_latencies_summary_count{operation="get",type="*extensions.DaemonSet"} 116 etcd_request_latencies_summary{operation="get",type="*extensions.Deployment",quantile="0.5"} NaN etcd_request_latencies_summary{operation="get",type="*extensions.Deployment",quantile="0.9"} NaN etcd_request_latencies_summary{operation="get",type="*extensions.Deployment",quantile="0.99"} NaN etcd_request_latencies_summary_sum{operation="get",type="*extensions.Deployment"} 535411 etcd_request_latencies_summary_count{operation="get",type="*extensions.Deployment"} 59 etcd_request_latencies_summary{operation="get",type="*extensions.Ingress",quantile="0.5"} NaN etcd_request_latencies_summary{operation="get",type="*extensions.Ingress",quantile="0.9"} NaN etcd_request_latencies_summary{operation="get",type="*extensions.Ingress",quantile="0.99"} NaN etcd_request_latencies_summary_sum{operation="get",type="*extensions.Ingress"} 97387 etcd_request_latencies_summary_count{operation="get",type="*extensions.Ingress"} 4 etcd_request_latencies_summary{operation="get",type="*extensions.PodSecurityPolicy",quantile="0.5"} NaN etcd_request_latencies_summary{operation="get",type="*extensions.PodSecurityPolicy",quantile="0.9"} NaN etcd_request_latencies_summary{operation="get",type="*extensions.PodSecurityPolicy",quantile="0.99"} NaN etcd_request_latencies_summary_sum{operation="get",type="*extensions.PodSecurityPolicy"} 132307 etcd_request_latencies_summary_count{operation="get",type="*extensions.PodSecurityPolicy"} 4 etcd_request_latencies_summary{operation="get",type="*extensions.ReplicaSet",quantile="0.5"} NaN etcd_request_latencies_summary{operation="get",type="*extensions.ReplicaSet",quantile="0.9"} NaN etcd_request_latencies_summary{operation="get",type="*extensions.ReplicaSet",quantile="0.99"} NaN etcd_request_latencies_summary_sum{operation="get",type="*extensions.ReplicaSet"} 1.372574e+06 etcd_request_latencies_summary_count{operation="get",type="*extensions.ReplicaSet"} 135 etcd_request_latencies_summary{operation="get",type="*extensions.ThirdPartyResource",quantile="0.5"} NaN etcd_request_latencies_summary{operation="get",type="*extensions.ThirdPartyResource",quantile="0.9"} NaN etcd_request_latencies_summary{operation="get",type="*extensions.ThirdPartyResource",quantile="0.99"} NaN etcd_request_latencies_summary_sum{operation="get",type="*extensions.ThirdPartyResource"} 145483 etcd_request_latencies_summary_count{operation="get",type="*extensions.ThirdPartyResource"} 2 etcd_request_latencies_summary{operation="get",type="*extensions.ThirdPartyResourceData",quantile="0.5"} NaN etcd_request_latencies_summary{operation="get",type="*extensions.ThirdPartyResourceData",quantile="0.9"} NaN etcd_request_latencies_summary{operation="get",type="*extensions.ThirdPartyResourceData",quantile="0.99"} NaN etcd_request_latencies_summary_sum{operation="get",type="*extensions.ThirdPartyResourceData"} 140938 etcd_request_latencies_summary_count{operation="get",type="*extensions.ThirdPartyResourceData"} 5 etcd_request_latencies_summary{operation="get",type="*rbac.ClusterRole",quantile="0.5"} NaN etcd_request_latencies_summary{operation="get",type="*rbac.ClusterRole",quantile="0.9"} NaN etcd_request_latencies_summary{operation="get",type="*rbac.ClusterRole",quantile="0.99"} NaN etcd_request_latencies_summary_sum{operation="get",type="*rbac.ClusterRole"} 371300 etcd_request_latencies_summary_count{operation="get",type="*rbac.ClusterRole"} 48 etcd_request_latencies_summary{operation="get",type="*rbac.ClusterRoleBinding",quantile="0.5"} NaN etcd_request_latencies_summary{operation="get",type="*rbac.ClusterRoleBinding",quantile="0.9"} NaN etcd_request_latencies_summary{operation="get",type="*rbac.ClusterRoleBinding",quantile="0.99"} NaN etcd_request_latencies_summary_sum{operation="get",type="*rbac.ClusterRoleBinding"} 48528 etcd_request_latencies_summary_count{operation="get",type="*rbac.ClusterRoleBinding"} 36 etcd_request_latencies_summary{operation="get",type="*rbac.Role",quantile="0.5"} NaN etcd_request_latencies_summary{operation="get",type="*rbac.Role",quantile="0.9"} NaN etcd_request_latencies_summary{operation="get",type="*rbac.Role",quantile="0.99"} NaN etcd_request_latencies_summary_sum{operation="get",type="*rbac.Role"} 10508 etcd_request_latencies_summary_count{operation="get",type="*rbac.Role"} 4 etcd_request_latencies_summary{operation="get",type="*rbac.RoleBinding",quantile="0.5"} NaN etcd_request_latencies_summary{operation="get",type="*rbac.RoleBinding",quantile="0.9"} NaN etcd_request_latencies_summary{operation="get",type="*rbac.RoleBinding",quantile="0.99"} NaN etcd_request_latencies_summary_sum{operation="get",type="*rbac.RoleBinding"} 2955 etcd_request_latencies_summary_count{operation="get",type="*rbac.RoleBinding"} 3 etcd_request_latencies_summary{operation="get",type="*storage.StorageClass",quantile="0.5"} NaN etcd_request_latencies_summary{operation="get",type="*storage.StorageClass",quantile="0.9"} NaN etcd_request_latencies_summary{operation="get",type="*storage.StorageClass",quantile="0.99"} NaN etcd_request_latencies_summary_sum{operation="get",type="*storage.StorageClass"} 70769 etcd_request_latencies_summary_count{operation="get",type="*storage.StorageClass"} 2 etcd_request_latencies_summary{operation="list",type="*[]api.ConfigMap",quantile="0.5"} NaN etcd_request_latencies_summary{operation="list",type="*[]api.ConfigMap",quantile="0.9"} NaN etcd_request_latencies_summary{operation="list",type="*[]api.ConfigMap",quantile="0.99"} NaN etcd_request_latencies_summary_sum{operation="list",type="*[]api.ConfigMap"} 629422 etcd_request_latencies_summary_count{operation="list",type="*[]api.ConfigMap"} 2 etcd_request_latencies_summary{operation="list",type="*[]api.Endpoints",quantile="0.5"} 2218 etcd_request_latencies_summary{operation="list",type="*[]api.Endpoints",quantile="0.9"} 17502 etcd_request_latencies_summary{operation="list",type="*[]api.Endpoints",quantile="0.99"} 90226 etcd_request_latencies_summary_sum{operation="list",type="*[]api.Endpoints"} 3.5723064e+07 etcd_request_latencies_summary_count{operation="list",type="*[]api.Endpoints"} 3042 etcd_request_latencies_summary{operation="list",type="*[]api.Event",quantile="0.5"} NaN etcd_request_latencies_summary{operation="list",type="*[]api.Event",quantile="0.9"} NaN etcd_request_latencies_summary{operation="list",type="*[]api.Event",quantile="0.99"} NaN etcd_request_latencies_summary_sum{operation="list",type="*[]api.Event"} 2.287489e+06 etcd_request_latencies_summary_count{operation="list",type="*[]api.Event"} 18 etcd_request_latencies_summary{operation="list",type="*[]api.LimitRange",quantile="0.5"} NaN etcd_request_latencies_summary{operation="list",type="*[]api.LimitRange",quantile="0.9"} NaN etcd_request_latencies_summary{operation="list",type="*[]api.LimitRange",quantile="0.99"} NaN etcd_request_latencies_summary_sum{operation="list",type="*[]api.LimitRange"} 589596 etcd_request_latencies_summary_count{operation="list",type="*[]api.LimitRange"} 7 etcd_request_latencies_summary{operation="list",type="*[]api.Namespace",quantile="0.5"} 1801 etcd_request_latencies_summary{operation="list",type="*[]api.Namespace",quantile="0.9"} 2868 etcd_request_latencies_summary{operation="list",type="*[]api.Namespace",quantile="0.99"} 2868 etcd_request_latencies_summary_sum{operation="list",type="*[]api.Namespace"} 7.182425e+06 etcd_request_latencies_summary_count{operation="list",type="*[]api.Namespace"} 636 etcd_request_latencies_summary{operation="list",type="*[]api.Node",quantile="0.5"} 5003 etcd_request_latencies_summary{operation="list",type="*[]api.Node",quantile="0.9"} 79089 etcd_request_latencies_summary{operation="list",type="*[]api.Node",quantile="0.99"} 187841 etcd_request_latencies_summary_sum{operation="list",type="*[]api.Node"} 2.45535351e+08 etcd_request_latencies_summary_count{operation="list",type="*[]api.Node"} 7483 etcd_request_latencies_summary{operation="list",type="*[]api.PersistentVolume",quantile="0.5"} NaN etcd_request_latencies_summary{operation="list",type="*[]api.PersistentVolume",quantile="0.9"} NaN etcd_request_latencies_summary{operation="list",type="*[]api.PersistentVolume",quantile="0.99"} NaN etcd_request_latencies_summary_sum{operation="list",type="*[]api.PersistentVolume"} 545599 etcd_request_latencies_summary_count{operation="list",type="*[]api.PersistentVolume"} 1 etcd_request_latencies_summary{operation="list",type="*[]api.PersistentVolumeClaim",quantile="0.5"} NaN etcd_request_latencies_summary{operation="list",type="*[]api.PersistentVolumeClaim",quantile="0.9"} NaN etcd_request_latencies_summary{operation="list",type="*[]api.PersistentVolumeClaim",quantile="0.99"} NaN etcd_request_latencies_summary_sum{operation="list",type="*[]api.PersistentVolumeClaim"} 629028 etcd_request_latencies_summary_count{operation="list",type="*[]api.PersistentVolumeClaim"} 1 etcd_request_latencies_summary{operation="list",type="*[]api.Pod",quantile="0.5"} 58137 etcd_request_latencies_summary{operation="list",type="*[]api.Pod",quantile="0.9"} 210363 etcd_request_latencies_summary{operation="list",type="*[]api.Pod",quantile="0.99"} 1.002154e+06 etcd_request_latencies_summary_sum{operation="list",type="*[]api.Pod"} 1.521469892e+09 etcd_request_latencies_summary_count{operation="list",type="*[]api.Pod"} 19508 etcd_request_latencies_summary{operation="list",type="*[]api.PodTemplate",quantile="0.5"} NaN etcd_request_latencies_summary{operation="list",type="*[]api.PodTemplate",quantile="0.9"} NaN etcd_request_latencies_summary{operation="list",type="*[]api.PodTemplate",quantile="0.99"} NaN etcd_request_latencies_summary_sum{operation="list",type="*[]api.PodTemplate"} 511785 etcd_request_latencies_summary_count{operation="list",type="*[]api.PodTemplate"} 1 etcd_request_latencies_summary{operation="list",type="*[]api.ReplicationController",quantile="0.5"} NaN etcd_request_latencies_summary{operation="list",type="*[]api.ReplicationController",quantile="0.9"} NaN etcd_request_latencies_summary{operation="list",type="*[]api.ReplicationController",quantile="0.99"} NaN etcd_request_latencies_summary_sum{operation="list",type="*[]api.ReplicationController"} 2.289743e+06 etcd_request_latencies_summary_count{operation="list",type="*[]api.ReplicationController"} 4 etcd_request_latencies_summary{operation="list",type="*[]api.ResourceQuota",quantile="0.5"} NaN etcd_request_latencies_summary{operation="list",type="*[]api.ResourceQuota",quantile="0.9"} NaN etcd_request_latencies_summary{operation="list",type="*[]api.ResourceQuota",quantile="0.99"} NaN etcd_request_latencies_summary_sum{operation="list",type="*[]api.ResourceQuota"} 503249 etcd_request_latencies_summary_count{operation="list",type="*[]api.ResourceQuota"} 10 etcd_request_latencies_summary{operation="list",type="*[]api.Secret",quantile="0.5"} NaN etcd_request_latencies_summary{operation="list",type="*[]api.Secret",quantile="0.9"} NaN etcd_request_latencies_summary{operation="list",type="*[]api.Secret",quantile="0.99"} NaN etcd_request_latencies_summary_sum{operation="list",type="*[]api.Secret"} 734726 etcd_request_latencies_summary_count{operation="list",type="*[]api.Secret"} 1 etcd_request_latencies_summary{operation="list",type="*[]api.Service",quantile="0.5"} 2423 etcd_request_latencies_summary{operation="list",type="*[]api.Service",quantile="0.9"} 8978 etcd_request_latencies_summary{operation="list",type="*[]api.Service",quantile="0.99"} 79524 etcd_request_latencies_summary_sum{operation="list",type="*[]api.Service"} 6.2743542e+07 etcd_request_latencies_summary_count{operation="list",type="*[]api.Service"} 5273 etcd_request_latencies_summary{operation="list",type="*[]api.ServiceAccount",quantile="0.5"} NaN etcd_request_latencies_summary{operation="list",type="*[]api.ServiceAccount",quantile="0.9"} NaN etcd_request_latencies_summary{operation="list",type="*[]api.ServiceAccount",quantile="0.99"} NaN etcd_request_latencies_summary_sum{operation="list",type="*[]api.ServiceAccount"} 547515 etcd_request_latencies_summary_count{operation="list",type="*[]api.ServiceAccount"} 1 etcd_request_latencies_summary{operation="list",type="*[]apps.StatefulSet",quantile="0.5"} 1572 etcd_request_latencies_summary{operation="list",type="*[]apps.StatefulSet",quantile="0.9"} 6042 etcd_request_latencies_summary{operation="list",type="*[]apps.StatefulSet",quantile="0.99"} 73153 etcd_request_latencies_summary_sum{operation="list",type="*[]apps.StatefulSet"} 3.0512066e+07 etcd_request_latencies_summary_count{operation="list",type="*[]apps.StatefulSet"} 3428 etcd_request_latencies_summary{operation="list",type="*[]autoscaling.HorizontalPodAutoscaler",quantile="0.5"} NaN etcd_request_latencies_summary{operation="list",type="*[]autoscaling.HorizontalPodAutoscaler",quantile="0.9"} NaN etcd_request_latencies_summary{operation="list",type="*[]autoscaling.HorizontalPodAutoscaler",quantile="0.99"} NaN etcd_request_latencies_summary_sum{operation="list",type="*[]autoscaling.HorizontalPodAutoscaler"} 918218 etcd_request_latencies_summary_count{operation="list",type="*[]autoscaling.HorizontalPodAutoscaler"} 6 etcd_request_latencies_summary{operation="list",type="*[]batch.CronJob",quantile="0.5"} NaN etcd_request_latencies_summary{operation="list",type="*[]batch.CronJob",quantile="0.9"} NaN etcd_request_latencies_summary{operation="list",type="*[]batch.CronJob",quantile="0.99"} NaN etcd_request_latencies_summary_sum{operation="list",type="*[]batch.CronJob"} 1.09446e+06 etcd_request_latencies_summary_count{operation="list",type="*[]batch.CronJob"} 3 etcd_request_latencies_summary{operation="list",type="*[]batch.Job",quantile="0.5"} NaN etcd_request_latencies_summary{operation="list",type="*[]batch.Job",quantile="0.9"} NaN etcd_request_latencies_summary{operation="list",type="*[]batch.Job",quantile="0.99"} NaN etcd_request_latencies_summary_sum{operation="list",type="*[]batch.Job"} 2.182617e+06 etcd_request_latencies_summary_count{operation="list",type="*[]batch.Job"} 12 etcd_request_latencies_summary{operation="list",type="*[]certificates.CertificateSigningRequest",quantile="0.5"} NaN etcd_request_latencies_summary{operation="list",type="*[]certificates.CertificateSigningRequest",quantile="0.9"} NaN etcd_request_latencies_summary{operation="list",type="*[]certificates.CertificateSigningRequest",quantile="0.99"} NaN etcd_request_latencies_summary_sum{operation="list",type="*[]certificates.CertificateSigningRequest"} 771938 etcd_request_latencies_summary_count{operation="list",type="*[]certificates.CertificateSigningRequest"} 1 etcd_request_latencies_summary{operation="list",type="*[]extensions.DaemonSet",quantile="0.5"} 2100 etcd_request_latencies_summary{operation="list",type="*[]extensions.DaemonSet",quantile="0.9"} 11647 etcd_request_latencies_summary{operation="list",type="*[]extensions.DaemonSet",quantile="0.99"} 26404 etcd_request_latencies_summary_sum{operation="list",type="*[]extensions.DaemonSet"} 5.8276395e+07 etcd_request_latencies_summary_count{operation="list",type="*[]extensions.DaemonSet"} 3649 etcd_request_latencies_summary{operation="list",type="*[]extensions.Deployment",quantile="0.5"} 4906 etcd_request_latencies_summary{operation="list",type="*[]extensions.Deployment",quantile="0.9"} 202694 etcd_request_latencies_summary{operation="list",type="*[]extensions.Deployment",quantile="0.99"} 202694 etcd_request_latencies_summary_sum{operation="list",type="*[]extensions.Deployment"} 9.4701252e+07 etcd_request_latencies_summary_count{operation="list",type="*[]extensions.Deployment"} 1010 etcd_request_latencies_summary{operation="list",type="*[]extensions.Ingress",quantile="0.5"} 3230 etcd_request_latencies_summary{operation="list",type="*[]extensions.Ingress",quantile="0.9"} 107940 etcd_request_latencies_summary{operation="list",type="*[]extensions.Ingress",quantile="0.99"} 585123 etcd_request_latencies_summary_sum{operation="list",type="*[]extensions.Ingress"} 2.806497591e+09 etcd_request_latencies_summary_count{operation="list",type="*[]extensions.Ingress"} 89541 etcd_request_latencies_summary{operation="list",type="*[]extensions.NetworkPolicy",quantile="0.5"} NaN etcd_request_latencies_summary{operation="list",type="*[]extensions.NetworkPolicy",quantile="0.9"} NaN etcd_request_latencies_summary{operation="list",type="*[]extensions.NetworkPolicy",quantile="0.99"} NaN etcd_request_latencies_summary_sum{operation="list",type="*[]extensions.NetworkPolicy"} 822594 etcd_request_latencies_summary_count{operation="list",type="*[]extensions.NetworkPolicy"} 1 etcd_request_latencies_summary{operation="list",type="*[]extensions.PodSecurityPolicy",quantile="0.5"} NaN etcd_request_latencies_summary{operation="list",type="*[]extensions.PodSecurityPolicy",quantile="0.9"} NaN etcd_request_latencies_summary{operation="list",type="*[]extensions.PodSecurityPolicy",quantile="0.99"} NaN etcd_request_latencies_summary_sum{operation="list",type="*[]extensions.PodSecurityPolicy"} 912447 etcd_request_latencies_summary_count{operation="list",type="*[]extensions.PodSecurityPolicy"} 1 etcd_request_latencies_summary{operation="list",type="*[]extensions.ReplicaSet",quantile="0.5"} 24306 etcd_request_latencies_summary{operation="list",type="*[]extensions.ReplicaSet",quantile="0.9"} 181020 etcd_request_latencies_summary{operation="list",type="*[]extensions.ReplicaSet",quantile="0.99"} 203649 etcd_request_latencies_summary_sum{operation="list",type="*[]extensions.ReplicaSet"} 2.72046309e+08 etcd_request_latencies_summary_count{operation="list",type="*[]extensions.ReplicaSet"} 3052 etcd_request_latencies_summary{operation="list",type="*[]extensions.ThirdPartyResource",quantile="0.5"} 2140 etcd_request_latencies_summary{operation="list",type="*[]extensions.ThirdPartyResource",quantile="0.9"} 103737 etcd_request_latencies_summary{operation="list",type="*[]extensions.ThirdPartyResource",quantile="0.99"} 278419 etcd_request_latencies_summary_sum{operation="list",type="*[]extensions.ThirdPartyResource"} 1.52579816e+08 etcd_request_latencies_summary_count{operation="list",type="*[]extensions.ThirdPartyResource"} 6888 etcd_request_latencies_summary{operation="list",type="*[]extensions.ThirdPartyResourceData",quantile="0.5"} 1779 etcd_request_latencies_summary{operation="list",type="*[]extensions.ThirdPartyResourceData",quantile="0.9"} 14197 etcd_request_latencies_summary{operation="list",type="*[]extensions.ThirdPartyResourceData",quantile="0.99"} 14197 etcd_request_latencies_summary_sum{operation="list",type="*[]extensions.ThirdPartyResourceData"} 8.772554e+06 etcd_request_latencies_summary_count{operation="list",type="*[]extensions.ThirdPartyResourceData"} 567 etcd_request_latencies_summary{operation="list",type="*[]policy.PodDisruptionBudget",quantile="0.5"} NaN etcd_request_latencies_summary{operation="list",type="*[]policy.PodDisruptionBudget",quantile="0.9"} NaN etcd_request_latencies_summary{operation="list",type="*[]policy.PodDisruptionBudget",quantile="0.99"} NaN etcd_request_latencies_summary_sum{operation="list",type="*[]policy.PodDisruptionBudget"} 3.906033e+06 etcd_request_latencies_summary_count{operation="list",type="*[]policy.PodDisruptionBudget"} 100 etcd_request_latencies_summary{operation="list",type="*[]rbac.ClusterRole",quantile="0.5"} NaN etcd_request_latencies_summary{operation="list",type="*[]rbac.ClusterRole",quantile="0.9"} NaN etcd_request_latencies_summary{operation="list",type="*[]rbac.ClusterRole",quantile="0.99"} NaN etcd_request_latencies_summary_sum{operation="list",type="*[]rbac.ClusterRole"} 1.93405e+06 etcd_request_latencies_summary_count{operation="list",type="*[]rbac.ClusterRole"} 3 etcd_request_latencies_summary{operation="list",type="*[]rbac.ClusterRoleBinding",quantile="0.5"} NaN etcd_request_latencies_summary{operation="list",type="*[]rbac.ClusterRoleBinding",quantile="0.9"} NaN etcd_request_latencies_summary{operation="list",type="*[]rbac.ClusterRoleBinding",quantile="0.99"} NaN etcd_request_latencies_summary_sum{operation="list",type="*[]rbac.ClusterRoleBinding"} 2.407216e+06 etcd_request_latencies_summary_count{operation="list",type="*[]rbac.ClusterRoleBinding"} 3 etcd_request_latencies_summary{operation="list",type="*[]rbac.Role",quantile="0.5"} NaN etcd_request_latencies_summary{operation="list",type="*[]rbac.Role",quantile="0.9"} NaN etcd_request_latencies_summary{operation="list",type="*[]rbac.Role",quantile="0.99"} NaN etcd_request_latencies_summary_sum{operation="list",type="*[]rbac.Role"} 2.038578e+06 etcd_request_latencies_summary_count{operation="list",type="*[]rbac.Role"} 2 etcd_request_latencies_summary{operation="list",type="*[]rbac.RoleBinding",quantile="0.5"} NaN etcd_request_latencies_summary{operation="list",type="*[]rbac.RoleBinding",quantile="0.9"} NaN etcd_request_latencies_summary{operation="list",type="*[]rbac.RoleBinding",quantile="0.99"} NaN etcd_request_latencies_summary_sum{operation="list",type="*[]rbac.RoleBinding"} 1.931307e+06 etcd_request_latencies_summary_count{operation="list",type="*[]rbac.RoleBinding"} 2 etcd_request_latencies_summary{operation="list",type="*[]settings.PodPreset",quantile="0.5"} NaN etcd_request_latencies_summary{operation="list",type="*[]settings.PodPreset",quantile="0.9"} NaN etcd_request_latencies_summary{operation="list",type="*[]settings.PodPreset",quantile="0.99"} NaN etcd_request_latencies_summary_sum{operation="list",type="*[]settings.PodPreset"} 1.199205e+06 etcd_request_latencies_summary_count{operation="list",type="*[]settings.PodPreset"} 1 etcd_request_latencies_summary{operation="list",type="*[]storage.StorageClass",quantile="0.5"} NaN etcd_request_latencies_summary{operation="list",type="*[]storage.StorageClass",quantile="0.9"} NaN etcd_request_latencies_summary{operation="list",type="*[]storage.StorageClass",quantile="0.99"} NaN etcd_request_latencies_summary_sum{operation="list",type="*[]storage.StorageClass"} 2.398306e+06 etcd_request_latencies_summary_count{operation="list",type="*[]storage.StorageClass"} 2 # HELP get_token_count Counter of total Token() requests to the alternate token source # TYPE get_token_count counter get_token_count 0 # HELP get_token_fail_count Counter of failed Token() requests to the alternate token source # TYPE get_token_fail_count counter get_token_fail_count 0 # HELP go_gc_duration_seconds A summary of the GC invocation durations. # TYPE go_gc_duration_seconds summary go_gc_duration_seconds{quantile="0"} 0.001287514 go_gc_duration_seconds{quantile="0.25"} 0.0018677470000000001 go_gc_duration_seconds{quantile="0.5"} 0.002117667 go_gc_duration_seconds{quantile="0.75"} 0.003951512 go_gc_duration_seconds{quantile="1"} 0.09878027800000001 go_gc_duration_seconds_sum 32.610498067 go_gc_duration_seconds_count 2615 # HELP go_goroutines Number of goroutines that currently exist. # TYPE go_goroutines gauge go_goroutines 885 # HELP go_memstats_alloc_bytes Number of bytes allocated and still in use. # TYPE go_memstats_alloc_bytes gauge go_memstats_alloc_bytes 2.63902024e+08 # HELP go_memstats_alloc_bytes_total Total number of bytes allocated, even if freed. # TYPE go_memstats_alloc_bytes_total counter go_memstats_alloc_bytes_total 3.41081993392e+11 # HELP go_memstats_buck_hash_sys_bytes Number of bytes used by the profiling bucket hash table. # TYPE go_memstats_buck_hash_sys_bytes gauge go_memstats_buck_hash_sys_bytes 5.945028e+06 # HELP go_memstats_frees_total Total number of frees. # TYPE go_memstats_frees_total counter go_memstats_frees_total 2.651573406e+09 # HELP go_memstats_gc_sys_bytes Number of bytes used for garbage collection system metadata. # TYPE go_memstats_gc_sys_bytes gauge go_memstats_gc_sys_bytes 1.271808e+07 # HELP go_memstats_heap_alloc_bytes Number of heap bytes allocated and still in use. # TYPE go_memstats_heap_alloc_bytes gauge go_memstats_heap_alloc_bytes 2.63902024e+08 # HELP go_memstats_heap_idle_bytes Number of heap bytes waiting to be used. # TYPE go_memstats_heap_idle_bytes gauge go_memstats_heap_idle_bytes 4.3302912e+07 # HELP go_memstats_heap_inuse_bytes Number of heap bytes that are in use. # TYPE go_memstats_heap_inuse_bytes gauge go_memstats_heap_inuse_bytes 3.0203904e+08 # HELP go_memstats_heap_objects Number of allocated objects. # TYPE go_memstats_heap_objects gauge go_memstats_heap_objects 1.937254e+06 # HELP go_memstats_heap_released_bytes_total Total number of heap bytes released to OS. # TYPE go_memstats_heap_released_bytes_total counter go_memstats_heap_released_bytes_total 3.56352e+06 # HELP go_memstats_heap_sys_bytes Number of heap bytes obtained from system. # TYPE go_memstats_heap_sys_bytes gauge go_memstats_heap_sys_bytes 3.45341952e+08 # HELP go_memstats_last_gc_time_seconds Number of seconds since 1970 of last garbage collection. # TYPE go_memstats_last_gc_time_seconds gauge go_memstats_last_gc_time_seconds 1.4919057147025465e+19 # HELP go_memstats_lookups_total Total number of pointer lookups. # TYPE go_memstats_lookups_total counter go_memstats_lookups_total 314949 # HELP go_memstats_mallocs_total Total number of mallocs. # TYPE go_memstats_mallocs_total counter go_memstats_mallocs_total 2.65351066e+09 # HELP go_memstats_mcache_inuse_bytes Number of bytes in use by mcache structures. # TYPE go_memstats_mcache_inuse_bytes gauge go_memstats_mcache_inuse_bytes 1200 # HELP go_memstats_mcache_sys_bytes Number of bytes used for mcache structures obtained from system. # TYPE go_memstats_mcache_sys_bytes gauge go_memstats_mcache_sys_bytes 16384 # HELP go_memstats_mspan_inuse_bytes Number of bytes in use by mspan structures. # TYPE go_memstats_mspan_inuse_bytes gauge go_memstats_mspan_inuse_bytes 5.084e+06 # HELP go_memstats_mspan_sys_bytes Number of bytes used for mspan structures obtained from system. # TYPE go_memstats_mspan_sys_bytes gauge go_memstats_mspan_sys_bytes 5.3248e+06 # HELP go_memstats_next_gc_bytes Number of heap bytes when next garbage collection will take place. # TYPE go_memstats_next_gc_bytes gauge go_memstats_next_gc_bytes 2.90363121e+08 # HELP go_memstats_other_sys_bytes Number of bytes used for other system allocations. # TYPE go_memstats_other_sys_bytes gauge go_memstats_other_sys_bytes 1.4693e+06 # HELP go_memstats_stack_inuse_bytes Number of bytes in use by the stack allocator. # TYPE go_memstats_stack_inuse_bytes gauge go_memstats_stack_inuse_bytes 5.275648e+06 # HELP go_memstats_stack_sys_bytes Number of bytes obtained from system for stack allocator. # TYPE go_memstats_stack_sys_bytes gauge go_memstats_stack_sys_bytes 5.275648e+06 # HELP go_memstats_sys_bytes Number of bytes obtained by system. Sum of all system allocations. # TYPE go_memstats_sys_bytes gauge go_memstats_sys_bytes 3.76091192e+08 # HELP http_request_duration_microseconds The HTTP request latencies in microseconds. # TYPE http_request_duration_microseconds summary http_request_duration_microseconds{handler="prometheus",quantile="0.5"} 170426.19 http_request_duration_microseconds{handler="prometheus",quantile="0.9"} 292473.122 http_request_duration_microseconds{handler="prometheus",quantile="0.99"} 1.001241986e+06 http_request_duration_microseconds_sum{handler="prometheus"} 1.1241217677620003e+09 http_request_duration_microseconds_count{handler="prometheus"} 4681 # HELP http_request_size_bytes The HTTP request sizes in bytes. # TYPE http_request_size_bytes summary http_request_size_bytes{handler="prometheus",quantile="0.5"} 111 http_request_size_bytes{handler="prometheus",quantile="0.9"} 111 http_request_size_bytes{handler="prometheus",quantile="0.99"} 111 http_request_size_bytes_sum{handler="prometheus"} 520383 http_request_size_bytes_count{handler="prometheus"} 4681 # HELP http_requests_total Total number of HTTP requests made. # TYPE http_requests_total counter http_requests_total{code="200",handler="prometheus",method="get"} 4681 # HELP http_response_size_bytes The HTTP response sizes in bytes. # TYPE http_response_size_bytes summary http_response_size_bytes{handler="prometheus",quantile="0.5"} 20300 http_response_size_bytes{handler="prometheus",quantile="0.9"} 20401 http_response_size_bytes{handler="prometheus",quantile="0.99"} 20425 http_response_size_bytes_sum{handler="prometheus"} 8.8227052e+07 http_response_size_bytes_count{handler="prometheus"} 4681 # HELP kubernetes_build_info A metric with a constant '1' value labeled by major, minor, git version, git commit, git tree state, build date, Go version, and compiler from which Kubernetes was built, and platform on which it is running. # TYPE kubernetes_build_info gauge kubernetes_build_info{buildDate="2017-04-04T00:32:53Z",compiler="gc",gitCommit="9212f77ed8c169a0afa02e58dce87913c6387b3e",gitTreeState="clean",gitVersion="v1.6.1+coreos.0",goVersion="go1.7.5",major="1",minor="6",platform="linux/amd64"} 1 # HELP process_cpu_seconds_total Total user and system CPU time spent in seconds. # TYPE process_cpu_seconds_total counter process_cpu_seconds_total 5790.75 # HELP process_max_fds Maximum number of open file descriptors. # TYPE process_max_fds gauge process_max_fds 1.048576e+06 # HELP process_open_fds Number of open file descriptors. # TYPE process_open_fds gauge process_open_fds 242 # HELP process_resident_memory_bytes Resident memory size in bytes. # TYPE process_resident_memory_bytes gauge process_resident_memory_bytes 4.41614336e+08 # HELP process_start_time_seconds Start time of the process since unix epoch in seconds. # TYPE process_start_time_seconds gauge process_start_time_seconds 1.49183638879e+09 # HELP process_virtual_memory_bytes Virtual memory size in bytes. # TYPE process_virtual_memory_bytes gauge process_virtual_memory_bytes 7.55318784e+08 # HELP ssh_tunnel_open_count Counter of ssh tunnel total open attempts # TYPE ssh_tunnel_open_count counter ssh_tunnel_open_count 0 # HELP ssh_tunnel_open_fail_count Counter of ssh tunnel failed open attempts # TYPE ssh_tunnel_open_fail_count counter ssh_tunnel_open_fail_count 0

The text was updated successfully, but these errors were encountered:

0xmichalis · 2017-06-04T10:37:26Z

@kubernetes/sig-api-machinery-misc

mikkeloscar · 2017-06-04T11:14:46Z

As a follow up, we consistently see that the metrics are there for the first hour when the APIserver comes up, but after that they are gone.

I imagine it's similar with a minikube setup, but I haven't gotten around to test it yet.

This has happened from 1.6.1 (when first reported) and through every update to 1.6.4.

Raffo · 2017-08-23T13:52:49Z

This issue seems to be solved for me, can you also confirm @mikkeloscar ?

mikkeloscar · 2017-08-23T16:13:31Z

Yup! We see metrics ever since we upgraded to v1.7.4.

k8s-github-robot added the needs-sig Indicates an issue or PR lacks a `sig/foo` label and requires one. label May 31, 2017

BugRoger mentioned this issue Jun 1, 2017

Alert KubernetesApiServerLatency Bugged sapcc/helm-charts#122

Closed

k8s-ci-robot added the sig/api-machinery Categorizes an issue or PR as relevant to SIG API Machinery. label Jun 4, 2017

k8s-github-robot removed the needs-sig Indicates an issue or PR lacks a `sig/foo` label and requires one. label Jun 4, 2017

mikkeloscar closed this as completed Aug 23, 2017

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

kube-apiserver metrics apiserver_request_latencies_summary NaN after upgrading to 1.6.1 #44329

kube-apiserver metrics apiserver_request_latencies_summary NaN after upgrading to 1.6.1 #44329

mikkeloscar commented Apr 11, 2017

0xmichalis commented Jun 4, 2017

mikkeloscar commented Jun 4, 2017

Raffo commented Aug 23, 2017

mikkeloscar commented Aug 23, 2017

kube-apiserver metrics apiserver_request_latencies_summary NaN after upgrading to 1.6.1 #44329

kube-apiserver metrics apiserver_request_latencies_summary NaN after upgrading to 1.6.1 #44329

Comments

mikkeloscar commented Apr 11, 2017

0xmichalis commented Jun 4, 2017

mikkeloscar commented Jun 4, 2017

Raffo commented Aug 23, 2017

mikkeloscar commented Aug 23, 2017