[metricbeat]kube-controller-manager metricset #12409

odacremolbap · 2019-06-04T08:48:25Z

Create kube-controller-manager metricset

Import kube-controller-manager prometheus metrics to the kubernetes module at metricbeat

CHANGELOG.next.asciidoc

metricbeat/module/kubernetes/kubecontroller/kubecontroller.go

metricbeat/module/kubernetes/kubecontroller/_meta/testdata/docs.plain-expected.json

metricbeat/module/kubernetes/kubecontroller/kubecontroller.go

jsoriano

This is looking good, but review types of metrics, at least the one that is making tests fail.

metricbeat/module/kubernetes/controllermanager/_meta/fields.yml

sayden

LGTM

jsoriano

I have added some suggestions to the documentation and reference config. Regarding documentation of fields take into account that we have the descriptions in the prometheus responses that can help.

Before merging we should check that all the fields are documented, I found some of them that seem undocumented, or documented for a previous version with paths for histograms/quantiles. The testdata framework can help as it checks for documented fields.

jsoriano · 2019-06-26T15:24:36Z

metricbeat/metricbeat.reference.yml

+  enabled: true
+  metricsets:
+    - controllermanager
+  hosts: ["http://controller-manager:10252"]


I'd leave it with localhost as we use to do, if deployed in master nodes in the host network it should work.

Suggested change

hosts: ["http://controller-manager:10252"]

hosts: ["http://localhost:10252"]

jsoriano · 2019-06-26T15:30:35Z

metricbeat/module/kubernetes/_meta/config.reference.yml

+  enabled: true
+  metricsets:
+    - controllermanager
+  hosts: ["http://controller-manager:10252"]


jsoriano · 2019-06-26T15:34:03Z

metricbeat/module/kubernetes/controllermanager/_meta/fields.yml

+          type: long
+        - name: retries.count
+          type: long
+          description: Unfinished processors


Suggested change

description: Unfinished processors

description: Number of retries handled by workqueue

jsoriano · 2019-06-26T15:35:22Z

metricbeat/module/kubernetes/controllermanager/_meta/fields.yml

+        - name: adds.count
+          type: long
+        - name: depth.count
+          type: long


description: Current depth of workqueue

jsoriano · 2019-06-26T15:36:55Z

metricbeat/module/kubernetes/controllermanager/_meta/fields.yml

+      fields:
+        - name: longestrunning.sec
+          type: double
+          description: Longest running processors


Suggested change

description: Longest running processors

description: Time the longest running processor has been running.

jsoriano · 2019-06-26T15:38:26Z

metricbeat/module/kubernetes/controllermanager/_meta/fields.yml

+          description: Longest running processors
+        - name: unfinished.sec
+          type: double
+          description: Unfinished processors


Suggested change

description: Unfinished processors

description: Time in unfinished processors. Large values can indicate stuck threads.

jsoriano · 2019-06-26T15:39:05Z

metricbeat/module/kubernetes/controllermanager/_meta/fields.yml

+          type: double
+          description: Unfinished processors
+        - name: adds.count
+          type: long


description: Number of adds handled by workqueue

jsoriano · 2019-06-26T15:43:57Z

metricbeat/module/kubernetes/controllermanager/_meta/fields.yml

+        - name: request.size.bytes.percentile.*
+          type: object
+          object_type: long
+          description: Request size percentiles


These percentiles are not used?

that's right, this is a new module, not being used yet.

Maybe we should remove these unused fields, as they will appear in documentation and people can expect them.

http.request.size.bytes.precentile is part of the metricset
you mean you want to remove it?

Ok, keep them if they are part of the metricset, this is what I meant with being used 🙂

I guess they are generated by:

"http_request_size_bytes": prometheus.Metric("http.request.size.bytes"),

jsoriano · 2019-06-26T15:46:03Z

metricbeat/module/kubernetes/controllermanager/_meta/fields.yml

+          type: long
+          format: bytes
+          description: Request size cumulative sum
+        - name: request.size.bytes.count


This metric is really called request.size.bytes according to the mapping in the code. I wonder why this is not being checked by the fields validator.

This is http.request.size.bytes.count.
Can you elaborate the comment?

Oh sorry, In the code I saw it as http.request.size.bytes, without the count:

"http_response_size_bytes": prometheus.Metric("http.response.size.bytes"),

But I guess this is an histogram 🤦‍♂️

jsoriano · 2019-06-26T15:49:24Z

metricbeat/module/kubernetes/controllermanager/controllermanager_test.go

+				ExpectedFile: "./_meta/test/metrics.controllermanager.1.14.expected",
+			},
+		},
+	)


I think this is not testing if fields are documented, we may consider changing it to the testdata framework. It would be to replace this with mbtest.TestDataFiles(t, "kubernetes", "controllermanager"), create a _meta/testdata/config.yml file and move the test and golden files to their location in _meta/testdata.

Please, open an issue for any global improvement.
This is using those helpers and test libraries as is today

Ok, I am fine with leaving this with prometheus test helpers, I was thinking that there were some undocumented fields but I was wrong.

jsoriano

It seems I was wrong about the undocumented fields, I got confused by histograms.
There are still some pending suggestions about the descriptions of the fields for the documentation, and I wouldn't add documentation for fields we don't collect, but I won't block this PR because of that. This can be fixed later if you prefer.

jsoriano · 2019-06-27T13:30:07Z

metricbeat/module/kubernetes/controllermanager/_meta/fields.yml

+        - name: request.size.bytes.percentile.*
+          type: object
+          object_type: long
+          description: Request size percentiles


Maybe we should remove these unused fields, as they will appear in documentation and people can expect them.

jsoriano · 2019-06-27T13:30:23Z

metricbeat/module/kubernetes/controllermanager/_meta/fields.yml

+          type: long
+          format: bytes
+          description: Request size cumulative sum
+        - name: request.size.bytes.count


Oh sorry, In the code I saw it as http.request.size.bytes, without the count:

"http_response_size_bytes": prometheus.Metric("http.response.size.bytes"),

But I guess this is an histogram 🤦‍♂️

jsoriano · 2019-06-27T13:33:41Z

metricbeat/module/kubernetes/controllermanager/controllermanager_test.go

+				ExpectedFile: "./_meta/test/metrics.controllermanager.1.14.expected",
+			},
+		},
+	)


Ok, I am fine with leaving this with prometheus test helpers, I was thinking that there were some undocumented fields but I was wrong.

Main issue I had was not such an issue.

kaiyan-sheng · 2019-06-27T14:47:44Z

metricbeat/module/kubernetes/controllermanager/README.md

+leader_election_master_status
+
+
+## Setup environment for manual tests


Thanks for adding this! 👍

first commit for kube controller metrics

a292d7d

odacremolbap added in progress Pull request is currently in progress. Metricbeat Metricbeat containers Related to containers use case Team:Integrations Label for the Integrations team labels Jun 4, 2019

odacremolbap self-assigned this Jun 4, 2019

odacremolbap added 3 commits June 4, 2019 17:48

Merge branch 'master' into task/kube-controller-metricset

b3a7aab

Merge branch 'master' into task/kube-controller-metricset

c881ccd

WIP, adding all metrics, yaml fields pending

d05dde2

andresrc added the [zube]: In Progress label Jun 11, 2019

odacremolbap added 4 commits June 14, 2019 12:43

Merge branch 'master' into task/kube-controller-metricset

716d02b

rebase

97fd855

add entry to CHANGELOG

f308f29

add expected data for tests

a5d4a2e

andresrc added the v7.3.0 label Jun 17, 2019

odacremolbap added 4 commits June 19, 2019 19:10

Merge branch 'master' into task/kube-controller-metricset

7d8758c

write yamls for fields

3976a38

add config and changelog

30402fa

Merge branch 'master' into task/kube-controller-metricset

1ed95b7

odacremolbap marked this pull request as ready for review June 20, 2019 08:57

odacremolbap added review and removed in progress Pull request is currently in progress. labels Jun 20, 2019

make fmt

571ea4b

odacremolbap requested review from a team as code owners June 20, 2019 10:25

mage fmt update

817a11a

jsoriano reviewed Jun 20, 2019

View reviewed changes

odacremolbap added 3 commits June 20, 2019 18:44

rename module

e6f5e66

add more metrics

cd64bc3

merge master

49aeaa0

update golden files

94408c9

jsoriano reviewed Jun 24, 2019

View reviewed changes

metricbeat/module/kubernetes/controllermanager/_meta/fields.yml Outdated Show resolved Hide resolved

metricbeat/module/kubernetes/controllermanager/_meta/fields.yml Outdated Show resolved Hide resolved

odacremolbap added 4 commits June 24, 2019 12:17

set is master metric to boolean

dc5ff4e

modify fields yaml types

b8cfc20

review more yaml types

0808af0

fields yaml for percentiles

b3d9ea7

jsoriano reviewed Jun 24, 2019

View reviewed changes

metricbeat/module/kubernetes/controllermanager/_meta/fields.yml Show resolved Hide resolved

metricbeat/module/kubernetes/controllermanager/_meta/fields.yml Show resolved Hide resolved

odacremolbap added 6 commits June 25, 2019 09:22

more yaml dev :-(

357c400

yaml

f86bfca

yaml

f2d8042

yaml

2d8942a

yaml

40d53ca

merge master

e8a705a

sayden approved these changes Jun 26, 2019

View reviewed changes

merge master

511dba3

jsoriano previously requested changes Jun 26, 2019

View reviewed changes

odacremolbap added 3 commits June 27, 2019 08:04

comments

6a506a2

set beta

5a46bf0

merge master

31cebe3

jsoriano reviewed Jun 27, 2019

View reviewed changes

kaiyan-sheng reviewed Jun 27, 2019

View reviewed changes

odacremolbap merged commit f3ba95a into elastic:master Jun 27, 2019

odacremolbap deleted the task/kube-controller-metricset branch June 27, 2019 14:50

zube bot added [zube]: Done and removed [zube]: In Progress labels Jun 27, 2019

andresrc removed the [zube]: Done label Jul 1, 2019

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[metricbeat]kube-controller-manager metricset #12409

[metricbeat]kube-controller-manager metricset #12409

odacremolbap commented Jun 4, 2019

jsoriano left a comment

sayden left a comment

jsoriano left a comment

jsoriano Jun 26, 2019

jsoriano Jun 26, 2019

jsoriano Jun 26, 2019

jsoriano Jun 26, 2019

jsoriano Jun 26, 2019

jsoriano Jun 26, 2019

jsoriano Jun 26, 2019

jsoriano Jun 26, 2019

odacremolbap Jun 27, 2019

jsoriano Jun 27, 2019

odacremolbap Jun 27, 2019

jsoriano Jun 27, 2019

jsoriano Jun 26, 2019

odacremolbap Jun 27, 2019

jsoriano Jun 27, 2019

jsoriano Jun 26, 2019 •

edited

Loading

odacremolbap Jun 27, 2019

jsoriano Jun 27, 2019

jsoriano left a comment

jsoriano Jun 27, 2019

jsoriano Jun 27, 2019

jsoriano Jun 27, 2019

kaiyan-sheng Jun 27, 2019

	hosts: ["http://controller-manager:10252"]
	hosts: ["http://localhost:10252"]

	description: Unfinished processors
	description: Number of retries handled by workqueue

	description: Longest running processors
	description: Time the longest running processor has been running.

	description: Unfinished processors
	description: Time in unfinished processors. Large values can indicate stuck threads.

		leader_election_master_status


		## Setup environment for manual tests

[metricbeat]kube-controller-manager metricset #12409

[metricbeat]kube-controller-manager metricset #12409

Conversation

odacremolbap commented Jun 4, 2019

jsoriano left a comment

Choose a reason for hiding this comment

sayden left a comment

Choose a reason for hiding this comment

jsoriano left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jsoriano Jun 26, 2019 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jsoriano left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jsoriano Jun 26, 2019 •

edited

Loading