[Metricbeat] Add GCP CloudSQL region filter #32943

gpop63 · 2022-08-31T20:46:47Z

What does this PR do?

Adds GCP cloudsql region filter.

Why is it important?

Checklist

My code follows the style guidelines of this project
I have commented my code, particularly in hard-to-understand areas
I have made corresponding changes to the documentation
I have made corresponding change to the default configuration files
I have added tests that prove my fix is effective or that my feature works
I have added an entry in CHANGELOG.next.asciidoc or CHANGELOG-developer.next.asciidoc.

Author's Checklist

[ ]

How to test this PR locally

Related issues

Closes [GCP Module] Region filtering support for cloudsql metricset #32906

Use cases

Screenshots

Logs

mergify · 2022-08-31T20:47:16Z

This pull request does not have a backport label.
If this is a bug or security fix, could you label this PR @gpop63? 🙏.
For such, you'll need to label your PR with:

The upcoming major version of the Elastic Stack
The upcoming minor version of the Elastic Stack (if you're not pushing a breaking change)

To fixup this pull request, you need to add the backport labels for the needed
branches, such as:

backport-v8./d.0 is the label to automatically backport to the 8./d branch. /d is the digit

elasticmachine · 2022-08-31T21:17:55Z

💚 Build Succeeded

the below badges are clickable and redirect to their specific view in the CI or DOCS

Expand to view the summary

Build stats

Start Time: 2022-09-19T11:20:26.299+0000
Duration: 52 min 47 sec

Test stats 🧪

Test	Results
Failed	0
Passed	1316
Skipped	87
Total	1403

💚 Flaky test report

Tests succeeded.

🤖 GitHub comments

Expand to view the GitHub comments

To re-run your PR in the CI, just comment with:

/test : Re-trigger the build.
/package : Generate the packages and run the E2E tests.
/beats-tester : Run the installation tests with beats-tester.
run elasticsearch-ci/docs : Re-trigger the docs validation. (use unformatted text in the comment!)

mergify · 2022-09-08T10:38:46Z

This pull request is now in conflicts. Could you fix it? 🙏
To fixup this pull request, you can check out it locally. See documentation: https://help.github.com/articles/checking-out-pull-requests-locally/

git fetch upstream
git checkout -b add_gcp-cloudsql_region_filter upstream/add_gcp-cloudsql_region_filter
git merge upstream/main
git push upstream add_gcp-cloudsql_region_filter

endorama · 2022-09-09T08:28:52Z

x-pack/metricbeat/module/gcp/metrics/metrics_requester.go

@@ -152,7 +157,7 @@ func (r *metricsRequester) getFilterForMetric(serviceName, m string) string {
 			// }

 			region := strings.TrimSuffix(r.config.Region, "*")
-			f = fmt.Sprintf(`%s AND resource.labels.zone = starts_with("%s")`, f, region)
+			f = fmt.Sprintf(`%s AND resource.labels.region = starts_with("%s")`, f, region)


Please do not include this change here, as it's not related to CloudSQL. This change is also a breaking change, so we need to take another approach.
I would add a comment, so we can remove this in the future: FIXME: this is using zone but should use region. This is a breaking change.

…gion_filter

This reverts commit e3ea3b8.

mergify · 2022-09-11T13:46:55Z

This pull request is now in conflicts. Could you fix it? 🙏
To fixup this pull request, you can check out it locally. See documentation: https://help.github.com/articles/checking-out-pull-requests-locally/

git fetch upstream
git checkout -b add_gcp-cloudsql_region_filter upstream/add_gcp-cloudsql_region_filter
git merge upstream/main
git push upstream add_gcp-cloudsql_region_filter

…gion_filter

mergify · 2022-09-12T10:51:50Z

This pull request is now in conflicts. Could you fix it? 🙏
To fixup this pull request, you can check out it locally. See documentation: https://help.github.com/articles/checking-out-pull-requests-locally/

git fetch upstream
git checkout -b add_gcp-cloudsql_region_filter upstream/add_gcp-cloudsql_region_filter
git merge upstream/main
git push upstream add_gcp-cloudsql_region_filter

…gion_filter

endorama

2 minor changes required then this can be merged

endorama · 2022-09-19T10:26:20Z

CHANGELOG.next.asciidoc

+- Add GCP CloudSQL region filter {pull}32943[32943]
+- Add network name to data.json {pull}32986[32986]
+- Add missing cluster metadata to k8s module metricsets {pull}32979[32979]


Suggested change

- Add GCP CloudSQL region filter {pull}32943[32943]

- Add network name to data.json {pull}32986[32986]

- Add missing cluster metadata to k8s module metricsets {pull}32979[32979]

- Add GCP CloudSQL region filter {pull}32943[32943]

endorama · 2022-09-19T10:27:30Z

x-pack/metricbeat/module/gcp/metrics/metrics_requester.go

@@ -198,6 +211,7 @@ func (r *metricsRequester) getFilterForMetric(serviceName, m string) string {

 		switch {
 		case r.config.Region != "":
+			// FIXME: using resource.labels.zone but should use region.


I though about this, let's remove it from here and do it for good separately. I'll open an issue to discuss how to address this.

Suggested change

// FIXME: using resource.labels.zone but should use region.

* add cloudsql region filter * add changelog entry * fix comment for golangci-lint * fix get filter for metric tests * better AlignersMapToGCP comment * revert to zone and add comment * Revert "fix get filter for metric tests" This reverts commit e3ea3b8. * add cloudsql region label * add regions filter * fix changelog * remove comment from metrics requester

gpop63 added 2 commits August 31, 2022 23:43

add cloudsql region filter

e9914b6

add changelog entry

b4d42c2

botelastic bot added the needs_team Indicates that the issue/PR needs a Team:* label label Aug 31, 2022

gpop63 added the Team:Cloud-Monitoring Label for the Cloud Monitoring team label Aug 31, 2022

botelastic bot removed the needs_team Indicates that the issue/PR needs a Team:* label label Aug 31, 2022

mergify bot assigned gpop63 Aug 31, 2022

gpop63 added 2 commits September 1, 2022 00:15

fix comment for golangci-lint

0e82c01

fix get filter for metric tests

e3ea3b8

better AlignersMapToGCP comment

117cfa7

endorama reviewed Sep 9, 2022

View reviewed changes

gpop63 added 3 commits September 10, 2022 19:20

Merge remote-tracking branch 'upstream/main' into add_gcp-cloudsql_re…

7353ae3

…gion_filter

revert to zone and add comment

1e85834

Revert "fix get filter for metric tests"

b02b11f

This reverts commit e3ea3b8.

Merge remote-tracking branch 'upstream/main' into add_gcp-cloudsql_re…

0b7d558

…gion_filter

gpop63 added 3 commits September 15, 2022 12:19

Merge remote-tracking branch 'upstream/main' into add_gcp-cloudsql_re…

69a3921

…gion_filter

add cloudsql region label

4e92cfb

add regions filter

35196aa

gpop63 marked this pull request as ready for review September 16, 2022 08:59

gpop63 requested a review from a team as a code owner September 16, 2022 08:59

endorama requested changes Sep 19, 2022

View reviewed changes

endorama mentioned this pull request Sep 19, 2022

[GCP] gcp.metrics always filter on resource.labels.zone even when only region is specified in the config file #33118

Closed

gpop63 added 2 commits September 19, 2022 14:19

fix changelog

46e33df

remove comment from metrics requester

2be3f1c

endorama approved these changes Sep 19, 2022

View reviewed changes

gpop63 merged commit f90a066 into elastic:main Sep 19, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[Metricbeat] Add GCP CloudSQL region filter #32943

[Metricbeat] Add GCP CloudSQL region filter #32943

gpop63 commented Aug 31, 2022

mergify bot commented Aug 31, 2022

elasticmachine commented Aug 31, 2022 •

edited by jenkins-beats-ci bot

Build stats

Test stats 🧪

mergify bot commented Sep 8, 2022

endorama Sep 9, 2022

mergify bot commented Sep 11, 2022

mergify bot commented Sep 12, 2022

endorama left a comment

endorama Sep 19, 2022

endorama Sep 19, 2022

[Metricbeat] Add GCP CloudSQL region filter #32943

[Metricbeat] Add GCP CloudSQL region filter #32943

Conversation

gpop63 commented Aug 31, 2022

What does this PR do?

Why is it important?

Checklist

Author's Checklist

How to test this PR locally

Related issues

Use cases

Screenshots

Logs

mergify bot commented Aug 31, 2022

elasticmachine commented Aug 31, 2022 • edited by jenkins-beats-ci bot

💚 Build Succeeded

Build stats

Test stats 🧪

💚 Flaky test report

🤖 GitHub comments

mergify bot commented Sep 8, 2022

endorama Sep 9, 2022

Choose a reason for hiding this comment

mergify bot commented Sep 11, 2022

mergify bot commented Sep 12, 2022

endorama left a comment

Choose a reason for hiding this comment

endorama Sep 19, 2022

Choose a reason for hiding this comment

endorama Sep 19, 2022

Choose a reason for hiding this comment

elasticmachine commented Aug 31, 2022 •

edited by jenkins-beats-ci bot