Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Metricbeat] Add GCP CloudSQL region filter #32943

Merged
merged 14 commits into from
Sep 19, 2022
3 changes: 3 additions & 0 deletions CHANGELOG.next.asciidoc
Original file line number Diff line number Diff line change
Expand Up @@ -86,6 +86,9 @@ https://github.com/elastic/beats/compare/v8.2.0\...main[Check the HEAD diff]
- Fix GCP storage field naming {pull}32806[32806]
- in module/windows/perfmon, changed collection method of the second counter value required to create a displayable value {pull}32305[32305]
- Fix and improve AWS metric period calculation to avoid zero-length intervals {pull}32724[32724]
- Add GCP CloudSQL region filter {pull}32943[32943]
- Add network name to data.json {pull}32986[32986]
- Add missing cluster metadata to k8s module metricsets {pull}32979[32979]
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
- Add GCP CloudSQL region filter {pull}32943[32943]
- Add network name to data.json {pull}32986[32986]
- Add missing cluster metadata to k8s module metricsets {pull}32979[32979]
- Add GCP CloudSQL region filter {pull}32943[32943]

- Add missing cluster metadata to k8s module metricsets {pull}32979[32979] {pull}33032[33032]

*Packetbeat*
Expand Down
2 changes: 2 additions & 0 deletions x-pack/metricbeat/module/gcp/constants.go
Original file line number Diff line number Diff line change
Expand Up @@ -24,6 +24,7 @@ const (
ServiceStorage = "storage"
ServiceFirestore = "firestore"
ServiceDataproc = "dataproc"
ServiceCloudSQL = "cloudsql"
)

//Paths within the GCP monitoring.TimeSeries response, if converted to JSON, where you can find each ECS field required for the output event
Expand Down Expand Up @@ -81,6 +82,7 @@ const (
ComputeResourceLabelZone = "resource.labels.zone"
GKEResourceLabelLocation = "resource.label.location"
StorageResourceLabelLocation = "resource.label.location"
CloudSQLResourceLabelRegion = "resource.labels.region"
)

// AlignersMapToGCP map contains available perSeriesAligner
Expand Down
14 changes: 14 additions & 0 deletions x-pack/metricbeat/module/gcp/metrics/metrics_requester.go
Original file line number Diff line number Diff line change
Expand Up @@ -190,6 +190,19 @@ func (r *metricsRequester) getFilterForMetric(serviceName, m string) string {
regionsFilter := r.buildRegionsFilter(r.config.Regions, gcp.StorageResourceLabelLocation)
f = fmt.Sprintf("%s AND %s", f, regionsFilter)
}
case gcp.ServiceCloudSQL:
if r.config.Region != "" && len(r.config.Regions) != 0 {
r.logger.Warnf("when region %s and regions config parameters are both provided, use region", r.config.Region)
}

switch {
case r.config.Region != "":
region := strings.TrimSuffix(r.config.Region, "*")
f = fmt.Sprintf("%s AND %s = starts_with(\"%s\")", f, gcp.CloudSQLResourceLabelRegion, region)
case len(r.config.Regions) != 0:
regionsFilter := r.buildRegionsFilter(r.config.Regions, gcp.CloudSQLResourceLabelRegion)
f = fmt.Sprintf("%s AND %s", f, regionsFilter)
}
default:
if r.config.Region != "" && r.config.Zone != "" {
r.logger.Warnf("when region %s and zone %s config parameter "+
Expand All @@ -198,6 +211,7 @@ func (r *metricsRequester) getFilterForMetric(serviceName, m string) string {

switch {
case r.config.Region != "":
// FIXME: using resource.labels.zone but should use region.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I though about this, let's remove it from here and do it for good separately. I'll open an issue to discuss how to address this.

Suggested change
// FIXME: using resource.labels.zone but should use region.

region := strings.TrimSuffix(r.config.Region, "*")
f = fmt.Sprintf(`%s AND resource.labels.zone = starts_with("%s")`, f, region)
case r.config.Zone != "":
Expand Down