Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Make compute_vm light metricset and add update cloud instance id #20889

Merged
merged 27 commits into from Sep 7, 2020
Merged

Make compute_vm light metricset and add update cloud instance id #20889

merged 27 commits into from Sep 7, 2020

Conversation

narph
Copy link
Contributor

@narph narph commented Sep 1, 2020 •
edited

What does this PR do?

  • rewrites the compute_vm to light metricset
  • maps the cloud.instance.id and cloud.instance.name fields only for compute_vm onlya
  • add azure.resource.name and azure.resource.id back to all metricsets

Why is it important?

  • rewrites the compute_vm to light metricset
  • maps the cloud.instance.id field

Checklist

  • My code follows the style guidelines of this project
  • I have commented my code, particularly in hard-to-understand areas
  • I have made corresponding changes to the documentation
  • I have made corresponding change to the default configuration files
  • I have added tests that prove my fix is effective or that my feature works
  • I have added an entry in CHANGELOG.next.asciidoc or CHANGELOG-developer.next.asciidoc.

Related issues

@botelastic botelastic bot added the needs_team Indicates that the issue/PR needs a Team:* label label Sep 1, 2020
@narph narph self-assigned this Sep 1, 2020
@narph narph added the Team:Platforms Label for the Integrations - Platforms team label Sep 1, 2020
@botelastic botelastic bot removed the needs_team Indicates that the issue/PR needs a Team:* label label Sep 1, 2020
@elasticmachine
Copy link
Collaborator

elasticmachine commented Sep 1, 2020 •
edited by jenkins-beats-ci bot

💚 Build Succeeded

Pipeline View Test View Changes Artifacts preview

Expand to view the summary

Build stats

  • Build Cause: [Pull request #20889 updated]

  • Start Time: 2020-09-07T08:42:31.642+0000

  • Duration: 62 min 39 sec

Test stats 🧪

Test Results
Failed 0
Passed 3602
Skipped 795
Total 4397

@narph narph marked this pull request as ready for review September 2, 2020 15:48
@elasticmachine
Copy link
Collaborator

Pinging @elastic/integrations-platforms (Team:Platforms)

@narph narph added [zube]: In Review needs_backport PR is waiting to be backported to other branches. labels Sep 2, 2020
kaiyan-sheng
kaiyan-sheng reviewed Sep 2, 2020
View reviewed changes
x-pack/metricbeat/module/azure/compute_vm/_meta/data.json
}
}
"id": "d5d9444a-1964-4d23-9c62-5463ecb16fe0",
"name": "obslinux"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Uh oh, we are missing the rest of the host fields.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@kaiyan-sheng , the values used to map those host fields are not retrieved in this event, most likely the next one containing those values will. as you can see only percentage_cpu is contained in the metrics.

ChrsMark
ChrsMark reviewed Sep 4, 2020
View reviewed changes
x-pack/metricbeat/module/azure/monitor_service.go
@@ -124,3 +124,24 @@ func (service *MonitorService) GetMetricValues(resourceId string, namespace stri
}
return metrics, interval, nil
}

// getResourceNameFormId maps resource group from resource ID
func getResourceNameFromId(path string) string {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe these functions could be moved to a utils.go file since look too generic?

ChrsMark
ChrsMark reviewed Sep 4, 2020
View reviewed changes
x-pack/metricbeat/module/azure/client_utils.go
@@ -198,3 +176,12 @@ func ContainsDimension(dimension string, dimensions []insights.LocalizableString
}
return false
}

func containsResource(resId string, resources []Resource) bool {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It this actually client's utils or is could be re-organised to a generic utils.go for the whole azure module?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it is clients_utils , would like to keep it that way for the moment as it might get too muddy if we add more functionality, once a function is reused at other levels as service etc then it needs rethinking.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍

ChrsMark
ChrsMark reviewed Sep 4, 2020
View reviewed changes
Copy link
Member

@ChrsMark ChrsMark left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Left some suggestions/questions mostly about code organization. Not being so familiar with this stuff but I don't see anything blocking this.

@narph narph requested a review from ChrsMark September 7, 2020 08:46
@narph narph merged commit f8c2477 into elastic:master Sep 7, 2020
@narph narph deleted the az-inven branch September 7, 2020 11:27
narph added a commit to narph/beats that referenced this pull request Sep 8, 2020
@narph
Make compute_vm light metricset and add update cloud instance id (ela…
4c2db11 
…stic#20889)

* mofidy doc

* rewrite

* fix

* temp

* tests

* work

* changelog

* fit tests

* update dashboards

(cherry picked from commit f8c2477)
@narph narph mentioned this pull request Sep 8, 2020
Merged
6 tasks
narph added a commit that referenced this pull request Sep 14, 2020
@narph
Cherry-pick #20889 to 7.x: Make compute_vm light metricset and add up…
6ae258e 
…date cloud instance id (#21015)

* Make compute_vm light metricset and add update cloud instance id (#20889)

* mofidy doc

* rewrite

* fix

* temp

* tests

* work

* changelog

* fit tests

* update dashboards

(cherry picked from commit f8c2477)

* solve conflicts
melchiormoulin pushed a commit to melchiormoulin/beats that referenced this pull request Oct 14, 2020
@narph @melchiormoulin
Make compute_vm light metricset and add update cloud instance id (ela…
27c5f5e 
…stic#20889)

* mofidy doc

* rewrite

* fix

* temp

* tests

* work

* changelog

* fit tests

* update dashboards
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@kaiyan-sheng kaiyan-sheng kaiyan-sheng left review comments

@ChrsMark ChrsMark ChrsMark approved these changes

Assignees

@narph narph

Labels
needs_backport PR is waiting to be backported to other branches. Team:Platforms Label for the Integrations - Platforms team [zube]: In Review
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

Map cloud.instance.id to the vm id in the compute_vm metricset
4 participants
@narph @elasticmachine @ChrsMark @kaiyan-sheng