[Azure] [compute_vm_scaleset] List exposed metrics; add dimension and metric_type metadata

[tetianakravchenko]

What does this PR do?

• remove duplicated fields
• instead of using wildcard in the field name - use predefined field names
  • for azure.dimensions - dimension name is predefined (vmname) - https://learn.microsoft.com/en-us/azure/azure-monitor/reference/supported-metrics/microsoft-compute-virtualmachinescalesets-metrics
  • list of metrics - https://learn.microsoft.com/en-us/azure/azure-monitor/reference/supported-metrics/microsoft-compute-virtualmachinescalesets-metrics
• used fields for dimensions:

  • cloud.region

  • azure.resource.id - this field includes azure.subscription_id, azure.resource.group, azure.resource.name, format:

    /subscriptions/{azure.subscription_id}/resourceGroups/{azure.resource.group}/providers/Microsoft.Compute/virtualMachineScaleSets/{azure.resource.name}
    

  • azure.dimensions.vmname

  • azure.dimensions.virtual_machine

  • azure.timegrain

• metrics - use predefined metrics - since in the configuration is defined list of metrics, descriptions are taken from https://learn.microsoft.com/en-us/azure/azure-monitor/reference/supported-metrics/microsoft-compute-virtualmachinescalesets-metrics and https://techcommunity.microsoft.com/t5/ask-the-performance-team/an-overview-of-troubleshooting-memory-issues/ba-p/372673

Checklist

• I have reviewed tips for building integrations and this pull request is aligned with them.
• I have verified that all data streams collect metrics or logs.
• I have added an entry to my package's changelog.yml file.
• I have verified that Kibana version constraints are current according to guidelines.

Author's Checklist

• [ ]

How to test this PR locally

Related issues

Screenshots

[elasticmachine]

💚 Build Succeeded

the below badges are clickable and redirect to their specific view in the CI or DOCS

Expand to view the summary

Build stats

• Start Time: 2023-08-16T16:18:32.395+0000

• Duration: 13 min 6 sec

Test stats 🧪

Test	Results
Failed	0
Passed	33
Skipped	0
Total	33

🤖 GitHub comments

Expand to view the GitHub comments

To re-run your PR in the CI, just comment with:

• /test : Re-trigger the build.

[elasticmachine]

🌐 Coverage report

Name	Metrics % (covered/total)	Diff
Packages	100.0% (3/3)	💚
Files	100.0% (3/3)	💚
Classes	100.0% (3/3)	💚
Methods	73.333% (22/30)	👍
Lines	100.0% (21/21)	💚
Conditionals	100.0% (0/0)	💚

[tetianakravchenko]

FYI: I couldn't get this metric
available in UI list:

[zmoog]

The changes look good.

Did you get a chance to collect the missing metric in your tests? I'm not an expert here, but I'm available if you wan to try to get it before merging.

[tetianakravchenko]

@zmoog

Did you get a chance to collect the missing metric in your tests? I'm not an expert here, but I'm available if you wan to try to get it before merging.

no, I wasn't able to get this one metric Memory\\Commit Limit. The problem is that this metric is not even available on the Azure > VM > Diagnostic settings.

Also I wasn't able to find the full list of metrics, that are supposed to be available in the azure documentation, maybe you know about such documentation?

[agithomas]

Referring to https://learn.microsoft.com/en-us/azure/azure-monitor/reference/supported-metrics/microsoft-compute-virtualmachinescalesets-metrics, which field holds LUN value?

[tetianakravchenko]

there is no such field exposed, list of dimensions is defined in configuration - https://github.com/elastic/integrations/blob/main/packages/azure_metrics/data_stream/compute_vm_scaleset/agent/stream/stream.yml.hbs#L71-L73

from my understanding - in this case metric should be grouped by vmname only, if only this dimension is defined

[tetianakravchenko]

I've added lun azure dimension in this PR - #7314

coult you please have another look on this PR?

[zmoog]

Also I wasn't able to find the full list of metrics, that are supposed to be available in the azure documentation, maybe you know about such documentation?

I can't find this metric either.

I suppose you already checked at https://learn.microsoft.com/en-us/azure/azure-monitor/reference/supported-metrics/metrics-index; I remember this page in your browser's tabs.

[tetianakravchenko]

Also I wasn't able to find the full list of metrics, that are supposed to be available in the azure documentation, maybe you know about such documentation?

I can't find this metric either.

I suppose you already checked at https://learn.microsoft.com/en-us/azure/azure-monitor/reference/supported-metrics/metrics-index; I remember this page in your browser's tabs.

yes, I've seen it, but it does not contain the guest VM metrics there

[agithomas]

LGTM!

[elasticmachine]

Package azure_metrics - 1.0.23 containing this change is available at https://epr.elastic.co/search?package=azure_metrics