[Azure] [Container_instance] Add dimensions and metric_type metadata for the container_instance data_stream #7127

tetianakravchenko · 2023-07-24T14:42:06Z

What does this PR do?

remove duplicated fields
instead of using wildcard in the field name - use predefined field names
- for azure.dimensions - dimension name is predefined - https://learn.microsoft.com/en-us/azure/azure-monitor/reference/supported-metrics/microsoft-containerinstance-containergroups-metrics
- list of metrics - https://learn.microsoft.com/en-us/azure/azure-monitor/reference/supported-metrics/microsoft-containerinstance-containergroups-metrics, default aggregation is avg
used fields for dimensions:
- cloud.region
- azure.resource.id - this field includes azure.subscription_id, azure.resource.group, azure.resource.name, format:
```
/subscriptions/{azure.subscription_id}/resourceGroups/{azure.resource.group}/providers/Microsoft.ContainerInstance/containerGroups/{azure.resource.name}
```
- azure.dimensions.container_name - this field is not available for 2 metrics: network_bytes_transmitted_per_second.avg and network_bytes_received_per_second.avg
- azure.timegrain
metrics - use predefined metrics - since in the configuration is defined list of metrics

Checklist

I have reviewed tips for building integrations and this pull request is aligned with them.
I have verified that all data streams collect metrics or logs.
I have added an entry to my package's changelog.yml file.
I have verified that Kibana version constraints are current according to guidelines.

Author's Checklist

[ ]

How to test this PR locally

Related issues

Screenshots

…ta_stream Signed-off-by: Tetiana Kravchenko <tetiana.kravchenko@elastic.co>

tetianakravchenko · 2023-07-24T14:48:41Z

@zmoog I couldn't find those fields - https://github.com/elastic/integrations/blob/main/packages/azure_metrics/data_stream/container_instance/fields/fields.yml#L1

could you please check if this field is not added by mistake?

elasticmachine · 2023-07-24T14:56:55Z

💚 Build Succeeded

the below badges are clickable and redirect to their specific view in the CI or DOCS

Expand to view the summary

Build stats

Start Time: 2023-07-27T08:13:34.635+0000
Duration: 15 min 8 sec

Test stats 🧪

Test	Results
Failed	0
Passed	134
Skipped	0
Total	134

🤖 GitHub comments

Expand to view the GitHub comments

To re-run your PR in the CI, just comment with:

/test : Re-trigger the build.

elasticmachine · 2023-07-24T14:57:08Z

🌐 Coverage report

Name	Metrics % (`covered/total`)	Diff
Packages	100.0% (`3/3`)	💚
Files	100.0% (`3/3`)	💚 2.831
Classes	100.0% (`3/3`)	💚 2.831
Methods	73.333% (`22/30`)	👎 -18.854
Lines	100.0% (`21/21`)	💚 9.022
Conditionals	100.0% (`0/0`)	💚

zmoog · 2023-07-25T09:24:29Z

could you please check if this field is not added by mistake?

Yep, I'll check it out!

packages/azure_metrics/data_stream/container_instance/fields/package-fields.yml

agithomas · 2023-07-25T11:02:29Z

packages/azure_metrics/changelog.yml

+  changes:
+    - description: Add dimension and metric_type metadata to the container_instance data_stream
+      type: enhancement
+      link: https://github.com/elastic/integrations/pull/5123


Please correct the PR link.

done - 7a77aa7

agithomas · 2023-07-25T11:02:54Z

packages/azure_metrics/changelog.yml

@@ -1,3 +1,8 @@
+- version: "1.0.17"
+  changes:
+    - description: Add dimension and metric_type metadata to the container_instance data_stream


Suggested change

- description: Add dimension and metric_type metadata to the container_instance data_stream

- description: Add dimension and metric_type metadata to the container_instance datastream

packages/azure_metrics/data_stream/container_instance/fields/package-fields.yml

agithomas · 2023-07-25T11:06:07Z

packages/azure_metrics/manifest.yml

@@ -1,6 +1,6 @@
 name: azure_metrics
 title: Azure Resource Metrics
-version: 1.0.16
+version: 1.0.17


Why not 1.1.0 ? Any specific reason ?

no specific reason, I did it similarly to the AWS package, also I've noticed that previous changes in this package were also added with 1.0.X

Enabling TSDB seems worth a bump to the minor version, but I have no strong opinions. I'll leave the decision to you, TSDB folks.

@zmoog there will be another PR to enable TSDB by default (similar to this PR and additionally adding stack restriction), this PR just adds metadata to the fields, that will be used by TSDB

Signed-off-by: Tetiana Kravchenko <tetiana.kravchenko@elastic.co>

agithomas

LGTM!

packages/azure_metrics/data_stream/container_instance/fields/package-fields.yml

Signed-off-by: Tetiana Kravchenko <tetiana.kravchenko@elastic.co>

mlunadia · 2023-07-28T11:44:00Z

@bturquet can you help us getting these reviewed? We have a long queue of Azure packages waiting on it. Merci

zmoog · 2023-07-28T18:25:33Z

packages/azure_metrics/manifest.yml

@@ -1,6 +1,6 @@
 name: azure_metrics
 title: Azure Resource Metrics
-version: 1.0.16
+version: 1.0.17


Enabling TSDB seems worth a bump to the minor version, but I have no strong opinions. I'll leave the decision to you, TSDB folks.

elasticmachine · 2023-08-02T08:20:35Z

Package azure_metrics - 1.0.17 containing this change is available at https://epr.elastic.co/search?package=azure_metrics

…for the container_instance data_stream (#7127) * add dimensions and metric_type metadata for the container_instance data_stream Signed-off-by: Tetiana Kravchenko <tetiana.kravchenko@elastic.co> * address review comments Signed-off-by: Tetiana Kravchenko <tetiana.kravchenko@elastic.co> * add agent.id field Signed-off-by: Tetiana Kravchenko <tetiana.kravchenko@elastic.co> * extend azure.container_instance field Signed-off-by: Tetiana Kravchenko <tetiana.kravchenko@elastic.co> --------- Signed-off-by: Tetiana Kravchenko <tetiana.kravchenko@elastic.co>

add dimensions and metric_type metadata for the container_instance da…

fe0bbd6

…ta_stream Signed-off-by: Tetiana Kravchenko <tetiana.kravchenko@elastic.co>

tetianakravchenko requested a review from a team as a code owner July 24, 2023 14:42

tetianakravchenko requested a review from zmoog July 24, 2023 14:44

tetianakravchenko requested a review from agithomas July 25, 2023 08:11

agithomas reviewed Jul 25, 2023

View reviewed changes

packages/azure_metrics/data_stream/container_instance/fields/package-fields.yml Show resolved Hide resolved

agithomas reviewed Jul 25, 2023

View reviewed changes

packages/azure_metrics/data_stream/container_instance/fields/package-fields.yml Show resolved Hide resolved

agithomas reviewed Jul 25, 2023

View reviewed changes

tetianakravchenko mentioned this pull request Jul 25, 2023

[Azure] TSDB enablement - track all metrics data streams #7140

Closed

31 tasks

tetianakravchenko added 2 commits July 26, 2023 10:24

address review comments

7a77aa7

Signed-off-by: Tetiana Kravchenko <tetiana.kravchenko@elastic.co>

Merge branch 'master' into azure-container-instance-tsdb

b674371

agithomas approved these changes Jul 26, 2023

View reviewed changes

tetianakravchenko commented Jul 26, 2023

View reviewed changes

packages/azure_metrics/data_stream/container_instance/fields/package-fields.yml Show resolved Hide resolved

tetianakravchenko added 2 commits July 27, 2023 09:40

add agent.id field

460951d

Signed-off-by: Tetiana Kravchenko <tetiana.kravchenko@elastic.co>

extend azure.container_instance field

a334441

Signed-off-by: Tetiana Kravchenko <tetiana.kravchenko@elastic.co>

zmoog approved these changes Jul 28, 2023

View reviewed changes

zmoog mentioned this pull request Jul 31, 2023

[Azure] [container_instance] Not correct metrics field #7158

Closed

andrewkroh added the Integration:Azure label Aug 1, 2023

tetianakravchenko merged commit 1ca98fb into elastic:main Aug 2, 2023
4 checks passed

tetianakravchenko deleted the azure-container-instance-tsdb branch August 2, 2023 08:00

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[Azure] [Container_instance] Add dimensions and metric_type metadata for the container_instance data_stream #7127

[Azure] [Container_instance] Add dimensions and metric_type metadata for the container_instance data_stream #7127

tetianakravchenko commented Jul 24, 2023 •

edited

tetianakravchenko commented Jul 24, 2023

elasticmachine commented Jul 24, 2023 •

edited

Build stats

Test stats 🧪

elasticmachine commented Jul 24, 2023 •

edited

zmoog commented Jul 25, 2023

agithomas Jul 25, 2023

tetianakravchenko Jul 26, 2023

agithomas Jul 25, 2023

agithomas Jul 25, 2023

tetianakravchenko Jul 26, 2023

zmoog Jul 28, 2023

tetianakravchenko Aug 2, 2023

agithomas left a comment

mlunadia commented Jul 28, 2023

zmoog Jul 28, 2023

elasticmachine commented Aug 2, 2023

	- description: Add dimension and metric_type metadata to the container_instance data_stream
	- description: Add dimension and metric_type metadata to the container_instance datastream

[Azure] [Container_instance] Add dimensions and metric_type metadata for the container_instance data_stream #7127

[Azure] [Container_instance] Add dimensions and metric_type metadata for the container_instance data_stream #7127

Conversation

tetianakravchenko commented Jul 24, 2023 • edited

What does this PR do?

Checklist

Author's Checklist

How to test this PR locally

Related issues

Screenshots

tetianakravchenko commented Jul 24, 2023

elasticmachine commented Jul 24, 2023 • edited

💚 Build Succeeded

Build stats

Test stats 🧪

🤖 GitHub comments

elasticmachine commented Jul 24, 2023 • edited

🌐 Coverage report

zmoog commented Jul 25, 2023

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

agithomas left a comment

Choose a reason for hiding this comment

mlunadia commented Jul 28, 2023

Choose a reason for hiding this comment

elasticmachine commented Aug 2, 2023

tetianakravchenko commented Jul 24, 2023 •

edited

elasticmachine commented Jul 24, 2023 •

edited

elasticmachine commented Jul 24, 2023 •

edited