Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[AWS] Add metric type to SNS, SQS and Billing data streams #6319

Merged

Conversation

constanca-m
Copy link
Contributor

What does this PR do?

Adds metric type to SNS, SQS and Billing data streams.

Billing:
From my understanding, most metrics are estimates, so they can go up and down. The ones that are not (UnblendedCost and UsageQuantity) consider a specific time window.

SNS:
Values are only considered for a specific time window, after that they reset for the next time window. This causes lots of ups and downs over time, and since the next value received for a specific field does not take into account the previous, all of them were considered gauge. Take this field (NumberOfMessagesPublished.sum) as an example:
Screenshot from 2023-05-25 09-34-52

SQS:
Same logic was applied. Example of a gauge field that might look like a counter from the description is messages.received. When we visualize that metric, we can see that is in fact a gauge:
Screenshot from 2023-05-25 10-22-39

Checklist

  • I have reviewed tips for building integrations and this pull request is aligned with them.
  • I have verified that all data streams collect metrics or logs.
  • I have added an entry to my package's changelog.yml file.
  • I have verified that Kibana version constraints are current according to guidelines.

Author's Checklist

  • Verification of data in visualisation after enabling TSDB flag in kibana
  • Verify if field mapping is correct in the data stream template.

How to test this PR locally

Refer to #6293.

Related issues

Relates to #6293.

Signed-off-by: constanca-m <constanca.manteigas@elastic.co>
Signed-off-by: constanca-m <constanca.manteigas@elastic.co>
Signed-off-by: constanca-m <constanca.manteigas@elastic.co>
@constanca-m constanca-m self-assigned this May 25, 2023
@constanca-m constanca-m requested a review from a team as a code owner May 25, 2023 09:11
Signed-off-by: constanca-m <constanca.manteigas@elastic.co>
@elasticmachine
Copy link

elasticmachine commented May 25, 2023

💚 Build Succeeded

the below badges are clickable and redirect to their specific view in the CI or DOCS
Pipeline View Test View Changes Artifacts preview preview

Expand to view the summary

Build stats

  • Start Time: 2023-06-05T15:34:59.730+0000

  • Duration: 47 min 22 sec

Test stats 🧪

Test Results
Failed 0
Passed 190
Skipped 4
Total 194

🤖 GitHub comments

Expand to view the GitHub comments

To re-run your PR in the CI, just comment with:

  • /test : Re-trigger the build.

@elasticmachine
Copy link

elasticmachine commented May 25, 2023

🌐 Coverage report

Name Metrics % (covered/total) Diff
Packages 100.0% (15/15) 💚
Files 93.75% (15/16) 👎 -3.296
Classes 93.75% (15/16) 👎 -3.296
Methods 85.921% (238/277) 👎 -5.922
Lines 85.925% (7387/8597) 👎 -6.146
Conditionals 100.0% (0/0) 💚

Copy link
Contributor

@agithomas agithomas left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

@lalit-satapathy
Copy link
Collaborator

Can this PR be merged after code-owner review?

@@ -1,4 +1,9 @@
# newer versions go on top
- version: "1.36.9"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: in #6365 we bumped minor version, here only patch one
why the difference? :)

@constanca-m constanca-m merged commit 9d18c59 into elastic:main Jun 5, 2023
3 checks passed
@elasticmachine
Copy link

Package aws - 1.40.0 containing this change is available at https://epr.elastic.co/search?package=aws

@constanca-m constanca-m deleted the add-sns-sqs-billing-metric-type branch June 6, 2023 06:54
sodhikirti07 pushed a commit that referenced this pull request Jun 15, 2023
* Add metric type to SNS.

Signed-off-by: constanca-m <constanca.manteigas@elastic.co>

* Add metric type to SQS.

Signed-off-by: constanca-m <constanca.manteigas@elastic.co>

* Add metric type to Billing.

Signed-off-by: constanca-m <constanca.manteigas@elastic.co>

* Update changelog.

Signed-off-by: constanca-m <constanca.manteigas@elastic.co>

* Update files.

Signed-off-by: constanca-m <constanca.manteigas@elastic.co>

---------

Signed-off-by: constanca-m <constanca.manteigas@elastic.co>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

5 participants