Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[AWS] [Billing] Document GetCostAndUsage API limitation #9290

Merged
merged 6 commits into from
Apr 8, 2024

Conversation

gpop63
Copy link
Contributor

@gpop63 gpop63 commented Mar 6, 2024

Overview

This change aims to document the limitation of the GetCostAndUsage API used by the billing metricset. Since it only allows grouping costs by two groups at once we can end up with duplicated data.

Checklist

  • I have reviewed tips for building integrations and this pull request is aligned with them.
  • I have verified that all data streams collect metrics or logs.
  • I have added an entry to my package's changelog.yml file.
  • I have verified that Kibana version constraints are current according to guidelines.

Author's Checklist

  • [ ]

How to test this PR locally

Related issues

Screenshots

@gpop63 gpop63 self-assigned this Mar 6, 2024
@elasticmachine
Copy link

elasticmachine commented Mar 7, 2024

🚀 Benchmarks report

To see the full report comment with /test benchmark fullreport

Copy link

Quality Gate passed Quality Gate passed

Kudos, no new issues were introduced!

0 New issues
0 Security Hotspots
No Coverage information No data about Coverage
No Duplication information No data about Duplication

See analysis details on SonarQube

@gpop63 gpop63 marked this pull request as ready for review March 7, 2024 10:02
@gpop63 gpop63 requested review from a team as code owners March 7, 2024 10:02
@gpop63 gpop63 requested a review from agithomas March 7, 2024 10:02

When you want to group by a combination of several tags and dimensions, like `SERVICE` along with various tags (e.g., `team`, `project`, `aws:createdBy`), you might see duplicated data. This happens because of a [limitation](https://docs.aws.amazon.com/aws-cost-management/latest/APIReference/API_GetCostAndUsage.html) with the `GetCostAndUsage` API, which only allows grouping costs by two different groups.

To avoid data duplication, it's recommended to group data using two groups at once.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is there a problem in using just one member in the group?

I guess, by group you meant - one tag + one dimension / two tags / two dimensions.

It would be best to explain this a little bit more, i think

It may be best to rephrase recommended to group data using two groups at once.. The word group when used in two different context makes it more confusing.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

To avoid data duplication, it's recommended to aggregate data using a combination of two attributes, which can be either: one tag and one dimension, two tags or two dimensions. do you think this makes more sense?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, this improves clarity.

Copy link

Quality Gate passed Quality Gate passed

Kudos, no new issues were introduced!

0 New issues
0 Security Hotspots
No Coverage information No data about Coverage
No Duplication information No data about Duplication

See analysis details on SonarQube

@elasticmachine
Copy link

💚 Build Succeeded

History

cc @gpop63

@ishleenk17
Copy link
Contributor

@gpop63 : Are we waiting for something here?

@gpop63 gpop63 merged commit a13cf83 into elastic:main Apr 8, 2024
5 checks passed
@elasticmachine
Copy link

Package aws - 2.14.1 containing this change is available at https://epr.elastic.co/search?package=aws

@BenB196
Copy link
Contributor

BenB196 commented Apr 8, 2024

Just an FYI,

which can be either: one tag and one dimension, two tags or two dimensions.

two tags or two dimensions.

This part isn't currently possible with Elastic's AWS integration. elastic/beats#38755 attempts to resolve this, but this doc changes might be confusing to users.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Document the limitations of AWS Billing integration in supporting multiple tags and dimensions.
7 participants