Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

New component: AWS Metrics Processor #38292

Open
andrebienemann opened this issue Mar 3, 2025 · 1 comment
Open

New component: AWS Metrics Processor #38292

andrebienemann opened this issue Mar 3, 2025 · 1 comment
Labels
Sponsor Needed New component seeking sponsor

Comments

@andrebienemann
Copy link

The purpose and use-cases of the new component

Although, it's possible to transition AWS CloudWatch metrics to Prometheus using the awsfirehosereceiver and prometheusexporter components, I see a need for a third component in the pipeline to transform CloudWatch metrics into a format better suited for Prometheus.

Specifically, I want to convert non-monotonic counters into monotonic ones to ensure proper usage in PromQL. Additionally (though optionally), I'd like the metrics to follow a more consistent naming convention, which the forementioned pipeline configuration does not guarantee.

While existing processors like metricstransformprocessor and deltatocumulativeprocessor could potentially be adapted for this use case, from a user perspective, it would be far more convenient (IMO) to have a dedicated component that seamlessly handles the transformation required when transmitting AWS CloudWatch metrics to Prometheus-compatible storage backends.

Example configuration for the component

TBD

Telemetry data types supported

Metrics

Code Owner(s)

No response

Sponsor (optional)

No response

Additional context

I created a PoC awsmetricsprocessor component that is incomplete but somewhat illustrates my proposal. Currently, delta-to-cumulative transformation is achieved using a cache that stores the latest values of the count and sum vectors. Name translation is handled through a mapping, which, while it may seem cumbersome, provides the benefit of adding consistency to an otherwise inconsistent metric naming convention in AWS. This approach can also be used to translate lables (dimensions) and enrich metrics with help text.

@andrebienemann andrebienemann added needs triage New item requiring triage Sponsor Needed New component seeking sponsor labels Mar 3, 2025
@atoulme atoulme removed the needs triage New item requiring triage label Mar 8, 2025
@atoulme
Copy link
Contributor

atoulme commented Mar 8, 2025

Thank you for your write up. Please come to a SIG meeting and explore how this component would work for the collector.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Sponsor Needed New component seeking sponsor
Projects
None yet
Development

No branches or pull requests

2 participants