You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Although, it's possible to transition AWS CloudWatch metrics to Prometheus using the awsfirehosereceiver and prometheusexporter components, I see a need for a third component in the pipeline to transform CloudWatch metrics into a format better suited for Prometheus.
Specifically, I want to convert non-monotonic counters into monotonic ones to ensure proper usage in PromQL. Additionally (though optionally), I'd like the metrics to follow a more consistent naming convention, which the forementioned pipeline configuration does not guarantee.
While existing processors like metricstransformprocessor and deltatocumulativeprocessor could potentially be adapted for this use case, from a user perspective, it would be far more convenient (IMO) to have a dedicated component that seamlessly handles the transformation required when transmitting AWS CloudWatch metrics to Prometheus-compatible storage backends.
Example configuration for the component
TBD
Telemetry data types supported
Metrics
Code Owner(s)
No response
Sponsor (optional)
No response
Additional context
I created a PoC awsmetricsprocessor component that is incomplete but somewhat illustrates my proposal. Currently, delta-to-cumulative transformation is achieved using a cache that stores the latest values of the count and sum vectors. Name translation is handled through a mapping, which, while it may seem cumbersome, provides the benefit of adding consistency to an otherwise inconsistent metric naming convention in AWS. This approach can also be used to translate lables (dimensions) and enrich metrics with help text.
The text was updated successfully, but these errors were encountered:
The purpose and use-cases of the new component
Although, it's possible to transition AWS CloudWatch metrics to Prometheus using the
awsfirehosereceiver
andprometheusexporter
components, I see a need for a third component in the pipeline to transform CloudWatch metrics into a format better suited for Prometheus.Specifically, I want to convert non-monotonic counters into monotonic ones to ensure proper usage in PromQL. Additionally (though optionally), I'd like the metrics to follow a more consistent naming convention, which the forementioned pipeline configuration does not guarantee.
While existing processors like
metricstransformprocessor
anddeltatocumulativeprocessor
could potentially be adapted for this use case, from a user perspective, it would be far more convenient (IMO) to have a dedicated component that seamlessly handles the transformation required when transmitting AWS CloudWatch metrics to Prometheus-compatible storage backends.Example configuration for the component
TBD
Telemetry data types supported
Metrics
Code Owner(s)
No response
Sponsor (optional)
No response
Additional context
I created a PoC awsmetricsprocessor component that is incomplete but somewhat illustrates my proposal. Currently, delta-to-cumulative transformation is achieved using a cache that stores the latest values of the count and sum vectors. Name translation is handled through a mapping, which, while it may seem cumbersome, provides the benefit of adding consistency to an otherwise inconsistent metric naming convention in AWS. This approach can also be used to translate lables (dimensions) and enrich metrics with help text.
The text was updated successfully, but these errors were encountered: