Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add support AWS Embedded Metric Format Version 0 #20314

Merged
merged 3 commits into from
Apr 25, 2023
Merged

Add support AWS Embedded Metric Format Version 0 #20314

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
5a3b0a5
Support AWS EMF Version 1
khanhntd Mar 25, 2023
60043c1
Add support for EMF v1
khanhntd Mar 25, 2023
d3a0532
Add timestamp for emf v0 when there is no measurement
khanhntd Mar 31, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
16 changes: 16 additions & 0 deletions .chloggen/add-support-emf-version-0.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
# One of 'breaking', 'deprecation', 'new_component', 'enhancement', 'bug_fix'
change_type: enhancement

# The name of the component, or a single word describing the area of concern, (e.g. filelogreceiver)
component: awsemfexporter

# A brief description of the change. Surround your text with quotes ("") if it needs to start with a backtick (`).
note: Add support AWS Embedded Metric Format Version 0

# One or more tracking issues related to the change
issues: [20314]

# (Optional) One or more lines of additional information to render under the primary note.
# These lines will be padded with 2 spaces and then inserted directly into the document.
# Use pipe (|) for multiline entries.
subtext:
7 changes: 4 additions & 3 deletions exporter/awsemfexporter/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -32,12 +32,13 @@ The following exporter configuration parameters are supported.
| `max_retries` | Maximum number of retries before abandoning an attempt to post data. | 1 |
| `dimension_rollup_option` | DimensionRollupOption is the option for metrics dimension rollup. Three options are available: `NoDimensionRollup`, `SingleDimensionRollupOnly` and `ZeroAndSingleDimensionRollup` | "ZeroAndSingleDimensionRollup" (Enable both zero dimension rollup and single dimension rollup) |
| `resource_to_telemetry_conversion` | "resource_to_telemetry_conversion" is the option for converting resource attributes to telemetry attributes. It has only one config onption- `enabled`. For metrics, if `enabled=true`, all the resource attributes will be converted to metric labels by default. See `Resource Attributes to Metric Labels` section below for examples. | `enabled=false` |
| `output_destination` | "output_destination" is an option to specify the EMFExporter output. Currently, two options are available. "cloudwatch" or "stdout" | `cloudwatch` |
| `detailed_metrics` | Retain detailed datapoint values in exported metrics (e.g instead of exporting a quantile as a statistical value, preserve the quantile's population) | `false` |
| `output_destination` | Specify the EMFExporter output. Currently, two options are available. "cloudwatch" or "stdout" | `cloudwatch` |
| `detailed_metrics` | Retain detailed datapoint values in exported metrics (e.g instead of exporting a quantile as a statistical value, preserve the quantile's population) | `false` |
| `version` | Send metrics to CloudWatchLogs with Embedded Metric Format in selected version [(e.g version 1 with _aws)](https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/CloudWatch_Embedded_Metric_Format_Specification.html#CloudWatch_Embedded_Metric_Format_Specification_structure), version 0 without _aws) | `1` |
| `parse_json_encoded_attr_values` | List of attribute keys whose corresponding values are JSON-encoded strings and will be converted to JSON structures in emf logs. For example, the attribute string value "{\\"x\\":5,\\"y\\":6}" will be converted to a json object: ```{"x": 5, "y": 6}``` | [ ] |
| [`metric_declarations`](#metric_declaration) | List of rules for filtering exported metrics and their dimensions. | [ ] |
| [`metric_descriptors`](#metric_descriptor) | List of rules for inserting or updating metric descriptors. | [ ] |
| `retain_initial_value_of_delta_metric` | This option specifies how the first value of a metric is handled. AWS EMF expects metric values to only contain deltas to the previous value. In the default case the first received value is therefor not sent to AWS but only used as a baseline for follow up changes to this metric. This is fine for high throughput metrics with stable labels (e.g. `requests{code=200}`). In this case it does not matter if the first value of this metric is discarded. However when your metric describes infrequent events or events with high label cardinality, then the exporter in default configuration would still drop the first occurrence of this metric. With this configuration value set to `true` the first value of all metrics will instead be send to AWS. | false |
| `retain_initial_value_of_delta_metric` | Specify how the first value of a metric is handled. AWS EMF expects metric values to only contain deltas to the previous value. In the default case the first received value is therefor not sent to AWS but only used as a baseline for follow up changes to this metric. This is fine for high throughput metrics with stable labels (e.g. `requests{code=200}`). In this case it does not matter if the first value of this metric is discarded. However when your metric describes infrequent events or events with high label cardinality, then the exporter in default configuration would still drop the first occurrence of this metric. With this configuration value set to `true` the first value of all metrics will instead be send to AWS. | false |

### metric_declaration
A metric_declaration section characterizes a rule to be used to set dimensions for exported metrics, filtered by the incoming metrics' labels and metric names.
Expand Down
12 changes: 10 additions & 2 deletions exporter/awsemfexporter/config.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,7 @@ package awsemfexporter // import "github.com/open-telemetry/opentelemetry-collec
import (
"errors"

"go.opentelemetry.io/collector/component"
"go.uber.org/zap"

"github.com/open-telemetry/opentelemetry-collector-contrib/internal/aws/awsutil"
Expand Down Expand Up @@ -79,15 +80,20 @@ type Config struct {
// Note that at the moment in order to use this feature the value "kubernetes" must also be added to the ParseJSONEncodedAttributeValues array in order to be used
EKSFargateContainerInsightsEnabled bool `mapstructure:"eks_fargate_container_insights_enabled"`

// ResourceToTelemetrySettings is the option for converting resource attrihutes to telemetry attributes.
// ResourceToTelemetrySettings is an option for converting resource attrihutes to telemetry attributes.
// "Enabled" - A boolean field to enable/disable this option. Default is `false`.
// If enabled, all the resource attributes will be converted to metric labels by default.
ResourceToTelemetrySettings resourcetotelemetry.Settings `mapstructure:"resource_to_telemetry_conversion"`

// DetailedMetrics is the options for retaining detailed datapoint values in exported metrics (e.g instead of exporting a quantile as a statistical value,
// DetailedMetrics is an option for retaining detailed datapoint values in exported metrics (e.g instead of exporting a quantile as a statistical value,
// preserve the quantile's population)
DetailedMetrics bool `mapstructure:"detailed_metrics"`

// Version is an option for sending metrics to CloudWatchLogs with Embedded Metric Format in selected version (with "_aws")
// https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/CloudWatch_Embedded_Metric_Format_Specification.html#CloudWatch_Embedded_Metric_Format_Specification_structure
// Otherwise, sending metrics as Embedded Metric Format version 0 (without "_aws")
Version string `mapstructure:"version"`

// logger is the Logger used for writing error/warning logs
logger *zap.Logger
}
Expand All @@ -102,6 +108,8 @@ type MetricDescriptor struct {
Overwrite bool `mapstructure:"overwrite"`
}

var _ component.Config = (*Config)(nil)
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This will validate if the Config declared by awsemfexporter Config has all the declared function by the interface


// Validate filters out invalid metricDeclarations and metricDescriptors
func (config *Config) Validate() error {
var validDeclarations []*MetricDeclaration
Expand Down
3 changes: 3 additions & 0 deletions exporter/awsemfexporter/config_test.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -59,6 +59,7 @@ func TestLoadConfig(t *testing.T) {
LogStreamName: "",
DimensionRollupOption: "ZeroAndSingleDimensionRollup",
OutputDestination: "cloudwatch",
Version: "1",
logger: zap.NewNop(),
},
},
Expand All @@ -79,6 +80,7 @@ func TestLoadConfig(t *testing.T) {
LogStreamName: "",
DimensionRollupOption: "ZeroAndSingleDimensionRollup",
OutputDestination: "cloudwatch",
Version: "1",
ResourceToTelemetrySettings: resourcetotelemetry.Settings{Enabled: true},
logger: zap.NewNop(),
},
Expand All @@ -100,6 +102,7 @@ func TestLoadConfig(t *testing.T) {
LogStreamName: "",
DimensionRollupOption: "ZeroAndSingleDimensionRollup",
OutputDestination: "cloudwatch",
Version: "1",
MetricDescriptors: []MetricDescriptor{{
MetricName: "memcached_current_items",
Unit: "Count",
Expand Down