Exporter Feature: OpenSearch Metrics

[edmondsiu0]

Component(s)

exporter/opensearch

Is your feature request related to a problem? Please describe.

Similar to issue #23611, this Issue is to suggest OpenSearch exporter to convert Otel metric data directly into OpenSearch.

This will remove the need of using dataprepper otel_metrics processor to reprocess the information before being sent into OpenSearch.

Describe the solution you'd like

Similar to the linked issue, but specific to metrics, such as:

exporters:
  opensearch/metrics:
    http:
      endpoint: https://opensearch.example.com:9200
    index: my-metrics-index

service:
  pipelines:
    metrics:
      receivers: [otlp]
      processors: [batch]
      exporters: [opensearch/metrics]

Describe alternatives you've considered

Dataprepper otel_metrics processor is able to perform this task but it is an external component. It would be good if this can be transformed directly within Otel.

Additional context

No response

[github-actions]

Pinging code owners:

• exporter/opensearch: @Aneurysm9 @MitchellGale @MaxKsyunz @YANG-DB

See Adding Labels via Comments if you do not have permissions to add labels yourself.

[crobert-1]

Just to confirm my understanding, this issue is a request for the OpenSearch exporter to add support for metrics. You're also saying that for this to work, it has to convert metrics into a format acceptable to OpenSearch. Am I understanding your request properly?

[edmondsiu0]

That is correct, thanks!

[crobert-1]

Does OpenSearch support any metric format other than Prometheus? I'm unfamiliar but trying to figure out if there are possible alternative options here.

From the help page I referenced, it looks like you may actually be able to just point OpenSearch at a collector instance using the Prometheus exporter. Would that work for your use case or am I missing something?

[scott-parillo]

We're in the very early phases of looking closely at Open Search as an alternative to Elastic and was just about to create a similar request when I found this entry. I know that Prometheus exporter could get the job done but we're hoping to avoid adding another service/backend to our solution.

Any idea how much effort it would take to enhance this exporter w/ metrics?

[crobert-1]

I know that Prometheus exporter could get the job done but we're hoping to avoid adding another service/backend to our solution.

Out of ignorance, would adding the Prometheus exporter to your collector configuration require more of a backend change than adding the OpenSearch exporter for metrics?

How would the functionality of an OpenSearch exporter that supports metrics differ from the Prometheus exporter? What functionality is missing that you're looking for?

Converting metrics to the Prometheus format in the collector is quite a bit of effort and prone to bugs, so it would be good to avoid duplicating this effort as much as possible.

[github-actions]

This issue has been inactive for 60 days. It will be closed in 60 days if there is no activity. To ping code owners by adding a component label, see Adding Labels via Comments, or if you are unsure of which component this issue relates to, please ping @open-telemetry/collector-contrib-triagers. If this issue is still relevant, please ping the code owners or leave a comment explaining why it is still relevant. Otherwise, please close it.

Pinging code owners:

• exporter/opensearch: @Aneurysm9 @MitchellGale @MaxKsyunz @YANG-DB

See Adding Labels via Comments if you do not have permissions to add labels yourself.