Elasticsearchexporter: dynamic index

[leewoobin789]

Is your feature request related to a problem? Please describe.
For future usecase:
According to the existing readme of Elasticsearchexporter, which is inactivated in the current collector version , there seems to be no dynamic index capability e.g. overwriting index name with prefix and namespace etc.
This feature would make another integration component like logstash obsolete and simplifies the overall architecture.

Describe the solution you'd like
Optional configuration: overwriting index name with predefined prefix and value of a ecs field name like elasticsearch plugin in fluentd.

Describe alternatives you've considered
None

Additional context

• configuration extension on elasticsearch exporter

[wajika]

index: should support ILM or template...

[github-actions]

Pinging code owners: @urso @faec @blakerouse. See Adding Labels via Comments if you do not have permissions to add labels yourself.

[leewoobin789]

@djaglowski @JaredTan95 i will be working on it. just telling you to avoid any redundant effort.

[leewoobin789]

Idea:

elasticsearch/log:
    endpoints: [http://localhost:9200]
    logs_index: myLogIndex
    logs_dynamic_index:
        disabled: false #default: true
        prefix: "staticPrefix-" # both static or attribute/resource based prefix (if one of prefix & suffix does not match, then fall back to logs_index"
        suffix: "-{resource['k8s.namespace']}" # both static or attribute/resource based suffix (if one of prefix & suffix does not match, then fall back to logs_index"

ex) staticPrefix-myLogIndex-someNamespace
there will be no breaking changes as it is addition of optional setting.

or

resource attribute based.

elasticsearch/log:
    endpoints: [http://localhost:9200]
    logs_index: myLogIndex
    logs_dynamic_index:
        disabled: false #default: true

which checks resource attribute named "elasticsearchexporter.logs_index_suffix", which is added to default index as a suffix.
this way, one could use resource processor of collector or add this attribute in applications to dynamically route it to different indices.

I actually prefer second one because it prevents many other error prone operations and checks only existence of dynamic or predefined resources

What do you think folks? :) @djaglowski @JaredTan95

[github-actions]

This issue has been inactive for 60 days. It will be closed in 60 days if there is no activity. To ping code owners by adding a component label, see Adding Labels via Comments, or if you are unsure of which component this issue relates to, please ping @open-telemetry/collector-contrib-triagers. If this issue is still relevant, please ping the code owners or leave a comment explaining why it is still relevant. Otherwise, please close it.

Pinging code owners:

• exporter/elasticsearch: @JaredTan95

See Adding Labels via Comments if you do not have permissions to add labels yourself.

[github-actions]

This issue has been inactive for 60 days. It will be closed in 60 days if there is no activity. To ping code owners by adding a component label, see Adding Labels via Comments, or if you are unsure of which component this issue relates to, please ping @open-telemetry/collector-contrib-triagers. If this issue is still relevant, please ping the code owners or leave a comment explaining why it is still relevant. Otherwise, please close it.

Pinging code owners:

• exporter/elasticsearch: @JaredTan95

See Adding Labels via Comments if you do not have permissions to add labels yourself.

[github-actions]

This issue has been closed as inactive because it has been stale for 120 days with no activity.