[system_otel]: Host cloud metadata fails even with cloud resource detectors

[graphaelli]

Integration Name

System OpenTelemetry Assets [system_otel]

Dataset Name

hostmetricsreceiver.otel

Integration Version

0.2.0

Agent Version

n/a

Agent Output Type

elasticsearch

Elasticsearch Version

serverless

OS Version and Architecture

amazon linux 2 x86_64

Software/API Version

nA

Error Message

Unexpected error from Elasticsearch: verification_exception - Found 1 problem
line 5:1: Unknown column [resource.attributes.cloud.instance.id], did you mean any of [resource.attributes.cloud.account.id, resource.attributes.cloud.provider, resource.attributes.cloud.platform]?

Event Original

No response

What did you do?

this is using the hostmetrics receiver with ec2 resourcedetector

What did you see?

What did you expect to see?

cloud metadata. I haven't been able to find anything (gcp, ec2, or azure) reporting cloud.instance.id so I'm not sure if this visualization ever works as is. I'd expect host.id to be the instance id.

Anything else?

No response

[rogercoll]

We are inserting this attribute manually in our onboarding configurations:

• https://github.com/elastic/elastic-agent/blob/main/deploy/helm/edot-collector/kube-stack/values.yaml#L262
• https://github.com/elastic/elastic-agent/blob/main/deploy/helm/edot-collector/kube-stack/managed_otlp/values.yaml#L299
• https://github.com/elastic/elastic-agent/blob/main/deploy/helm/edot-collector/kube-stack/managed_otlp/logs-values.yaml#L127

As not being an OpenTelemetry/SemConv attribute, I agree that we should remove it from both, dashboards and ingestion configurations.

[mlunadia]

This is a bug let's prioritise this one which seems to be straight forward to solve.

[jmmcorreia]

@mlunadia Do you agree with @rogercoll comment that we should not be using this attribute due to it not being part of the semantic conventions? Or are you looking for the dashboard to be fixed and keep the attribute ?

[mlunadia]

The bar is not whether an attribute is part of the semantic conventions. Many attributes implemented in upstream components are not defined in the semantic conventions or declared stable if they are.

If a corresponding semantic convention exists, we should use it. Ideally this is obtained with zero configuration from the relevant component, such as the hostmetrics receiver. In the worst case, some configuration may be required on the upstream component to get it.

What we should avoid is adding attributes that do not exist in the upstream specification or implementation.

So, yes let's fix the dashboard to do what was the original intention but using attributes we can get natively from the component without adding them in.