Add otel.scope.schema_url to non-OTLP exporters

[pellared]

Fixes #2320

Related to (follows):

• prometheus: handle schema URL and scope attributes opentelemetry-specification#4505

The Instrumentation Scope Attributes for non-OTLP formats already says what exporters can do with scope attributes.

Changes

Add otel.scope.schema_url attribute to non-OTLP exporters that represents an instrumentation scope schema URL.

Remarks

Schema URL is defined as part of instrumentation scope in the spec: https://github.com/open-telemetry/opentelemetry-specification/blob/main/specification/common/instrumentation-scope.md

However, in OTLP it is always on a "higher" level for reasons unknown to me (maybe it was historically introduced before the concept of the instrumentation scope?): https://github.com/open-telemetry/opentelemetry-proto/blob/c30610041736aa5c0077b156f27b09e878b797ea/opentelemetry/proto/trace/v1/trace.proto#L58-L64

We could consider also naming it otel.schema_url. However, with the pre-work of https://github.com/open-telemetry/opentelemetry-specification/blob/main/specification/common/instrumentation-scope.md and open-telemetry/opentelemetry-specification#4505 I prefer otel.scope.schema_url.

[pellared]

Undrafting as this needs to be reviewed/discussed even though it is not ready to be merged.

[joaopgrassi]

This I don't agree with - the source of the data is OTel and OTel recommends namespacing everything - main/docs/general/naming.md#recommendations-for-application-developers.

So good otel instrumentation should not put name on scope and name on data, it would put something like foo.name on the scope attribute and bar.name on the telemetry item.

If they put the same name on both, they want to pass the same thing twice - this seems redundant and ambigous. And while it's allowed, it makes no sense.

I also agree here with @lmolkova. If the deal is because of prometheus compatibility, we already have that defined in the other document: https://github.com/open-telemetry/opentelemetry-specification/blob/ac137b349b006dc926da91c6a6619217737921cc/specification/compatibility/prometheus_and_openmetrics.md#instrumentation-scope. So I'm not sure this needs to be defined in semconv.

[pellared]

I think this PR is ready for an actual review.

CC @open-telemetry/prometheus-interoperability

[thompson-tomo]

Still discussing the attributes topic but found another small thing.

[ArthurSens]

We managed to revert the feature in Prometheus before it made it into a release: prometheus/prometheus#16842

It should be safe to change the spec

[pellared]

It should be safe to change the spec

Can you please how do we want to change the spec?
What do you changes do you have in mind, the ones from this PR or some follow-up in Spec repo?

[pellared]

OTel Logs SIG meeting notes (from my memory):

For most use-cases non-OTLP exporters could simply put scope attributes to attributes/tags without any modifications.

However, for places where OTel's nested structure (resource, scope, datapoint) when translating is important adding an otel.scope. prefix can be allowed (it would be especially handy for Collector receivers). Probably this should be an opt-in behavior for the non-OTLP exporters.

It looks like we do not need to describe anything in the Semantic Conventions.

@lmolkova, @trask, @austinlparker, feel free to edit my comment if I missed anything or made any mistake.

[pellared]

Maybe the existing description in Instrumentation Scope Attributes for non-OTLP formats is good enough?

I changed this PR so it only adds otel.scope.schema_url which is not controversial and I also plan create a similar PR for (https://github.com/open-telemetry/opentelemetry-specification/blob/main/specification/common/mapping-to-non-otlp.md#instrumentation-scope-attributes.

[pellared]

Do we want/need to make it "development" first?

[ArthurSens]

It should be safe to change the spec

Can you please how do we want to change the spec? What do you changes do you have in mind, the ones from this PR or some follow-up in Spec repo?

What I meant here is that we were able to revert our commitment to translating Scope attributes to otel_scope_[attr] before it gets released in Prometheus. This means we can adjust the spec, e.g., using otel_scope_attr_[attr] instead if we so desire and that won't be a breaking change in Prometheus. And by spec, I mean the Prometheus and OpenMetrics compatibility.

[pellared]

What I meant here is that we were able to revert our commitment to translating Scope attributes to otel_scope_[attr] before it gets released in Prometheus. This means we can adjust the spec, e.g., using otel_scope_attr_[attr] instead if we so desire and that won't be a breaking change in Prometheus. And by spec, I mean the Prometheus and OpenMetrics compatibility.

@ArthurSens, with the conversation we had I think it is find to keep otel_scope_[attr]. There is a question whether users would not simply prefer [attr] instead. This could be potentially configurable. In my opinion this can be discussed separately by @open-telemetry/prometheus-interoperability. I am sorry for my bothersome way of working...

[ArthurSens]

@ArthurSens, with the conversation we had I think it is find to keep otel_scope_[attr]. There is a question whether users would not simply prefer [attr] instead. This could be potentially configurable. In my opinion this can be discussed separately by @open-telemetry/prometheus-interoperability. I am sorry for my bothersome way of working...

No problems at all! I'm glad we have the chance to align, please don't hesitate to keep tagging us when things aren't clear