[Feature] OpenTelemetry support (?)

[clemherreman]

Describe the feature you'd like

Hello,

Is there an integration of open-telemetry/opentelemetry into the dd-trace SDK, so that we may use the OTEL collector + Datadog exporter instead of the Datadog Agent ?

Hello,

Is your feature request related to a problem?

At my job we are using happily Datadog currently. For interoperability purpose, we want our collection to be OTEL/OTLP only.

This is reachable by using open-telemetry/opentelemetry-php + https://github.com/open-telemetry/opentelemetry-php-instrumentation + some auto-instrumentation if available.

However, this is rather new and, while it's great to be able to build exactly what you need, I want to spend my effort using my observability toolsuite, instead of building it. And DD-Trace is really good at that, brings a lot out-of-the-box and suits 90% of what I need to instrument.

Describe alternatives you've considered

I saw that there is an ongoing PR #2332 implementing the OpenTelemetry API into DD-Trace SDK.

Is it a strong goal to reach for Datadog? Or is it possible that OpenTelemetry support (and that PR) could be dropped?

In the first case: do you have an targeted ETA for that support, even very vague?

Such estimate would permit me to organize & schedule our migration toward an OTEL-only collection :)

Thank you for your answer !

Additional context

No response

[pierotibou]

Hello @clemherreman , @PROFeNoM is actively working on #2332 and it is our goal to deliver it this quarter now that the OTEL API isn't in beta anymore.

[clemherreman]

Thank you for the very quick answer @pierotibou.
This quarter means Q4 2023 right ?

[pierotibou]

@clemherreman yes, we should merge this PR at most by the end of 2023 indeed. :) I mean - to be clear - we plan only to work on this drop in replacement, so merging the PR you've linked. Nothing else for now as we've already worked on w3c trace context propagation, 128 bits trace ids to be compatible with OTEL

[clemherreman]

@pierotibou end of 2023 ? great news :)

If you have time, I have a couple more questions:

• you mentioned that you only plan to work on OpenTelemetry API #2332. In the description of that PR, there is a list of features, titled "Not supported (yet 👀):". However, some items are marked as TBC or WIP.

Q: Does it mean that it is the targeted scope of that PR, or is it a list of things that will no be implemented?

• you mentionned w3c trace context propagation, so I read Propagating PHP Trace Context, but also OpenTelemetry in Datadog, Custom Instrumentation with the OpenTelemetry API& OpenTelemetry Collector Datadog Exporter

Q: If I understood correctly, this means that dd-trace-php is implementing (via #2332) the OTEL API in it, so that it sends traces, spans & metrics the way OTEL sends it, allowing a "vanilla" OTEL collector to collect it? And without having to change the way the code is instrumented (and enjoying all the auto-instrumentation dd-trace-php already does)?

Q: Does #2332 takes care of trace propagation context, so that traces collected with that OTEL collector, will be correlated to collected logs too?

Thank you for your time :)

[PROFeNoM]

Hi @clemherreman, I'll try to answer the best I can :)

Does it mean that it is the targeted scope of that PR, or is it a list of things that will not be implemented?

The first thing to note is that our goal is to provide as much support and interoperability as possible with OTel - That said, some features aren't supported by Datadog yet (e.g., Span Events), or are in the process of being supported (e.g., Span Links). The main scope targeted by that PR is to provide support for the Tracing API, i.e., we want customers to be able to be vendor-agnostic regarding instrumentation & interoperability while leveraging Datadog's product suite in a zero-touch manner. We eventually want to implement as much as possible when the time is right and keep cohesion with other Datadog's tracers.

If I understood correctly, this means that dd-trace-php is implementing (via #2332) the OTEL API in it so that it sends traces, spans & metrics the way OTEL sends it, allowing a "vanilla" OTEL collector to collect it?

There are multiple aspects here. As mentioned in one of the document you linked, there are two ways to get traces to the Datadog backend: using the Datadog Exporter or using the Datadog Agent. Currently, the latter is not possible when instrumenting using the OTel API - This PR provides this capability, allowing proprietary products to be used on top of this data (e.g., Continuous Profiler). However, it is important to note that the term "metrics" is broad and that OTel's metrics API will not be supported at this time. I'll be reaching out to my colleagues to get more information on how Datadog's backend will handle this issue.

And without changing how the code is instrumented (and enjoying all the auto-instrumentation dd-trace-php already does)?

Yes, the goal is for you to keep everything the same code-wise. We want you to be able to enjoy dd-trace-php's auto-instrumentation, OTel's auto-instrumentation, and DD/OTel's manual instrumentation.

Does #2332 take care of trace propagation context so that traces collected with that OTEL collector will be correlated to collected logs, too?

Logs should correlate to traces if you are using Datadog's exporter with an application instrumented with the OTel library. There was an effort in this PR to try and support trace propagation context, which was successful on test applications.

[PROFeNoM]

Hi @clemherreman!

We just released v0.94.0, which includes support for the OTel API. We appreciate your patience and eagerly await your feedback 😃

[PROFeNoM]

I am closing this issue as the feature has been delivered. However, please feel free to open a new issue if you encounter any problems with the API.