OpenTelemetry Traces support

[marcotc]

What does this PR do?

The PR is divided into compartmentalized commits, if you prefer reviewing one commit at a time.

This PR allows ddtrace users to use Datadog Tracing and OpenTelemetry Traces together.

OpenTelemetry Traces created by the host application will be collected by ddtrace and sent to the Datadog UI without any code changes.

Datadog and OpenTelemetry spans will form a single trace, with correct parenting being handled by ddtrace.

To activate the OpenTelemetry Traces collection, run this line:

require 'datadog/opentelemetry'

Motivation

OpenTelemetry is a vendor-agnostic instrumentation specification that is being widely supported across the tech community today.

We should support users that want to keep using this vendor-agnostic API while using Datadog to process the telemetry data created.

Additional Notes

This is the first version of OpenTelemetry Traces support. It will be released as experimental.

This PR includes:

• Testing: Unit test coverage.
• Testing: An integration app exercising OpenTelemetry trace interweaving.
• Testing: Run the OpenTelemetry Ruby repository test suite with ddtrace integration enabled (bundle exec rake upstream:opentelemetry).
• Documentation about how to configure Datadog APM with OpenTelemetry, including current limitations.
• Telemetry collection for OpenTelemetry (a boolean flag for when OpenTelemetry integration is enabled).

Examples (from the integration app)

OpenTelemetry root span, with a Datadog child span:

Datadog root span, with an OpenTelemetry child span:

And because we handle trace parenting now, a baseline test case to ensure this PR didn't break pure OpenTelemetry parenting:

How to test the change?

• The easiest way is to run the integration/apps/opentelemetry integration app, to see real spans in the Datadog UI.
• bundle exec rake upstream:opentelemetry ensures no OpenTelemetry contract has been violated by this integration.

[codecov-commenter]

Codecov Report

Merging #2496 (848a5e6) into master (b8f4cec) will decrease coverage by 0.02%.
The diff coverage is 95.02%.

@@            Coverage Diff             @@
##           master    #2496      +/-   ##
==========================================
- Coverage   98.06%   98.04%   -0.02%     
==========================================
  Files        1135     1143       +8     
  Lines       61702    62033     +331     
  Branches     2777     2801      +24     
==========================================
+ Hits        60505    60822     +317     
- Misses       1197     1211      +14     

Impacted Files	Coverage Δ
lib/datadog/tracing/configuration/ext.rb	100.00% <ø> (ø)
lib/datadog/tracing/metadata/tagging.rb	95.23% <50.00%> (-2.27%)	⬇️
lib/datadog/opentelemetry/sdk/propagator.rb	82.35% <82.35%> (ø)
lib/datadog/opentelemetry/sdk/span_processor.rb	83.87% <83.87%> (ø)
lib/datadog/opentelemetry/api/context.rb	91.42% <91.42%> (ø)
lib/datadog/core/telemetry/collector.rb	100.00% <100.00%> (ø)
lib/datadog/kit/appsec/events.rb	100.00% <100.00%> (ø)
lib/datadog/opentelemetry.rb	100.00% <100.00%> (ø)
lib/datadog/opentelemetry/api/trace/span.rb	100.00% <100.00%> (ø)
lib/datadog/opentelemetry/sdk/configurator.rb	100.00% <100.00%> (ø)
... and 11 more

📣 We’re building smart automated test selection to slash your CI/CD build times. Learn more

[delner]

Rebased on the latest master and added some documentation. Waiting for a green build.

[delner]

Reviewed this with @marcotc at a high level.

There's a lot to digest in the details (which I'm not fully knowledgeable), but overall the general strategy to patch into OTel over re-implementing the API seems sound; less brittle in regards to changes within OTel (needing to duplicate a lot of implementation to meet spec). It's also reassuring to see the OTel test suite passing in combination with these changes in our CI.

This should be sufficient for experimental support; we'll want to performance test this for stability and run in other environments to validate it before promoting support to "beta".

All-in-all awesome job on this @marcotc ! Really excited to see this in action! 😄