Implement span started and live health metrics

[JonasKunz]

Implements the semantic conventions SDK health metrics otel.sdk.span.live and otel.sdk.span.endedotel.sdk.span.started metrics.

Follows the approach of #7265 that these metrics are currently disabled by default, but can be enabled via the SdkTracerProviderBuilder.

Some examples of insights gained from these metrics:

• The effective span sampling rate
• Detection of span-leaks due to missing end() calls (via a steadily increasing otel.sdk.span.live count)
• Mis-configured / Mis-instrumentation detection indication via excessive span counts

[codecov]

Codecov Report

Attention: Patch coverage is 89.42308% with 11 lines in your changes missing coverage. Please review.

Project coverage is 89.84%. Comparing base (317c002) to head (24a7a0a).

Files with missing lines	Patch %	Lines
...sdk/trace/internal/SemConvSpanInstrumentation.java	89.55%	4 Missing and 3 partials ⚠️
...ava/io/opentelemetry/sdk/trace/SdkSpanBuilder.java	50.00%	1 Missing and 1 partial ⚠️
...emetry/sdk/trace/internal/SpanInstrumentation.java	50.00%	1 Missing and 1 partial ⚠️

Additional details and impacted files

@@             Coverage Diff              @@
##               main    #7430      +/-   ##
============================================
+ Coverage     89.82%   89.84%   +0.02%     
- Complexity     6998     7026      +28     
============================================
  Files           798      801       +3     
  Lines         21199    21297      +98     
  Branches       2055     2062       +7     
============================================
+ Hits          19041    19135      +94     
- Misses         1496     1498       +2     
- Partials        662      664       +2     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[JonasKunz]

This test is pretty much a copy of PropagatedSpanTest.doNotCrash. I don't think it's very useful, but it keeps the CodeCov bot happy

[jack-berg]

Solid stuff. One recommendation about renaming to *Instrumentation - since there's going to be more and more internal telemetry, we should try to standardize naming / patterns where we can.

Thanks!

[jack-berg]

I know this is the contract of other components, but I wonder if we may want to think bigger, and start accepting Supplier<OpenTelemetry> for internal monitoring. That would position us well if there are semantic conventions that involve emitting traces or logs or events for internal monitoring.

[JonasKunz]

Fixed in 74fdbae. I changed it just at the API-level, the implementation still directly uses Supplier<MeterProvider> for now, because this is the only part used currently..

I went for setInternalTelemetryOpenTelemetry as name, because I feel setOpenTelemetry would be confusing: It could make it look like the counterpart to SdkOpenTelemetryBuilder.setTracerProvider.

[jack-berg]

Let's follow the pattern of ExporterInstrumentation:

• Let's have a class called SpanInstrumentation (or maybe TracerProviderInstrumentation)
• Class should have a constructor which accepts InternalTelemetryVersion and Supplier<MeterProvider> and initializes accordingly.

[JonasKunz]

Changed in 6f4b0af.

However, I kept SpanInstrumentation as an interface instead of class with a static create method for now: It doesn't contain any logic except for create, which is why I think making it a class and adding delegation like ExporterInstrumentation would be unnecessary boilerplate. It however does allow us to change to this model later if we need it without touching the consumers of SpanInstrumentation.

[jack-berg]

Should we mark endAlreadyReported concurrent safe? If a span is ended multiple times on different threads, then the state of these metrics could become corrupted. On the other hand, SdkSpan#endInternal is already protected by a lock, so the only thing that could cause is is concurrent calls to end a span + NonRecordSpan, which isn't protected by a lock.

[JonasKunz]

I added a @GuardedBy("this"), is this what you meant? It was already concurrent safe because recordSpanEnd() is synchronized.
Also this is in fact necessary, because for NonRecordingSpan end() isn't synchronized.

[JonasKunz]

I'm moving this back to draft, because this implementation actally violates the spec. See this comment.

We'll fix the definition of the metrics first on that Sem-Conv PR, and then update this one after reaching consensus there.

[JonasKunz]

Ready for review again, as the spec changes from this merged semconv PR have now been applied:

• The otel.sdk.span.ended metric was replaced with otel.sdk.span.started
• otel.sdk.span.live was changed to not be collected for non-recording spans
• Added a new otel.span.parent.origin to otel.sdk.span.started. This allows us to derive many cool insights, such as:
  • average number of spans per trace
  • effective trace sampling rate (versus the effective span-sampling rate)
  • effective trace sampling rate for traces starting in the current service
  • effective trace sampling rate for traces with a remote parent