.NET 9.0 Native Trace Instrumentation Support for HttpClient as per OTel specification

[vishweshbankwar]

Current State

Currently, users depend on the instrumentation library provided by the OpenTelemetry .NET repository to enable the enrichment of activities created during HttpClient requests, following the OpenTelemetry specification. This instrumentation library enriches activities by subscribing to DiagnosticSource events generated by HttpClient. However, this approach has certain limitation and performance overhead of Diagnostic Source listeners and reflection.

Feature Request

We are requesting the addition of native instrumentation support for HttpClient in .NET 9.0. This enhancement will complement the out-of-the-box OTel metrics instrumentation introduced in .NET 8.0.

Design Considerations

When designing this feature, we need to consider the following aspects:

1. Support for Enrichment and Filtering of Spans: Ensure that the native instrumentation supports the enrichment and filtering of HttpClient spans.

2. Custom Propagators: Address the differences in API between propagators in OpenTelemetry and the DistributedContextPropagator. Determine how custom propagators should be handled in this context.

[ghost]

Tagging subscribers to this area: @dotnet/ncl
See info in area-owners.md if you want to be subscribed.

Issue Details

---

Current State

Currently, users depend on the instrumentation library provided by the OpenTelemetry .NET repository to enable the enrichment of activities created during HttpClient requests, following the OpenTelemetry specification. This instrumentation library enriches activities by subscribing to DiagnosticSource events generated by HttpClient. However, this approach has certain limitation.

Feature Request

We are requesting the addition of native instrumentation support for HttpClient in .NET 9.0. This enhancement will complement the out-of-the-box OTel metrics instrumentation introduced in .NET 8.0.

Design Considerations

When designing this feature, we need to consider the following aspects:

1. Support for Enrichment and Filtering of Spans: Ensure that the native instrumentation supports the enrichment and filtering of HttpClient spans.

2. Custom Propagators: Address the differences in API between propagators in OpenTelemetry and the DistributedContextPropagator. Determine how custom propagators should be handled in this context.

Author:	vishweshbankwar
Assignees:	-
Labels:	area-System.Net.Http
Milestone:	-

[vishweshbankwar]

@tarekgh @cijothomas FYI

[cijothomas]

However, this approach has certain open-telemetry/opentelemetry-dotnet#4830.

And the performance overhead of ds listeners/reflection etc.

[ghost]

Tagging subscribers to this area: @tarekgh, @tommcdon, @pjanotti
See info in area-owners.md if you want to be subscribed.

Issue Details

---

Current State

Currently, users depend on the instrumentation library provided by the OpenTelemetry .NET repository to enable the enrichment of activities created during HttpClient requests, following the OpenTelemetry specification. This instrumentation library enriches activities by subscribing to DiagnosticSource events generated by HttpClient. However, this approach has certain limitation.

Feature Request

We are requesting the addition of native instrumentation support for HttpClient in .NET 9.0. This enhancement will complement the out-of-the-box OTel metrics instrumentation introduced in .NET 8.0.

Design Considerations

When designing this feature, we need to consider the following aspects:

1. Support for Enrichment and Filtering of Spans: Ensure that the native instrumentation supports the enrichment and filtering of HttpClient spans.

2. Custom Propagators: Address the differences in API between propagators in OpenTelemetry and the DistributedContextPropagator. Determine how custom propagators should be handled in this context.

Author:	vishweshbankwar
Assignees:	-
Labels:	area-System.Diagnostics.Tracing, area-System.Net.Http, untriaged
Milestone:	-

[antonfirsov]

Moved this to area-System.Diagnostics.Tracing since it seems to touch higher level distributed tracing concepts.

/cc @noahfalk

[ghost]

Tagging subscribers to this area: @dotnet/ncl
See info in area-owners.md if you want to be subscribed.

Issue Details

---

Current State

Currently, users depend on the instrumentation library provided by the OpenTelemetry .NET repository to enable the enrichment of activities created during HttpClient requests, following the OpenTelemetry specification. This instrumentation library enriches activities by subscribing to DiagnosticSource events generated by HttpClient. However, this approach has certain limitation and performance overhead of Diagnostic Source listeners and reflection.

Feature Request

We are requesting the addition of native instrumentation support for HttpClient in .NET 9.0. This enhancement will complement the out-of-the-box OTel metrics instrumentation introduced in .NET 8.0.

Design Considerations

When designing this feature, we need to consider the following aspects:

1. Support for Enrichment and Filtering of Spans: Ensure that the native instrumentation supports the enrichment and filtering of HttpClient spans.

2. Custom Propagators: Address the differences in API between propagators in OpenTelemetry and the DistributedContextPropagator. Determine how custom propagators should be handled in this context.

Author:	vishweshbankwar
Assignees:	-
Labels:	enhancement, area-System.Net.Http
Milestone:	9.0.0

[tarekgh]

Talked offline with @noahfalk and changed it back to http as the work will be in http.

[antonfirsov]

@vishweshbankwar a question regarding the timing of tag injection and enrichment:

opentelemetry-dotnet's HttpHandlerDiagnosticListener sets some of the tags at System.Net.Http.HttpRequestOut.Start, wile others are being set at System.Net.Http.HttpRequestOut.Stop. Similarly, enrichment is possible both at the start (EnrichWithHttpRequestMessage) and at the end (EnrichWithHttpResponseMessage & EnrichWithException) of the request. If in HttpClients instrumentation we could emit everything at the end instead, that would allow us to design a simpler, more compact tracing enrichment API which would be more consistent with our current Metrics enrichment offering. I wonder if there is a practical necessity for knowing the tag values at System.Net.Http.HttpRequestOut.Start? Are you aware of use-cases which actually depend on this?

[vishweshbankwar]

@vishweshbankwar a question regarding the timing of tag injection and enrichment:

opentelemetry-dotnet's HttpHandlerDiagnosticListener sets some of the tags at System.Net.Http.HttpRequestOut.Start, wile others are being set at System.Net.Http.HttpRequestOut.Stop. Similarly, enrichment is possible both at the start (EnrichWithHttpRequestMessage) and at the end (EnrichWithHttpResponseMessage & EnrichWithException) of the request. If in HttpClients instrumentation we could emit everything at the end instead, that would allow us to design a simpler, more compact tracing enrichment API which would be more consistent with our current Metrics enrichment offering. I wonder if there is a practical necessity for knowing the tag values at System.Net.Http.HttpRequestOut.Start? Are you aware of use-cases which actually depend on this?

In the current implementation which is based on the diagnostic source callbacks, there was no way to allow enrich via single API (due to different callbacks with different information). If that constraint is not valid for instrumentation within the library, it makes sense to have a simpler API.