SuppressDownstreamInstrumentation option in OpenTelemetry.Instrumentation.GrpcNetClient instrumentation

[vishweshbankwar]

Issue

The usage of SuppressDownstreamInstrumentation isn't supported when used with the rc/stable versions of OpenTelemetry.Instrumentation.Http instrumentation. The limitation comes from the changes introduced in open-telemetry/opentelemetry-dotnet#5077.

SuppressDownstreamInstrumentation relies on OpenTelemetry.Instrumentation.Http instrumentation to avoid overriding the injected ActivityContext by OpenTelemetry.Instrumentation.GrpcNetClient instrumentation. However, since OpenTelemetry.Instrumentation.Http doesn't depend on the SDK, this suppression isn't possible.

As a result, When OpenTelemetry.Instrumentation.GrpcNetClient is used with OpenTelemetry.Instrumentation.Http, the outgoing request will always contain the traceparent header set from ActivityContext of HttpClient Activity instead of GrpcClient Activity .

Workaround

Use beta versions of OpenTelemetry.Instrumentation.Http instrumentation if SuppressDownstreamInstrumentation is needed

Update(4/15): The workaround shared above is not recommended to be used due to GHSA-vh2m-22xx-q94f. Alternate options needs to be further evaluated.

[reyang]

However, since OpenTelemetry.Instrumentation.Http doesn't depend on the SDK, this suppression isn't possible.

Would this solve the issue open-telemetry/opentelemetry-dotnet#960 (comment)?

[specialforest]

Use beta versions of OpenTelemetry.Instrumentation.Http instrumentation if SuppressDownstreamInstrumentation is needed.

I guess you had a specific version in mind, likely beta versions of 1.6.0.

[vishweshbankwar]

@specialforest - Yes. However, this has changed recently due to GHSA-vh2m-22xx-q94f. All versions of OpenTelemetry.Instrumentation.Http lower than 1.8.1 have been marked deprecated. As of now, we do not have an alternate for this feature. I have updated the description.

[gao-artur]

The SuppressDownstreamInstrumentation is documented as "not working". It makes an impression that setting it to true will be a no-op, but in fact, it suppresses both Grpc and Http client spans. This breaks the telemetry for those users who set it to true before 1.6.0. I think this property should be changed to no-op.

[gao-artur]

Sorry, I was wrong. It doesn't suppress the Grpc client span. It "disappears" the Http client span, making the Grpc server span an orphan.