JaegerExportProtocol.UdpCompactThrift connects UdpClient within constructor

[AlexanderInova]

Bug Report

List of all OpenTelemetry NuGet
packages and versions:

• OpenTelemetry.Exporter.Jaeger 1.5.0
• OpenTelemetry.Extensions.Hosting 1.5.0
• OpenTelemetry.Instrumentation.AspNetCore 1.5.0-beta.1

Runtime version: net7.0

Symptom

OpenTelemetry.Exporter.Jaeger initialization crashes application when the underlying UdpClient fails to "connect" for some reason.

What is the expected behavior?

The unavailability of a tracing service like Jaeger should imho not lead to more or less the complete cluster/application being taken out of service.

What is the actual behavior?

The plugin "OpenTelemetry.Exporter.JaegerExporter" initializes and opens a UdpClient already in the constructor. This leads, for example, to an immediate crash of an ASP.NET Core application if any form of transient error occurs, e.g. the configured DNS cannot be accessed temporarily.

Unhandled exception. System.Net.Sockets.SocketException (0x00002AF9): No such host is known.
   at System.Net.Dns.GetHostEntryOrAddressesCore(String hostName, Boolean justAddresses, AddressFamily addressFamily, Int64 startingTimestamp)
   at System.Net.Dns.GetHostAddresses(String hostNameOrAddress, AddressFamily family)
   at System.Net.Sockets.UdpClient.Connect(String hostname, Int32 port)
   at OpenTelemetry.Exporter.Jaeger.Implementation.JaegerUdpClient.Connect()
   at OpenTelemetry.Exporter.JaegerExporter..ctor(JaegerExporterOptions options, TProtocolFactory protocolFactory, IJaegerClient client)
   at OpenTelemetry.Exporter.JaegerExporter..ctor(JaegerExporterOptions options)
   at OpenTelemetry.Trace.JaegerExporterHelperExtensions.BuildJaegerExporterProcessor(JaegerExporterOptions options, IServiceProvider serviceProvider)
   at OpenTelemetry.Trace.JaegerExporterHelperExtensions.<>c__DisplayClass2_0.<AddJaegerExporter>b__1(IServiceProvider sp)
   at OpenTelemetry.Trace.TracerProviderBuilderExtensions.<>c__DisplayClass8_0.<AddProcessor>b__0(IServiceProvider sp, TracerProviderBuilder builder)
   at OpenTelemetry.Trace.OpenTelemetryDependencyInjectionTracingServiceCollectionExtensions.ConfigureTracerProviderBuilderCallbackWrapper.ConfigureBuilder(IServiceProvider serviceProvider, TracerProviderBuilder tracerProviderBuilder)
   at OpenTelemetry.Trace.TracerProviderSdk..ctor(IServiceProvider serviceProvider, Boolean ownsServiceProvider)
   at OpenTelemetry.Trace.TracerProviderBuilderBase.<>c.<.ctor>b__3_0(IServiceProvider sp)
   at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteVisitor`2.VisitCallSiteMain(ServiceCallSite callSite, TArgument argument)
   at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteRuntimeResolver.VisitRootCache(ServiceCallSite callSite, RuntimeResolverContext context)
   at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteVisitor`2.VisitCallSite(ServiceCallSite callSite, TArgument argument)
   at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteRuntimeResolver.Resolve(ServiceCallSite callSite, ServiceProviderEngineScope scope)
   at Microsoft.Extensions.DependencyInjection.ServiceProvider.CreateServiceAccessor(Type serviceType)
   at System.Collections.Concurrent.ConcurrentDictionary`2.GetOrAdd(TKey key, Func`2 valueFactory)
   at Microsoft.Extensions.DependencyInjection.ServiceProvider.GetService(Type serviceType, ServiceProviderEngineScope serviceProviderEngineScope)
   at Microsoft.Extensions.DependencyInjection.ServiceProviderServiceExtensions.GetService[T](IServiceProvider provider)
   at OpenTelemetry.Extensions.Hosting.Implementation.TelemetryHostedService.Initialize(IServiceProvider serviceProvider)
   at OpenTelemetry.Extensions.Hosting.Implementation.TelemetryHostedService.StartAsync(CancellationToken cancellationToken)
   at Microsoft.Extensions.Hosting.Internal.Host.StartAsync(CancellationToken cancellationToken)
   at Microsoft.Extensions.Hosting.HostingAbstractionsHostExtensions.RunAsync(IHost host, CancellationToken token)
   at Microsoft.Extensions.Hosting.HostingAbstractionsHostExtensions.RunAsync(IHost host, CancellationToken token)
   at Microsoft.Extensions.Hosting.HostingAbstractionsHostExtensions.Run(IHost host)

Reproduce

1. Add the Jaeger exporter with JaegerExportProtocol.UdpCompactThrift and ensure the given DNS of the AgentHost is not resolvable at the moment.

    builder.Services.AddOpenTelemetry()
      .WithTracing(tracerProviderBuilder =>
          .AddAspNetCoreInstrumentation()
          .AddJaegerExporter(options =>
          {
            options.Protocol = JaegerExportProtocol.UdpCompactThrift;
            options.AgentHost = "some-host-that-does-not-exist";
            options.AgentPort = 6831;
          })
        )
      )

2. Start the web application

Additional Context

I encountered this problem when I started my application with a utility command. This would normally initialize the application just briefly, do a task, and then shut it down again. However, since the application is thus started outside the cluster for which it is intended, the corresponding DNS records under which, for example, Jaeger is resolved, are not available in this environment.

Notes

The responsibility of a constructor is to initialize an object and set its initial state, not to handle network-related functionality, such as resolving domain names. The UdpClient could, for example, be "connected" on the first invocation of the Export method instead.

Workaround

The current work-around is to set the used Protocol to JaegerExportProtocol.HttpBinaryThrift due to the fact that the underlying JaegerHttpClient.Connect() method doesn't do anything and thus letting the service run as expected.

[utpilla]

@Alexanerinova JaegerExporter is scheduled to be open-telemetry/opentelemetry-specification#2858 and we wouldn't be making any fixes to it. Users are advised to move to OTLP Exporter. The getting started with Jaeger tutorial shows how to use OTLP Exporter to export traces to Jaeger.

[AlexanderInova]

@utpilla Ok, thank you.

It might be a good idea to mark the corresponding nuget package OpenTelemetry.Exporter.Jaeger, as well as the AddJaegerExporter(..) extension method, as deprecated, as there seems to be way more documentation and tutorial out there showing how to use this exporter as for the new way. An obvious deprecation warning within the Nuget Manager and/or code itself might help users to immediately see that this should not be used anymore, while also giving existing users a heads up.