Enable Access to Dependency Request Objects in DependencyCollector

[michaelwstark]

Opening this item per comments on the pull request -- consolidating discussions referenced items.

#issues:
#820
#747
#587
#897

Please reference the issues and pull request for additional details and\or discussion.

Consumers of the dependency collection package would like to be able to populate additional data in the tracked telemetry through implementing ITelemetryInitializer classes. At present, this is not possible given only the data passed in the call to Initialize() is the dependency telemetry item itself with pre-defined fields populated from within the dependency collector.

The ask is to track key data in the dependency collection and provide access to it as part of the ITelemetry item passed to the initializers.

Here is a sample of intended use -- reporting a debug-id value from an http response:

public class SampleTelemetryInitializer : ITelemetryInitializer
{
    private const string DebugHeaderName = "debug-id";

    public void Initialize(ITelemetry telemetry)
    {
        var dependencyTelemetry = telemetry as DependencyTelemetry;
        if (dependencyTelemetry != null && dependencyTelemetry.TryGetOperationDetail("HttpResponse", out var reponseObject))
        {
            IEnumerable<string> debugData = null;

            var response = reponseObject as HttpResponseMessage;
            if (response?.Headers?.TryGetValues(DebugHeaderName, out debugData) ?? false)
            {
                if (debugData != null)
                {
                    dependencyTelemetry.Properties.Add(DebugHeaderName, string.Join(",", debugData));
                }
            }
        }
    }
}

[christianacca]

I'm trying to use this enhancement but not getting anywhere. Maybe I'm missing some additional configuration?

More information

I've registered a custom telemetry initializer and confirm that it's Initialize method is being called (see code listing). However, the call to dependency.TryGetOperationDetail("HttpResponse", out response) always returns false.

Is there any additional configuration I need to add to my ApplicationInsights.config to enable this enhancement?

Note: I am using version 2.7.2 of the app insight libraries.

Thanks

Code listing

    public class DebugDependencyDataTelemtryInitializer: ITelemetryInitializer
    {
        public void Initialize(ITelemetry telemetry)
        {
            var dependency = telemetry as DependencyTelemetry;
            if (dependency == null) return;

            object response;
            bool found = dependency.TryGetOperationDetail("HttpResponse", out response);
            if (found)
            {
                Console.Out.WriteLine("response = {0}", response);
            }
        }
    }

[pschaeflein]

The issue mentioned by @christianacca is happening to me.

@lmolkova Should this issue be closed? Is there a different issue tracking a but rather than the implementation?

[cijothomas]

@pschaeflein Its possible that Initialize() is called more than once, and in the first call, dependency.TryGetOperationDetail("HttpResponse", out response). this call may not get the response. Initialize() will be called again with the correct response..

Can you check if this is whats happening in your case? We'll investigate more once you share more details, if posible a small repro app.

[pschaeflein]

Yes, Initialize is being called twice, however neither call includes ResponseHeaders. (The rawObjects field always contains only 1 item, the request object.)

I will try to get a repro together. I'm running a bot hosted locally at the moment. (Production will be on Azure App Service.) Below is a list of the NuGet dependencies. Should this environment be supported?

[michaelwstark]

@pschaeflein When you say that it is hosted locally -- what is the hosting process? E.g. IIS, IIS Express or an Azure Emulator of some kind, etc

[pschaeflein]

Sorry, IISExpress/Win10

[cijothomas]

The response headers are read and populated correctly in .NET Framework.
https://github.com/Microsoft/ApplicationInsights-dotnet-server/blob/develop/Src/DependencyCollector/Shared/Implementation/HttpDesktopDiagnosticSourceListener.cs#L86
https://github.com/Microsoft/ApplicationInsights-dotnet-server/blob/develop/Src/DependencyCollector/Shared/Implementation/HttpProcessing.cs#L385

However, I dont see the same for .NET Core.
https://github.com/Microsoft/ApplicationInsights-dotnet-server/blob/develop/Src/DependencyCollector/Shared/HttpCoreDiagnosticSourceListener.cs

@michaelwstark Any chance you can take a look? Was it intentional or just happened to have left out for .net core?

[lmolkova]

@cijothomas

ResponseHeaders are set when response was received, but response object is not available - this is .NET Fx case only.

On .NET Core response object is always available (if response was received). It is set here

• for netcore2.0:
  https://github.com/Microsoft/ApplicationInsights-dotnet-server/blob/develop/Src/DependencyCollector/Shared/HttpCoreDiagnosticSourceListener.cs#L398
• for netcore 1.1 and 1.0
  https://github.com/Microsoft/ApplicationInsights-dotnet-server/blob/develop/Src/DependencyCollector/Shared/HttpCoreDiagnosticSourceListener.cs#L463

@pschaeflein could it be the case that there were no response (DNS issue, timeout exception, etc)?

[michaelwstark]

@pschaeflein My personal experience during development and integration with my services was that I was never able to get anything set when running in a IIS Express host process. Not sure what the key differences are\were (or whether the conclusion is entirely accurate), but the minute I actually deployed it to an development environment running in IIS everything worked as expected. Not ideal. I'll sanity check that this continues to be true for me later today and report what version my services are currently consuming. I did just confirm that all the telemetry is reporting accurately in all of our non-IIS Express environments though (and has been very useful).

@lmolkova \ @cijothomas - Any idea why these values would be null in IIS Express from your domain knowledge?

[lmolkova]

@michaelwstark this is not something I was aware of.

if anyone has or will at some point have a repro, would you mind sending it over to me (email in the profile) or posting it somewhere? I'll check.