Skip to content
This repository has been archived by the owner on Feb 12, 2021. It is now read-only.

Home

Jump to bottom
Ruslan Hamzatov edited this page May 3, 2019 · 24 revisions

!! Correlation Tracking does not work with an AsyncAppender by default !!

Project Structure

Correlation projects

  • Correlation.Core contains the CorrelationManager and CorrelationScope classes to assign and manage business flow identifiers (sets of related correlation IDs, so called a "correlation scope").
  • Correlation.IIS contains IIS HttpModule to restore or create a new correlation scope in any ASP.NET based application (MVC, Web API, Web Forms, etc.).
  • Correlation.WCF contains WCF Service and Client behaviors to restore or create a new correlation scope for both WCF services and WCF clients.
  • Correlation.Http contains HTTP DelegatingHandler to include the correlation identifier into the HTTP request.
  • Correlation.MassTransit contains: ** MassTransit Consume observer to restore or create a new correlation scope when consuming a message; ** MassTransit Publish observer to include the correlation identifier into the MassTransit message.

Tracing projects (Logging)

  • Tracing.IIS contains an IIS HttpModule for logging a request and its corresponding response (using Log4net).
  • Tracing.WCF contains WCF Service and Client behaviors for logging incoming and outgoing messages and exceptions.
  • Tracing.Http contains HTTP DelegatingHandler for logging an outgoing HTTP request and its corresponding response.
  • Tracing.MassTransit contains MassTransit Consume and Publish observers for logging incoming and outgoing RabbitMQ messages.

Linking projects

  • CorrelationTracking (depends on Correlation.Core, Correlation.Log4net) hooks up log4net for logging.
  • Correlation.Log4net contains a correlation scope interceptor to pass correlation IDs to Kibana through log4net logical thread properties.

Configuration

Common

nuget packages

PM> Install-Package Albumprinter.CorrelationTracking

Program.cs

using Albumprinter.CorrelationTracking;
using log4net.Config;

public class Program
{
	static void Main(string[] args)
	{
		XmlConfigurator.Configure();
		CorrelationTrackingConfiguration.Initialize();
	}
}

log4net.config

<?xml version="1.0" encoding="utf-8"?>
<configuration>

  <configSections>
    <section name="log4net" type="log4net.Config.Log4NetConfigurationSectionHandler, log4net" />
  </configSections>

  <log4net>
    <appender name="RemoteSyslogAppender" type="log4net.Appender.RemoteSyslogAppender">
      <layout type="log4net.Layout.SerializedLayout, log4net.Ext.Json">
        <decorator type="log4net.Layout.Decorators.StandardTypesDecorator, log4net.Ext.Json" />
        <default />
        <member value="ProcessId|%property{X-ProcessId}" />
        <member value="CorrelationId|%property{X-CorrelationId}" />
        <member value="RequestId|%property{X-RequestId}" />
      </layout>
      <!-- for IPv4 machines defaults to: 
      <remoteAddress value="127.0.0.1" />
      <remotePort value="514" />
      -->
    </appender>
    <root>
      <appender-ref ref="RemoteSyslogAppender" />
    </root>
  </log4net>

</configuration>

Dotnet Core 2.0 Lambda Api with Serilog

nuget package

PM> Install-Package Albumprinter.CorrelationTracking.Correlation.Core
PM> Install-Package Albumprinter.CorrelationTracking.Correlation.Serilog

LocalEntryPoint.cs

using Albelli.Customer.Api.Configurations;
using JetBrains.Annotations;
using Microsoft.AspNetCore;
using Microsoft.AspNetCore.Hosting;

namespace Albelli.Customer.Api
{
    /// <summary>
    ///     The Main function can be used to run the ASP.NET Core application locally using the Kestrel webserver.
    /// </summary>
    [UsedImplicitly]
    public class LocalEntryPoint
    {
        public static void Main(string[] args)
        {
            BuildWebHost(args).Run();
        }

        public static IWebHost BuildWebHost(string[] args) =>
            WebHost.CreateDefaultBuilder(args)
            .ConfigureAppConfig()
            .ConfigureLocalLogging()
            .UseStartup<Startup>()
            .Build();
    }
}

LambdaEntryPoint.cs

using Albelli.Customer.Api.Configurations;
using Amazon.Lambda.AspNetCoreServer;
using JetBrains.Annotations;
using Microsoft.AspNetCore.Hosting;

namespace Albelli.Customer.Api
{
    /// <inheritdoc />
    /// <summary>
    ///     This class extends from APIGatewayProxyFunction which contains the method FunctionHandlerAsync which is the
    ///     actual Lambda function entry point. The Lambda handler field should be set to
    ///     Albelli.Customer.Api::Albelli.Customer.Api.LambdaEntryPoint::FunctionHandlerAsync
    /// </summary>
    [UsedImplicitly]
    public class LambdaEntryPoint : APIGatewayProxyFunction
    {
        protected override void Init(IWebHostBuilder builder)
        {
            builder.ConfigureAppConfig().ConfigureLambdaLogging().UseStartup<Startup>();
        }
    }
}

LoggingConfiguration.cs

using Albumprinter.CorrelationTracking.Correlation.Serilog;
using Microsoft.AspNetCore.Hosting;
using Microsoft.Extensions.Logging;
using Serilog;
using Serilog.Sinks.SystemConsole.Themes;

namespace Albelli.Customer.Api.Configurations
{
    internal static class LoggingConfiguration
    {
        internal static IWebHostBuilder ConfigureLocalLogging(this IWebHostBuilder hostBuilder) =>
            hostBuilder.ConfigureLogging(
                (hostingContext, logging) =>
                {
                    var logger = new LoggerConfiguration()
                                 .Enrich.FromLogContext()
                                 .WriteTo.Console(
                                     outputTemplate: "{Timestamp:yyyy-MM-dd HH:mm:ss.fff zzz} [{SourceContext}] [{Level}] [{Properties:j}]{Message}{NewLine}{Exception}",
                                     theme: AnsiConsoleTheme.Code)
                                 .AddCorrelationProperties()
                                 .CreateLogger();

                    Log.Logger = logger;

                    logging.ClearProviders();
                    logging.AddSerilog(logger);
                });

        internal static IWebHostBuilder ConfigureLambdaLogging(this IWebHostBuilder hostBuilder) =>
            hostBuilder.ConfigureLogging(
                (hostingContext, logging) =>
                {
                    var logger = new LoggerConfiguration()
                                 .ReadFrom.Configuration(hostingContext.Configuration)
                                 .AddCorrelationProperties()
                                 .CreateLogger();

                    Log.Logger = logger;

                    logging.ClearProviders();
                    logging.AddSerilog(logger);
                });
    }
}

app.settings

...
  "Serilog": {
    "MinimumLevel": {
      "Default": "Debug"
    },
    "Enrich": [ "FromLogContext" ],
    "WriteTo": [
      {
        "Name": "SumoLogic",
        "Args": {
          "endpointUrl": "https://endpoint1.collection.eu.sumologic.com/****",
          "sourceName": "CustomerApi",
          "sourceCategory": "AP/Local/CustomerApi",
          "textFormatter": "Serilog.Formatting.Json.JsonFormatter, Serilog",
          "outputTemplate": "{Timestamp:yyyy-MM-dd HH:mm:ss.fff zzz} [{SourceContext}] [{Level}] [{Properties:j}]{Message}{NewLine}{Exception}"
        }
      }
    ]
  }
...

Startup.cs

using Serilog;

namespace Albelli.Customer.Api
{
    [UsedImplicitly]
    public class Startup
    {
        public Startup(IConfiguration config)
        {
            Configuration = config;
        }

        private IConfiguration Configuration { get; }

        public void ConfigureServices(IServiceCollection services)
        {
            ...
        }

        public void Configure(IApplicationBuilder app, IApplicationLifetime appLifetime)
        {
			...
            app.UseMiddleware<CorrelationTrackingMiddleware>();

        }
    }
}

CorrelationTrackingMiddleware.cs stored in Albelli.Correlation.Http.Server

Dotnet Core 2.0 Lambda Function SNSEvent with Serilog

nuget package

PM> Install-Package Albumprinter.CorrelationTracking.Correlation.AmazonSns.Lambda
PM> Install-Package Albumprinter.CorrelationTracking.Correlation.Serilog

Handler.cs

using Albumprinter.CorrelationTracking.Correlation.AmazonSns.Lambda;
using Albumprinter.CorrelationTracking.Correlation.Serilog;
using Amazon.Lambda.Core;
using Amazon.Lambda.SNSEvents;
using Serilog;

// Assembly attribute to enable the Lambda function's JSON input to be converted into a .NET class.
[assembly: LambdaSerializer(typeof(JsonSerializer))]

// Lambda entry points can only be 128 characters long
// ReSharper disable once CheckNamespace
namespace Lambda
{
    public class Handler
    {

        private static ServiceProvider ServiceProvider { get; set; }

        public async Task Handle(SNSEvent snsEvent)
        {
			...
            var record = snsEvent.Records.Single();

            using (record.TrackCorrelationId())
            {
                var address = JsonConvert.DeserializeObject<Address>(record.Sns.Message);
                await addressAddedEventProcessor.Process(address);
            }
        }

        

        private static void ConfigureServices(IServiceCollection services)
        {
           ...
            // Add logging
            Log.Logger = new LoggerConfiguration()
                         .ReadFrom.Configuration(configuration)
                         .AddCorrelationProperties()
                         .CreateLogger();
            services.AddSingleton(new LoggerFactory().AddSerilog());
            services.AddLogging();
		...
        }

    }
}

app.settings

...
  "Serilog": {
    "MinimumLevel": {
      "Default": "Debug"
    },
    "Enrich": [ "FromLogContext" ],
    "WriteTo": [
      {
        "Name": "SumoLogic",
        "Args": {
          "endpointUrl": "https://endpoint1.collection.eu.sumologic.com/****",
          "sourceName": "CustomerApi",
          "sourceCategory": "AP/Local/CustomerApi",
          "textFormatter": "Serilog.Formatting.Json.JsonFormatter, Serilog",
          "outputTemplate": "{Timestamp:yyyy-MM-dd HH:mm:ss.fff zzz} [{SourceContext}] [{Level}] [{Properties:j}]{Message}{NewLine}{Exception}"
        }
      }
    ]
  }
...

IIS

nuget package

PM> Install-Package Albumprinter.CorrelationTracking.IIS
depends on
PM> Install-Package Albumprinter.CorrelationTracking
PM> Install-Package Albumprinter.CorrelationTracking.Correlation.IIS
PM> Install-Package Albumprinter.CorrelationTracking.Tracing.IIS

global.asax

using Albumprinter.CorrelationTracking;
using log4net.Config;

public class Global: HttpApplication
{
	protected void Application_Start()
	{
		XmlConfigurator.Configure();
		CorrelationTrackingConfiguration.Initialize();
	}
}

web.config

<?xml version="1.0" encoding="utf-8"?>
<configuration>
  <appSettings>
    <!-- OPTIONAL -->
    <add key="Log4NetHttpModule:DeniedUrls" value="^[^:]+$" />
    <add key="Log4NetHttpModule:AllowedUrls" value="/(api|v\\d+)/|\\.(asmx|svc)(\\?|$)" />
    <add key="Log4NetHttpModule:AllowedHeaders" value="" />
    <!-- /OPTIONAL -->
  </appSettings>
  <system.web>
    <httpModules>
      <add name="CorrelationHttpModule" type="Albumprinter.CorrelationTracking.Correlation.IIS.CorrelationHttpModule, Albumprinter.CorrelationTracking.Correlation.IIS" />
      <add name="Log4NetHttpModule" type="Albumprinter.CorrelationTracking.Tracing.IIS.Log4NetHttpModule, Albumprinter.CorrelationTracking.Tracing.IIS" />
    </httpModules>
  </system.web>
  <system.webServer>
    <validation validateIntegratedModeConfiguration="false" />
    <modules runAllManagedModulesForAllRequests="true">
      <remove name="CorrelationHttpModule" />
      <add name="CorrelationHttpModule" type="Albumprinter.CorrelationTracking.Correlation.IIS.CorrelationHttpModule, Albumprinter.CorrelationTracking.Correlation.IIS" />
      <remove name="Log4NetHttpModule" />
      <add name="Log4NetHttpModule" type="Albumprinter.CorrelationTracking.Tracing.IIS.Log4NetHttpModule, Albumprinter.CorrelationTracking.Tracing.IIS" />
    </modules>
  </system.webServer>
</configuration>

HttpClient

nuget package

PM> Install-Package Albumprinter.CorrelationTracking.Http
depends on
PM> Install-Package Albumprinter.CorrelationTracking
PM> Install-Package Albumprinter.CorrelationTracking.Correlation.Http
PM> Install-Package Albumprinter.CorrelationTracking.Tracing.Http
CSharp
using System.Net.Http;
using Albumprinter.CorrelationTracking;
using Albumprinter.CorrelationTracking.Http;
using Albumprinter.CorrelationTracking.Correlation.Http;
using Albumprinter.CorrelationTracking.Tracing.Http;
using log4net.Config;

namespace Sample
{
    class Program
    {
        static void Main(string[] args)
        {
            XmlConfigurator.Configure();
            CorrelationTrackingConfiguration.Initialize();

            HttpClient client = new HttpClient().UseCorrelationTracking();
            // OR
            HttpClient client = HttpClientFactory.Create(
                new CorrelationDelegatingHandler(),
                new Log4NetDelegatingHandler(true));
        }
    }
}

RestClient

Please put the following code in your application in order to resolve the problem with different versions of RestSharp:

internal static class CorrelationTrackingRestExtensions
{
	public static RestClient UseCorrelationTracking(this RestClient client)
	{
		client.Authenticator = new CorrelationTrackingInterceptor(client.Authenticator);
		return client;
	}

	public static TRequest UseCorrelationTracking<TRequest>(this TRequest request) where TRequest : IRestRequest
	{
		dynamic correlationScope = CallContext.LogicalGetData(@"Albumprinter.CorrelationTracking.Correlation.Core.CorrelationScope");
		var correlationId = correlationScope?.CorrelationId as Guid? ?? Guid.NewGuid();
		request.AddHeader(@"X-CorrelationId", correlationId.ToString());
		return request;
	}

	private sealed class CorrelationTrackingInterceptor : IAuthenticator
	{
		private readonly IAuthenticator authenticator;

		public CorrelationTrackingInterceptor(IAuthenticator authenticator = null)
		{
			this.authenticator = authenticator;
		}

		public void Authenticate(IRestClient client, IRestRequest request)
		{
			request.UseCorrelationTracking();
			authenticator?.Authenticate(client, request);
		}
	}
}
CSharp
using System;
using System.Runtime.Remoting.Messaging;
using RestSharp;
using RestSharp.Authenticators;
using log4net.Config;
using Albumprinter.CorrelationTracking;

namespace Sample
{
    class Program
    {
        static void Main(string[] args)
        {
            XmlConfigurator.Configure();
            CorrelationTrackingConfiguration.Initialize();

            IRestClient client = new RestClient().UseCorrelationTracking();
        }
    }
}

MassTransit

nuget package

PM> Install-Package Albumprinter.CorrelationTracking.MassTransit
depends on
PM> Install-Package Albumprinter.CorrelationTracking
PM> Install-Package Albumprinter.CorrelationTracking.Correlation.MassTransit
PM> Install-Package Albumprinter.CorrelationTracking.Tracing.MassTransit

MassTransit

using Albumprinter.CorrelationTracking;
using Albumprinter.CorrelationTracking.MassTransit;
using Albumprinter.CorrelationTracking.Correlation.MassTransit;
using Albumprinter.CorrelationTracking.Tracing.MassTransit;
using log4net.Config;
using MassTransit;

namespace Sample
{
    class Program
    {
        static void Main(string[] args)
        {
            XmlConfigurator.Configure();
            CorrelationTrackingConfiguration.Initialize();

            var bus = Bus.Factory.CreateUsingRabbitMq(sbc => { });

            bus.UseCorrelationTracking();
            // OR
            bus.UseCorrelationObserver();
            bus.UseLog4NetObserver();

            bus.Start();
        }
    }
}

Albumprinter.Messaging (with MassTransit)

nuget package

PM> Install-Package Albumprinter.Messaging.MassTransit

CSharp

using Albumprinter.CorrelationTracking;
using Albumprinter.CorrelationTracking.Correlation.MassTransit;
using Albumprinter.CorrelationTracking.MassTransit;
using Albumprinter.CorrelationTracking.Tracing.MassTransit;
using Albumprinter.Messaging.MassTransit;
using log4net.Config;

namespace Sample
{
    class Program
    {
        static void Main(string[] args)
        {
            XmlConfigurator.Configure();
            CorrelationTrackingConfiguration.Initialize();

            var rabbitMqMassTransitFactoryConfig = RabbitMqMassTransitFactoryConfig.FromConfig();
            var rabbitMqMassTransitFactory = new RabbitMqMassTransitFactory(rabbitMqMassTransitFactoryConfig);
            rabbitMqMassTransitFactory.BusCreated += (sender, bus) =>
            {
                bus.UseCorrelationTracking();
                // OR
                bus.UseCorrelationObserver();
                bus.UseLog4NetObserver();
            };

            var messageBus = new MassTransitMessageBus(rabbitMqMassTransitFactory);
        }
    }
}

TopShelf (with MassTransit)

Clone this wiki locally