What's the future of log abstractions like ASP.NET 5 ILogger, LibLog, EventSource, ...

[cwe1ss]

Hi everyone! This is a rather philosophical question but still, it would be great if I could get some opinions from you guys!

I'm currently working on a quite big new "microservices"-like project with many services. Routing all log entries into one central repository is a key to success in my opinion. Therefore I've been reading a lot about this topic lately and unfortunately the more I read the less I like the current logging ecosystem in .NET.

It's quite common nowadays to rely on many different libraries - NuGet makes it incredibly easy to just drop in one library after the other into your project. Before systems like LibLog came up, there were hard dependencies on log4net or others - so LibLog and so on were a great step forward.

However I feel like what was done wrong with the direct depencies to logging frameworks is now just put up one level and repeated for "logger abstractions" or "logging emitters" (in other words: classes like a typical ILogger that actually >create< log entries)

I'm working on a greenfield project but still I already had to deal with the following logging emitters:

• LibLog (in IdentityServer3)
• Castle ILogger (in a library called "asp.net boilerplate")
• Microsoft.Owin.Logging (especially for authentication libraries)
• System.Diagnostics.Trace / TraceSource (Owin logs there by default and it seems to be the preferred way for Azure Web Apps if you want to use some built-in debugging functionality)
• System.Diagnostics.EventSource (we are using Azure Service Fabric to host our services and the .NET framework itself emits useful information through these)
• this new ILogger in Microsoft.Extensions.Logging (I do like the implementation, however I'm not happy with the fact that there is yet another ILogger)

None of these emitters actually store logs somewhere and if you don't configure them they just log into void. I'm having a really hard time to figure out the best way to get all of them into one repository. I'm currently routing most of them through Serilog (by writing a few adapters) but still, it doesn't work well (I don't get some EventSource entries, ...). What also frustrates me about this is the fact that you can't easily reuse this logic throughout your applications since every application depends on different abstractions. This makes it really painful in a microservice environment. I don't want my developers to think about this topic whenever they create a new service.

I'm also frustrated that all of this configuration is just for one technology stack in a very big logging architecture. There are so many other things that have to be aggregated as well. (infrastructure logging like nagios, windows event logs, IIS logs, ...)

So I don't understand why there isn't one good solution in the .NET framework itself for such an important topic. System.Diagnostics.Tracing.EventSource seems to be the most recent solution but it feels like it doesn't get enough attention and the tooling around it isn't very good in my opinion. Its rather static nature also makes it somehow hard to adopt in a system with many components. Much has been improved lately, but there are still many things that are hard to understand. E.g. would the ASP.NET 5 ecosystem have one EventSource per library? ... I still haven't got my head around it completely...

And if there's no sufficient solution in the framework itself, why haven't we as a community managed to reach 1 or 2 de-facto standards within the 15 years of .NET?

What do you think where we should head as a community?

Should we invest more into a built-in system like "EventSource" which seems to be the recommended solution from the .NET team? Its adoption within the framework itself basically makes it a "problem" we have to solve anyway in my opinion.

Should a system like LibLog be able to aggregate some of these other abstractions??

Should we actually go "back" and have hard dependencies on a system like Serilog that can emit and store events?

Or do we need yet another layer which basically does the opposite as the existing systems and acts like an aggregater who can read events from Trace, TraceSource, EventSource, LibLog, Common.Logging, ASP.NET 5 ILogger, ... and offers them in one central stream which then can be consumed by e.g. log4net/serilog - and how would such a system handle DLL dependencies??

And really important - where would be a good place to discuss this issue with the major stakeholders? Because the most important thing in my opinion is to NOT end up with yet another ILogger.
@nblumhardt, @vancem, @davidfowl, @damianh - As owners/main contributors to some of these systems, I hope it's ok to cc you for this question especially. Is this something you think should be discussed in more detail??

Or do you think I'm seeing a problem where there isn't one? :)

I'd really appreciate your feedback!!

[muratg]

#328

[herecydev]

The ILogger provided in this repo is supposed to be a common logging abstraction. What I have done typically is use the abstraction in my libraries everywhere and log all the things. Then in my composition root I bind up where I need that to be. Sometime the console logger feels correct so I can simply use the provider in this repo.
If you need to have something more persistent like you are describing then it should be super simple to create something for it. By creating logging abstractions that are the lowest set of common features, it means the learning barrier is quite low.
It sounds like everything you're asking for can be achieved by writing an adapter that can be reused across your components, if you're looking to OSS it I'm more than happy to help with that also.

[cwe1ss]

thx for your comment, @herecydev ! As long as you only have one (or very few) logging abstractions, everything is ok and it would be exactly as you've described it. This is the scenario I would like to see in the future.

However I'm talking about the fact that there is an increasing amount of logger abstractions and application developers have no choice than to write/find adapters for every library. Of course, this isn't the most complex task but it doesn't add any business value and is lost time IMO.

I feel like there should have been more discussion with the corefx team to consolidate the existing Microsoft solutions. .NET Core would have been a great chance to deprecate/remove old technologies (e.g. Trace.Write) and promote one solution that is used by all .NET stacks. This would have helped to remove the need for 3rd party libraries like LibLog or CommonLogging.

Now one can just hope that this ILogger will be good enough for this purpose and that other library vendors will move to this library when they migrate to .NET Core.

PS: I would like to add a few more examples to my list of current abstractions:

• CoreFX recently added yet another mechanism to System.Diagnostics: DiagnosticSource. As far as I understand, this was introduced for tools like Glimpse. I haven't yet figured out why EventSource hasn't been used for this.
• System.Diagnostics now has Debug.*, Trace.*, TraceSource, EventSource, DiagnosticSource - IMO it is very hard to know when to use which of these.
• MassTransit doesn't use any well-known logging abstraction and uses its own ILog

[herecydev]

There are a few logging abstractions but the idea is to stick with one, as you change the implementations out during the composition root. I like the ILogger here, it fits my purposes and is supported by Microsoft. If it doesn't work to your taste then you can swap it out for another. The beauty of this is that it should feel simple and easy to do so as it barely does anything (encapsulating common functionality that other loggers do).

The things you have listed are good candidates for implementations of ILogger. You're still using your ILogger abstraction in the places where you want to log (you would never call these directly). And then as part of your composition you are directing where you want the flow of logs to go.

For the mass transit situation it's an established framework with lots of users so It's harder to just say swap over. But as far as I can see nobody has suggest it to them yet. Potential problems with including RC into mass transit is the logging has evolved and is still undergoing changes so you risk alienating users. There is a proposal for using serilog so they're obviously considering the use of a "more common" abstraction.

I'm sure we will start seeing more exposure of ILogger out in the wild as it matures into RTM but if you don't suggest it to the library authors don't expect them to go out on whim and implement it. They want to know what the community wants...

[damianh]

Because .net doesn't support structural typing like GO's interfaces and diamond dependencies are problematic I am of the very strong opinion that each independently versioned library that can log stuff, should define it's own ILog abstraction within itself. This keeps things loosely coupled at the cost of the the consumer having to create adapters. This is the premise of LibLog (though it tries to be helpful to by wiring adapters up automatically by reflection, this should only be seen as a bonus).

The integration packages for Common.Logging is a train wreck.

If this issue https://github.com/dotnet/roslyn/issues/7844 is implemented, then what will happen is a series of ILog interfaces will be defined in various places and eventually a few will rise to the top. Library authors can then support these structures without having a dependency. This is the only way to solve this.

I look forward to the day I can kill LibLog.

[cwe1ss]

@damianh thx for sharing your thoughts! May I ask you a few more questions:

• Do you think that Microsoft.Extensions.Logging.Abstractions is the same concept as Common.Logging? Would you also not want to take a dependency on this library as a library developer? (due to the same versioning issues and so on?)
• What do you think about EventSource from the .NET framework? This would allow every library to write its own logging api and be completely independent from other libraries. However as a consumer you could use one technology (e.g EventListener) to subscribe to all interesting libraries.

and thx for pointing out the roslyn link - I didn't know about that!

[phatboyg]

EventSource is a really complicated and continuously broken way to use logging. We tried it when we first moved to Azure Cloud Services, and it's been nothing but pain and suffering. Registration, connecting listeners, all that stuff just overcomplicates what should be easy for developers to understand and use and EventSource/ETW doesn't fall into any "ease of use" curriculum of which I'm aware.

We use the approach of having our own ILog, for internal logging, and then integration with the popular loggers (NLog, log4net, Serilog).

[cwe1ss]

Chris, that's my concern with EventSource as well. The concept sounds good for client applications where you only need it for actual troubleshooting scenarios. But for server-side logging, where you want to send your application logs to some place all the time, it just seems way too complicated. I'm wondering if it's "broken by design" for server applications or if it just doesn't get enough love.

I really appreciate you guys taking the time to discuss this with me. I think this is a very important topic and maybe we find a way to improve it!

For my current project, we decided to use Microsoft.Extensions.Logging.Abstractions for creating events in our libraries. But since there aren't many sinks for it yet, we forward everything to Serilog and will use their sinks to store/display the logs (Although Serilog 2 is also still in beta and many sinks don't support it yet). this logic is in one place though so we can easily switch to e.g. M.E.L providers later.

We will try to write/find adapters for other ILogger (Trace, MassTransit.Logging, ...) and forward those to M.E.L or Serilog.

We will not monitor any .Net EventSource providers for now. However, we will need to find a way to get some logs from Service Fabric, which uses EventSource. I think we will use Azure Diagnostics to store it in Azure Storage and we will try to fetch it from there.

We don't yet have a good solution for performance/metrics - most likely we'll use Stackify. But that's a totally different story.

[avanderhoorn]

@cwe1ss I sympathize with your struggle and wish I had better answers for you. I think some of the problem you are describing is there because we didn't had a common logging abstraction from the beginning and others because one logging abstraction doesn't rule them all.

History
As you have observed, prior to Core, no logging abstraction has really taken off. I think part of this reason was that there were multiple options (with no clear winner) and for the average consumer there wasn't ever really a clear benefit for using such an abstraction. Additionally, prior to Nuget, managing dependencies wasn't all that much fun, so framework authors tended to internalize their logging dependencies. This is my no means an exhaustive history but when I think about the "how did we get here" they seem like the high points.

Currently
With the arrival of Core, we get the opportunity to provide a common logging abstraction from the beginning. This will hopefully help improve things for anything on Core or which chooses to take a dependency on it. To your point, this doesn't solve the problem of having Microsoft.Extensions.Logging vs EventSource/ETW vs DiagnosticSource. Having Microsoft.Extensions.Logging as part of Core, shows the commitment to how important getting this abstraction right is.

EventSource/ETW
Here we need to start thinking about the various needs of logging in different scenarios. Specifically, EventSource/ETW are (if you get to know them) amazingly powerful logging tools. It goes all the way through your application, through to the very depths of the underlying machine. The shear volume of information you can get access to is massive and it is designed to consumed out of process at high volume.

EventSource/ETW becomes even more powerful when augmented with data from the frameworks you use and your application. In a core world it is intended that if you need to consume EventSource/ETW level data, you setup a Microsoft.Extensions.Logging sink to push to EventSource/ETW. Here you will all the information you could ever want in one place.

You might be asking why not just settle on using EventSource/ETW instead of Microsoft.Extensions.Logging. The problem here is that EventSource/ETW has been around for a very long time and updating programming models/interfaces here to match what we want from a modern logging platform isn't really that viable. Hence, you could look at Microsoft.Extensions.Logging existence as being this update, but done in such a way that it doesn't force you to use EventSource/ETW if you didn't want to.

DiagnosticSource
DiagnosticSource is designed to complement complement what we think of as logging, but is in fact solving a very different problem. Most logging platforms (Microsoft.Extensions.Logging and EventSource/ETW included) are geared towards human readable statements and for data that can efficiently/reliably sent out of process (i.e. to a log file).

Traditionally this has been fine, but with the advent of newer diagnostics tools (like Glimpse) and richer requirements from others (like Visual Studios various diagnostics tools), getting access to non string data and actual rich objects has become a must. Hence DiagnosticSource was created for consumers who exist in process, are interested in distinct events (i.e. begin/end request, begin/end action, etc) and want access rich objects (i.e. the actual HttpContext object, etc).

This requirement can't simply be filed by Microsoft.Extensions.Logging and EventSource/ETW and is really trying to enable a while new class of scenarios. If you are interested in knowing more about DiagnosticSource, I'm happy to fill you in more.

Framework Authors
With all the choices that are at play, framework/library authors have a lot of things to consider when choosing which logging system you will pick. If you are simply targeting core, then you will most likely use Microsoft.Extensions.Logging. But If you are trying to target legacy consumers as well, then it becomes harder.

Some who have ties to the operating system or azure infrastructure will choose to work with EventSource/ETW and others who are targeting OWIN based infrastructure might go with Microsoft.Owin.Logging. This all leads to the onus being on the consumer to decide which sink they want to ultimately push their data to.

I hope this helps to provide a bit more context to struggles you (and other in situations like yours) are having. We do hear you and we are trying to improve things for the future, but even though we whole platform is being reset, logging is one of those things that is hard to completely reset when you start to introduce legacy components or the underlying hosting environment/infrastructure.

[damianh]

@cwe1ss

Do you think that Microsoft.Extensions.Logging.Abstractions is the same concept as Common.Logging? Would you also not want to take a dependency on this library as a library developer? (due to the same versioning issues and so on?)

Yes, I hold that opinion currently. Strong naming is still a problem in .net, though I'm not quite sure how less so in CoreCLR.

What do you think about EventSource from the .NET framework?

I hold the same opinion as @phatboyg on this.

[cwe1ss]

Thank you for this detailed post, Anthony! It's great to get some background on DiagnosticSource - I haven't found much about it yet. Do you know of any other detailed posts about it?

I read a lot about EventSource lately and I agree that it is a very powerful tool. I really like the concept and the fact that it is used by the framework itself. It would be a great place to get everything you need. But as you've said Microsoft.Extensions.Logging seems to be the easier API to use. That's why we now decided to use this in our code to create log events.

From a conceptual point, I keep thinking that forwarding M.E.L to EventSource would be the right thing. (I even wrote a forwarder, in case someone needs it: #328).
There's just not much stuff around to handle EventSource in a good way on the server. "Semantic Logging" seems to be the only serious solution, but it doesn't support the new EventSource features which are a must-have IMO (they promised to share their plans regarding this within the next weeks).

I'd be willing to help but I don't feel like there will be many other interested people. So there won't be a big community and there won't be many sinks I guess (which of course isn't that important because writing sinks is quite easy).
Do you see any interest or upcoming improvements for this within Microsoft?

My goal now is to get as much stuff as possible into M.E.L (our own logs, 3rd party ILogger) and to keep the code which actually configures the sinks in one library that's used by all of our applications. This way it should be fairly easy to switch from Serilog to EventSource.