Adding new Override's on the fly

[shvez]

Hi, there.
I'm newbie with serilog and it is not clear for me why it is not possible to create new overrides and reload them dynamically. we use this feature of log4net when we are looking for bug reasons or check some new functionality to reduce amount of logging.

is this some architecture limit or just your lib does not implement it because of some consideration?
is the a way to workaround this limit?

[nblumhardt]

Hi! Thanks for dropping by. Back when level overrides were designed, this feature wasn't considered useful enough to warrant the additional complexity that implementing it would cost.

I agree that there are some scenarios in which it would be useful, of course - but dynamic filtering with LoggingFilterSwitch from Serilog.Expressions can be used instead in most of them (esp. when debugging).

You may need to do some more plumbing work to wire it in, however, the basic idea is that in debugging scenarios you set the global minimum level to Verbose or Debug, and then dynamically change the switch to include or exclude events based on @l and SourceContext, e.g.:

@l in ['Fatal', 'Error', 'Warning', 'Information'] or SourceContext like 'Microsoft.AspNetCore.%'

HTH!

[shvez]

@nblumhardt thank you for you answer. I did ask about that in many places but did not get answer. I think it would be really useful to have it somewhere in tutorials, because for people that got used to log4net/nlog this idea is hidden

[bart-vmware]

@nblumhardt Would you be interested in a PR that changes watching individual overrides into one watcher for the set of overrides? The reason I'm asking is I'm investigating how to dynamically change minimum log levels at runtime via configuration, similarly to https://github.com/PawelGerr/Thinktecture.Logging.Configuration#with-serilog, which lists the following limitation:

Limitation: Serilog creates a watcher (i.e. a LoggingLevelSwitch) for configuration keys only that are present when building the logger. For example: if there is no configuration for MinimumLevel:Override:Thinktecture when CreateLogger() is called then you won't be able to change the log level for this category during runtime.

I traced this down to Serilog.Settings.Configuration.ConfigurationReader.SubscribeToLoggingLevelChanges, which I believe the current behavior is originating from.

I'm one of the developers of Steeltoe, a framework that provides an endpoint to dynamically change minimum log levels at runtime. The way it currently works is by replacing ConsoleLogger, but we found it broke several times on new ASP.NET releases. Therefore I'm looking into overriding the levels through configuration instead, which we suspect to be more reliable and increase flexibility for different logging frameworks. My configuration-based prototype works great with the built-in ASP.NET logging, but is problematic when Serilog is used for the reason described above.

The picture below shows the feature in action from a management tool. The logger categories from this minimal sample service already have 140 active categories. In large applications, there are typically thousands of logger categories active, so the expression-based approach does not seem very appealing to me.

[nblumhardt]

Hey @bart-vmware! That's a super-cool level control panel 😎 - thanks for dropping by.

I'm not sure what our chances will be of adding this to Serilog without compromising performance or multicore contention. I haven't looked into it for a while, so you might spot some options I've missed, though. (The problem isn't just a missing watcher in Configuration, but limitations in Serilog on updating existing leveled loggers, IIRC.)

The best way to try moving this forward without burning too much time on it might be to hack together a proof-of-concept approach, and write up what you find in a new issue in serilog/serilog - we can then run through it and try to make a call on whether more effort is likely to yield success. Let me know what you think.

[shvez]

@bart-vmware if you follow an advice please add me to the issue in serilog repo. I'm very interested in results. thank you

[bart-vmware]

@nblumhardt Thanks for the pointers. I've created serilog/serilog#1764 to show my approach.
@shvez I'm unable to tag you in the other repo, so please subscribe to the PR yourself.

[terryaney]

@bart-vmware I've read through your PR thread. Curious as your final comment wasn't totally clear, how are you managing this? Did you give up supporting dynamic updates when serilog is used? Dynamically generate filter statements?

[bart-vmware]

We decided to stick with our original implementation that doesn't use filters. Instead, we capture each time a logger instance is created and keep all their names with levels in an in-memory dictionary.