new feature: dynamic logging level per file or class

[KjellKod]

Being able to change the logging level that is active but on a file or class (or some other unique identifier)

--- The purpose is to be able to at runtime specify that instances (files, class or similar) can get increased or decreased active logging levels.

[KjellKod]

👍 LogRhythm wants this (but not that much). I have not heard anyone else that demands this.

--> Candidate for closing.

[zelk]

It's really helpful when debugging parts of a codebase so that you can set the general loglevel at info but to debug or verbose on some parts of the code that you are looking closer at. Are there other ways to achieve that?

[KjellKod]

Right now you could do it easily by a custom sink... I guess it would come down to either a (g3sinks) sink solution or an new g3log feature.

I would be happy to juggle ideas with someone. Right now it's still too vague of an idea for me to grasp what's really needed.

Of course pull requests are always welcomed ;)

[zelk]

The best example of a very flexible such system is the log4j. It has been ported to a lot of languages but log4cpp, log4xx and log4plus did not do a great job at finding out how to apply this to C++ IMHO.

I wrote a logging system for C# a few years back and used a system where you would create one static Logger per class. My system called sinks LogWriters and then the setup for the system would allow things like:

myConsoleLogWriter.SetLogLevel(LogLevel.Warning);
myConsoleLogWriter.SetLogLevel("MySystem.MySubSystem.*", LogLevel.Debug);
myConsoleLogWriter.SetLogLevel(typeof(MySpecificClass), LogLevel.Verbose);
mySocketLogWriter.SetLogLevel(LogLevel.Info);

...where "MySystem.MySubSystem.*" matched the fully qualified class name, which in C# includes the namespace + class name.

Obviously, the setup and matching was done during setup and caches of which loglevel should be used for each logger and logwriter was precalculated. I think I used lazy initialization for that but I expect that you might not like that since you focus hard on worst case latency (but done on separate logging thread, it might be fine with lazy init). The system also has support for changing all the configuration live, in which case the precalculated caches that becomes invalid are flushed.

I have not made any efforts in porting my lib to C++ yet.

Tell me if you want the code.

[KjellKod]

Nice!

So in your example was `myConsole' below the static instance?

myConsoleLogWriter.SetLogLevel(LogLevel.Warning);

I think the hard part would be to globally set the log levels per file and/or/ module.

I think for this to be useful there need to be one global API which sets log levels per module.

It's easy to have an object that you can change logging level at. However from a TierIV or debug help perspective I think was is needed is on global API.

Example One.cpp should be on DEBUG level where TWO.cpp we only want to have WARNING. I.e

How should that API look? What should the input arguments be? (File name , log level range)?

[zelk]

"myConsoleLogWriter" in the example is an appender or a sink if you will.

The log system I created is just used in 3-4 systems that I am/was all somewhat part of the development around and I made some design decisions that might not be good for all use cases.

So, I had a LogManager that was not a forced singleton but I had separate class with a static singleton with a default instance and the systems I created only ever used one LogManager through this class.

The LogManager keeps track of:

• all the Loggers (one per class).
• all the LogWriters (think appenders or sinks).

Each LogWriter has a default LogLevel but also a set of LogLevels per Logger and a set of LogLevels per fully qualified class name (string match - like "MySystem.MySubSystem") - works better in Java and C# because name spaces are heavily used and first class citizens of the reflection system.

All or most things lazily initialized in a way that the cached info is quickly accessible when the actual log calls are coming through.

So... in this system, you need to think of Loggers as a per-class instance that does not know anything about which LogWriters they will end up logging to. They log through the LogManager class and the LogManager forwards the call only to the LogWriters with the right severity level for that specific logger.

I have not spent time on how to make this beautifully in C++.

In each class in the user's code, this is what happened:

class MyAwesomeClass
{
    static Logger log_ = L.O.G(typeof(MyAwesomeClass));
    void MyFuction()
    {
        log_.Info("Hello Log!");
    }
}

The "L" was the class with a single instance of the LogManager that I mentioned. The static "O" method retrieved the default LogManager instance (a shorthand for GetDefaultLogManager). The LogManager has a GetLogger for your class type method and a shorthand "G". I thought that it was cute to use:
L.O.G(typeof(myClass))
...to do all the init instead of:
LogSystem.GetDefaultLogManager().GetLogger(typeof(myClass))
Tell me if you want the code.

[KjellKod]

Aha. Now it makes sense.

So the filtering happened in the LogManager then? On a string input that represented the class.

I assume on the inside you then had a map or similar to change logging levels and to do lookup for each incoming call to see if it was OK to send forward (for actual logging)?

The concept is similar to the dynamic logging levels in g3log. It only filters globally on a logging level but the same concept could be used to filter on class as well. The LogMessage struct already contains information about file, function and line.

If you have code to share that doesn't break some license boundary then please do so. I will not borrow any code unless it's public domain already. (Or cleared with the person/company that owns the code).

My email is
Hedstrom at KjellKod dot cc

Sent from my iPhone

On May 27, 2016, at 5:21 AM, zelk notifications@github.com wrote:

"myConsoleLogWriter" in the example is an appender or a sink if you will.

The log system I created is just used in 3-4 systems that I am/was all somewhat part of the development around and I made some design decisions that might not be good for all use cases.

So, I had a LogManager that was not a forced singleton but I had separate class with a static singleton with a default instance and the systems I created only ever used one LogManager through this class.

The LogManager keeps track of:

all the Loggers (one per class - but obviously not per instance of the class).
all the LogWriters (appenders or sinks).
Each LogWriter has a default LogLevel but also a set of LogLevel per Logger and a set of LogLevel per class name (string match - like "MySystem.MySubSystem").

All or most things lazily initialized in a way that the cached info is quickly accessible when the actual log calls are coming through.

So... in this system, you need to think of Loggers as a per-class instance that does not know anything about which LogWriters they will end up logging to. They log through the LogManager class and it forwards the call to the LogWriters with the right severity level for that specific logger.

I have not spent time on how to make this beautifully in C++.

In each class in the user's code, this is what happened:

class MyAwesomeClass
{
static Logger log_ = L.O.G(typeof(MyAwesomeClass));
void MyFuction()
{
log_.Info("Hello Log!");
}
}

The "L" was the class with a single instance of the LogManager that I mentioned. The static "O" method retrieved the default LogManager instance (a shorthand for GetDefaultLogManager). The LogManager has a GetLogger for your class type method and a shorthand "G". I thought that it was cute to use L.O.G(typeof(myClass)) to do all the init instead of LogSystem.GetDefaultLogManager().GetLogger(typeof(myClass)).

Tell me if you want the code.

—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub, or mute the thread.

[mobileben]

Not sure if anything ever came about this. I was curious about any support for level control on the sink level. I know I can add it myself for a custom sink. Not sure if that is the recommended methodology here for that. Given how sinks are pretty "loosely" defined, I would suspect this is the case.

[KjellKod]

Not sure what you mean with that the g3log sinks are loosely defined.
In either case, no, I did not hear back from @zelk.

Unless there are many people that are requesting this then it'll not be implemented by me. Of course, I'll always evaluate pull requests.

[KjellKod]

I.e. right now this feature request is nearing its expiration period where I'll close the request.

[mobileben]

@KjellKod Sorry for the lack of clarity. What I meant was that there is no sub-classing of a Sink base class as opposed to specifying the Sink and the method.

I was also sampling with spdlog and that approach is taken for customizing sinks.

[KjellKod]

If anything subclassing is a restriction. There is no restriction on the sinks used in g3log apart from that there must be a function that receives the log message (or a std string)

I.e

• if you want to subclass - go ahead
• if you want to re-use but through composition (a better choice imo) - go ahead
• if you want to make your own class or struct, or re-use code from a completely different code base as your sink - go ahead

[mobileben]

I wasn't saying this was a bad thing. Just keep in mind that someone new is trying to get their bearings around how to properly use the library. I am also looking at spdlog and easylogging. g3log was my last stop, so coming from the others, I was a bit used to how they were implementing custom sinks.

[KjellKod]

No that's cool. No offense. And I didn't mean any offense either.

Since you are looking into the sinks you might want to take a peek at g3sinks

[KjellKod]

No action going forward towards this feature request. It can be solved with custom sinks or more painfully with another architecture