log/slog: enable setting level on default log.Logger

[zbrogz]

Edit: Official proposal in this comment

When slog.SetDefault() is run, it sets the log package's default Logger's output as well, configured to log at LevelInfo. While this is probably a good default behavior, having a way to configure the level would be useful (such as LevelError), especially considering that log.Printf() by default logs to stderr, not stdout.

While slog.NewLogLogger() exists, it returns a *log.Logger, which cannot be used to update the default logger of the log package, as far as I can tell. (log.SetOuput(w io.Writer) exists, but log.SetDefault(*log.Logger) does not).

I propose exporting handleWriter to make this use case easier:

package main

import (
	"log"
	"log/slog"
	"os"
)

func main() {
	l := slog.New(slog.NewJSONHandler(os.Stdout, &slog.HandlerOptions{
		Level: slog.LevelInfo,
	}))

	slog.SetDefault(l) // configures log package to print with LevelInfo
	
	capturePC := log.Flags()&(log.Lshortfile|log.Llongfile) != 0
	log.SetOutput(&slog.HandlerWriter{l.Handler(), slog.LevelError, capturePC}) // configures log package to print with LevelError
}

Currently I can accomplish this by copying the handleWriter code, but it's a bit verbose in my opinion:

package main

import (
	"context"
	"log"
	"log/slog"
	"os"
	"runtime"
	"time"
)

// handlerWriter is an io.Writer that calls an  slog.Handler.
// It is used to link the default log.Logger to the default slog.Logger.
type handlerWriter struct {
	h         slog.Handler
	level     slog.Level
	capturePC bool
}

func (w *handlerWriter) Write(buf []byte) (int, error) {
	if !w.h.Enabled(context.Background(), w.level) {
		return 0, nil
	}
	var pc uintptr
	if w.capturePC {
		// skip [runtime.Callers, w.Write, Logger.Output, log.Print]
		var pcs [1]uintptr
		runtime.Callers(4, pcs[:])
		pc = pcs[0]
	}

	// Remove final newline.
	origLen := len(buf) // Report that the entire buf was written.
	if len(buf) > 0 && buf[len(buf)-1] == '\n' {
		buf = buf[:len(buf)-1]
	}
	r := slog.NewRecord(time.Now(), w.level, string(buf), pc)
	return origLen, w.h.Handle(context.Background(), r)
}

func main() {
	l := slog.New(slog.NewJSONHandler(os.Stdout, &slog.HandlerOptions{
		Level: slog.LevelInfo,
	}))

	slog.SetDefault(l) // configures log package to print with LevelInfo

	capturePC := log.Flags()&(log.Lshortfile|log.Llongfile) != 0
	log.SetOutput(&handlerWriter{l.Handler(), slog.LevelError, capturePC}) // configures log package to print with LevelError
}

A couple more alternatives I've thought of:

• slog.SetDefaultLogLogger(l, slog.LevelError), possibly confusing naming, but more concise than using HandlerWriter, not requiring computation of capturePC.
• Using slog.NewLogLogger(), explained above.
• slog.SetDefault() updated to use l.Handler().Level() rather LevelInfo. Would be a breaking change.
• slog.SetDefaultLogLoggerLevel(slog.LevelError), noop if slog.SetDefault() has not been called yet?

[ianlancetaylor]

CC @jba

[AndrewHarrisSPU]

Here's something (gnarly): https://go.dev/play/p/h29CkeqMUOt

The idea is to pass in HandlerOptions and return a modified version that employs some subterfuge:

• The emitted level is changed by the ReplaceAttr mechanism.
• Since this is not really changing a Record.Level until after Enabled, the solution has to lie to Enabled. It uses a custom slog.Leveler, remapping the boolean result of the expected comparison back to the levels slog.LevelInfo or slog.Levelnfo+1. As a result, the same boolean result is generated in the log handler's Enabled calls.

This doesn't preserve the log package's formatting, but does allow log to log at a different level.

[jba]

What if the default handler used a LevelVar that was exported as a global?

package slog

var DefaultLevel LevelVar

Would that work?

[AndrewHarrisSPU]

Would that work?

I think so, it would work like this?

Invoking slog.SetDefault:

	// must configure both slog HandlerOptions and the log package to emit source location
	h := slog.NewJSONHandler(w, &slog.HandlerOptions{
		AddSource: true,
	})
	log.SetFlags(log.Llongfile)

	// configure log package calls to emit at ERROR
	slog.DefaultLevel.Set(slog.LevelError)

	// install the new Handler
	slog.SetDefault(slog.New(h))

Without invoking slog.SetDefault:

	// configure just the log package to emit source location
	log.SetFlags(log.Llongfile)

	// configure log package calls to emit at ERROR
	slog.DefaultLevel.Set(slog.LevelError)

	// configure `log` output writer
	log.SetOutput(w)

I wonder if an example including log.SetFlags and log.SetOutput in slog docs would be helpful, just to mention where the log package needs some setup in these cases.

[panjf2000]

From my point of view, I'd suggest adding a new API like func SetDefaultWithLevel(*Logger, Level) for slog which will do the exact same thing as slog.SetDefault does, except that it uses the given level instead of LevelInfo for log package because setting the logging level for the log's default Logger ought to be a disposable operation along with setting the output in most cases, thus they are best done in conjunction, it doesn't make much sense to split it into two steps.

Neither a global variable DefaultLevel nor a new API like slog.SetDefaultLogLoggerLevel is a good choice for this scenario, these alternatives will enable developers to update the default level at will even after only one call to slog.SetDefault, from which we benefit nothing other than a variable being constantly changed. What's worse is that it may confuse users when the value of DefaultLevel is retrieved in one place and it doesn't match the behavior of the default logger (or it's not the value that was set initially) because it might get updated elsewhere.

[gopherbot]

Change https://go.dev/cl/525096 mentions this issue: log/slog: add SetDefaultWithLevel to enable setting level on the default logger in log

[panjf2000]

I've drafted a CL for this, check out if it's OK?

[panjf2000]

Holding the CL until this proposal gets accepted.

[zbrogz]

What if the default handler used a LevelVar that was exported as a global?

I think that would work, but a better name might be DefaultLogLoggerLevel, since DefaultLevel may imply that calling NewJSONHandler() will give a handler that uses that same level as the default. Another name could be DefaultHandlerLevel, though presumably it would be used outside the default handler as well, such as when SetDefault() is called.

Examples:

1. Setting level on default log logger while using non-default handler

l := slog.New(slog.NewJSONHandler(os.Stdout, &slog.HandlerOptions{
	Level: slog.LevelInfo,
}))

slog.DefaultLogLoggerLevel.Set(slog.LevelError)
slog.SetDefault(l)

2. Setting level on default log logger while using default handler

slog.DefaultLogLoggerLevel.Set(slog.LevelError) // Would require that `handlerWriter.level` is a `Leveler` rather than `Level` for this to take any effect without calling `slog.SetDefault()`

// slog.SetDefault(slog.Default()) // deadlock?

Edit: On second thought, I think my second example is conflating setting the default slog handler level with the log logger's level.

2. Setting level on default handler level

slog.DefaultLogLoggerLevel.Set(slog.LevelError)

This indicates to me that both defaultHandler.Enabled() and SetDefault should use DefaultLogLoggerLevel.Level() instead of LevelInfo

[zbrogz]

From my point of view, I'd suggest adding a new API like func SetDefaultWithLevel(*Logger, Level)

This would work for my use case, but I don't believe it would solve setting the default handler's level, which I think makes sense to solve with a shared solution since they are related. I also think it might be a bit confusing that the Level passed in is independent from the Logger since it may have its own Level (via .Handler().Enabled()).

[AndrewHarrisSPU]

Edit: On second thought, I think my second example is conflating setting the default slog handler level with the log logger's level.

I share this sentiment - "Default" might be confusing here. SetDefault isn't really specific to the bridge between log and slog packages, it's global slog state. The bridge is maybe a set of global state from both packages (configured by slog.SetDefault, log.SetFlags, log.SetOutput). The level that log emits at is irregularly placed - it only matters to log but is configured in slog. Would names like LogBridgeLevel or SetLogBridge(*Logger, Level) be clearer?

I agree with every reason for Set...(*Logger, Level), but not about the problems with a global variable. I think the issues arise from global state, not from having a package identifier for it - Set...(*Logger, Level) could be used to create the same range of problems.

[tiaguinho]

Since level is an attribute of HandlerOptions, I think we could create a SetLevel method on Logger struct that change the value on HandlerOptions.
This way, it doesn't matter if Logger.handler is of type defaultHandler, JSONHandler or TextHandler.

[AndrewHarrisSPU]

@tiaguinho
‘Handler.Enabled’ reports the comparison of two input levels, one associated with the Handler, and the other associated with a single logging event. HandlerOptions provides a way to configure one input, but the other is currently fixed at INFO when the ‘log’ package makes logging calls in the bridged setup.

[tiaguinho]

@AndrewHarrisSPU this is not fixed in TextHandler and JSONHandler. Both call h.commonHandler.enabled(level), which is this.

func (h *commonHandler) enabled(l Level) bool {
	minLevel := LevelInfo
	if h.opts.Level != nil {
		minLevel = h.opts.Level.Level()
	}
	return l >= minLevel
}

The problem is with defaultHandler, where the implementation of Enabled is this:

func (*defaultHandler) Enabled(_ context.Context, l Level) bool {
	return l >= LevelInfo
}

If this is changed to the same implementation in TextHandler and JSONHandler, having a SetLevel method on Logger struct to change the level on the HandlerOptions will work.
Also, inside the SetDefault function, we could change the fixed LevelInfo to use the level configured at the handler.

[AndrewHarrisSPU]

In both implementations the comparison is eventLevel >= handlerLevel. It's true that the defaultHandler doesn't have a dynamic handlerLevel, it just emits at or above LevelInfo, but that's a different point than is raised in this proposal. This proposal notes that the eventLevel is fixed in handlerWriter for any event generated by log:

	log.SetOutput(&handlerWriter{l.Handler(), LevelInfo, capturePC})

So there's no way with any honest handler implementation to adjust the level of stuff coming from log.

[AndrewHarrisSPU]

Just an opinion from a slog superfan:

• The use case for this proposal seems narrow and well-defined - for a partial or incremental migration from log to slog, it'd be useful have a way to modulate log to emit at a high or low verbosity.

• The use case for a default, global programLevel (to borrow a term from the doc example) was tossed around a few times in the (epic) discussions about slog. Consistently slog has held to the principle of having a default but configurable Logger, but not a global programLevel. Allowing anyone holding the default Logger to modulate the handlerLevel would be like having a globally mutable programLevel.

To draw a contrast, I think this proposal identifies something that is currently static and unavailable to configure. Offering a solution maybe shouldn't be read as endorsing global state, it's just that any solution must be global.

[jba]

I think we could create a SetLevel method on Logger struct

@tiaguinho Two principles of slog's design are:

1. Loggers don't have levels. In fact, A Logger is just a wrapper around a Handler.
2. Handlers don't have levels either. That is, there is no SetLevel method on the Handler interface. Instead, there is a more general Enabled method, which may be implemented by checking against a minimum level, but may not. The built-in handlers happen to have a minimum level, configured in HandlerOptions, but another handler may not.

Although a lot of people have asked for these, it seems that the current design works well, so we don't have plans to change it.

[jba]

Thanks to all the participants here for thinking a lot of this through. The situation is confusing, because there are two directions to the slog-log "bridge," as @AndrewHarrisSPU calls it.

1. Before your application calls slog.SetDefault (or if it never does), calls to slog.Info and the like go through the default log.Logger. This is supposed to be a temporary state of affairs: you're just starting to use slog and you haven't gotten around to picking a handler yet. The output you get will not be a list of key-value pairs. It isn't even feasible to parse it, because the message isn't quoted so there's no solid way of separating it from the key-value pairs that follow. This is the case that uses the "default handler," which is implemented by an unexported type. The default handler is like the other two built-in handlers in that it has a minimum level, which is hard-coded to be Info. (The name "default handler" is a bad one; it is only the initial default handler, and is replaced when SetDefault is called.)

2. After your application calls slog.SetDefault, the direction reverses: all calls to log.Printf and friends—that is, all calls to the default log.Logger—will go through slog at Info level. This is intended as a convenience: all your existing code that uses log, and all the third-party packages using log that you import now or will import in the future, are going through the handler of the default logger you set. This situation may not be temporary, because third-party packages might use log forever and never switch to slog. This is the case that uses the unexported handlerWriter type.

In both cases there is a hard-coded LevelInfo: in case 1, it is the minimum level of the default handler, and in case 2, it is the logging level of calls to the default log.Logger. In both cases, it should be rare to want to change this value, or so we thought when we designed slog. Case 1 is temporary, lasting just until you set your own slog.Logger. And in case 2, it seems likely that most uses of the log package will be informational, not just for errors or debug output. And if one package does happen to use it just for errors, and another just for debugging, what will you do if your program imports both? Info seems like the right compromise.

One other point: these two cases are (normally) mutually exclusive: your program has either called SetDefault or it hasn't. I added that parenthetical because you could grab the default handler with

dh := slog.Default().Handler()

and hold on to it after you call SetDefault and then use it as a handler again. But the result is likely to be weird:

func main() {
    dh := slog.Default().Handler()                                                       
    slog.SetDefault(slog.New(slog.NewTextHandler(os.Stdout, nil)))                       
    slog.Info("hello")
    logger := slog.New(dh)                                                               
    logger.Warn("hi")
}

produces the output

time=2023-09-06T18:59:18.481-04:00 level=INFO msg=hello                                  
time=2023-09-06T18:59:18.481-04:00 level=INFO msg="WARN hi"   

So I don't think that's a situation worth worrying about.

To summarize:

• Only one of the two hard-coded levels is active at a time.
• It should be rare to want to change them.

That is why I suggested a single global LevelVar. It would be the minimum level of the default handler in case 1, and the logging level of calls to the default log.Logger in case 2. It is a minimal change—a single new symbol in the API—that covers both cases, and it is "out of the way" in the documentation, in the variables section.

Contrast that with adding a SetDefaultWIthLevel function. Although that is technically a nicer solution because it provides a direct way to change a previously hardcoded value, it only handles case 2, and it's going to show up right after SetDefault in the docs, even though most people will never need it.

So a global LevelVar is hackier, but simpler and less obtrusive.

[zbrogz]

@jba Thanks for the write up. I'm in favor of a single global LevelVar like you suggest. If there is consensus, I think it is worth discussing naming options.

[jba]

@panjf2000 I realize I didn't address two of your concerns with a global LevelVar, so let me do that here.

will enable developers to update the default level at will even after only one call to slog.SetDefault, from which we benefit nothing other than a variable being constantly changed.

The handlerWriter will have a reference to the LevelVar, so each time it is set it will affect behavior. This is how LevelVar is designed to be used. The same would be true if you passed a HandlerOptions{Level: aLevelVar} to a built-in handler.

What's worse is that it may confuse users when the value of DefaultLevel is retrieved in one place and it doesn't match the behavior of the default logger (or it's not the value that was set initially) because it might get updated elsewhere.

Again, the current value is the one that is in use.

Given my answers here and my rationale above, are you okay with the global?

[panjf2000]

Thanks for the clarification!

Only one leftover for me, the case 1 you referred to in the previous comment, it's still not so clear to me how the global variable is going to steer the default logger of log toward the customized logging level before we call slog.SetDefault?

@jba

[jba]

@panjf2000 instead of the current code

func (*defaultHandler) Enabled(_ context.Context, l Level) bool {
    return l >= LevelInfo
}

it will read from the global:

func (*defaultHandler) Enabled(_ context.Context, l Level) bool {
    return l >= whateverWeCallTheGlobal.Level()
}

[jba]

What should we call this global LevelVar? That is the hard part. I suggested DefaultLevel but that is confusing. Maybe LogLoggerLevel?

[jba]

how the global variable is going to steer the default logger of log toward the customized logging level before we call slog.SetDefault?

@panjf2000 maybe I didn't understand your question. But note that before SetDefault is called, there is no meaningful way to talk about the level of a log.Logger. See https://go.dev/play/p/kV4HauAYkLq:

func main() {
	slog.Debug("debug")
	slog.Info("info")
	slog.Warn("warn")
	log.Printf("log")
}

outputs

2009/11/10 23:00:00 INFO info
2009/11/10 23:00:00 WARN warn
2009/11/10 23:00:00 log

The only place that a level matters is the minimum level of defaultHandler, hence my code snippet just above.

[seankhliao]

I like LogLoggerLevel, it can be explained as "the level for the handler that outputs through log", and "the level log calls make".
Though it may also make sense to split it into 2 vars?

[panjf2000]

how the global variable is going to steer the default logger of log toward the customized logging level before we call slog.SetDefault?

@panjf2000 maybe I didn't understand your question. But note that before SetDefault is called, there is no meaningful way to talk about the level of a log.Logger. See https://go.dev/play/p/kV4HauAYkLq:

func main() {
	slog.Debug("debug")
	slog.Info("info")
	slog.Warn("warn")
	log.Printf("log")
}

outputs

2009/11/10 23:00:00 INFO info
2009/11/10 23:00:00 WARN warn
2009/11/10 23:00:00 log

The only place that a level matters is the minimum level of defaultHandler, hence my code snippet just above.

Yes, I was aware of that, and that's also my confusion because you said "slog.SetDefaultWitlLevel only handles case 2", which made me wonder how the global variable would help handle case 1. Maybe I didn't fully comprehend your comment about case 1, but it would be nice to have more details about the global variable handling it, thanks!

@jba

[AndrewHarrisSPU]

The current doc for slog.SetDefault says:

After this call, output from the log package's default Logger (as with log.Print, etc.) will be logged at LevelInfo using l's Handler.

LogLoggerLevel works really well here, I think:

After this call, output from the log package's default Logger (as with log.Print, etc.) will be logged at [LogLoggerLevel] using l's Handler.

The current doc for slog.Default doesn't say much along the lines of case 1 of #62418 (comment). This could be another place to mention LogLoggerLevel.

I think an example for LogLoggerLevel that hits both of the cases would be really helpful.

[jba]

@zbrogz, can you add a line at the top of your original post that refers to the above comment?

[zbrogz]

@jba Yep, updated.

[jba]

I'd like to amend this to be a function: SetLogLoggerLevel(Level) Level sets the value of a global (which is now unexported), and returns the old value.

Any objections?

[zbrogz]

@jba What's the rationale? I slightly prefer the existing LogLoggerLevel since it's a shorter name and also allows checking what the level is currently at via LogLoggerLevel.Level(). That said, either way works for my specific use case.

[jba]

Someone might assign to the global, which would be a bug. That can't happen with the function.

[zbrogz]

Ah I see. What about making LogLoggerLevel a getter?

func LogLoggerLevel() *LevelVar

One thing I didn't mention before is that SetLogLoggerLevel(Level) Level returning the old value seems strange to me. For what purpose would that serve?

[jba]

Exactly the one you mentioned! So you don't also need a getter.

[zbrogz]

I see. What if there's a use case where you want to know what the current level is without setting it? Not a use case I necessarily care about, just thinking about implications for other users of the API.

Admittedly, your solution is a bit more concise:

slog.SetLogLoggerLevel(slog.LevelError)

vs

slog.LogLoggerLevel().Set(slog.LevelError)

[jba]

I agree, I think LogLoggerLevel() *LevelVar is a bit nicer.

[panjf2000]

So we now expose two new APIs for this?

[jba]

On the other hand, SetLogLoggerLevel is that it gives us more implementation freedom.

[panjf2000]

I think we can make SetLogLoggerLevel return the previous level when the input level is one of LevelDebug, LevelInfo, LevelWarn, and LevelError, otherwise, it does nothing and just returns the current level, just like runtime.GOMAXPROCS.

[rsc]

This proposal has been added to the active column of the proposals project
and will now be reviewed at the weekly proposal review meetings.
— rsc for the proposal review group

[rsc]

Have all remaining concerns about this proposal been addressed?

Add the following to log/slog:

// LogLoggerLevel controls the level for the bridge to the [log] package.
//
// Before [SetDefault] is called, when slog top-level logging functions call the default [log.Logger],
// this controls the minimum level for those calls. By default it is Info, so calls to [slog.Debug]
// (as well as top-level logging calls at lower levels) will not passed to the log.Logger. After calling
//    slog.LogLoggerLevel.Set(slog.LevelDebug)
// calls to slog.Debug will be passed to the log.Logger.
//
// After [SetDefault] is called, when calls to the default [log.Logger] are passed to the
// slog default handler, this controls the level at which those calls are logged.
var LogLoggerLevel LevelVar

An implementation is in https://go.dev/cl/525096

[zbrogz]

Yes, that works for me!

[panjf2000]

I thought we would change the global variable to a function, according to what @jba suggested.

Also, I got no responses to my follow-up comment.

[jba]

Yes, it's going to be a function.

I think we can make SetLogLoggerLevel return the previous level when the input level is one of LevelDebug, LevelInfo, LevelWarn, and LevelError, otherwise, it does nothing and just returns the current level, just like runtime.GOMAXPROCS.

@panjf2000, runtime.GOMAXPROCS benefits from the fact that some arguments make no sense. Any integer can be a level, so we can't do the same thing.

[rsc]

No change in consensus, so accepted. 🎉
This issue now tracks the work of implementing the proposal.
— rsc for the proposal review group

// SetLogLoggerLevel controls the level for the bridge to the [log] package.
//
// Before [SetDefault] is called, slog top-level logging functions call the default [log.Logger].
// In that mode, SetLogLoggerLevel sets the minimum level for those calls.
// By default the minimum level is Info, so calls to [Debug]
// (as well as top-level logging calls at lower levels)
// will not passed to the log.Logger. After calling
// slog.LogLoggerLevel.Set(slog.LevelDebug)
// calls to [Debug] will be passed to the log.Logger.
//
// After [SetDefault] is called, calls to the default [log.Logger] are passed to the
// slog default handler. In that mode,
// SetLogLoggerLevel sets the level at which those calls are logged.
// That is, after calling
// slog.LogLoggerLevel.Set(slog.LevelDebug)
// A call to [log.Printf] will result in a structured log at level [LevelDebug].
var LogLoggerLevel LevelVar

[jba]

The doc in the acceptance comment is a bit garbled. The proposed API is a function, not a global variable.
Here is the corrected version:

// SetLogLoggerLevel controls the level for the bridge to the [log] package.
//
// Before [SetDefault] is called, slog top-level logging functions call the default [log.Logger].
// In that mode, SetLogLoggerLevel sets the minimum level for those calls.
// By default the minimum level is Info, so calls to [Debug]
// (as well as top-level logging calls at lower levels)
// will not passed to the log.Logger. After calling
// slog.SetLogLoggerLevel(slog.LevelDebug)
// calls to [Debug] will be passed to the log.Logger.
//
// After [SetDefault] is called, calls to the default [log.Logger] are passed to the
// slog default handler. In that mode,
// SetLogLoggerLevel sets the level at which those calls are logged.
// That is, after calling
// slog.SetLogLoggerLevel(slog.LevelDebug)
// A call to [log.Printf] will result in a structured log at level [LevelDebug].
//
// SetLogLoggerLevel returns the previous value.
func SetLogLoggerLevel(Level) Level

[3052]

I find TextHandler awkward to work with:

h := slog.NewTextHandler(os.Stderr, &slog.HandlerOptions{
   Level: slog.LevelDebug,
})
slog.SetDefault(slog.New(h))

especially if you want to modify the default format. until Go 1.22 hits, a better option for me was just implementing Handler:

package slog

import (
   "context"
   "fmt"
   "log/slog"
)

type Handler struct { Level slog.Level }

func (Handler) WithAttrs([]slog.Attr) slog.Handler { return nil }

func (Handler) WithGroup(string) slog.Handler { return nil }

func (h Handler) Enabled(_ context.Context, l slog.Level) bool { return l >= h.Level }

func (Handler) Handle(_ context.Context, r slog.Record) error {
   fmt.Print(r.Message)
   r.Attrs(func(a slog.Attr) bool {
      fmt.Print(" ", a)
      return true
   })
   fmt.Println()
   return nil
}

note the entire interface has to be implemented because of #61892. then usage is more ergonomic:

h := Handler{slog.LevelDebug}
slog.SetDefault(slog.New(h))