New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
log/slog: structured, leveled logging #56345
Comments
This comment has been hidden.
This comment has been hidden.
This comment has been hidden.
This comment has been hidden.
This comment has been hidden.
This comment has been hidden.
This is a huge API surface without any real production testing (AIUI). Perhaps it might be better to land it under golang.org/x for some time? Eg, like context, xerrors changes. |
It's available under golang.org/x/exp/slog |
This comment has been hidden.
This comment has been hidden.
I love most of what this does, but I don't support its addition as it stands. Specifically, I have issues with the option to use inline key-value pairs in the log calls. I believe the attributes system alone is fine. Logging does not need the breakage that key-value args like that allow. The complexity in the documentation around Log should be a warning sign.
The suggestion was that potential problems with key-value misalignment will all be solved by vet checks. As I mentioned in this thread of the discussion, relying on vet should be viewed a warning as to potential problems with the design, not a part of the design itself. Vet should not be a catch-all, and we should do what we can to avoid requiring vet warnings to safely develop go. An accidentally deleted/missed or added/extra argument in key value logging would offset the keys and values after it. That could easily bog down a logging system trying to index all the new "keys" it is getting. It could also lead to data being exposed in a way that it should not be. I acknowledge that neither of these examples are all that likely in a well defined system, or somewhere with good practices around code reviewing etc.. But they are possible. |
@deefdragon Doesn't this concern apply to Printf as well? Is the difference the dependency on these logs by external systems..? |
Based on that the Go standard library is very often being recommended as source of idiomatic code, and this package aspires to be merged as part of it, I would like you explain to me the involvement of context package. If some object uses logger, isn't it its dependency? Shouldn't we make the dependencies explicit? Isn't this logger smuggling bad practice? If passing the logger by context is idiomatic, is *sql.DB too? Why has the logger stored context? It violates the most famous sentence from the documentation for context
Logger in context containing its own context inside ... Frankly, I'm a bit confused. |
@hherman1 The same concern does apply to printf, tho it's not as bad compared to logging. With printf, most statements are consumed as a single chunk, and only consumed locally by the programmer. Being misaligned is easy enough for a human to notice, parse, and correct. In the case of Sprintf, where it might not be consumed by the programmer, and instead be used as an argument to something, the "testing" during development that is making sure the program starts would likely catch most misalignment. Being off by one in a log is much harder to catch as it has no real impact in the program's execution. You only notice there is an issue when you have to go through your logs. |
I think I share some of @prochac 's concerns regarding context. Maybe I'm being a bit of a luddite, but recommending that the logger is passed around inside context rather than via explicit dependency injection, smells a bit funny to me. Context, from what I have always followed, is for request-scoped information, rather than dependencies. And the more clarity surfacing dependencies the better. IE just assuming the right logger is in context, and getting the default one because it's still getting some logger |
I think there are two approaches here:
I've used both patterns frequently in high-scale production services; and both have their places. I'd definitely like to see slog promote context-propagated logging as the observability benefits are huge. |
Appreciate your explanation @v3n . I'm still having a slightly hard time understanding the benefit of the logger itself being passed around in context. I understand propagating the log correlation information via context, and we currently use the otelzap implementation that does this sort of thing via ErrorContext(ctx, ...) etc logging methods. I like the WithContext methods proposed here, passing the context to the logger, in similar fashion. It's more the logger itself being passed around inside the context that feels a bit odd to me The zap and otelzap libraries do allow for the same kind of thing, whereby you can get the logger from context etc (and I'm sure others do), it's just this being in the std library it's more of a recommendation for this kind of pattern |
I still want a standard handler for |
@deefdragon, we'll have a vet check for that. |
@seankhliao, such a handler seems easy to write, and it's not clear to me yet whether there is enough demand to include it. Let's hold off for now; we can always add it later. |
@prochac, that is a design principle, not a hard-and-fast rule. It is there to steer people away from buggy code, but that has to be weighed against other factors. In this case, we knew that passing tracing information to logging was an important feature, but we didn't want to add a context argument to every log output method. This was our solution. |
@mminklet, scoping a logger to a request is a common pattern, and is probably the main application of the ability to add a Logger to a context. It doesn't preclude dependency injection; if that works for you, stick with it. |
This is a significant proposal. @jba can you do a video talk on this. And, perhaps, a blog post? |
@jba As I said in my original post, I don't think that's a good solution.
|
I like this in general. One API nit from an experiment in s3-upload-proxy: it would be good to have a way to convert a string into to the level (say you want to allow users set an environment variable like LOG_LEVEL=debug and have that translated to Other libraries (logrus, zerolog, zap) call that function |
|
The additional '+'/'-' terms put a twist on this, I think it'd be nice to have. (I had this laying around: https://go.dev/play/p/Izwzgd8Kmc9) |
Three comments on the proposal: One thing which irritated me with zap was the existence of both sugared and attribute log methods. My second observation is that we have 10 Attr constructors, one for each common type we want to log, + any. Finally, I think it is very good (but perhaps overdue) that we are moving towards a canonical production strength logging library in stdlib. Most libraries need some level of logging, if only for debugging. And not having a standard interface of |
I disagree. There is only one
The answer is, "hopefully." With the current implementation, you can only reduce the API surface at a considerable time penalty. But that may change before this API is frozen. See this item in the discussion.
According to my analysis, |
This proposal has been added to the active column of the proposals project |
I'm with @prochac and @mminklet in that passing logger in context seems awkward. However, I see your point about context propagation and @v3n point:
But then @jba (apologies if this was already discussed in #54763, I tried to & failed to find it there) did you consider something like context-propagated log-context, as opposed to propagating the whole logger? I frequently wish I could do easily is adding log-context to an already existing logger, like: logger = slow.WithContext(ctx).With("method", r.Method ...)
.....
err := doSomething(c ctx) err {
doSomethingElse(c ctx) {
somethingVeryDeep(c ctx) {
// do some other work
slog.AddToContext(ctx, "workResult", ok)
}
somethingVeryDeepAgain(c ctx) {
// do some work
slog.AddToContext(ctx, "otherWorkResult", ok)
}
} ()
} ()
if err != nil {
logger.Error("requestFailed") // `slog` extract values for me.
} This would allow me to log once per request/error instead of tens of scattered log-lines which I then need to correlate with request/span ids. I think this could also support https://go-review.googlesource.com/c/proposal/+/444415/5/design/56345-structured-logging.md#754 ctx, span := tracer.Start(ctx, name, opts) since
|
There may be a hint of namespace clobbring in ReplaceAttr, where matching on a key string is invoked before group prefixes are applied (example: https://go.dev/play/p/yFNXLz3gklD). I think it's a reasonable behavior; the version of A version of |
Everyone, minor changes to slog continue to be discussed in other proposals, not all of which refer back to here. If you're interested, you can search for them. Most would benefit from more eyeballs. |
@jba (I apologize if this isn't the right place to ask these questions, but I didn't want to edit the wiki without asking.)
|
@telemachus, thanks. I updated the initial sentences of the wiki page with the status and location changes, and a disclaimer. |
Format Group values like a []Attr, rather than a *Attr. Also, use fmt.Append in Value.append. Updates golang#56345. Change-Id: I9db1a8ec47f8e99c1ac3225d78e152013116bff3 Reviewed-on: https://go-review.googlesource.com/c/go/+/479515 Run-TryBot: Jonathan Amsterdam <jba@google.com> Reviewed-by: Alan Donovan <adonovan@google.com> TryBot-Result: Gopher Robot <gobot@golang.org>
Comparing two Values with == is sensitive to the internal representation of Values, and may not correspond to equality on the Go values they represent. For example, StringValue("X") != StringValue(strings.ToUpper("x")) because Go ends up doing a pointer comparison on the data stored in the Values. So make Values non-comparable by adding a non-comparable field. Updates golang#56345. Change-Id: Ieedbf454e631cda10bc6fcf470b57d3f1d2182cc Reviewed-on: https://go-review.googlesource.com/c/go/+/479516 Run-TryBot: Jonathan Amsterdam <jba@google.com> Reviewed-by: Alan Donovan <adonovan@google.com> TryBot-Result: Gopher Robot <gobot@golang.org>
Hey all, over on #59292, we're considering removing |
Hey, |
Yes.
It can get suggested in a new issue, but this comment suggests that it wouldn't get accepted. I'm currently undecided whether I should ask for it or propose go-logr/logr as the "owner" of the key for storing a logger in a context. The latter approach has the advantage that with some help by wrappers, two-way compatibility between clients using the logr API and those using the slog API + logr for context access could be achieved (with some caveats).
It also wouldn't be interoperable with other libraries. |
That's unfortunate. I'm not sure I agree with (or understand correctly) this sentence:
IMHO it would be worth it. Involving a third party package feels weird and complicated, when |
What I've implemented in the past is that my |
The implication of having a
This is not different from |
I'd argue the consensus is that libraries should NOT log, in the current state of things. Or if they do, it's with 1) their own logger, 2) an exposed way to turn off/override their logger. I don't see that changing with slog in its current state. And I'd say it's fine, slog doesn't have to fix everything that's imperfect with Go's logging story. This is a stretch, and I think out of scope for the current proposal, but has it been considered to implement some kind of namespacing to make slog more library friendly? // In the library
var namespace = struct{}{}
func LibNameSlogNamespace() any {
return namespace
}
// Can be used by a library, or by sections of your code that you want to namespace
slog.FromContextWithNamespace(context.Context, any) *slog.Logger
// slog.FromContextWithNamespace(ctx, MyLibNamespace)
// In the caller code
HandlerOptions{
Namespaces: map[any]slog.Level{
LibNameSlogNamespace(): slog.LevelError,
}
} Log4j has a similar behaviour. I think this would solve a real itch with logging across the whole ecosystem. |
@jba Is there any way to set default Level? The
Perhaps i.e func NewDefaultHandler(w io.Writer) Handler {
return (HandlerOptions{}).NewDefaultHandler(w)
}
func (opts HandlerOptions) NewDefaultHandler(w io.Writer) Handler {
lg := log.New(w, "", log.LstdFlags)
return newDefaultHandler(lg.Output)
} |
The existence of |
The whole purpose of a The pro and con of this is that the caller doesn't need to configure individual loggers for everything that might have relevant log output down the call chain. It's a con because one might also get undesired log output. |
@Thiht, that is what |
The logger-in-a-context discussion is worth having, but not here. If someone wants to champion it, make a proposal. |
@shaj13, we think of the default handler as a bridge to the Others have asked about this, so let me explain the rationale in more detail. We wanted to make sure even the simplest uses of slog would do something reasonable. Even if your program just uses But if you decide to use At that point you should use |
Support for output prefixes and format customization was already requested and rejected, several times I think. The solution is to write your own formatter or use a wrapped |
I still strongly disagree with the argument not to provide any customization of the default logger. For example, I adopted slog because in my development environment, I just want log levels (and the ability to filter by level). In production, I want full structured JSON output, so here installing the built-in JsonHandler makes sense. In my development environment, the default logger is a lot more readable than the TextHandler. Writing my own handler feels like overkill when all I want is to see my debug-level log messages. |
Will slog be in the standard library with the next version of Golang released? |
I wonder if it would make sense to keep the exp package in sync. It's much easier for people to use in their experiments than copying code from the Go source tree (or recompiling Go from tip after every update). |
After creating my own clone of the package, I realize it may not be trivial though... |
We propose a new package providing structured logging with levels. Structured logging adds key-value pairs to a human-readable output message to enable fast, accurate processing of large amounts of log data.
See the design doc for details.
The text was updated successfully, but these errors were encountered: