This discussion has led to a proposal and is now finished. Please comment on the proposal.
We would like to add structured logging with levels to the standard library. Structured logging is the ability to output logs with machine-readable structure, typically key-value pairs, in addition to a human-readable message. Structured logs can be parsed, filtered, searched and analyzed faster and more reliably than logs designed only for people to read. For many programs that aren't run directly by person, like servers, logging is the main way for developers to observe the detailed behavior of the system, and often the first place they go to debug it. Logs therefore tend to be voluminous, and the ability to search and filter them quickly is essential.
In theory, one can produce structured logs with any logging package:
In practice, this is too tedious and error-prone, so structured logging packages provide an API for expressing key-value pairs. This draft proposal contains such an API.
We also propose generalizing the logging "backend." The
Lastly, we include levels in our design, in a way that accommodates both traditional named levels and logr-style verbosities.
Our goals are:
What Does Success Look Like?
Go has many popular structured logging packages, all good at what they do. We do not expect developers to rewrite their existing third-party structured logging code en masse to use this new package. We expect existing logging packages to coexist with this one for the foreseeable future.
We have tried to provide an API that is pleasant enough to prefer to existing packages in new code, if only to avoid a dependency. (Some developers may find the runtime tracing integration compelling.) We also expect newcomers to Go to become familiar with this package before learning third-party packages, so they will naturally prefer it.
But more important than any traction gained by the "frontend" is the promise of a common "backend." An application with many dependencies may find that it has linked in many logging packages. If all of the packages support the handler interface we propose, then the application can create a single handler and install it once for each logging library to get consistent logging across all its dependencies. Since this happens in the application's main function, the benefits of a unified backend can be obtained with minimal code churn. We hope that this proposal's handlers will be implemented for all popular logging formats and network protocols, and that every common logging framework will provide a shim from their own backend to a handler. Then the Go logging community can work together to build high-quality backends that all can share.
Logrus, one of the first structured logging packages, showed how an API could add structure while preserving the formatted printing of the
Zap grew out of Uber's frustration with the slow log times of their high-performance servers. It showed how a logger that avoided allocations could be very fast.
zerolog reduced allocations even further, but at the cost of reducing the flexibility of the logging backend.
All the above loggers include named levels along with key-value pairs. Logr and Google's own glog use integer verbosities instead of named levels, providing a more fine-grained approach to filtering high-detail logs.
Overview of the Design
Here is a short program that uses some of the new API:
It begins by setting the default logger to one that writes log records in an easy-to-read format similar to logfmt . (There is also a built-in handler for JSON.)
The program then outputs three log messages augmented with key-value pairs. The first logs at the Info level, passing a single key-value pair along with the message. The second logs at the Error level, passing an
The third produces the same output as the second, but more efficiently. Functions like
Interaction Between Existing and New Behavior
The main method is
Beta Was this translation helpful? Give feedback.