Go package for routing, formatting and publishing events produced by a program.
While Go's standard log package is handy it definitely lacks crucial features, like the ability to control the output format of the events for example. There are many packages that provides logger implementations for Go but they often expose complex APIs and were not designed to be efficient in terms of CPU and memory usage.
The events package attempts to address these problems by providing high level
abstractions with highly efficient implementations. But it also goes further,
offering a new way to think about what logging is in a program, starting with
the package name, events
, which expresses what this problem is about.
During its execution, a program produces events, and these events need to be
captured, routed, formatted and published to a persitence system in order to
be later analyzed.
The package was inspired by this post from Dave Cheney. It borrowed a lot of the ideas but tried to find the sweet spot between Dave's idialistic view of what logging is supposed to be, and production constraints that we have here at Segment.
At the core of the package is the Event
type. Instances of this type carry
the context in which the event was generated and the inforation related to
the event.
Events are passed from the sources that trigger them to handlers, which are
types implementing the Handler
interface:
type Handler interface {
HandleEvent(*Event)
}
The sub-packages provide implementations of handlers that publish events to various formats and locations.
The Logger
type is a source of events, the program uses loggers to generate
events with an API that helps the developer express its intent. Unlike a lot of
logging libraries, the logger doesn't support levels of messages, instead it
exposes a Log
and Debug
methods. Events generated by the Log
method are
always produced by the logger, while those generated by Debug
may be turned
on or off if necessary.
The package also exposes a default logger via top-level functions which cover
the needs of most programs. The Log
and Debug
functions support fmt-style
formatting but augment the syntax with features that make it simpler to generate
meaningful events. Refer to the package's documentation to learn more about it.
The standard log
package doesn't give much flexibility when it comes to its
logger type. It is a concrete type and there is no Logger
interface which
would make it easy to plugin different implementations in packages that need to
log events. Unfortunately many of these packages have hard dependencies on the
standard logger, making it hard to capture their events and produce them in
different formats.
However, the events/log
package is a shim between the standard log
package,
and a stream of events. It exposes an API compatible with the standard library,
and automatically configures the log
package to reroute the messages it emits
as events to the default logger.
Event handlers are the abstraction layer that allows to connect event sources to
arbitrary processing pipelines.
The sub-packages provides pre-defiend implementations of handlers.
The events/text
package provides the implementation of an event handler which
formats the event it receives in a human-readable format.
The events/ecslogs
package provides the implementation of an event handler
which formats the events it receives in a format that is understood by ecs-logs.
We said the logger doesn't support log levels, however these levels have proven
useful to get a signal on a program misbehaving when it starts emitting tons of
ERROR level messages.
However, the program doesn't have to express what the severity level is in order
to get the right behavior. The events/ecslogs
package analyzes the events it
receives and guess what the level should be, here are the rules:
- By default events are set to the INFO level.
- If an event was generated from a
Debug
call then handler sets the event level to DEBUG. - If the event's arguments contains at least one value that satisfies the
error
interface then the level is set to ERROR. These rules allow for the best of both worlds, giving the program a small and expressive API to produce events while maintaining compatibility with our existing tools.
The sub-packages have side-effects when they are importaed, both events/text
and events/ecslogs
set the default logger's handler. The events/text
package
sets a handler if the program's output is a terminal, while events/ecslogs
does the opposite.
This approach mimics what we've achieved in many parts of our software stack and
has proven to be good defaults, doing the right thing whether the program is
dealing with a production or development environment.