This library is covered in an AWS Open Source blog post: Splitting an application’s logs into multiple streams: a Fluent tutorial
This library was created to demonstrate a somewhat experimental idea. If you end up using it (or write your own similar code), please plus one this issue to let us know. Thoughts/comments/feedback also welcome.
Go Code that wraps the fluent-logger-golang in a struct that implements io.Writer.
This means it can be used as the underlying io stream for many loggers. See main.go for full example usage.
Simple example with the popular sirupsen/logrus logger:
import (
"fmt"
"github.com/awslabs/fluent-golang-io-writer/logger"
"github.com/fluent/fluent-logger-golang/fluent"
"github.com/sirupsen/logrus"
)
func main() {
// configure logrus to output as JSON
logrus.SetFormatter(&logrus.JSONFormatter{})
logrus.SetLevel(logrus.DebugLevel)
// create a FluentWriter instance
fluentLogger, err := logger.NewFluentWriter(fluent.Config{}, "app", []string{"level"})
if err != nil {
fmt.Println(err)
os.Exit(1)
}
// Set logrus to use it
logrus.SetOutput(fluentLogger)
// Now use logrus as normal!
logrus.WithFields(logrus.Fields{
"animal": "walrus",
}).Info("A walrus appears")
}
logger.NewFluentWriter(fluent.Config{}, "app", []string{"level"})
Above, we see the call to the constructor for the library. The first argument is the Fluent Logger Golang config object which configures the connection to Fluentd/Bit. The second argument is a prefix for the tags given to logs emitted by this writer. The third argument is a list of keys in the log messages whose values will be the suffix of the tag. This library relies on the fact that the logs produced by Logrus will be JSON formatted. Thus, it will find the level key in each log message and append this to the tag prefix to construct the final tag. In practice, logs will be emitted with tags as follows:
app.debugfor the debug logsapp.infofor the info logsapp.warningfor the warn logsapp.errorfor the error logsapp.fatalfor the fatal logs
Logs which do not have a level field or which can not be parsed as JSON will simply be given the tag app.
This project is licensed under the Apache-2.0 License.