Decide on a logging package

[atc0005]

I've been using the standard library log package and while it works, it seems to be lacking the basic leveled logging features I've come to rely upon over the years (e.g., logger from shell scripts, Python's logging module, ...).

There are a lot of options available, but I'm going to focus first on trying those which are most actively supported and used by some of the bigger teams/projects.

[atc0005]

Some of the packages I hope to evaluate (and or look further into):

• https://golang.org/pkg/log/#New

  • I've been using the default logger thus far

• https://golang.org/pkg/log/syslog/

  • I will likely use this at some point, but I'd like to use a package which supports dual-logging to both syslog and a local file, although trying syslog (which could create the local log for us) and falling back to a standard log file is also acceptable

• https://github.com/goinggo/tracelog

  • William Kennedy from Ultimate Go Training / Ardan Labs maintains this "wrapper" around the log standard library package
  • https://www.ardanlabs.com/blog/2013/11/using-log-package-in-go.html

• https://github.com/sirupsen/logrus

  • very popular
  • actively maintained
  • used by Docker
  • https://www.datadoghq.com/blog/go-logging/
  • https://www.scalyr.com/blog/getting-started-quickly-with-go-logging
  • https://github.com/sirupsen/logrus/blob/master/hooks/syslog/README.md

• https://godoc.org/github.com/juju/loggo

  • Used by Ubuntu for Juju?

• https://github.com/inconshreveable/log15

  • supports logging to syslog
  • https://godoc.org/github.com/inconshreveable/log15
  • unfortunately, this project appears mostly inactive

• https://github.com/rs/zerolog

  • leveled logging
  • structured logging
  • active maintenance?

• https://github.com/romana/rlog

  • https://stackoverflow.com/a/46236419

• https://godoc.org/github.com/alexcesaro/log/stdlog

  • https://stackoverflow.com/a/23172322

---

• https://stackoverflow.com/questions/16895651/how-to-implement-level-based-logging-in-golang
  • https://stackoverflow.com/a/31076772

[atc0005]

sirupsen/logrus and goinggo/tracelog are both very appealing, the former more so because it is actively maintained and likely has sufficient features to cover all of my use cases.

[atc0005]

This one looked good, but based on issues and pull requests it looks to be abandoned:

https://github.com/op/go-logging

[atc0005]

https://godoc.org/github.com/inconshreveable/log15#hdr-Library_Use

log15 is intended to be useful for library authors as a way to provide configurable logging to users of their library. Best practice for use in a library is to always disable all output for your logger by default and to provide a public Logger instance that consumers of your library can configure. Like so:

package yourlib

import "github.com/inconshreveable/log15"

var Log = log.New()

func init() {
    Log.SetHandler(log.DiscardHandler())
}

Users of your library may then enable it if they like:

import "github.com/inconshreveable/log15"
import "example.com/yourlib"

func main() {
    handler := // custom handler setup
    yourlib.Log.SetHandler(handler)
}

[atc0005]

Worth noting: The inconshreveable/log15 package author used Sirupsen/logrus as an inspiration for their package design.

Unfortunately it doesn't look like the library is receiving active support (quite a few issues/pull requests languishing).

[atc0005]

https://github.com/avelino/awesome-go#logging

Worth looking at later if sirupsen/logrus doesn't work out.

[atc0005]

https://github.com/apsdehal/go-logger

The formatting options remind me of Python's logging module.

[atc0005]

Next steps (to make it clearer to myself):

• https://github.com/sirupsen/logrus
• https://godoc.org/github.com/sirupsen/logrus
• https://www.scalyr.com/blog/getting-started-quickly-with-go-logging
• https://www.datadoghq.com/blog/go-logging/

[atc0005]

On a related note: does flaggy support limiting options for a flag parameter to a slice of options?

Ex: logging level

[atc0005]

Had some initial stumbling blocks with getting logrus setup/working with the syslog hook, but this code seems to work as intended with v1.4.2 of logrus (the latest at this time of writing):

package main

import (
	"log/syslog"

	// Use `log` if we are going to override the default `log`, otherwise
	// import without an "override" if we want to use the `logrus` name.
	// https://godoc.org/github.com/sirupsen/logrus
	log "github.com/sirupsen/logrus"

	// Official examples use either `lSyslog` or `logrus_syslog`
	// https://godoc.org/github.com/sirupsen/logrus/hooks/syslog
	lSyslog "github.com/sirupsen/logrus/hooks/syslog"
  )

  func main() {
	// https://godoc.org/github.com/sirupsen/logrus#New
	// https://godoc.org/github.com/sirupsen/logrus#Logger
	//log       := logrus.New()
	hook, err := lSyslog.NewSyslogHook("", "", syslog.LOG_INFO, "")

	if err == nil {
		// https://github.com/sirupsen/logrus#hooks
		// https://github.com/sirupsen/logrus/blob/master/hooks/syslog/README.md
		// Seems to require `log.AddHook(hook)`` vs `log.Hooks.Add(hook)`
		log.AddHook(hook)
	}

	log.Info("A walrus appears")

	log.WithFields(log.Fields{
			"animal": "walrus",
			"size":   10,
	}).Info("A group of walrus emerges from the ocean")

	log.WithFields(log.Fields{
			"omg":    true,
			"number": 122,
	}).Warn("The group's number increased tremendously!")

	log.WithFields(log.Fields{
			"omg":    true,
			"number": 100,
	}).Fatal("The ice breaks!")
  }

I worked from the https://github.com/atc0005/go-logrustest repo to figure out the syntax. When I went back and fully read over the logrus README.md file, I found that most of my stumbling blocks had been covered (to some degree or another).

Yet another case where it is better to try and read the manual from front-to-back before beginning.

[atc0005]

Some ideas regarding selection of log output based on current environment:

https://callistaenterprise.se/blogg/teknik/2017/08/02/go-blog-series-part10/

[atc0005]

Scratch notes for validating whether a support logging format was chosen on the command-line:

https://github.com/jessevdk/go-flags/blob/c0795c8afcf41dd1d786bebce68636c199b3bb45/option.go#L251

	if len(option.Choices) != 0 {
		found := false

		for _, choice := range option.Choices {
			if choice == *value {
				found = true
				break
			}
		}

		if !found {
			allowed := strings.Join(option.Choices[0:len(option.Choices)-1], ", ")

			if len(option.Choices) > 1 {
				allowed += " or " + option.Choices[len(option.Choices)-1]
			}

			return newErrorf(ErrInvalidChoice,
				"Invalid value `%s' for option `%s'. Allowed values are: %s",
				*value, option, allowed)
		}
	}

This existing function is probably a better fit for the current state of the code however:

elbow/matches.go

Line 74 in 785805f

func inFileExtensionsPatterns(ext string, exts []string) bool {

[atc0005]

• https://play.golang.org/p/0ybaFl_b9OX
• https://stackoverflow.com/a/57366702/903870

package main

import (
	"flag"
	"fmt"
	"os"
)

func main() {
	required := []string{"b", "s"}
	os.Setenv("ENV_B", "test")
	flag.String("b", os.Getenv("ENV_B"), "b value")
	flag.Parse()

	seen := make(map[string]bool)
	flag.VisitAll(func(f *flag.Flag) {
		if f.Value.String() != "" {
			seen[f.Name] = true
		}
	})
	for _, req := range required {
		if !seen[req] {
			// or possibly use `log.Fatalf` instead of:
			fmt.Fprintf(os.Stderr, "missing required -%s argument/flag\n", req)
			os.Exit(2) // the same exit code flag.Parse uses
		}
	}
}

Idea for validating that non-empty values have been provided for ALL command-line flags (using the stdlib flag package).

[atc0005]

On a related note: does flaggy support limiting options for a flag parameter to a slice of options?

Ex: logging level

I didn't spot the option when I looked. The only flag package which seemed to do so was jessevdk/go-flags:

https://github.com/jessevdk/go-flags#example

var opts struct {
	// Slice of bool will append 'true' each time the option
	// is encountered (can be set multiple times, like -vvv)
	Verbose []bool `short:"v" long:"verbose" description:"Show verbose debug information"`

	// Example of automatic marshalling to desired type (uint)
	Offset uint `long:"offset" description:"Offset"`

	// Example of a callback, called each time the option is found.
	Call func(string) `short:"c" description:"Call phone number"`

	// Example of a required flag
	Name string `short:"n" long:"name" description:"A name" required:"true"`

	// Example of a flag restricted to a pre-defined set of strings
	Animal string `long:"animal" choice:"cat" choice:"dog"`

	// Example of a value name
	File string `short:"f" long:"file" description:"A file" value-name:"FILE"`

	// Example of a pointer
	Ptr *int `short:"p" description:"A pointer to an integer"`

	// Example of a slice of strings
	StringSlice []string `short:"s" description:"A slice of strings"`

	// Example of a slice of pointers
	PtrSlice []*string `long:"ptrslice" description:"A slice of pointers to string"`

	// Example of a map
	IntMap map[string]int `long:"intmap" description:"A map from string to int"`
}

[atc0005]

Need to pick back up converting the other files in the project to using logrus-based log calls.

[atc0005]

TODO: Need to move file-based logging into main() ?

https://stackoverflow.com/questions/32619318/logging-to-a-file-in-golang

[atc0005]

Additional info:

• Who closes the file handler if i use file logger sirupsen/logrus#706
• when write to file, need to close by myself? sirupsen/logrus#619

[atc0005]

TODO: Need to move file-based logging into main() ?

https://stackoverflow.com/questions/32619318/logging-to-a-file-in-golang

Some good ideas here:

https://stackoverflow.com/a/43827612/903870

The use of the sync package, the clean wrapper are both good. I don't see a Close() call however.

[atc0005]

For now, I'm bundling a copy of *os.File with the Config object and checking for nil value to determine whether I've set something. Feels crude, broken somehow, but I can revisit this later once I learn more.

[atc0005]

Going to go with logrus for this project. I can pick another package for the next project in order to get some experience with several different ones.

Will use the use-go-flags-and-logrus branch to complete the work for this issue.