Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

getlantern/lantern#2232 Code review fixes #96

Merged
merged 5 commits into from
Feb 25, 2015
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion crosscompile.bash
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ VERSION="`git describe --abbrev=0 --tags --exact-match || git rev-parse --short
BUILD_DATE="`date -u +%Y%m%d%.%H%M%S`"
export VERSION_STRING="$VERSION ($BUILD_DATE)"
LOGGLY_TOKEN="469973d5-6eaf-445a-be71-cf27141316a1"
LDFLAGS="-w -X main.version $VERSION -X main.buildDate $BUILD_DATE -X main.logglyToken LOGGLY_TOKEN"
LDFLAGS="-w -X main.version $VERSION -X main.buildDate $BUILD_DATE -X github.com/getlantern/flashlight/logging.logglyToken $LOGGLY_TOKEN"
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This never worked, so nothing ever actually got logged to Loggly. I fixed it to embed the token correctly and verified that we're getting data to Loggly now.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks Ox! that's a testing-only token, right?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nope, that's the production token.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm... that's a thing. Since that's the only thing one needs to post to loggly I was going to suggest we remove it from the public view and add it to too-many-secrets but then I realized it will be embedded into the binary anyway, so I guess we may as well leave it public for now.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, by definition it has to be public because our clients will use it to post.

echo "Building flashlight version $VERSION ($BUILD_DATE)"
# gox -ldflags="-w -X main.version $VERSION -X main.buildDate $BUILD_DATE -X main.logglyToken LOGGLY_TOKEN" -osarch="linux/386 linux/amd64 windows/386 darwin/amd64" github.com/getlantern/flashlight
# Compile for Mac
Expand Down
11 changes: 3 additions & 8 deletions src/github.com/getlantern/flashlight/flashlight.go
Original file line number Diff line number Diff line change
Expand Up @@ -60,13 +60,8 @@ func main() {
}

func doMain() {
defer logging.Close()
i18nInit()

// Passing version and build date to the logging package. They'll be reported
// to loggly.
logfile := logging.Setup(version, buildDate)
defer logfile.Close()

configureSystemTray()
displayVersion()

Expand Down Expand Up @@ -145,7 +140,7 @@ func runClientProxy(cfg *config.Config) {

// Using a goroutine because we'll be using waitforserver and at this time
// the proxy is not yet ready.
go logging.Configure(cfg)
go logging.Configure(cfg, version, buildDate)
proxiedsites.Configure(cfg.ProxiedSites)

if hqfd == nil {
Expand All @@ -168,7 +163,7 @@ func runClientProxy(cfg *config.Config) {
hqfdc := hqfd.DirectHttpClient()
geolookup.Configure(hqfdc)
statserver.Configure(hqfdc)
logging.Configure(cfg)
logging.Configure(cfg, version, buildDate)
}
}
}()
Expand Down
165 changes: 83 additions & 82 deletions src/github.com/getlantern/flashlight/logging/logging.go
Original file line number Diff line number Diff line change
Expand Up @@ -23,40 +23,27 @@ import (
"github.com/getlantern/wfilter"
)

const (
logTimestampFormat = "Jan 02 15:04:05.000"
)

var (
log = golog.LoggerFor("flashlight")

LogTimestampFormat = "Jan 02 15:04:05.000"
lang string
tz string
versionToLoggly string
cfgMutex sync.Mutex
logFile *rotator.SizeRotator
cfgMutex sync.Mutex

// logglyToken is populated at build time by crosscompile.bash. During
// development time, logglyToken will be empty and we won't log to Loggly.
logglyToken string

// A wrapper of the loggly client for the current session.
logglyClient *logglyErrorWriter
errorOut io.Writer
debugOut io.Writer

lastAddr string
)

func init() {
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All the basic logging stuff can be done without any external data, so I'm doing it in init() now.

lang, _ = jibber_jabber.DetectLanguage()
tz = time.Local.String()
}

func Setup(version string, buildDate string) *rotator.SizeRotator {

if version == "" {
panic("You can't use an empty version.")
}

if buildDate == "" {
panic("You can't use an empty build date.")
}

versionToLoggly = fmt.Sprintf("%v (%v)", version, buildDate)

logdir := appdir.Logs("Lantern")
log.Debugf("Placing logs in %v", logdir)
if _, err := os.Stat(logdir); err != nil {
Expand All @@ -67,108 +54,122 @@ func Setup(version string, buildDate string) *rotator.SizeRotator {
}
}
}
file := rotator.NewSizeRotator(filepath.Join(logdir, "lantern.log"))
logFile = rotator.NewSizeRotator(filepath.Join(logdir, "lantern.log"))
// Set log files to 1 MB
file.RotationSize = 1 * 1024 * 1024
logFile.RotationSize = 1 * 1024 * 1024
// Keep up to 20 log files
file.MaxRotation = 20
logFile.MaxRotation = 20

// Loggly has its own timestamp so don't bother adding it in message,
// moreover, golog always write each line in whole, so we need not to care about line breaks.
errorOut := timestamped(NonStopWriter(os.Stderr, file))
errorOut = timestamped(io.MultiWriter(os.Stderr, logFile))
debugOut = timestamped(io.MultiWriter(os.Stdout, logFile))
golog.SetOutputs(errorOut, debugOut)
}

func Configure(cfg *config.Config, version string, buildDate string) {
if logglyToken == "" {
log.Debugf("No logglyToken, not sending error logs to Loggly")
} else {
log.Debugf("Sending error logs to Loggly")
logglyClient = &logglyErrorWriter{loggly.New(logglyToken)}
errorOut = NonStopWriter(errorOut, logglyClient)
return
}

debugOut := timestamped(NonStopWriter(os.Stdout, file))
golog.SetOutputs(errorOut, debugOut)
return file
}

func Configure(cfg *config.Config) (err error) {
cfgMutex.Lock()
defer cfgMutex.Unlock()

if logglyClient == nil {
return fmt.Errorf("No loggly client to configure.")
if version == "" {
log.Error("No version configured, Loggly won't include version information")
return
}

if cfg.Addr == "" {
return fmt.Errorf("No proxy address provided.")
if buildDate == "" {
log.Error("No build date configured, Loggly won't include build date information")
return
}

if err = waitforserver.WaitForServer("tcp", cfg.Addr, 10*time.Second); err != nil {
return fmt.Errorf("Proxy never came online at %s: %q", cfg.Addr, err)
}
if cfg.Addr != lastAddr {
lastAddr = cfg.Addr

var cli *http.Client
if cli, err = util.HTTPClient(cfg.CloudConfigCA, cfg.Addr); err != nil {
return fmt.Errorf("Could not create proxy HTTP client %q.", err)
}
if cfg.Addr == "" {
log.Error("No known proxy, won't report to Loggly")
removeLoggly()
return
}

logglyClient.l.SetHTTPClient(cli)
err := waitforserver.WaitForServer("tcp", cfg.Addr, 10*time.Second)
if err != nil {
log.Errorf("Proxy never came online at %v, not logging to Loggly", cfg.Addr)
removeLoggly()
return
}

var client *http.Client
client, err = util.HTTPClient(cfg.CloudConfigCA, cfg.Addr)
if err != nil {
log.Errorf("Could not create proxied HTTP client, not logging to Loggly: %v", err)
removeLoggly()
return
}

log.Debugf("Sending error logs to Loggly via proxy at %v", cfg.Addr)

lang, _ := jibber_jabber.DetectLanguage()
logglyWriter := &logglyErrorWriter{
lang: lang,
tz: time.Local.String(),
versionToLoggly: fmt.Sprintf("%v (%v)", version, buildDate),
client: loggly.New(logglyToken),
}
logglyWriter.client.SetHTTPClient(client)
addLoggly(logglyWriter)
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Configure() now builds a logglyWriter on the fly every time that the cfg.Addr has changed. logglyWriter has all the state that it needs (e.g. country, tz, etc.) and none of that is global anymore.

}
}

return nil
func Close() error {
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Instead of returning the logfile so that main can close it, we just expose a Close() function which main can use instead.

return logFile.Close()
}

// timestamped adds a timestamp to the beginning of log lines
func timestamped(orig io.Writer) io.Writer {
return wfilter.LinePrepender(orig, func(w io.Writer) (int, error) {
return fmt.Fprintf(w, "%s - ", time.Now().In(time.UTC).Format(LogTimestampFormat))
return fmt.Fprintf(w, "%s - ", time.Now().In(time.UTC).Format(logTimestampFormat))
})
}

type logglyErrorWriter struct {
l *loggly.Client
func addLoggly(logglyWriter io.Writer) {
golog.SetOutputs(io.MultiWriter(errorOut, logglyWriter), debugOut)
}

func (w logglyErrorWriter) Write(b []byte) (int, error) {
return writeToLoggly(w.l, "ERROR", string(b))
func removeLoggly() {
golog.SetOutputs(errorOut, debugOut)
}
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

addLoggly and removeLoggly allow adding a logglyWriter to the base errorOut Writer. This way, we can add remove Loggly whenever we're ready (e.g. once Configure() has been called) and we can change the Loggly settings simply by creating a new logglyWriter and adding that.


func writeToLoggly(l *loggly.Client, level string, msg string) (int, error) {
type logglyErrorWriter struct {
lang string
tz string
versionToLoggly string
client *loggly.Client
}

func (w logglyErrorWriter) Write(b []byte) (int, error) {
extra := map[string]string{
"logLevel": level,
"logLevel": "ERROR",
"osName": runtime.GOOS,
"osArch": runtime.GOARCH,
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I took out nonStopWriter because I noticed that my errors weren't making it to the console. I replaced it with io.MultiWriter, which has the downside that it stops as soon as any of the writers had an error, but the upside is that it works and my output makes it to the console.

"osVersion": "",
"language": lang,
"language": w.lang,
"country": globals.GetCountry(),
"timeZone": tz,
"version": versionToLoggly,
"timeZone": w.tz,
"version": w.versionToLoggly,
}

m := loggly.Message{
"extra": extra,
"message": msg,
"message": string(b),
}
err := l.Send(m)

err := w.client.Send(m)
if err != nil {
return 0, err
}
return len(msg), nil
}

type nonStopWriter struct {
writers []io.Writer
}

func (t *nonStopWriter) Write(p []byte) (n int, err error) {
for _, w := range t.writers {
n, _ = w.Write(p)
}
return len(p), nil
}

// NonStopWriter creates a writer that duplicates its writes to all the
// provided writers, even if errors encountered while writting.
func NonStopWriter(writers ...io.Writer) io.Writer {
w := make([]io.Writer, len(writers))
copy(w, writers)
return &nonStopWriter{w}
return len(b), nil
}