Skip to content

Latest commit

 

History

History
391 lines (272 loc) · 16.1 KB

configure-logs-context-go.mdx

File metadata and controls

391 lines (272 loc) · 16.1 KB
title tags metaDescription redirects freshnessValidatedDate
Go agent logs in context
Logs
Enable log management in New Relic
Logs in context for Go
For apps monitored by the Go agent, learn how to link log data with related data across the rest of the New Relic platform.
/docs/enable-logs-context-go
/docs/logs/new-relic-logs/enable-logs-context/enable-logs-context-go
/docs/logs/enable-logs/logs-context-go/configure-logs-context-go
/docs/logs/enable-log-monitoring-new-relic/logs-context-go/configure-logs-context-go
/docs/logs/enable-log-management-new-relic/logs-context-go
/docs/logs/enable-log-management-new-relic/logs-context-go/configure-logs-context-go
/docs/logs/enable-log-management-new-relic/configure-logs-context/configure-logs-context-go
never

With our Go language agent, you can get logs in context, which lets you see your app logs in the context of your other New Relic data. For general information on this feature, see APM logs in context.

Got lots of Go logs? Check out our [tutorial on how to optimize and manage them](/docs/tutorial-large-logs/get-started-managing-large-logs/).

Configuring logs in context in the agent [#automatic]

Modifications to the Go agent's config options are needed in order to use the following logs in context features.

For most users, using the automatic log forwarding built into the Go agent is the best solution. It's easy to set up and works great for a majority of use cases. Learn about log forwarding limitations.

Automatic application log forwarding is now enabled by default in Go agent version 3.20.0 or higher. If your applications use these Go agent versions, you don't need to change your config options to enable automatic log forwarding.

If you're using an older version of the Go agent, you'll need to modify its config to enable application log forwarding:

app, err := newrelic.NewApplication(
  newrelic.ConfigAppLogForwardingEnabled(true),
)

For users with more custom logging needs, or users who want to send more than 10,000 logs per 60 second cycle, or 833 logs per 5 second cycle, to New Relic we recommend that you manually set up log forwarding. Once you have a log forwarder configured to send logs to New Relic, allow the Go agent to enrich your logs for forwarding by modifying its config to include this setting:

app, err := newrelic.NewApplication(
  newrelic.ConfigAppLogDecoratingEnabled(true),
)

If you are using Go agent version v3.20.0 or higher, you need to disable application log forwarding in order to avoid duplicating the logs collected by New Relic.

app, err := newrelic.NewApplication(
  newrelic.ConfigAppLogDecoratingEnabled(true),
  newrelic.ConfigAppLogForwardingEnabled(false),
)

Installing a logs-in-context plugin

Once your agent is configured to send logs to New Relic, install a logs in context plugin to instrument your logging library.

The logWriter library is an io.Writer that automatically integrates the latest New Relic Logs in Context features into the go standard library logger. Follow these steps to install it in your application. This library requires the installed version of the Go agent to be 3.19.1 or higher.

  1. Add the logWriter package to your module.

    go get github.com/newrelic/go-agent/v3/integrations/logcontext-v2/logWriter
  2. Import the logWriter package in the file where you initialize your logger.

    import (
      "log"
    
      "github.com/newrelic/go-agent/v3/integrations/logcontext-v2/logWriter"
    )
  3. Create a logWriter object. It must be passed a valid io.Writer, which is where logs will be written to, and an initialized go-agent application. The following example writes to standard out.

    writer := logWriter.New(os.Stdout, newRelicApp)
  4. Create a logger object with your new logWriter object as the logger's output destination.

    logger := log.New(&writer, "", log.Default().Flags())

At this point, any log written with the logger created will be handled by the go-agent based on your logging config settings.

Transactions

If you want to capture the context of a transaction, a new logWriter object and logger object must be created. To create a new logWriter object for your transaction use either the WithTransaction() or WithContext() functions depending on how the transaction is passed to your function. These functions will make a new copy of the original logWriter object with the transaction context included.

If your function receives a context wrapped with a transaction, use the WithContext() function:

txnWriter := writer.WithContext(myWrappedContext)

If your function receives a transaction pointer, then use the WithTransaction() function:

txnWriter := writer.WithTransaction(myTransaction)

Make sure to always create a new logger with a new logWriter for each transaction. This prevents the possibility of asynchronous processes accessing the same logger object with different contextual information.

txnLogger := log.New(txnWriter, "", log.Default().Flags())

Troubleshooting

The logWriter tool is designed to fail silently. To enable debugging information, call the DebugLogging(true) method on your logWriter object. When an error occurs in in the logWriter, it will always print your unedited log line on the first line. When DebugLogging is enabled, it will print an error message on the following line if an error occurs.

writer.DebugLogging(true)

The zerologWriter library is an io.Writer that automatically integrates the latest New Relic Logs in Context features into zerolog. Follow these steps to install it in your application. This library requires the installed version of the go-agent to be 3.19.1 or higher.

1. Add the zerologWriter package to your module.

```sh
go get github.com/newrelic/go-agent/v3/integrations/logcontext-v2/zerologWriter
```
  1. Import the zerologWriter package in the file where you initialize your zerolog logger.

    import (
      "log"
    
      "github.com/newrelic/go-agent/v3/integrations/logcontext-v2/zerologWriter"
    )
  2. Create a zerologWriter object. It must be passed a valid io.Writer, which is where logs will be written to, and an initialized go-agent application. The following example writes to standard out.

    writer := logWriter.New(os.Stdout, newRelicApp)
  3. Create a logger object with your new zerologWriter object as the logger's output destination.

    logger := zerolog.New(writer)

At this point, any log written with the logger created will be handled by the go-agent based on your logging config settings.

Transactions

If you want to capture the context of a transaction, a new zerologWriter object and logger object must be created. To create a new zerologWriter object for your transaction use either the WithTransaction() or WithContext() functions depending on how the transaction is passed to your function. These functions will make a new copy of the original zerologWriter object with the transaction context included.

If your function receives a context wrapped with a transaction, use the WithContext() function:

txnWriter := writer.WithContext(myWrappedContext)

If your function receives a transaction pointer, then use the WithTransaction() function:

txnWriter := writer.WithTransaction(myTransaction)

Make sure to always create a new logger with a new zerologWriter for each transaction. This prevents the possibility of asynchronous processes accessing the same logger object with different contextual information.

txnLogger := logger.Output(txnWriter)

Troubleshooting

The zerologWriter tool is designed to fail silently. To enable debugging information, call the DebugLogging(true) method on your zerologWriter object. When an error occurs in in the zerologWriter, it will always print your unedited log line on the first line. When DebugLogging is enabled, it will print an error message on the following line if an error occurs.

writer.DebugLogging(true)

The nrlogrus plugin enables automatic logs in context ingestion with the logrus logging framework. Once your application is configured to use it, the Go agent will automatically ingest any logs written in logrus. This requires Go agent version 3.18.0 or higher.

  1. Add the nrlogrus package to your module.

    go get github.com/newrelic/go-agent/v3/integrations/logcontext-v2/nrlogrus
    1. Import the nrlogrus package in the file where you initialize your logrus logger.
    import (
      "github.com/newrelic/go-agent/v3/integrations/logcontext-v2/nrlogrus"
    
      "github.com/sirupsen/logrus"
    )
  2. Configure and create a new nrlogrus formatter. It must be passed a valid application, and any valid logrus formatter. The formatter passed will determine the appearance of the logs that get written.

    nrlogrusFormatter := nrlogrus.NewFormatter(newRelicApplication, &logrus.TextFormatter{})
  3. Set your logger's formatter to the newly create nrlogrus formatter.

    log := logrus.New()
    log.SetFormatter(nrlogrusFormatter)

At this point, any log written with the logger created will be handled by the go-agent based on your logging config settings.

Transactions

When instrumenting logs inside of a transaction, you must pass that transaction to logrus as a context. This will create a new logger object for that transaction that links logs handled by the agent to the transaction and spans they occured in.

If you have a context that contains a transaction:

txnLogger := log.WithContext(yourWrappedContext)

Otherwise, you can create a context and pass it to logrus:

txnLogger := log.WithContext(newrelic.NewContext(context.Background(), txn))

The nrzap plugin enables automatic logs in context ingestion with the Zap logging framework. Once your application is configured to use it, the go-agent will automatically ingest any logs written in Zap. This requires the installed go-agent version to be 3.22.1 or higher.

  1. Add the nrzap package to your module.

    go get github.com/newrelic/go-agent/v3/integrations/logcontext-v2/nrzap
  2. Import the nrzap package in the file where you initialize your Zap logger.

    import (
      "github.com/newrelic/go-agent/v3/integrations/logcontext-v2/nrzap"
    
      "go.uber.org/zap"
        "go.uber.org/zap/zapcore"
    )
  3. Configure and create a new Zap core object for your applications logging. Because the core object never needs to change, and only needs to be wrapped, it's a good practice to keep a pointer to this object available so you can re-use it to create wrapped Zap cores more efficiently.

    core := zapcore.NewCore(zapcore.NewJSONEncoder(zap.NewProductionEncoderConfig()), zapcore.AddSync(os.Stdout), zap.InfoLevel)
  4. Wrap the Zap core object to capture and send logs to New Relic. Note that as long as the wrap function is passed a valid Zap core, the core it returns will be valid, even if an error is returned. For this reason, it's important to check the error type.

    backgroundCore, err := nrzap.WrapBackgroundCore(core, app)
    if err != nil && err != nrzap.ErrNilApp {
      panic(err)
    }
    
    backgroundLogger := zap.New(backgroundCore)

Transactions

When instrumenting logs inside of a transaction, you will need to create a new Zap core and logger. This can be done by calling the WrapTransactionCore() function.

txn := app.StartTransaction("nrzap example transaction")
  txnCore, err := nrzap.WrapTransactionCore(core, txn)
  if err != nil && err != nrzap.ErrNilTxn {
  	panic(err)
  }

  txnLogger := zap.New(txnCore)

The nrslog plugin enables automatic logs in context ingestion with the standard library slog framework. Once your application is configured to use it, the Go agent will automatically ingest any logs written with your wrapped slog logger. This requires Go agent version 3.30.0 or higher.

  1. Add the nrslog package to your module.

    go get github.com/newrelic/go-agent/v3/integrations/logcontext-v2/nrslog
  2. Import the nrslog package in the file where you initialize your slog logger.

    import (
      "github.com/newrelic/go-agent/v3/integrations/logcontext-v2/nrslog"
    
      "log/slog"
    )
  3. Configure and create a new nrslog handler. It must be passed a valid New Relilc application pointer, an io.Writer, and a pointer to an slog.HandlerOptions object. Convenience functions are packaged in nrslog to automatically instrument the text and json handlers that come out of the box in slog. In this example, we will wrap slog.TextHandler.

    instrumentedTextHandler := nrslog.TextHandler(app, os.Stdout, &slog.HandlerOptions{})
  4. Create a new logger with your instrumented handler.

    logger := slog.New(instrumentedTextHandler)

At this point, any log written with this new logger will be handled by the go-agent based on your logging config settings.

Transactions

When instrumenting logs inside of a transaction, you must pass that transaction to nrslog. The preferred way to do this is by passing it inside of a context to slog. If you have a context that contains a transaction:

txnCtx := newrelic.NewContext(context.Background(), txn)
logger.InfoContext(txnCtx, "My log message")

Secure your data [#obfuscation]

Your logs may include sensitive information protected by HIPAA or other compliance protocols. By default we obfuscate number patterns that appear to be for items such as credit cards or Social Security numbers, but you may need to hash or mask additional information.

For more information, see our documentation about obfuscation expressions and rules. You can hash or mask your log data by using the New Relic UI or by using NerdGraph, our GraphQL API.

What's next? [#what-next]

After you set up logs in context, make the most of your logging data: