GUIDE.md

New Relic Go Agent Guide

• Installation
• Config and Application
• Logging
  • logrus
• Transactions
• Segments
  • Datastore Segments
  • External Segments
• Attributes
• Cross Application Tracing
  • Upgrading Applications to Support Cross Application Tracing
• Custom Metrics
• Custom Events
• Request Queuing

Installation

Installing the Go Agent is the same as installing any other Go library. The simplest way is to run:

go get github.com/newrelic/go-agent

Then import the github.com/newrelic/go-agent package in your application.

Config and Application

• config.go
• application.go

In your main function or in an init block:

config := newrelic.NewConfig("Your Application Name", "__YOUR_NEW_RELIC_LICENSE_KEY__")
app, err := newrelic.NewApplication(config)

Find your application in the New Relic UI. Click on it to see the Go runtime tab that shows information about goroutine counts, garbage collection, memory, and CPU usage.

If you are working in a development environment or running unit tests, you may not want the Go Agent to spawn goroutines or report to New Relic. You're in luck! Set the config's Enabled field to false. This makes the license key optional.

config := newrelic.NewConfig("Your Application Name", "")
config.Enabled = false
app, err := newrelic.NewApplication(config)

Logging

• log.go

The agent's logging system is designed to be easily extensible. By default, no logging will occur. To enable logging, assign the Config.Logger field to something implementing the Logger interface. A basic logging implementation is included.

To log at debug level to standard out, set:

config.Logger = newrelic.NewDebugLogger(os.Stdout)

To log at info level to a file, set:

w, err := os.OpenFile("my_log_file", os.O_CREATE|os.O_APPEND|os.O_WRONLY, 0644)
if nil == err {
  config.Logger = newrelic.NewLogger(w)
}

logrus

• _integrations/nrlogrus/nrlogrus.go

If you are using logrus and would like to send the agent's log messages to its standard logger, import the github.com/newrelic/go-agent/_integrations/nrlogrus package, then set:

config.Logger = nrlogrus.StandardLogger()

Transactions

• transaction.go
• Naming Transactions
• More info on Transactions

Transactions time requests and background tasks. Each transaction should only be used in a single goroutine. Start a new transaction when you spawn a new goroutine.

The simplest way to create transactions is to use Application.StartTransaction and Transaction.End.

txn := app.StartTransaction("transactionName", responseWriter, request)
defer txn.End()

If the response writer is provided when calling StartTransaction, you can then use txn.WriteHeader as a drop in replacement for the standard library's http.ResponseWriter.WriteHeader function. We strongly recommend doing so, as this both enables Cross Application Tracing support and ensures that attributes are added to the Transaction event capturing the response size and status code.

The response writer and request parameters are optional. Leave them nil to instrument a background task.

txn := app.StartTransaction("backgroundTask", nil, nil)
defer txn.End()

The transaction has helpful methods like NoticeError and SetName. See more in transaction.go.

If you are using http.ServeMux, use WrapHandle and WrapHandleFunc. These wrappers automatically start and end transactions with the request and response writer. See instrumentation.go.

http.HandleFunc(newrelic.WrapHandleFunc(app, "/users", usersHandler))

To access the transaction in your handler, use type assertion on the response writer passed to the handler.

func myHandler(w http.ResponseWriter, r *http.Request) {
	if txn, ok := w.(newrelic.Transaction); ok {
		txn.NoticeError(errors.New("my error message"))
	}
}

Segments

• segments.go

Find out where the time in your transactions is being spent! Each transaction should only track segments in a single goroutine.

Segment is used to instrument functions, methods, and blocks of code. A segment begins when its StartTime field is populated, and finishes when its End method is called.

segment := newrelic.Segment{}
segment.Name = "mySegmentName"
segment.StartTime = newrelic.StartSegmentNow(txn)
// ... code you want to time here ...
segment.End()

StartSegment is a convenient helper. It creates a segment and starts it:

segment := newrelic.StartSegment(txn, "mySegmentName")
// ... code you want to time here ...
segment.End()

Timing a function is easy using StartSegment and defer. Just add the following line to the beginning of that function:

defer newrelic.StartSegment(txn, "mySegmentName").End()

Segments may be nested. The segment being ended must be the most recently started segment.

s1 := newrelic.StartSegment(txn, "outerSegment")
s2 := newrelic.StartSegment(txn, "innerSegment")
// s2 must be ended before s1
s2.End()
s1.End()

A zero value segment may safely be ended. Therefore, the following code is safe even if the conditional fails:

var s newrelic.Segment
if txn, ok := w.(newrelic.Transaction); ok {
	s.StartTime = newrelic.StartSegmentNow(txn),
}
// ... code you wish to time here ...
s.End()

Datastore Segments

Datastore segments appear in the transaction "Breakdown table" and in the "Databases" tab.

• datastore.go
• More info on Databases tab

Datastore segments are instrumented using DatastoreSegment. Just like basic segments, datastore segments begin when the StartTime field is populated and finish when the End method is called. Here is an example:

s := newrelic.DatastoreSegment{
	// Product is the datastore type.  See the constants in datastore.go.
	Product: newrelic.DatastoreMySQL,
	// Collection is the table or group.
	Collection: "my_table",
	// Operation is the relevant action, e.g. "SELECT" or "GET".
	Operation: "SELECT",
}
s.StartTime = newrelic.StartSegmentNow(txn)
// ... make the datastore call
s.End()

This may be combined into a single line when instrumenting a datastore call that spans an entire function call:

defer newrelic.DatastoreSegment{
	StartTime:  newrelic.StartSegmentNow(txn),
	Product:    newrelic.DatastoreMySQL,
	Collection: "my_table",
	Operation:  "SELECT",
}.End()

External Segments

External segments appear in the transaction "Breakdown table" and in the "External services" tab. Version 1.11.0 of the Go agent also adds support for Cross Application Tracing (CAT), which will result in external segments also appearing in the "Service maps" tab and being linked in transaction traces when both sides of the request have traces.

• More info on External Services tab
• More info on Cross Application Tracing

External segments are instrumented using ExternalSegment. There are three ways to use this functionality:

1. Using StartExternalSegment to create an ExternalSegment before the request is sent, and then calling ExternalSegment.End when the external request is complete.

   For CAT support to operate, an http.Request must be provided to StartExternalSegment, and the ExternalSegment.Response field must be set before ExternalSegment.End is called or deferred.

   For example:

   func external(txn newrelic.Transaction, req *http.Request) (*http.Response, error) {
     s := newrelic.StartExternalSegment(txn, req)
     response, err := http.DefaultClient.Do(req)
     s.Response = response
     s.End()
     return response, err
   }

2. Using NewRoundTripper to get a http.RoundTripper that will automatically instrument all requests made via http.Client instances that use that round tripper as their Transport. This option results in CAT support transparently, provided the Go agent is version 1.11.0 or later.

   For example:

   client := &http.Client{}
   client.Transport = newrelic.NewRoundTripper(txn, nil)
   resp, err := client.Get("http://example.com/")

   Note that, as with all segments, the round tripper returned must only be used in the same goroutine as the transaction.

3. Directly creating an ExternalSegment via a struct literal with an explicit URL or Request, and then calling ExternalSegment.End. This option does not support CAT, and may be removed or changed in a future major version of the Go agent. As a result, we suggest using one of the other options above wherever possible.

   For example:

   func external(txn newrelic.Transaction, url string) (*http.Response, error) {
     defer newrelic.ExternalSegment{
       StartTime: newrelic.StartSegmentNow(txn),
       URL:   url,
     }.End()
   
     return http.Get(url)
   }

Attributes

Attributes add context to errors and allow you to filter performance data in Insights.

You may add them using the Transaction.AddAttribute method.

txn.AddAttribute("key", "value")
txn.AddAttribute("product", "widget")
txn.AddAttribute("price", 19.99)
txn.AddAttribute("importantCustomer", true)

• More info on Custom Attributes

Some attributes are recorded automatically. These are called agent attributes. They are listed here:

• attributes.go

To disable one of these agents attributes, AttributeResponseCode for example, modify the config like this:

config.Attributes.Exclude = append(config.Attributes.Exclude, newrelic.AttributeResponseCode)

• More info on Agent Attributes

Cross Application Tracing

New Relic's Cross Application Tracing feature, or CAT for short, links transactions between applications in APM to help identify performance problems within your service-oriented architecture. Support for CAT was added in version 1.11.0 of the Go agent.

As CAT uses HTTP headers to track requests across applications, the Go agent needs to be able to access and modify request and response headers both for incoming and outgoing requests.

Upgrading Applications to Support Cross Application Tracing

Although many Go applications instrumented using older versions of the Go agent will not require changes to enable CAT support, we've prepared this checklist that you can use to ensure that your application is ready to take advantage of the full functionality offered by New Relic's CAT feature:

1. Ensure that incoming HTTP requests both parse any incoming CAT headers, and output the required outgoing CAT header:

   1. If you use WrapHandle or WrapHandleFunc to instrument a server that uses http.ServeMux, no changes are required.

   2. If you use either of the Go agent's Gin or Gorilla integrations, no changes are required.

   3. If you use another framework or http.Server directly, you will need to ensure that:

      1. All calls to StartTransaction include the response writer and request, and
      2. Transaction.WriteHeader is used instead of calling WriteHeader directly on the response writer, as described in the transactions section of this guide.

2. Convert any instances of using an ExternalSegment literal directly to either use StartExternalSegment or NewRoundTripper, as described in the external segments section of this guide.

3. Ensure that calls to StartExternalSegment provide an http.Request.

4. Ensure that the Response field is set on ExternalSegment values before making or deferring calls to ExternalSegment.End.

Custom Metrics

• More info on Custom Metrics

You may create custom metrics via the RecordCustomMetric method.

app.RecordCustomMetric(
	"CustomMetricName", // Name of your metric
	132,                // Value
)

Note: The Go Agent will automatically prepend the metric name you pass to RecordCustomMetric ("CustomMetricName" above) with the string Custom/. This means the above code would produce a metric named Custom/CustomMetricName. You'll also want to read over the Naming Transactions and Metrics section below for advice on coming up with appropriate metric names.

Custom Events

You may track arbitrary events using custom Insights events.

app.RecordCustomEvent("MyEventType", map[string]interface{}{
	"myString": "hello",
	"myFloat":  0.603,
	"myInt":    123,
	"myBool":   true,
})

Request Queuing

If you are running a load balancer or reverse web proxy then you may configure it to add a X-Queue-Start header with a Unix timestamp. This will create a band on the application overview chart showing queue time.

• More info on Request Queuing

Error Reporting

You may track errors using the Transaction.NoticeError method. The easiest way to get started with NoticeError is to use errors based on Go's standard error interface.

txn.NoticeError(errors.New("my error message"))

NoticeError will work with any sort of object that implements Go's standard error type interface -- not just errorStrings created via errors.New.

If you're interested in sending more than an error message to New Relic, the Go Agent also offers a newrelic.Error struct.

txn.NoticeError(newrelic.Error{
	Message: "my error message",
	Class:   "IdentifierForError",
	Attributes: map[string]interface{}{
		"important_number": 97232,
		"relevant_string":  "zap",
	},
})

Using the newrelic.Error struct requires you to manually marshall your error data into the Message, Class, and Attributes fields. However, there's two advantages to using the newrelic.Error struct.

First, by setting an error Class, New Relic will be able to aggregate errors in the Error Analytics section of APM. Second, the Attributes field allows you to send through key/value pairs with additional error debugging information (also exposed in the Error Analytics section of APM).

Advanced Error Reporting

You're not limited to using Go's built-in error type or the provided newrelic.Error struct. The Go Agent provides three error interfaces

type StackTracer interface {
	StackTrace() []uintptr
}

type ErrorClasser interface {
	ErrorClass() string
}

type ErrorAttributer interface {
	ErrorAttributes() map[string]interface{}
}

If you implement any of these on your own error structs, the txn.NoticeError method will recognize these methods and use their return values to provide error information.

For example, you could implement a custom error struct named MyErrorWithClass

type MyErrorWithClass struct {

}

Then, you could implement both an Error method (per Go's standard error interface) and an ErrorClass method (per the Go Agent ErrorClasser interface) for this struct.

func (e MyErrorWithClass) Error() string { return "A hard coded error message" }

// ErrorClass implements the ErrorClasser interface.
func (e MyErrorWithClass) ErrorClass() string { return "MyErrorClassForAggregation" }

Finally, you'd use your new error by creating a new instance of your struct and passing it to the NoticeError method

txn.NoticeError(MyErrorWithClass{})

While this is an oversimplified example, these interfaces give you a great deal of control over what error information is available for your application.

Naming Transactions and Metrics

You'll want to think carefully about how you name your transactions and custom metrics. If your program creates too many unique names, you may end up with a Metric Grouping Issue (or MGI).

MGIs occur when the granularity of names is too fine, resulting in hundreds or thousands of uniquely identified metrics and transactions. One common cause of MGIs is relying on the full URL name for metric naming in web transactions. A few major code paths may generate many different full URL paths to unique documents, articles, page, etc. If the unique element of the URL path is included in the metric name, each of these common paths will have its own unique metric name.

For More Help

There's a variety of places online to learn more about the Go Agent.

The New Relic docs site contains a number of useful code samples and more context about how to use the Go Agent.

New Relic's discussion forums have a dedicated public forum for the Go Agent.

When in doubt, the New Relic support site is the best place to get started troubleshooting an agent issue.