NewRelic Logback Appender

Appender implements HTTP integration with API provided by NewRelic (see Introduction to the Log API). Log API expects data in JSON format, which is generated by encoders and layouts from library Logstash Logback Encoder.

Dependencies

Appender depends on following components:

  <dependencies>
    <dependency>
      <groupId>ch.qos.logback</groupId>
      <artifactId>logback-classic</artifactId>
      <version>1.4.6</version>
    </dependency>
    <dependency>
      <groupId>net.logstash.logback</groupId>
      <artifactId>logstash-logback-encoder</artifactId>
      <version>7.3</version>
      <scope>runtime</scope>
    </dependency>
    <dependency>
      <groupId>com.fasterxml.uuid</groupId>
      <artifactId>java-uuid-generator</artifactId>
      <version>4.1.0</version>
      <scope>runtime</scope>
    </dependency>
  </dependencies>

Appender is built by Maven3 on Java17.

Usage

As the appender implements Logback's AppenderBase, it expects that an application uses Logback for logging. So Logback can be found in the path, or it is explicitly defined as a dependency, as shown below.

To use NewRelic Logback Appender in the application add it as a dependency into a project.

    <dependencies>
        <dependency>
            <groupId>ch.qos.logback</groupId>
            <artifactId>logback-classic</artifactId>
            <version>1.4.6</version>
        </dependency>
        <dependency>
            <groupId>ua.rapp</groupId>
            <artifactId>nr-logback-appender</artifactId>
            <version>0.1.0</version>
            <scope>runtime</scope>
        </dependency>
    </dependencies>

Then add the appender into logback.xml configuration file. The appender supports next configuration properties:

Property name	Description (default value)
* apiKey	Log API requires a valid API key to be sent along with log data. Register NewRelic account or use existing one.
* host	NewRelic address, like https://log-api.eu.newrelic.com for Europe. See details Introduction to the Log API.
* url	URL path to API, like /log/v1.
* layout	The appender uses net.logstash.logback.layout.LoggingEventCompositeJsonLayout to configure a log message structure in JSON format. Some details of the layout configuration see below. Documentation is on Composite Encoder/Layout.
maxPayloadSize	Limit of a raw (uncompressed) log size (bytes) to be sent to NewRelic (1024000). Also see limits imposed by NewRelic Log API on Introduction to the Log API.
maxUncompressedSize	The appender sends data compressed by GZIP if a log message is more than maxUncompressedSize (1024). Internally, there is used a limit of minimum 256 bytes for compressed data. So, if maxUncompressedSize is less, 256 bytes will be applied as a lower limit.
retryNumber	If the appender failed to send data to NewRelic, it will try resending the same data retryNumber times (3).
retryInterval	In a case of failure the interval in seconds between retries to resend data (2).

* - mandatory properties

Log message structure

NewRelic expects log data in JSON format with a mandatory message attribute. Other attributes are optional, though it is common to have in the logs also timestamp and level. These and other 'standard' fields supported by Logback can be added into a pattern of net.logstash.logback.layout.LoggingEventCompositeJsonLayout. The layout supports several providers that inject attributes directly into JSON message, for instance, timestamp, version, uuid. If log's MDC is not empty, the content of MDC's map is added into JSON as well.

Here it is an example of Logback configuration logback.xml:

<?xml version="1.0" encoding="UTF-8" ?>
<configuration>
    <appender name="NEWRELIC" class="ua.rapp.logback.appenders.NewRelicAppender">
        <host>${NR_HOST:-https://log-api.newrelic.com}</host>
        <url>${NR_URL:-/log/v1}</url>
        <apiKey>${NR_API_KEY:-set-valid-api-key}</apiKey>
        <layout class="net.logstash.logback.layout.LoggingEventCompositeJsonLayout">
            <providers>
                <timestamp/>
                <version/>
                <uuid/>
                <pattern>
                    <pattern>
                        {
                        "timestamp": "%date{ISO8601}",
                        "logger": "%logger",
                        "level": "%level",
                        "message": "%message"
                        }
                    </pattern>
                </pattern>
                <mdc/>
                <tags/>
            </providers>
        </layout>
    </appender>

    <root level="TRACE">
        <appender-ref ref="NEWRELIC"/>
    </root>
</configuration>

With SLF4J logger and MDC defined and used as the following:

private static final Logger LOG = LoggerFactory.getLogger("newrelic-logger-function");
// ...
MDC.put("invocationId", UUID.randomUUID().toString());
MDC.put("app-name", "console");
MDC.put("errorMessage", "Missed ID attribute!");
LOG.error("User [555] logged in failed.");

the resulting JSON will look like this:

{
  "@timestamp": "2023-04-22T14:09:07.2011033Z",
  "@version": "1",
  "app-name": "console",
  "errorMessage": "Missed ID attribute!",
  "invocationId": "0d16c981-d342-43f1-8b89-4e3e3fde8974",
  "level": "ERROR",
  "logger": "newrelic-logger-function",
  "message": "User [555] logged in failed.",
  "newrelic.source": "api.logs",
  "timestamp": 1682172547201,
  "uuid": "24212215-c075-4644-a122-15c075f6447f"
}

where attributes @timestamp, @version, and uuid are added by corresponding Logstash Logback layout providers <timestamp/>, <version/>, and <uuid/>. Attributes app-name, invocationId, and errorMessage are added by <mdc/> provider.