temporal-newrelic

A couple of useful classes to integrate NewRelic into Temporal Java SDK applications

Workflow & Activity executions as NewRelic Transaction

By default, workflow execution and activity methods will not be traced by NewRelic (like it happens with web requests), so manual instrumentation is required to hint NewRelic about this kind of "background executions".

In order to support this without messing up business logic code, Temporal has an interface to intercept calls to activities and workflows by wrapping those executions with interceptors implementing WorkflowInboundCallsInterceptor and ActivityInboundCallsInterceptor. By implementing those interface and adding required NewRelic instrumentation, we will achieve visibility of those executions in NewRelic platform.

To enable this, we only need to configure TraceWorkerInterceptor when configuring WorkerFactory, like this:

WorkflowClient workflowClient = ...
var factory =
    WorkerFactory.newInstance(
        workflowClient,
        WorkerFactoryOptions.newBuilder()
            .setWorkerInterceptors(new TraceWorkerInterceptor())
            .build());

Moreover, if Distributed Tracing is enabled for your application, it will automatically use it, see below.

Distributed Tracing

Temporal has an interface to enable context information propagation between executions (threads). By implementing this interface we can pass through all Distributed Tracing information. This includes not only from the client to the workers, but also between workflows and activities.

To enable this you only need to add NewRelicDistributedTraceContextPropagator to ``, during initialization of workflow clients like this:

 WorkflowServiceStubs service = ...
 var workflowClientOptionsBuilder = WorkflowClientOptions.newBuilder();
 workflowClientOptionsBuilder.setContextPropagators(
      List.of(new NewRelicDistributedTraceContextPropagator()));
 WorkflowClient.newInstance(service, workflowClientOptionsBuilder.validateAndBuildWithDefaults())

Any unhandled error happening inside NewRelicDistributedTraceContextPropagator will probably mess up the workflow execution, so I do recommend to wrap this propagator with SilentWrapperContextPropagator to log and discard any unhandled error related with propagation (not the workflow execution itself), like this:

var propagator = SilentWrapperContextPropagator.wrap(new NewRelicDistributedTraceContextPropagator());

Metric Scope

Temporal also has the capability to report its internal metrics to external services. To be able to send this information to NewRelic as custom metrics you can configure MetricScopeReporter as following:

WorkflowServiceStubs service = WorkflowServiceStubs.newServiceStubs(WorkflowServiceStubsOptions.newBuilder()
            .setMetricsScope(MetricScopeReporter.getScope())
            .validateAndBuildWithDefaults());

This will produce metrics starting with the pattern Custom/temporalio/. At the moment there is support for:

• timer: Custom/temporalio/timer
• gauge: Custom/temporalio/gauge
• counter: Custom/temporalio/counter

After that is appended the following relevant information:

• namespace
• worker_type
• task_queue
• workflow_type
• operation_name
• signal_name
• activity_type
• query_type
• exception
• status_code

If any of this information is not present on the reported record, the word none will take its place instead of null. For example:

MetricScopeReporter.instance().reportCounter("name", Map.of(), 1L);

will append the following metric: Custom/temporalio/timer/none/none/none/none/none/none/none/none/none/none/name with the given value.

Furthermore, timers will be reported using NewRelic.recordResponseTimeMetric, counters with NewRelic.recordResponseTimeMetric and finally gauges with NewRelic.recordMetric.

How to install

If you prefer to use maven central releases, you can find it here. Also, if you support Jitpack.io you can find it here