Skip to content
This repository has been archived by the owner on Jul 10, 2023. It is now read-only.

thundra-io/thundra-agent-nodejs

Repository files navigation

Thundra Node.js Agent

OpenTracing Badge Coverage Status CircleCI

Trace your marvelous nodejs projects with async monitoring by Thundra!

Check out example projects for a quick start and Thundra docs for more information.

Contents

Installation

npm install @thundra/core --save

Configuration

You can configure Thundra using environment variables or module initialization parameters.

Environment variables have higher precedence over initialization parameters.

Check out the configuration part of our docs for more detailed information.

1. Most Useful Environment variables

Name Type Default Value
THUNDRA_APIKEY string -
THUNDRA_AGENT_APPLICATION_NAME string -
THUNDRA_AGENT_APPLICATION_STAGE string -
THUNDRA_AGENT_TRACE_DISABLE bool false
THUNDRA_AGENT_METRIC_DISABLE bool true
THUNDRA_AGENT_LOG_DISABLE bool true
THUNDRA_AGENT_TRACE_REQUEST_SKIP bool false
THUNDRA_AGENT_TRACE_RESPONSE_SKIP bool false
THUNDRA_AGENT_LAMBDA_TIMEOUT_MARGIN number -
THUNDRA_AGENT_REPORT_REST_BASEURL string https://collector.thundra.io/v1
THUNDRA_AGENT_REPORT_CLOUDWATCH_ENABLE bool false

Usage

Integration Options for Containers and VMs

export THUNDRA_APIKEY=<YOUR-THUNDRA-API-KEY>
export THUNDRA_AGENT_APPLICATION_NAME=<YOUR-APP-NAME>

For Dockerfile, you just replace export with ENV.

For more information see the doc

Express

const thundra = require("@thundra/core");
const express = require('express');

const app = express();

app.get('/', function (req,res) {
   res.send("Response")
});
app.listen(3000);

Hapi

const thundra = require("@thundra/core");
const Hapi = require('@hapi/hapi');

thundra.init();

const startServer = async () => {
    const server = Hapi.server({
        ...
    });

    server.route([{
        method: 'GET',
        path: '/',
        handler: (request, h) => {
            return 'Response';
        }
    }]);
    
    await server.start();
}

startServer();

Koa

const thundra = require("@thundra/core");
const Koa = require('koa');

thundra.init();
const app = new Koa();

app.use(async (ctx, next) => {
  await next();
  ctx.body = 'Hello Thundra!';
});
app.listen(3000)

Google Pubsub

Publish
const { PubSub } = require('@google-cloud/pubsub');

const projectId = 'your_google_cloud_project_id';
const topicName = 'your_google_cloud_pubsub_topic';

const pubsub = new PubSub({ projectId });

/* 
* if topic allready exists 
* const topic = await pubsub.topic(topicName)
**/
const topic = await pubsub.createTopic(topicName);

const date = new Date().toString();
const dataBuffer = Buffer.from(JSON.stringify({date}));

const result = await topic.publishMessage({ data: dataBuffer });
Subscription
Asynchronous Pull
const thundra = require("@thundra/core");
thundra.init();

const { PubSub, Subscription } = require('@google-cloud/pubsub');

const projectId = 'your_google_cloud_project_id';
const topicName = 'your_google_cloud_pubsub_topic';
const subscriptionName = 'your_google_cloud_pubsup_subscription_name';

const pubsub = new PubSub({ projectId });

(async() => {
    
    /* 
    * if subscription allready exists 
    * const subscription = pubsub.subscription(subscriptionName);
    **/
    const [subscription] = await pubsub.topic(topicName).createSubscription(subscriptionName);
    
    const messageHandler = message => {
      try {
        ...
        message.ack();
      } catch (err) {
        ...
        message.nack();
      }
    };
    
    subscription.on(`message`, messageHandler);
})().catch(error => console.log(error));
Synchronous Pull
const { v1 } = require('@google-cloud/pubsub');

const subClient = new v1.SubscriberClient();

const projectId = 'your_google_cloud_project_id';
const subscriptionName = 'your_google_cloud_pubsup_subscription_name';

const formattedSubscription = subClient.subscriptionPath(
  projectId,
  subscriptionName
);

const request = {
  subscription: formattedSubscription,
  maxMessages: 10,
};

...

const result = await subClient.pull(request);
const [response] = result;

const ackIds = [];
for (const message of response.receivedMessages) {
  ...
  ackIds.push(message.ackId);
}

if (ackIds.length !== 0) {
  const ackRequest = {
    subscription: formattedSubscription,
    ackIds: ackIds,
  };

  await subClient.acknowledge(ackRequest);
}
...

Integration Options for AWS Lambda

Using Layers

Integrating Thundra using AWS Lambda Layers is the recommended (and easier) way to get started with Thundra. For latest layer version(layer arn) and details of the integration see the doc

Without Layers

Just require this module, pass your api key to it and wrap your handler:

const thundra = require("@thundra/core")({ apiKey: "<YOUR-THUNDRA-API-KEY>" });

exports.handler = thundra((event, context,callback) => {
    callback(null, "Hello Thundra!");
});

Thundra will monitor your AWS lambda function and report automatically!

context.done, context.succeed and context.fail are also supported:

const thundra = require("@thundra/core")({ apiKey: "<YOUR-THUNDRA-API-KEY>" });

exports.handler = thundra((event, context) => {
    context.succeed("Hello Thundra!");
});

NOTES

  • In order to activate AWS Step Functions trace, THUNDRA_AGENT_LAMBDA_AWS_STEPFUNCTIONS environment variable should be set true.
  • In order to activate AWS AppSync trace, THUNDRA_AGENT_LAMBDA_AWS_APPSYNC environment variable should be set true.
  • For other integrations' configuration, please take a look environment variables table at the end.

Frameworks

The following frameworks are supported by Thundra:

Framework Supported Version Auto-tracing Supported
AWS Lambda All
Express >=3.0.0
Hapi >=16.0.0
  • - [âś“]
Koa >=2.0.0
  • - [âś“]

Integrations

Thundra provides out-of-the-box instrumentation (tracing) for following libraries.

Library Supported Version
logging Fully supported
aws-sdk >=2.0.0
elasticsearch >=10.5.0
http Fully supported
https Fully supported
http2 Fully supported
ioredis >=2.0.0
redis >=2.6.0
mongodb >=1.0.0
mysql >=2.0.0
mysql2 >=1.5.0
pg >=6.0.0
amqp 0.9.1 >=0.5.0
@google-cloud/pubsub >=1.2
@google-cloud/bigquery >=5.0

Async Monitoring with Zero Overhead

By default, Thundra agent reports by making an HTTPS request. This adds an overhead to your lambda function.

Instead, you can setup async monitoring in 2 minutes and monitor your lambda functions with zero overhead!

Check out our async monitoring example at our example projects for a quick start.

Log Support

You can monitor your logs using Thundra and enjoy the three pillars of observability in one place!

const thundra = require("@thundra/core");

const logger = thundra.createLogger();

exports.handler = thundra({
    apiKey: "MY_APIKEY",
})((event, context, callback) => {
    logger.info("Hello %s", "Thundra");
    callback(null, "Hello Thundra!");
});

You can also set the name of a logger while creating it (default name is default):

const logger = thundra.createLogger({loggerName: "Bob"});

Logger's name will be visible in Thundra's trace chart.

How to use Thundra loggers

You can log by two different ways.

1. Using trace, debug, info, warn, error, fatal methods

All these methods support printf-like format. Same as Node's util.format.

const thundra = require("@thundra/core");
const logger = thundra.createLogger();

logger.trace("Hey, I %s things", "trace");
logger.debug("Someone is %s %d"," debugging", 2);
logger.info("Get some info","and more");
logger.warn("I am warning you %s", "!!!");
logger.error("Error Error Error...");
logger.fatal("FATALITY");

2. Using log method

Pass an object with level and message fields:

const thundra = require("@thundra/core");
const logger = thundra.createLogger();

logger.log({
    level: "trace",
    message: "Hey, I am tracing."
});

You can also pass level as a string, this way you can use printf-like formatting:

logger.log("trace", "Hey, I am %s", "tracing.");

level can be one of the following: "trace", "debug", "info", "warn", "error", "fatal"

Log Levels

In increasing precedence: trace, debug, info, warn, error, fatal.

You can set the log level by setting the environment variable THUNDRA_AGENT_LOG_LOGLEVEL to one of the following:

  • trace
  • debug
  • info
  • warn
  • error
  • fatal
  • none

For instance, if THUNDRA_AGENT_LOG_LOGLEVEL is:

  • debug, only debug and higher precedence logs will be reported.
  • none, none of the logs will be reported.

Trace ColdStart

You can enable coldstart tracing by setting THUNDRA_AGENT_LAMBDA_TRACE_COLDSTART_ENABLE environment variable to true (which is disabled by default). When coldstart tracing is enabled, during coldstart (first invocation of the Lambda container), imports/requires are traced, and they are reported with their module names, file names, types and durations under the init span of the invocation trace chart to give you insights about which modules are the highest coldstart overhead.

By default, top 100 imports/requires with the highest duration are reported (others are filtered out) and top import/require limit can be configured by setting THUNDRA_AGENT_LAMBDA_TRACE_COLDSTART_MODULE_LOAD_DURATION_TOP environment variable (which is 100 by default as mentioned).

Another traced import/require limit is the depth. By default, imports/requires are traced until depth 10 at max and the max depth can be configured by setting THUNDRA_AGENT_LAMBDA_TRACE_COLDSTART_MODULE_LOAD_DEPTH_MAX environment variable (which is 10 by default as mentioned).

Another optional import/require filtering mechanism is duration. Even though there is no duration based filtering applied by default, you can specify minimum duration for imports/requires to be traced by setting THUNDRA_AGENT_LAMBDA_TRACE_COLDSTART_MODULE_LOAD_DURATION_MIN environment variable.

Mask Sensitive Data

You can specify the keys to be masked in the trace by passing the key names (separated by comma (,) if there are multiple) through the THUNDRA_AGENT_REPORT_MASKED_KEYS environment variable. Here, key names can be string for exact match or regexp pattern.

For example,

THUNDRA_AGENT_REPORT_MASKED_KEYS=password

masks all the keys/properties whose names exactly match to password.

As another example,

THUNDRA_AGENT_REPORT_MASKED_KEYS=/.*password.*/

masks all the keys/properties whose names contain (partially match) password.

If there are multiple key names or patterns you want to specify, you can separate them by comma (,).

THUNDRA_AGENT_REPORT_MASKED_KEYS=/.*password.*/,/.*secret.*/

By default, masked data is replaced with *****. But if you want to remove the masked data completely, you can set THUNDRA_AGENT_REPORT_HIDE environment variable to true.

Warmup Support

You can cut down cold starts easily by deploying our lambda function thundra-lambda-warmup.

By default, Thundra agent doesn't recognize warmup requests by default, but you can enable it by setting THUNDRA_AGENT_LAMBDA_WARMUP_WARMUPAWARE environment variable to true

Check out this part in our docs for more information.

All Environment Variables

Name Type Default Value Description
THUNDRA_APIKEY string -
THUNDRA_AGENT_DISABLE bool false
THUNDRA_AGENT_DEBUG_ENABLE bool false
THUNDRA_AGENT_TRACE_DISABLE bool false
THUNDRA_AGENT_METRIC_DISABLE bool true
THUNDRA_AGENT_LOG_DISABLE bool true
THUNDRA_AGENT_REPORT_REST_BASEURL string https://collector.thundra.io/v1
THUNDRA_AGENT_REPORT_REST_TRUSTALLCERTIFICATES bool false
THUNDRA_AGENT_REPORT_REST_LOCAL bool false
THUNDRA_AGENT_REPORT_CLOUDWATCH_ENABLE bool false
THUNDRA_AGENT_REPORT_SIZE_MAX number 32 * 1024 (32 KB)
THUNDRA_AGENT_REPORT_MASKED_KEYS string - Comma (,) separated key names (can be string or regexp) to be masked in the trace
THUNDRA_AGENT_REPORT_HIDE bool false Hides masked keys instead of masking them
THUNDRA_AGENT_LAMBDA_HANDLER string -
THUNDRA_AGENT_LAMBDA_WARMUP_WARMUPAWARE bool false
THUNDRA_AGENT_LAMBDA_TIMEOUT_MARGIN number -
THUNDRA_AGENT_LAMBDA_ERROR_STACKTRACE_MASK bool false
THUNDRA_AGENT_TRACE_REQUEST_SKIP bool false
THUNDRA_AGENT_TRACE_RESPONSE_SKIP bool false
THUNDRA_AGENT_LAMBDA_TRACE_KINESIS_REQUEST_ENABLE bool false
THUNDRA_AGENT_LAMBDA_TRACE_FIREHOSE_REQUEST_ENABLE bool false
THUNDRA_AGENT_LAMBDA_TRACE_CLOUDWATCHLOG_REQUEST_ENABLE bool false
THUNDRA_AGENT_LAMBDA_TRACE_COLDSTART_ENABLE bool false
THUNDRA_AGENT_LAMBDA_TRACE_MODULE_LOAD_DURATION_TOP number 100
THUNDRA_AGENT_LAMBDA_TRACE_MODULE_LOAD_DURATION_MIN number -1
THUNDRA_AGENT_LAMBDA_TRACE_MODULE_LOAD_DEPTH_MAX number 10
THUNDRA_AGENT_LAMBDA_AWS_STEPFUNCTIONS bool false
THUNDRA_AGENT_LAMBDA_AWS_APPSYNC bool false
THUNDRA_AGENT_APPLICATION_ID string -
THUNDRA_AGENT_APPLICATION_INSTANCEID string -
THUNDRA_AGENT_APPLICATION_REGION string -
THUNDRA_AGENT_APPLICATION_NAME string -
THUNDRA_AGENT_APPLICATION_STAGE string -
THUNDRA_AGENT_APPLICATION_DOMAINNAME string -
THUNDRA_AGENT_APPLICATION_CLASSNAME string -
THUNDRA_AGENT_APPLICATION_VERSION string -
THUNDRA_AGENT_APPLICATION_TAG any -
THUNDRA_AGENT_INVOCATION_SAMPLE_ONERROR bool false
THUNDRA_AGENT_INVOCATION_REQUEST_TAGS string -
THUNDRA_AGENT_INVOCATION_RESPONSE_TAGS string -
THUNDRA_AGENT_TRACE_INSTRUMENT_DISABLE bool false
THUNDRA_AGENT_TRACE_INSTRUMENT_TRACEABLECONFIG string -
THUNDRA_AGENT_TRACE_INSTRUMENT_FILE_PREFIX string -
THUNDRA_AGENT_TRACE_SPAN_LISTENERCONFIG string -
THUNDRA_AGENT_TRACE_SPAN_COUNT_MAX number 200
THUNDRA_AGENT_SAMPLER_TIMEAWARE_TIMEFREQ number 300000
THUNDRA_AGENT_SAMPLER_COUNTAWARE_COUNTFREQ number 100
THUNDRA_AGENT_TRACE_INTEGRATIONS_DISABLE bool false
THUNDRA_AGENT_TRACE_INTEGRATIONS_AWS_INSTRUMENT_ONLOAD bool false
THUNDRA_AGENT_TRACE_INTEGRATIONS_AWS_SNS_MESSAGE_MASK bool false
THUNDRA_AGENT_TRACE_INTEGRATIONS_AWS_SNS_TRACEINJECTION_DISABLE bool false
THUNDRA_AGENT_TRACE_INTEGRATIONS_AWS_SQS_MESSAGE_MASK bool false
THUNDRA_AGENT_TRACE_INTEGRATIONS_AWS_SQS_TRACEINJECTION_DISABLE bool false
THUNDRA_AGENT_TRACE_INTEGRATIONS_AWS_LAMBDA_PAYLOAD_MASK bool false
THUNDRA_AGENT_TRACE_INTEGRATIONS_AWS_LAMBDA_TRACEINJECTION_DISABLE bool false
THUNDRA_AGENT_TRACE_INTEGRATIONS_AWS_DYNAMODB_STATEMENT_MASK bool false
THUNDRA_AGENT_TRACE_INTEGRATIONS_AWS_DYNAMODB_RESULT_MASK bool false
THUNDRA_AGENT_TRACE_INTEGRATIONS_AWS_DYNAMODB_TRACEINJECTION_ENABLE bool false
THUNDRA_AGENT_TRACE_INTEGRATIONS_AWS_ATHENA_STATEMENT_MASK bool false
THUNDRA_AGENT_TRACE_INTEGRATIONS_HTTP_BODY_MASK bool false
THUNDRA_AGENT_TRACE_INTEGRATIONS_HTTP_BODY_SIZE_MAX number 10 * 1024 (10 KB)
THUNDRA_AGENT_TRACE_INTEGRATIONS_HTTP_HEADERS_MASK bool false
THUNDRA_AGENT_TRACE_INTEGRATIONS_HTTP_RESPONSE_BODY_MASK bool true
THUNDRA_AGENT_TRACE_INTEGRATIONS_HTTP_RESPONSE_BODY_SIZE_MAX number 10 * 1024 (10 KB)
THUNDRA_AGENT_TRACE_INTEGRATIONS_HTTP_RESPONSE_HEADERS_MASK bool false
THUNDRA_AGENT_TRACE_INTEGRATIONS_HTTP_URL_DEPTH number 1
THUNDRA_AGENT_TRACE_INTEGRATIONS_HTTP_TRACEINJECTION_DISABLE bool false
THUNDRA_AGENT_TRACE_INTEGRATIONS_HTTP_ERROR_ON4XX_DISABLE bool false
THUNDRA_AGENT_TRACE_INTEGRATIONS_HTTP_ERROR_ON5XX_DISABLE bool false
THUNDRA_AGENT_TRACE_INTEGRATIONS_REDIS_COMMAND_MASK bool false
THUNDRA_AGENT_TRACE_INTEGRATIONS_RDB_STATEMENT_MASK bool false
THUNDRA_AGENT_TRACE_INTEGRATIONS_RDB_RESULT_MASK bool false
THUNDRA_AGENT_TRACE_INTEGRATIONS_ELASTICSEARCH_BODY_MASK bool false
THUNDRA_AGENT_TRACE_INTEGRATIONS_ELASTICSEARCH_PATH_DEPTH number 1
THUNDRA_AGENT_TRACE_INTEGRATIONS_MONGODB_COMMAND_MASK bool false
THUNDRA_AGENT_TRACE_INTEGRATIONS_AWS_EVENTBRIDGE_DETAIL_MASK bool false
THUNDRA_AGENT_TRACE_INTEGRATIONS_AWS_SES_MAIL_MASK bool false
THUNDRA_AGENT_TRACE_INTEGRATIONS_AWS_SES_MAIL_DESTINATION_MASK bool false
THUNDRA_AGENT_TRACE_INTEGRATIONS_RABBITMQ_MESSAGE_MASK bool false
THUNDRA_AGENT_TRACE_INTEGRATIONS_GOOGLE_PUBSUB_MESSAGE_MASK bool false
THUNDRA_AGENT_TRACE_INTEGRATIONS_GOOGLE_BIGQUERY_RESPONSE_SIZE_MAX number 1 * 1024 (1 KB)
THUNDRA_AGENT_TRACE_INTEGRATIONS_GOOGLE_BIGQUERY_QUERY_MASK bool false
THUNDRA_AGENT_TRACE_INTEGRATIONS_GOOGLE_BIGQUERY_RESPONSE_MASK bool false
THUNDRA_AGENT_LOG_CONSOLE_DISABLE bool false
THUNDRA_AGENT_LOG_LOGLEVEL string TRACE
THUNDRA_AGENT_LAMBDA_DEBUGGER_ENABLE bool false
THUNDRA_AGENT_LAMBDA_DEBUGGER_PORT number 1111
THUNDRA_AGENT_LAMBDA_DEBUGGER_LOGS_ENABLE bool false
THUNDRA_AGENT_LAMBDA_DEBUGGER_WAIT_MAX number 60000
THUNDRA_AGENT_LAMBDA_DEBUGGER_IO_WAIT number 60000
THUNDRA_AGENT_LAMBDA_DEBUGGER_BROKER_PORT number 444
THUNDRA_AGENT_LAMBDA_DEBUGGER_BROKER_HOST string debug.thundra.io
THUNDRA_AGENT_LAMBDA_DEBUGGER_SESSION_NAME string default
THUNDRA_AGENT_LAMBDA_DEBUGGER_AUTH_TOKEN string -
THUNDRA_AGENT_TRACE_INTEGRATIONS_HAPI_DISABLE bool false
THUNDRA_AGENT_TRACE_INTEGRATIONS_KOA_DISABLE bool false
THUNDRA_AGENT_TRACE_INTEGRATIONS_GOOGLE_PUBSUB_DISABLE bool false

Module initialization parameters

Name Type Default Value
apiKey string -
disableThundra bool false
plugins array [ ]

How to build

Webpack is used as a module bundler.

To build the project,

npm install
npm run build

How to test

Tests are written using Jest.

To run tests,

npm run test

Changelog

Please see the CHANGELOG file.