tracing not working after moving from commonjs to es2020

[guidev]

Expected behaviour

Actual behaviour

Steps to reproduce
We changed our typescript configuration from commonjs to es2020.

The resulting javascript now looks like this:

/* ~/config/datadog.js */

import tracer from 'dd-trace';
tracer.init({
    logInjection: true,
    profiling: true,
    appsec: true,
    runtimeMetrics: true
});
export default tracer;

previous commonjs version:

/* ~/config/datadog.js */

"use strict";
var __importDefault = (this && this.__importDefault) || function (mod) {
    return (mod && mod.__esModule) ? mod : { "default": mod };
};
Object.defineProperty(exports, "__esModule", { value: true });
const dd_trace_1 = __importDefault(require("dd-trace"));
dd_trace_1.default.init({
    logInjection: true,
    profiling: true,
    appsec: true,
    runtimeMetrics: true
});
exports.default = dd_trace_1.default;

The app.js file now looks like this:

/* ~/app.js */

import './config/datadog.js';
import express from 'express';
...

previous commonjs version:

/* ~/app.js */
"use strict";
var __importDefault = (this && this.__importDefault) || function (mod) {
    return (mod && mod.__esModule) ? mod : { "default": mod };
};
Object.defineProperty(exports, "__esModule", { value: true });
require("./config/datadog");

const express_1 = __importDefault(require("express"));

....

We stopped receiving traces on our dashboard.

I've checked the guide at https://docs.datadoghq.com/tracing/trace_collection/dd_libraries/nodejs/?tab=containers, tried using the one liner

import 'dd-trace/init';

but it fails with the following error:

 node dist/app.js

node:internal/process/esm_loader:89
    internalBinding('errors').triggerUncaughtException(
                              ^

Error [ERR_MODULE_NOT_FOUND]: Cannot find module '/Users/gg/projects/my-project-api/node_modules/dd-trace/init' imported from /Users/gg/projects/my-project-api/dist/app.js
Did you mean to import dd-trace/init.js?
    at new NodeError (node:internal/errors:387:5)
    at finalizeResolution (node:internal/modules/esm/resolve:404:11)
    at moduleResolve (node:internal/modules/esm/resolve:965:10)
    at defaultResolve (node:internal/modules/esm/resolve:1173:11)
    at nextResolve (node:internal/modules/esm/loader:173:28)
    at ESMLoader.resolve (node:internal/modules/esm/loader:852:30)
    at ESMLoader.getModuleJob (node:internal/modules/esm/loader:439:18)
    at ModuleWrap.<anonymous> (node:internal/modules/esm/module_job:76:40)
    at link (node:internal/modules/esm/module_job:75:36) {
  code: 'ERR_MODULE_NOT_FOUND'
}

Environment

• Operation system:
• Node.js version: v16.15.1
• Tracer version: 2.11.0
• Agent version: latest 7.x
• Relevant library versions:

[sabrenner]

The one-liner import should really only work in explicitly typescript files or with a bundler, not just a plain .js file (even if it uses ESM), which to my understanding is what you have currently (import 'dd-trace/init' would be transpiled into something like require('dd-trace/init')). So, I believe the error for the second way you're seeing isn't unexpected (although, if I misunderstood your setup, let me know!).

As for the first approach - the contents of ./config/datadog.js and how it is imported in ./app.js look fine, provided it is the first thing imported (although, if you're using ESM and typescript, but these are JS files that aren't transpiled, maybe that could be part of the problem, and they should be .ts files instead). Would you be able to provide any other context into what else changed when you converted from CommonJS to ESM (i.e. something that would prevent traces from being sent to the agent/showing up in your dashboard)?

[rochdev]

@guidev There is experimental support for ESM that can be enabled with node --loader dd-trace/loader-hook.mjs. It's worth noting that this doesn't replace your current logic and should be added on top of what you currently do. We're still working on it but it should work well enough for most apps. If you try it, please let me know if your traces are back exactly as before or if anything is still missing.

[guidev]

I've just noticed that we're still receiving all traces, but it looks like that express isn't recognised anymore.

[dyarleniber]

@guidev I had the same issue with missing HTTP routes in the APM dashboards at the 2.11 version.
I resolved this by upgrading to version 3.0.0 and moving the library import to the top of my entry point file. (I believe that initializing the lib at the beginning of the entry point file is also recommended).

[jamesone]

I've just noticed that we're still receiving all traces, but it looks like that express isn't recognised anymore.

@guidev Did you figure out why this was happening? And even better, did you figure out a way to fix it?

I am also having the same issue after moving from commonjs to native ESM in node

[rochdev]

@jamesone How are you initializing dd-trace? Also, did you add our ESM loader to the CLI arguments?

[guidev]

@jamesone

Adding --loader dd-trace/loader-hook.mjs solved the issue for me:

node --loader dd-trace/loader-hook.mjs app.js

[jamesone]

Hey @guidev I tried adding the loader (I am on Node 16.16.0) and I am now getting another error:

 node --experimental-specifier-resolution=node --experimental-loader dd-trace/loader-hook build/src/index.js
(node:47552) ExperimentalWarning: --experimental-loader is an experimental feature. This feature could change at any time
(Use `node --trace-warnings ...` to show where the warning was created)
(node:47552) ExperimentalWarning: The Node.js specifier resolution flag is experimental. It could change or be removed at any time.
(node:47552) DeprecationWarning: Obsolete loader hook(s) supplied and will be ignored: getFormat, getSource
node:internal/errors:465
    ErrorCaptureStackTrace(err);
    ^

TypeError [ERR_INVALID_URL_SCHEME]: The URL must be of scheme file
    at new NodeError (node:internal/errors:372:5)
    at fileURLToPath (node:internal/url:1422:11)
    at checkIfDisallowedImport (node:internal/modules/esm/resolve:1061:23)
    at defaultResolve (node:internal/modules/esm/resolve:1189:23)
    at resolve (file:///Users/james/dev/proj/node_modules/import-in-the-middle/hook.mjs:58:21)
    at ESMLoader.resolve (node:internal/modules/esm/loader:580:30)
    at ESMLoader.getModuleJob (node:internal/modules/esm/loader:294:18)
    at ModuleWrap.<anonymous> (node:internal/modules/esm/module_job:80:40)
    at link (node:internal/modules/esm/module_job:78:36) {
  code: 'ERR_INVALID_URL_SCHEME'
}
error Command failed with exit code 1.

I also tried:

• --experimental-loader dd-trace/loader-hook.mjs
• --loader dd-trace/loader-hook.mjs

These both produced the same error as above.

@guidev which version of node did you successfully use the loader with?

[guidev]

I've seen the same error randomly a couple of times, but now I'm running it successfully on node v16.17.0.

Make sure to have the latest version of dd-trace -> #2311

[jamesone]

It seems like there is an issue with the import-in-the-middle library, it cannot handle import assertions. i.e. import contracts from './package.json' assert { type: 'json' };

[jamesone]

This issue has been fixed in: https://github.com/DataDog/dd-trace-js/tree/v3.3.1 🕺