Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(instrumentation): add support for esm module #2846

Closed
wants to merge 3 commits into from
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -30,6 +30,7 @@ For experimental package changes, see the [experimental CHANGELOG](experimental/

* feat(sdk-trace): re-export sdk-trace-base in sdk-trace-node and web [#3319](https://github.com/open-telemetry/opentelemetry-js/pull/3319) @legendecas
* feat: enable tree shaking [#3329](https://github.com/open-telemetry/opentelemetry-js/pull/3329) @pkanal
* feat(instrumentation): add support for esm module [#2846](https://github.com/open-telemetry/opentelemetry-js/pull/2846) @vmarchaud

### :bug: (Bug Fix)

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -263,6 +263,11 @@ If nothing is specified the global registered provider is used. Usually this is
There might be usecase where someone has the need for more providers within an application. Please note that special care must be takes in such setups
to avoid leaking information from one provider to the other because there are a lot places where e.g. the global `ContextManager` or `Propagator` is used.

## Instrumentation for ESM Module In NodeJS (experimental)

As the module loading mechanism for ESM is different than CJS, you need to select a custom loader so instrumentation can load hook on the esm module it want to patch. To do so, you must provide the `--experimental-loader=import-in-the-middle/hook.mjs` flag to the `node` binary, alternatively you can set the `NODE_OPTIONS` environment variable to `--experimental-loader=import-in-the-middle/hook.mjs`.
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would recommend creating a wrapper for the hook so you can have a loader hook name that's clearly in the otel namespace rather than the unfamiliar import-in-the-middle naming. That's what we do at Datadog. We tell users to use dd-trace/loader-hook.js which just loads the IITM hook: https://github.com/DataDog/dd-trace-js/blob/master/loader-hook.mjs

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the suggestion, i would like other opinion on this since this would result in using --experimental-loader=@opentelemetry-instrumentation/hook.mjs instead. Also since the instrumentation is built for multiple platforms i'm not sure this would not create more problems as we support more (deno comes to my mind)

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The hook could be a separate package. My suggestion is just to make the connection to otel clearer in some way.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@open-telemetry/javascript-maintainers I would like in put on this, i'm not sure what's the best way to do this:

  • have a dedicated package that re-export ittm
  • exporting it from the instrumentation package but it need to be excluded from the browser build, however i'm not sure where to place the file in this case ? inside /plaform/node ? at the root ? Since user would need to require it directly, that means we need it to be at the root of build

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Between those options i'd rather have a dedicated package. Honestly though I think it's fine to just use RITM directly. It's just one more thing for us to build, maintain, and publish

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I can guess that @Qard being at the other end by maintaining ITTM doesn't want to get questions about ESM instrumentation within otel but maybe i'm mistaken ? I agree that its more complicated to have one empty package on our end just for this.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

--experimental-loader=@opentelemetry-instrumentation/hook.mjs looks good to me. I didn't find a new dedicated package necessary here. The hook entry files to be published will be rather small. Reducing the number of OTEL packages can alleviate the burden of setting up an OTEL application.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

--experimental-loader=@opentelemetry-instrumentation/hook.mjs looks good to me. I didn't find a new dedicated package necessary here. The hook entry files to be published will be rather small. Reducing the number of OTEL packages can alleviate the burden of setting up an OTEL application.

Do you mean --experimental-loader=@opentelemetry/instrumentation/hook.mjs

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My concern is more just users being confused about needing --experimental-loader=import-in-the-middle/hook.mjs which is loading some other module than opentelemetry which they are likely unfamiliar with. Having an opentelemetry-specific proxy to it makes it clearer that this is a thing opentelemetry wants from the user. It also doesn't leave you tied to some specific external library you don't control. If at some point in the future IITM no longer does what you need or maintenance dies or whatever else, you can simply change the contents of your hook file without needing to ask all your users to update their CLI args. Changing CLI args sounds like it should be a trivial thing, but stuff like that often gets encoded deep in infra configs and places like that which creates unnecessary friction if changes need to be made in the future. It's just a better idea to have an entrypoint for that which you control.

Copy link

@smnbbrv smnbbrv Jan 13, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would also choose the way of having own wrapper for loading a non-core external library. If we were speaking about browser, it would probably make sense to keep the external dep as is to reduce the generated client's bundle size in case this lib is reused outside of opentelemetry as well. For node it's irrelevant, there are no cons of wrapping it

As the ESM module loader from NodeJS is experimental, so is our support for it. Feel free to provide feedback or report issues about it.

## License

Apache 2.0 - See [LICENSE][license-url] for more information.
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -47,7 +47,9 @@
"tdd": "npm run tdd:node",
"tdd:node": "npm run test -- --watch-extensions ts --watch",
"tdd:browser": "karma start",
"test": "nyc ts-mocha -p tsconfig.json 'test/**/*.test.ts' --exclude 'test/browser/**/*.ts'",
"test:cjs": "nyc ts-mocha -p tsconfig.json 'test/**/*.test.ts' --exclude 'test/browser/**/*.ts'",
"test:esm": "nyc node --experimental-loader=import-in-the-middle/hook.mjs ../../../node_modules/mocha/bin/mocha 'test/node/*.test.mjs' test/node/*.test.mjs",
"test": "npm run test:cjs && npm run test:esm",
vmarchaud marked this conversation as resolved.
Show resolved Hide resolved
"test:browser": "nyc karma start --single-run",
"version": "node ../../../scripts/version-update.js",
"watch": "tsc --build --watch tsconfig.all.json",
Expand All @@ -68,6 +70,7 @@
"url": "https://github.com/open-telemetry/opentelemetry-js/issues"
},
"dependencies": {
"import-in-the-middle": "^1.3.4",
"require-in-the-middle": "^5.0.3",
"semver": "^7.3.2",
"shimmer": "^1.2.1"
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,7 @@
*/

import * as RequireInTheMiddle from 'require-in-the-middle';
import * as ImportInTheMiddle from 'import-in-the-middle';
import * as path from 'path';
import { ModuleNameTrie, ModuleNameSeparator } from './ModuleNameTrie';

Expand All @@ -23,6 +24,12 @@ export type Hooked = {
onRequire: RequireInTheMiddle.OnRequireFn
};

/**
* We are forced to re-type there because ImportInTheMiddle is exported as normal CJS
* in the JS files but transpiled ESM (with a default export) in its typing.
*/
const ESMHook = ImportInTheMiddle as unknown as typeof ImportInTheMiddle.default;

/**
* Whether Mocha is running in this process
* Inspired by https://github.com/AndreasPizsa/detect-mocha
Expand All @@ -35,12 +42,12 @@ const isMocha = ['afterEach','after','beforeEach','before','describe','it'].ever
});

/**
* Singleton class for `require-in-the-middle`
* Singleton class for `require-in-the-middle` and `import-in-the-middle`
* Allows instrumentation plugins to patch modules with only a single `require` patch
* WARNING: Because this class will create its own `require-in-the-middle` (RITM) instance,
* WARNING: Because this class will create its own RITM and IITM instance,
* we should minimize the number of new instances of this class.
* Multiple instances of `@opentelemetry/instrumentation` (e.g. multiple versions) in a single process
* will result in multiple instances of RITM, which will have an impact
* will result in multiple instances of RITM/ITTM, which will have an impact
* on the performance of instrumentation hooks being applied.
*/
export class RequireInTheMiddleSingleton {
Expand All @@ -52,23 +59,22 @@ export class RequireInTheMiddleSingleton {
}

private _initialize() {
RequireInTheMiddle(
// Intercept all `require` calls; we will filter the matching ones below
null,
{ internals: true },
(exports, name, basedir) => {
// For internal files on Windows, `name` will use backslash as the path separator
const normalizedModuleName = normalizePathSeparators(name);
const onHook = (exports: any, name: string, basedir: string | undefined | void) => {
// For internal files on Windows, `name` will use backslash as the path separator
const normalizedModuleName = normalizePathSeparators(name);

const matches = this._moduleNameTrie.search(normalizedModuleName, { maintainInsertionOrder: true });
const matches = this._moduleNameTrie.search(normalizedModuleName, { maintainInsertionOrder: true });

for (const { onRequire } of matches) {
exports = onRequire(exports, name, basedir);
}

return exports;
for (const { onRequire } of matches) {
exports = onRequire(exports, name, basedir ? basedir : undefined);
}
);

return exports;
};
// Intercept all `require` calls; we will filter the matching ones below
RequireInTheMiddle(null, { internals: true }, onHook);
// We can give no module to patch but this signature isn't exposed in typings
new ESMHook(null as any, { internals: true }, onHook);
}

/**
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,47 @@
/*
* Copyright The OpenTelemetry Authors
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/

import * as assert from 'assert'
import { InstrumentationBase, InstrumentationNodeModuleDefinition } from '../../build/src/index.js';

describe('when loading esm module', () => {
it('should patch module file', async () => {
class TestInstrumentation extends InstrumentationBase {
constructor(onPatch, onUnpatch) {
super('my-esm-instrumentation', '0.1.0');
}

init() {
return [
new InstrumentationNodeModuleDefinition(
'my-esm-module',
['*'],
(exports, version) => {
exports.myConstant = 43;
exports.myFunction = () => 'another';
}
)
];
}
}

const instrumentation = new TestInstrumentation();
instrumentation.enable();
const exported = await import('my-esm-module');
assert.deepEqual(exported.myConstant, 43);
assert.deepEqual(exported.myFunction(), 'another');
});
});

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.