Irp type implementation

doc/api/esm.md

@@ -55,10 +55,6 @@ property:
 
 ## Notable differences between `import` and `require`
 
-### Only Support for .mjs
-
-ESM must have the `.mjs` extension.
-
 ### Mandatory file extensions
 
 You must provide a file extension when using the `import` keyword.
@@ -157,37 +153,128 @@ The resolver has the following properties:
 
 ### Resolver Algorithm
 
-The algorithm to resolve an ES module specifier is provided through
-_ESM_RESOLVE_:
+The algorithm to load an ES module specifier is given through the
+**ESM_RESOLVE** method below. It returns the resolved URL for a
+module specifier relative to a parentURL, in addition to the unique module
+format for that resolved URL given by the **ESM_FORMAT** routine.
+
+The _"esm"_ format is returned for an ECMAScript Module, while the
+_"legacy"_ format is used to indicate loading through the legacy
+CommonJS loader. Additional formats such as _"wasm"_ or _"addon"_ can be
+extended in future updates.
 
-**ESM_RESOLVE**(_specifier_, _parentURL_)
-> 1. Let _resolvedURL_ be _undefined_.
-> 1. If _specifier_ is a valid URL then,
+In the following algorithms, all subroutine errors are propogated as errors
+of these top-level routines.
+
+_isMain_ is **true** when resolving the Node.js application entry point.
+
+**ESM_RESOLVE(_specifier_, _parentURL_, _isMain_)**
+> 1. Let _resolvedURL_ be **undefined**.
+> 1. If _specifier_ is a valid URL, then
 >    1. Set _resolvedURL_ to the result of parsing and reserializing
 >       _specifier_ as a URL.
-> 1. Otherwise, if _specifier_ starts with _"/"_, _"./"_ or _"../"_ then,
+> 1. Otherwise, if _specifier_ starts with _"/"_, then
+>    1. Throw an _Invalid Specifier_ error.
+> 1. Otherwise, if _specifier_ starts with _"./"_ or _"../"_, then
 >    1. Set _resolvedURL_ to the URL resolution of _specifier_ relative to
 >       _parentURL_.
 > 1. Otherwise,
 >    1. Note: _specifier_ is now a bare specifier.
 >    1. Set _resolvedURL_ the result of
 >       **PACKAGE_RESOLVE**(_specifier_, _parentURL_).
-> 1. If the file at _resolvedURL_ does not exist then,
+> 1. If the file at _resolvedURL_ does not exist, then
 >    1. Throw a _Module Not Found_ error.
-> 1. Return _resolvedURL_.
-
-**PACKAGE_RESOLVE**(_packageSpecifier_, _parentURL_)
-> 1. Assert: _packageSpecifier_ is a bare specifier.
-> 1. If _packageSpecifier_ is a Node.js builtin module then,
+> 1. Let _format_ be the result of **ESM_FORMAT**(_url_, _isMain_).
+> 1. Load _resolvedURL_ as module format, _format_.
+
+PACKAGE_RESOLVE(_packageSpecifier_, _parentURL_)
+> 1. Let _packageName_ be *undefined*.
+> 1. Let _packageSubpath_ be *undefined*.
+> 1. If _packageSpecifier_ is an empty string, then
+>    1. Throw an _Invalid Specifier_ error.
+> 1. If _packageSpecifier_ does not start with _"@"_, then
+>    1. Set _packageName_ to the substring of _packageSpecifier_ until the
+>       first _"/"_ separator or the end of the string.
+> 1. Otherwise,
+>    1. If _packageSpecifier_ does not contain a _"/"_ separator, then
+>       1. Throw an _Invalid Specifier_ error.
+>    1. Set _packageName_ to the substring of _packageSpecifier_
+>       until the second _"/"_ separator or the end of the string.
+> 1. Let _packageSubpath_ be the substring of _packageSpecifier_ from the
+>    position at the length of _packageName_ plus one, if any.
+> 1. Assert: _packageName_ is a valid package name or scoped package name.
+> 1. Assert: _packageSubpath_ is either empty, or a path without a leading
+>    separator.
+> 1. If _packageSubpath_ contains any _"."_ or _".."_ segments or percent
+>    encoded strings for _"/"_ or _"\"_ then,
+>    1. Throw an _Invalid Specifier_ error.
+> 1. If _packageSubpath_ is empty and _packageName_ is a Node.js builtin
+>    module, then
 >    1. Return the string _"node:"_ concatenated with _packageSpecifier_.
-> 1. While _parentURL_ contains a non-empty _pathname_,
+> 1. While _parentURL_ is not the file system root,
 >    1. Set _parentURL_ to the parent folder URL of _parentURL_.
 >    1. Let _packageURL_ be the URL resolution of the string concatenation of
->       _parentURL_, _"/node_modules/"_ and _"packageSpecifier"_.
->    1. If the file at _packageURL_ exists then,
->       1. Return _packageURL_.
+>       _parentURL_, _"/node_modules/"_ and _packageSpecifier_.
+>    1. If the folder at _packageURL_ does not exist, then
+>       1. Set _parentURL_ to the parent URL path of _parentURL_.
+>       1. Continue the next loop iteration.
+>    1. Let _pjson_ be the result of **READ_PACKAGE_JSON**(_packageURL_).
+>    1. If _packageSubpath_ is empty, then
+>       1. Return the result of **PACKAGE_MAIN_RESOLVE**(_packageURL_,
+>          _pjson_).
+>    1. Otherwise,
+>       1. Return the URL resolution of _packageSubpath_ in _packageURL_.
 > 1. Throw a _Module Not Found_ error.
 
+PACKAGE_MAIN_RESOLVE(_packageURL_, _pjson_)
+> 1. If _pjson_ is **null**, then
+>    1. Throw a _Module Not Found_ error.
+> 1. If _pjson.main_ is a String, then
+>    1. Let _resolvedMain_ be the concatenation of _packageURL_, "/", and
+>       _pjson.main_.
+>    1. If the file at _resolvedMain_ exists, then
+>       1. Return _resolvedMain_.
+> 1. If _pjson.type_ is equal to _"esm"_, then
+>    1. Throw a _Module Not Found_ error.
+> 1. Let _legacyMainURL_ be the result applying the legacy
+>    **LOAD_AS_DIRECTORY** CommonJS resolver to _packageURL_, throwing a
+>    _Module Not Found_ error for no resolution.
+> 1. If _legacyMainURL_ does not end in _".js"_ then,
+>    1. Throw an _Unsupported File Extension_ error.
+> 1. Return _legacyMainURL_.
+
+**ESM_FORMAT(_url_, _isMain_)**
+> 1. Assert: _url_ corresponds to an existing file.
+> 1. Let _pjson_ be the result of **READ_PACKAGE_BOUNDARY**(_url_).
+> 1. If _pjson_ is **null** and _isMain_ is **true**, then
+>    1. Return _"legacy"_.
+> 1. If _pjson.type_ exists and is _"esm"_, then
+>    1. If _url_ does not end in _".js"_ or _".mjs"_, then
+>       1. Throw an _Unsupported File Extension_ error.
+>    1. Return _"esm"_.
+> 1. Otherwise,
+>    1. If _url_ ends in _".mjs"_, then
+>       1. Return _"esm"_.
+>    1. Otherwise,
+>       1. Return _"legacy"_.
+
+READ_PACKAGE_BOUNDARY(_url_)
+> 1. Let _boundaryURL_ be _url_.
+> 1. While _boundaryURL_ is not the file system root,
+>    1. Let _pjson_ be the result of **READ_PACKAGE_JSON**(_boundaryURL_).
+>    1. If _pjson_ is not **null**, then
+>       1. Return _pjson_.
+>    1. Set _boundaryURL_ to the parent URL of _boundaryURL_.
+> 1. Return **null**.
+
+READ_PACKAGE_JSON(_packageURL_)
+> 1. Let _pjsonURL_ be the resolution of _"package.json"_ within _packageURL_.
+> 1. If the file at _pjsonURL_ does not exist, then
+>    1. Return **null**.
+> 1. If the file at _packageURL_ does not parse as valid JSON, then
+>    1. Throw an _Invalid Package Configuration_ error.
+> 1. Return the parsed JSON source of the file at _pjsonURL_.
+
 [Node.js EP for ES Modules]: https://github.com/nodejs/node-eps/blob/master/002-es-modules.md
 [`module.createRequireFromPath()`]: modules.html#modules_module_createrequirefrompath_filename
 [ESM Minimal Kernel]: https://github.com/nodejs/modules/blob/master/doc/plan-for-new-modules-implementation.md

lib/internal/errors.js

@@ -864,13 +864,6 @@ E('ERR_MISSING_ARGS',
 E('ERR_MISSING_DYNAMIC_INSTANTIATE_HOOK',
   'The ES Module loader may not return a format of \'dynamic\' when no ' +
   'dynamicInstantiate function was provided', Error);
-E('ERR_MODULE_NOT_FOUND', (module, base, legacyResolution) => {
-  let msg = `Cannot find module '${module}' imported from ${base}.`;
-  if (legacyResolution)
-    msg += ' Legacy behavior in require() would have found it at ' +
-           legacyResolution;
-  return msg;
-}, Error);
 E('ERR_MULTIPLE_CALLBACK', 'Callback called multiple times', Error);
 E('ERR_NAPI_CONS_FUNCTION', 'Constructor must be a function', TypeError);
 E('ERR_NAPI_INVALID_DATAVIEW_ARGS',
@@ -973,10 +966,10 @@ E('ERR_UNKNOWN_CREDENTIAL', '%s identifier does not exist: %s', Error);
 E('ERR_UNKNOWN_ENCODING', 'Unknown encoding: %s', TypeError);
 
 // This should probably be a `TypeError`.
-E('ERR_UNKNOWN_FILE_EXTENSION', 'Unknown file extension: \'%s\' imported ' +
-  'from %s', Error);
 E('ERR_UNKNOWN_MODULE_FORMAT', 'Unknown module format: %s', RangeError);
 E('ERR_UNKNOWN_SIGNAL', 'Unknown signal: %s', TypeError);
+E('ERR_UNSUPPORTED_FILE_EXTENSION', 'Unsupported file extension: \'%s\' ' +
+  'imported from %s', Error);
 
 E('ERR_V8BREAKITERATOR',
   'Full ICU data not installed. See https://github.com/nodejs/node/wiki/Intl',

lib/internal/modules/esm/default_resolve.js

@@ -1,46 +1,30 @@
 'use strict';
 
-const { URL } = require('url');
-const CJSmodule = require('internal/modules/cjs/loader');
 const internalFS = require('internal/fs/utils');
 const { NativeModule } = require('internal/bootstrap/loaders');
 const { extname } = require('path');
 const { realpathSync } = require('fs');
 const { getOptionValue } = require('internal/options');
 const preserveSymlinks = getOptionValue('--preserve-symlinks');
 const preserveSymlinksMain = getOptionValue('--preserve-symlinks-main');
-const {
-  ERR_MODULE_NOT_FOUND,
-  ERR_UNKNOWN_FILE_EXTENSION
-} = require('internal/errors').codes;
+const { ERR_UNSUPPORTED_FILE_EXTENSION } = require('internal/errors').codes;
 const { resolve: moduleWrapResolve } = internalBinding('module_wrap');
 const { pathToFileURL, fileURLToPath } = require('internal/url');
 
 const realpathCache = new Map();
 
-function search(target, base) {
-  try {
-    return moduleWrapResolve(target, base);
-  } catch (e) {
-    e.stack; // cause V8 to generate stack before rethrow
-    let error = e;
-    try {
-      const questionedBase = new URL(base);
-      const tmpMod = new CJSmodule(questionedBase.pathname, null);
-      tmpMod.paths = CJSmodule._nodeModulePaths(
-        new URL('./', questionedBase).pathname);
-      const found = CJSmodule._resolveFilename(target, tmpMod);
-      error = new ERR_MODULE_NOT_FOUND(target, fileURLToPath(base), found);
-    } catch {
-      // ignore
-    }
-    throw error;
-  }
-}
-
 const extensionFormatMap = {
   '__proto__': null,
-  '.mjs': 'esm'
+  '.mjs': 'esm',
+  '.js': 'esm'
+};
+
+const legacyExtensionFormatMap = {
+  '__proto__': null,
+  '.js': 'cjs',
+  '.json': 'cjs',
+  '.mjs': 'esm',
+  '.node': 'cjs'
 };
 
 function resolve(specifier, parentURL) {
@@ -51,10 +35,14 @@ function resolve(specifier, parentURL) {
     };
   }
 
-  let url = search(specifier,
-                   parentURL || pathToFileURL(`${process.cwd()}/`).href);
-
   const isMain = parentURL === undefined;
+  if (isMain)
+    parentURL = pathToFileURL(`${process.cwd()}/`).href;
+
+  const resolved = moduleWrapResolve(specifier, parentURL, isMain);
+
+  let url = resolved.url;
+  const legacy = resolved.legacy;
 
   if (isMain ? !preserveSymlinksMain : !preserveSymlinks) {
     const real = realpathSync(fileURLToPath(url), {
@@ -67,14 +55,14 @@ function resolve(specifier, parentURL) {
   }
 
   const ext = extname(url.pathname);
+  let format = (legacy ? legacyExtensionFormatMap : extensionFormatMap)[ext];
 
-  let format = extensionFormatMap[ext];
   if (!format) {
     if (isMain)
-      format = 'cjs';
+      format = legacy ? 'cjs' : 'esm';
     else
-      throw new ERR_UNKNOWN_FILE_EXTENSION(url.pathname,
-                                           fileURLToPath(parentURL));
+      throw new ERR_UNSUPPORTED_FILE_EXTENSION(fileURLToPath(url),
+                                               fileURLToPath(parentURL));
   }
 
   return { url: `${url}`, format };

lib/internal/modules/esm/loader.js

@@ -14,7 +14,7 @@ const ModuleJob = require('internal/modules/esm/module_job');
 const defaultResolve = require('internal/modules/esm/default_resolve');
 const createDynamicModule = require(
   'internal/modules/esm/create_dynamic_module');
-const translators = require('internal/modules/esm/translators');
+const { translators } = require('internal/modules/esm/translators');
 
 const FunctionBind = Function.call.bind(Function.prototype.bind);
 
@@ -32,6 +32,9 @@ class Loader {
     // Registry of loaded modules, akin to `require.cache`
     this.moduleMap = new ModuleMap();
 
+    // map of already-loaded CJS modules to use
+    this.cjsCache = new Map();
+
     // The resolver has the signature
     //   (specifier : string, parentURL : string, defaultResolve)
     //       -> Promise<{ url : string, format: string }>

lib/internal/modules/esm/module_job.js

@@ -23,7 +23,7 @@ class ModuleJob {
 
     // This is a Promise<{ module, reflect }>, whose fields will be copied
     // onto `this` by `link()` below once it has been resolved.
-    this.modulePromise = moduleProvider(url, isMain);
+    this.modulePromise = moduleProvider.call(loader, url, isMain);
     this.module = undefined;
     this.reflect = undefined;
 

lib/internal/modules/esm/translators.js

@@ -23,7 +23,7 @@ const StringReplace = Function.call.bind(String.prototype.replace);
 const debug = debuglog('esm');
 
 const translators = new SafeMap();
-module.exports = translators;
+exports.translators = translators;
 
 function initializeImportMeta(meta, { url }) {
   meta.url = url;
@@ -35,7 +35,7 @@ async function importModuleDynamically(specifier, { url }) {
 }
 
 // Strategy for loading a standard JavaScript module
-translators.set('esm', async (url) => {
+translators.set('esm', async function(url) {
   const source = `${await readFileAsync(new URL(url))}`;
   debug(`Translating StandardModule ${url}`);
   const module = new ModuleWrap(stripShebang(source), url);
@@ -52,9 +52,14 @@ translators.set('esm', async (url) => {
 // Strategy for loading a node-style CommonJS module
 const isWindows = process.platform === 'win32';
 const winSepRegEx = /\//g;
-translators.set('cjs', async (url, isMain) => {
+translators.set('cjs', async function(url, isMain) {
   debug(`Translating CJSModule ${url}`);
   const pathname = internalURLModule.fileURLToPath(new URL(url));
+  const cached = this.cjsCache.get(url);
+  if (cached) {
+    this.cjsCache.delete(url);
+    return cached;
+  }
   const module = CJSModule._cache[
     isWindows ? StringReplace(pathname, winSepRegEx, '\\') : pathname];
   if (module && module.loaded) {
@@ -74,7 +79,7 @@ translators.set('cjs', async (url, isMain) => {
 
 // Strategy for loading a node builtin CommonJS module that isn't
 // through normal resolution
-translators.set('builtin', async (url) => {
+translators.set('builtin', async function(url) {
   debug(`Translating BuiltinModule ${url}`);
   // slice 'node:' scheme
   const id = url.slice(5);

lib/internal/process/esm_loader.js

@@ -33,9 +33,7 @@ exports.importModuleDynamicallyCallback = async function(wrap, specifier) {
 };
 
 let loaderResolve;
-exports.loaderPromise = new Promise((resolve, reject) => {
-  loaderResolve = resolve;
-});
+exports.loaderPromise = new Promise((resolve) => loaderResolve = resolve);
 
 exports.ESMLoader = undefined;