[historical reference] module.resolver alternative

[caridy]

As part of the exercise, we ended up with a handful of possibilities around package.json. The three top alternatives that were considered:

1. shim
2. modules.root
3. modules.resolver

We decided to push for option 2 from above, knowing that we could pivot if needed.

I think it is worth to compile the other two alternatives, for historical reasons. This issue is a recollection of option 3 thoughts.

disclaimer: this is @caridy's recollection of memories from the discussions with @dherman and @wycats; might be missing some important pieces.

Rationale

Today, node only support extending the loading mechanism at the process level, applying AOP on require.extensions to support new extensions, and other goodies. This is the mechanism used by babel-register, and you can see how it works here.

The problem with this mechanism, and the babel-register approach, is that it affects all modules required by your program, with no distinction.

Note: part of our analysis was to look into existing NPM pkgs, to see how many of them have babel-register as a dependency, to corroborate our hypothesis that since this is a process-level mechanism, packages should not attempt to patch the loading mechanism of the process.

Note: Because of the work that we have done on the module loader spec, we will love to have a custom resolver mechanism in node that can be configured per package.

Questions

• Can a package control its resolution process?
• Can a package dictates how to evaluate modules depending on the runtime conditions?
• How can we align node resolving mechanism with the loader spec?

Note: From the forward looking point of view, once we get the loader in browsers, it will be easy to control almost every aspect of the fetching, parsing, and instantiation process for every module to be evaluated, including segmentation per packages.

Proposal

Any package (or a folder with a package.json) could choose its own resolver, which is used by new versions of node to determine the path and type of the module in question.

modules.resolver directive

Add a new directive called modules.resolver in package.json, e.g.:

{
     ...
     "modules.resolver": "./path/to/resolver.js",
     ...
}

This directive can also be a reference from another package declared as a dependency:

{
     ...
     "modules.resolver": "node-awesome-resolver/lib/standard",
     ...
}

As a result, once the package is resolved by node for the first time, and package.json cached in memory, the module denoted by modules.resolver will also be evaluated and cache.

Node Resolution Algo

When requiring or importing a module, node will determine what package (a folder) contains the module in question, and whether or not that package contains a custom resolver. If a resolver is available, node will call the default export function from the resolver, by passing the requested path, and some meta information about the package.

Example 1: ES Standard Package

An example of a very dummy resolver:

export default function (requested, pkg) {
    return {
       type: 'standard',
       path: pathlib.relative(requested, pkg.root),
    }
};

As a result, any requested module from this package will be parsed as ES Standard Module, and will be resolved from the root of the package.

Example 2: Poly-Package

export default function (requested, pkg) {
    return {
       type: 'standard',
       path: pathlib.relative(requested, pkg.root + '/src'),
    }
};

As a result, any requested module from this package will be parsed as ES Standard Module from the src/ folder in new versions of node that will understand the modules.resolver directive, otherwise it will fallback to the normal resolution for CJS modules relative to the package root.

Example 2: .mjs vs .js

export default function (requested, pkg) {
    const ext = pathlib.extname(required);
    const file = pathlib.relative(requested, pkg.root);
    let t = 'cjs';
    if (ext === '.mjs') {
         t = 'standard';
    } else if (fslib.existsSync(requested + '.mjs')) {
         t = 'standard';
         file += '.mjs';
    }
    return {
         type: t,
         path: file,
    };
};

As a result, for any requested module from this package, if the extension is explicitly .mjs, or a file exists with .mjs extension, it be parsed as ES Standard Module, otherwise it will fallback to cjs.

Features

• Any package can define its own resolver without affecting the node process
• Resolvers can be shared via NPM
• Node can provide one default resolver
• Let the winner to emerge from the community
• It is a one-line to be added into package.json
• Poly-packages
• Apps and Modules requiring files from arbitrary folders can be added.
• Flexible mechanism to add more features in the future (e.g.: transpilation)
• WHATWG Loader resolver hook can tap into node's resolver, for easy integration.
• Testing compat-mode for poly-packages is possible.

Resolver API

Some preliminary work around the API of the resolver is specified here using TypeScript:

interface Package {
  root: string;
  json: PackageJSON
}

interface PackageJSON {
  // you know ... the object representing package.json
}

interface ResolvedModule {
  type: 'standard' | 'cjs';
  path: string; // absolute path
}

interface Resolver {
  resolve(requested: string, fromPackage: PackageJSON): ResolvedModule;
}