Not working with native ES modules

[cullylarson]

I'm moving a project over to using node's native ES modules (enabled with the --experimental-modules flag). After updating my code, module-alias is no longer working. I tried adding this to the root of my app (this is the method I was using before transitioning to esm):

require('module-alias/register')

I tried changing it to:

import 'module-alias/register'

I tried requiring when starting the server:

node --experimental-modules -r module-alias/register server/app.js

The first aliased import in my app is this:

import {responseError} from '@app/lib/response'

I'm getting this error from it:

internal/modules/esm/default_resolve.js:69
  let url = moduleWrapResolve(specifier, parentURL);
            ^

Error: Cannot find package '@app/lib' imported from server/app.js
    at Loader.resolve [as _resolve] (internal/modules/esm/default_resolve.js:69:13)
    at Loader.resolve (internal/modules/esm/loader.js:70:33)
    at Loader.getModuleJob (internal/modules/esm/loader.js:143:40)
    at ModuleWrap.<anonymous> (internal/modules/esm/module_job.js:43:40)
    at link (internal/modules/esm/module_job.js:42:36)

The relevant lines in my package.json are:

"_moduleAliases": {
    "@app": "./server"
},

I'm starting the app like this:

node --experimental-modules server/app.js

module-alias worked fine using CommonJS. The only change I made to the code was changing requires to imports.

[jdt3969]

@cullylarson Ran into the same issue.

It appears that the new esm code isn't running the _resolveFilename which is the core of this library. Based on the docs it looks as though they are moving off of this library's hack and onto a standard feature: https://github.com/nodejs/node/blob/master/doc/api/esm.md#experimental-loader-hooks It's still experimental though.

I rewrote and reduced a lot based on known things within my library but this code is working for me:

// custom-loader.mjs
import path from 'path';
import npmPackage from './package.json';

const getAliases = () => {

  const base = process.cwd();

  const aliases = npmPackage.aliases || {};

  const absoluteAliases = Object.keys(aliases).reduce((acc, key) =>
    aliases[key][0] === '/'
      ? acc
      : { ...acc, [key]: path.join(base, aliases[key]) },
    aliases)

  return absoluteAliases;

}

const isAliasInSpecifier = (path, alias) => {
  return path.indexOf(alias) === 0
    && (path.length === alias.length || path[alias.length] === '/')
}

const aliases = getAliases();

export const resolve = (specifier, parentModuleURL, defaultResolve) => {
  
  const alias = Object.keys(aliases).find((key) => isAliasInSpecifier(specifier, key));

  const newSpecifier = alias === undefined
    ? specifier
    : path.join(aliases[alias], specifier.substr(alias.length));

  return defaultResolve(newSpecifier, parentModuleURL);
}

Then:
node --no-warnings --experimental-modules --es-module-specifier-resolution=node --loader ./custom-loader.mjs index.js

[cullylarson]

@jdt3969 Thanks for sharing. I'll give it a try on my next node project.

[Kehrlann]

Very interesting ! So if I understand correctly, you must just provide a .js file in a --loader flag ; and that .js file be a module that exports a resolve function, which basically does what module-alias does. (There might be a chicken and egg problem if you want to use module-alias programmatically though)

So it'd be very easy to do a pull request (wink wink, nudge nudge) that exports a neat little module that wraps module-aliases' resolve function, which can then be used like (rough idea, semantics TBD) :

node --no-warnings --experimental-modules --es-module-specifier-resolution=node --loader ./node_modules/module-alias/es6-loader.js index.js

@cullylarson @jdt3969 interested in doing a PR ?

@ilearnio any additional thoughts ?

[cullylarson]

@Kehrlann Thanks for the kind invitation to do a PR. I really appreciate the way you presented it. I'm near a deadline on a project right now and about to start another, otherwise I would take you up on the offer.

[kirkouimet]

You guys think we can do this without using the --loader flag and instead do it programmatically? Now that Module._resolveFilename isn't being used for import statements I wonder what is. May dive into it this week.

[eouia]

Not yet solved natively without execution flags?

[JakobJingleheimer]

@kirkouimet @eouia the node ems documentation linked above suggests no, the flag is required.

[JakobJingleheimer]

@jdt3969's above solution worked for me after I realised it was looking for a property in package.json called aliases instead of module-alias's documented _moduleAliases (I prefer jdt3969's aliases).

I tidied up the above code and added some pre-computing of values that can't change during execution: gist

EDIT: ⚠️ This appears to have broken somewhere between node v15.1.0 and v15.6.0 (jdt3969's too)

This was due to an error in my own source-code, which node ESM erroneously reported as coming from alias-loader.mjs

[TimDaub]

This is really interesting and the gist looks promising @jshado1.

But I'm wondering if this functionality can already be used when publishing a module to npm. I guess not, because how am I supposed to specify --experimental-loader=./alias-loader.mjs ./index.mjs as a user of said module?

[JakobJingleheimer]

@TimDaub Thanks!

But I'm wondering if this functionality can already be used when publishing a module to npm.

You probably shouldn't as loaders are imminently changing.

how am I supposed to specify --experimental-loader=./alias-loader.mjs ./index.mjs as a user of said module?

I'm not sure I understand what you mean—are you talking about where your module would itself depends on the loader, or someone consuming your module would need to consume it via --experimental-loader? For both, the answer would be roughly the same: as loaders currently stand in their experimental form, the consuming user would need to manually add it to their command (ex in package.json's "scripts") regardless of "who" is using it.

[jaschaio]

@jdt3969's above solution worked for me after I realised it was looking for a property in package.json called aliases instead of module-alias's documented _moduleAliases (I prefer jdt3969's aliases).

Worked for me as well, though it's important to note that you now need to use --experimental-loader=./custom-loader.mjs. I added as well --experimental-json-modules so that it works with .json files.

[TimDaub]

I'm not sure I understand what you mean—are you talking about where your module would itself depends on the loader, or someone consuming your module would need to consume it via --experimental-loader?

Later. Imagine I published a package that relied on starting node with --experimental-loader but then I'd also publish it to npm for others to use it. Now they'd have to be aware that now their project too as to be started with --experimental-loader.

the consuming user would need to manually add it to their command (ex in package.json's "scripts") regardless of "who" is using it.

That makes it unusable for npm packages until a version of node is out that removes the flag.

[szydlovski]

I understand that this is kind of an issue with Node itself, but it would be nice if the README included a warning about this package not being compatible with native ES modules. I just spent an hour questioning my sanity only to find out that the feature I've been hopelessly debugging was never intended to work at all.

[Kehrlann]

@szydlovski good point, would you want to submit a PR?

[szydlovski]

@Kehrlann Sure, here's a proposal

[JakobJingleheimer]

@TimDaub "unusable" does not seem accurate at all, and such CLI flags are quite common in many, many libraries. Support for a non-CLI option has been briefly discussed as a possibility in future, but if it happens, it would likely be quite a bit down the road. How loaders in Node.js will (almost surely) work once loader chaining is supported would be something like

$> node --loader https-loader --loader typescript-loader ./your-app.ts

[TimDaub]

Having to mention in the installation instruction on the library level that the using application should be initiated with a loader flag makes it unusabe.

It's fragile declaring this information in a readme. On the library level, the loader dependency should be described in the package.json and any using higher level package should interpret the lower libraries' loaders through the package.json.

But I guess that's out of scope for module-alias.

[JakobJingleheimer]

That sounds a bit dramatic. It's called a README (in screaming capitals) for a reason…. This is classic RTFM.

Module alias is obsolete now anyway with import mapping.

[TheDirigible]

This problem has been plaguing me for years because I want to write good quality code that is shared between node & browser.
I finally found a system that works:

• Place 'nesm.js' in the root of your project
• [optional] Place the 'nesm' shell script in your path, make it executable
• Run scripts with: 'nesm file_to_run.js' or 'node path/to/nesm.js -- file_to_run.js'

'nesm.js'

/*
 * esm and module-alias do not play nicely together.
 * this precise arrangement is the only way I found to make it work.
 * you can run this from anywhere in your project hierarchy.
 * you can use args, and use in npm scripts.
 * encourage the node.js devs to make this work natively.  ux matters.
 * ---- CAVEATS
 * will not work with "type":"module"
 * ---- SETUP
 * place 'nesm.js' in the root of your project
 * [optional] place the 'nesm' shell script in your path, make it executable
 * ---- USAGE
 * > nesm file_to_run.js
 * to run without the nesm shell script:
 * > node path/to/nesm.js -- file_to_run.js
 * to run with nodemon:
 * > nodemon -- path/to/nesm.js -- file_to_run.js
*/
require = require('esm')(module);   // eslint-disable-line no-global-assign
require('module-alias/register');   // must come after esm for some reason

let runNext;
for(const arg of process.argv) {
    if(runNext) {
        let filename = arg;
        if(filename[0]!='.' && filename[0]!='/') filename = './'+filename;
        require(filename);
        break;
    }
    runNext = (arg=='--');
}

'nesm' shell script

#!/bin/bash
if [ -z $1 ]; then
    echo "Node esm runner.  Usage: nesm file_to_run.js"
    exit 1
fi
baseDir=$( pwd )
while [ ! -f "$baseDir/nesm.js" ]; do
    if [ ${#baseDir} -le 1 ]; then
        echo "nesm.js not found in folder ancestry"
        exit 1
    fi
    baseDir="$(dirname "$baseDir")"
done
file1=$(realpath $1);
node $baseDir/nesm.js -- $file1

[euberdeveloper]

@cullylarson Ran into the same issue.

It appears that the new esm code isn't running the _resolveFilename which is the core of this library. Based on the docs it looks as though they are moving off of this library's hack and onto a standard feature: https://github.com/nodejs/node/blob/master/doc/api/esm.md#experimental-loader-hooks It's still experimental though.

I rewrote and reduced a lot based on known things within my library but this code is working for me:

// custom-loader.mjs
import path from 'path';
import npmPackage from './package.json';

const getAliases = () => {

  const base = process.cwd();

  const aliases = npmPackage.aliases || {};

  const absoluteAliases = Object.keys(aliases).reduce((acc, key) =>
    aliases[key][0] === '/'
      ? acc
      : { ...acc, [key]: path.join(base, aliases[key]) },
    aliases)

  return absoluteAliases;

}

const isAliasInSpecifier = (path, alias) => {
  return path.indexOf(alias) === 0
    && (path.length === alias.length || path[alias.length] === '/')
}

const aliases = getAliases();

export const resolve = (specifier, parentModuleURL, defaultResolve) => {
  
  const alias = Object.keys(aliases).find((key) => isAliasInSpecifier(specifier, key));

  const newSpecifier = alias === undefined
    ? specifier
    : path.join(aliases[alias], specifier.substr(alias.length));

  return defaultResolve(newSpecifier, parentModuleURL);
}

Then: node --no-warnings --experimental-modules --es-module-specifier-resolution=node --loader ./custom-loader.mjs index.js

You should have said that in package.json you expected the key aliases and no more _moduleAliases...

[JakobJingleheimer]

You should have said that in package.json you expected the key aliases and no more _moduleAliases...

@euberdeveloper I did #59 (comment) you can also see it on line 9 of the code. You're welcome btw.

[euberdeveloper]

In this code there is a problem with Windows paths.

@cullylarson Ran into the same issue.
It appears that the new esm code isn't running the _resolveFilename which is the core of this library. Based on the docs it looks as though they are moving off of this library's hack and onto a standard feature: https://github.com/nodejs/node/blob/master/doc/api/esm.md#experimental-loader-hooks It's still experimental though.
I rewrote and reduced a lot based on known things within my library but this code is working for me:

// custom-loader.mjs
import path from 'path';
import npmPackage from './package.json';

const getAliases = () => {

  const base = process.cwd();

  const aliases = npmPackage.aliases || {};

  const absoluteAliases = Object.keys(aliases).reduce((acc, key) =>
    aliases[key][0] === '/'
      ? acc
      : { ...acc, [key]: path.join(base, aliases[key]) },
    aliases)

  return absoluteAliases;

}

const isAliasInSpecifier = (path, alias) => {
  return path.indexOf(alias) === 0
    && (path.length === alias.length || path[alias.length] === '/')
}

const aliases = getAliases();

export const resolve = (specifier, parentModuleURL, defaultResolve) => {
  
  const alias = Object.keys(aliases).find((key) => isAliasInSpecifier(specifier, key));

  const newSpecifier = alias === undefined
    ? specifier
    : path.join(aliases[alias], specifier.substr(alias.length));

  return defaultResolve(newSpecifier, parentModuleURL);
}

Then: node --no-warnings --experimental-modules --es-module-specifier-resolution=node --loader ./custom-loader.mjs index.js

You should have said that in package.json you expected the key aliases and no more _moduleAliases...

I made an npm module using this code that fixes also that problem, check it out here: esm-module-alias

[JakobJingleheimer]

Heads up: the node loader API function signatures have changed, and the 2nd argument is an (optional) object (not parentURL). Supplying an argument of invalid type will trigger an exception. Please see the Node.js docs for current signatures.

[euberdeveloper]

@JakobJingleheimer

Could you please link me the signature in the doc?
Does this mean that my noduek should behave differently depending on the version of nodejs?

[JakobJingleheimer]

https://nodejs.org/api/esm.html#hooks

I'm not sure what version(s) of Node.js the change was backported to, but at the very least, my code from ~2 years ago and what you included in your npm package will not work in the v18 (LTS) or v19 (current) of node.

[euberdeveloper]

https://nodejs.org/api/esm.html#hooks

I'm not sure what version(s) of Node.js the change was backported to, but at the very least, my code from ~2 years ago and what you included in your npm package will not work in the v18 (LTS) or v19 (current) of node.

My repo does some tests with also versions of nodejs 18 and 19, and it seems to work

[foges]

For anyone else coming across this looking to support wildcard imports replace:

const isAliasInSpecifier = (path, alias) => {
  return path.indexOf(alias) === 0
    && (path.length === alias.length || path[alias.length] === '/')
}

With

const isAliasInSpecifier = (path, alias) => {
  return path.indexOf(alias) === 0
}