NX Plugin throws when importing ESM (even with workaround)

[JulioC]

Current Behavior

There is a known issue #15682 on using ESM dependencies in NX Plugins, which requires the following workaround:

const { someImport } = await (Function('return import("some-esm-package")')() as Promise<typeof import('some-esm-package')>);

The problem is this won't work for some dependencies (ie chalk@5), with node throwing on the module import with either A dynamic import callback was not specified. or Invalid host defined options errors.

I've tried reproducing this using nx-examples repository, but initially it worked fine there. After a lot of search comparing that repository with my own, I found out the issue only happens when @angular/cli@~16.2.0 is not installed as a dependency (!!!)

I was unable to find anything further on the issue, but it looks like there is some code in nx that checks for that dependency, by calling require.resolve on it. I'm not sure if that might cause any side effect (ie running the dependency code somehow?) or if nx changes behavior based on that...

Expected Behavior

ESM should be able to load, even if the workaround for #15682 is required.

GitHub Repo

https://github.com/JulioC/nx-examples/tree/bug/esm-import-in-plugins

Steps to Reproduce

1. Download and install the linked repository (make sure you are on branch bug/esm-import-in-plugins)
2. Run the example generator yarn nx g @nx-example/nx-workspace-plugin:example to see the issue
3. Checkout previous commit in that branch to see the working version, where @angular/cli is still installed (d141243dbde12a582125affc5b4341efea71cdf5)

Nx Report

npx nx report 
>  NX  Falling back to ts-node for local typescript execution. This may be a little slower.
  - To fix this, ensure @swc-node/register and @swc/core have been installed

 >  NX   Report complete - copy this into the issue template

   Node   : 18.16.0
   OS     : win32-x64
   yarn   : 1.22.19

   nx                 : 16.7.4
   @nx/js             : 16.7.4
   @nx/jest           : 16.7.4
   @nx/linter         : 16.7.4
   @nx/workspace      : 16.7.4
   @nx/angular        : 16.7.4
   @nx/cypress        : 16.7.4
   @nx/devkit         : 16.7.4
   @nx/eslint-plugin  : 16.7.4
   @nx/react          : 16.7.4
   @nrwl/tao          : 16.7.4
   @nx/web            : 16.7.4
   @nx/webpack        : 16.7.4
   nx-cloud           : 16.3.0
   typescript         : 5.1.3
   ---------------------------------------
   Community plugins:
   @ngrx/component-store : 16.0.0
   @ngrx/effects         : 16.0.0
   @ngrx/entity          : 16.0.0
   @ngrx/router-store    : 16.0.0
   @ngrx/store           : 16.0.0
   @ngrx/store-devtools  : 16.0.0
   ---------------------------------------
   Local workspace plugins:
         @nx-example/nx-workspace-plugin

Failure Logs

nx g @nx-example/nx-workspace-plugin:example
>  NX  Falling back to ts-node for local typescript execution. This may be a little slower.
  - To fix this, ensure @swc-node/register and @swc/core have been installed

>  NX  Generating @nx-example/nx-workspace-plugin:example


 >  NX   Invalid host defined options

Package Manager Version

No response

Operating System

• macOS
• Linux
• Windows
• Other (Please specify)

Additional Information

• Issue happens on Windows or WSL Ubuntu
• Issue happens idependently of using ts-node or @swc-node/register and @swc/core for typescript execution

[JulioC]

I've dug deeper into this issue, and found out the cause is a known issue in v8-compile-cache (zertosh/v8-compile-cache#41).

Nx v8-compile-cache uses when it detects the workspace is not using angular, by trying to load @angular/cli.

nx/packages/nx/bin/nx.ts

Line 55 in 7508e7b

require('v8-compile-cache');

I believe that dependency is really necessary, but it would be nice to have a flag to turn it off as a workaround for this case.

[JulioC]

There is a env flag to disable v8-compile-cache: DISABLE_V8_COMPILE_CACHE=1

I've updated the workaround from #15682 to include a warning about this:

/**
 * Workaround for Import ESM modules in plugin
 *
 * @see {@link https://github.com/nrwl/nx/issues/18974} related issue
 * @see {@link https://github.com/nrwl/nx/issues/15682} related issue
 * @see {@link https://gist.github.com/passbyval/b85f79381816c197c5c651b7c0b00d5e} for alternative approach
 *
 * @example
 * const { default: chalk } = await requireEsm<typeof import('chalk')>('chalk');
 */
export async function requireEsm<T>(module: string) {
  try {
    return await (Function(`return import("${module}")`)() as Promise<T>);
  } catch (e) {
    if (e instanceof TypeError && 'code' in e && e.code === 'ERR_VM_DYNAMIC_IMPORT_CALLBACK_MISSING') {
      console.error([
        'Nx failed to import ESM module due to https://github.com/nrwl/nx/issues/18974',
        'Add DISABLE_V8_COMPILE_CACHE=1 to your .env'
      ].join('\n'));
    }
    throw e;
  }
}

[AgentEnder]

Hey Julio! I'm attaching a quick fix for generators + format that will close this issue out. Plugins still won't work if they try to import ESM during the project graph creation process, but generators, executors etc should be compatible with it now.

[JulioC]

Awesome, thanks!

I believe the issue will still happen with custom executors, but the user-side workaround with DISABLE_V8_COMPILE_CACHE=1 has much less impact than any change you can make on Nx-side.

[AgentEnder]

It shouldn't hit custom executors since they are in a different process which doesn't hit the v8 compile cache, hence why vitest was always working

[igilham]

The fix didn't work for me, as documented in #18721.

The work-around does work.

[github-actions]

This issue has been closed for more than 30 days. If this issue is still occuring, please open a new issue with more recent context.