nx migrate fails with Yarn berry #16338

gremlin896 · 2023-04-17T11:51:13Z

Current Behavior

Running nx migrate latest with yarn@3.5.0 fails.

Expected Behavior

nx migrate latest should run without any errors

GitHub Repo

https://github.com/nrwl/nx-examples

Steps to Reproduce

Run the following commands (assuming volta is installed)

volta install yarn@3
yarn
nx migrate latest

Nx Report

>  NX   Report complete - copy this into the issue template

   Node : 16.17.0
   OS   : darwin arm64
   yarn : 3.5.0

   nx                      : 15.9.0-rc.2
   @nrwl/js                : 15.9.0-rc.2
   @nrwl/jest              : 15.9.0-rc.2
   @nrwl/linter            : 15.9.0-rc.2
   @nrwl/workspace         : 15.9.0-rc.2
   @nrwl/angular           : 15.9.0-rc.2
   @nrwl/cli               : 15.9.0-rc.2
   @nrwl/cypress           : 15.9.0-rc.2
   @nrwl/devkit            : 15.9.0-rc.2
   @nrwl/eslint-plugin-nx  : 15.9.0-rc.2
   @nrwl/react             : 15.9.0-rc.2
   @nrwl/tao               : 15.9.0-rc.2
   @nrwl/web               : 15.9.0-rc.2
   @nrwl/webpack           : 15.9.0-rc.2
   @nrwl/nx-cloud          : 15.3.3
   typescript              : 4.9.5
   ---------------------------------------
   Community plugins:
   @ngrx/component-store : 15.3.0
   @ngrx/effects         : 15.3.0
   @ngrx/entity          : 15.3.0
   @ngrx/router-store    : 15.3.0
   @ngrx/store           : 15.3.0
   @ngrx/store-devtools  : 15.3.0

Failure Logs

/bin/sh: /var/folders/tf/tjgc6tp54151mmr3jxmgcdr40000gq/T/tmp-32796-hV5BmtocRoSJ/node_modules/.bin/nx: No such file or directory
Command failed: /var/folders/tf/tjgc6tp54151mmr3jxmgcdr40000gq/T/tmp-32796-hV5BmtocRoSJ/node_modules/.bin/nx _migrate latest

Additional Information

After checking the temporary folders listed in the errors, there is no node_modules folder.

The text was updated successfully, but these errors were encountered:

pumano · 2023-05-01T07:06:56Z

I use yarn 3.3.x and also 3.5.x and all works good

ChrisSargent · 2023-05-18T19:14:37Z

Also having the same issue. Tried doing it from v15 then manually upgraded to v16 and tried it, same result as above:

On yarn 3.5.1.

>  NX   Report complete - copy this into the issue template

   Node   : 18.16.0
   OS     : darwin x64
   yarn   : 3.5.1
   Hasher : Native
   
   nx                 : 16.2.1
   @nx/js             : 16.2.1
   @nx/jest           : 16.2.1
   @nx/linter         : 16.2.1
   @nx/workspace      : 16.2.1
   @nx/devkit         : 16.2.1
   @nx/eslint-plugin  : 16.2.1
   @nx/node           : 16.2.1
   @nx/react          : 16.2.1
   @nrwl/tao          : 16.2.1
   @nx/web            : 16.2.1
   @nx/webpack        : 16.2.1
   nx-cloud           : 16.0.5
   typescript         : 5.0.4

ciceropablo · 2023-05-21T17:11:32Z

I have the same issue. Any news?

 >  NX   Report complete - copy this into the issue template

   Node : 18.15.0
   OS   : darwin x64
   yarn : 3.5.1
   
   nx                      : 15.9.0
   @nrwl/js                : 15.9.0
   @nrwl/jest              : 15.9.0
   @nrwl/linter            : 15.9.0
   @nrwl/workspace         : 15.9.0
   @nrwl/cli               : 15.9.0
   @nrwl/cypress           : 15.9.0
   @nrwl/devkit            : 15.9.0
   @nrwl/eslint-plugin-nx  : 15.9.0
   @nrwl/react             : 15.9.0
   @nrwl/tao               : 15.9.0
   @nrwl/vite              : 15.9.0
   typescript              : 4.9.5

dlarroder · 2023-05-24T06:24:18Z

I have the same issue, any updates on this?

>  NX   Report complete - copy this into the issue template

   Node : 18.16.0
   OS   : darwin arm64
   yarn : 3.2.4
   
   nx : 15.6.3
   @nrwl/angular : Not Found
   @nrwl/cypress : 15.6.3
   @nrwl/detox : Not Found
   @nrwl/devkit : 15.6.3
   @nrwl/esbuild : Not Found
   @nrwl/eslint-plugin-nx : 15.6.3
   @nrwl/expo : Not Found
   @nrwl/express : 15.6.3
   @nrwl/jest : 15.6.3
   @nrwl/js : 15.6.3
   @nrwl/linter : 15.6.3
   @nrwl/nest : Not Found
   @nrwl/next : 15.6.3
   @nrwl/node : 15.6.3
   @nrwl/nx-cloud : 15.0.3
   @nrwl/nx-plugin : Not Found
   @nrwl/react : 15.6.3
   @nrwl/react-native : Not Found
   @nrwl/rollup : 15.6.3
   @nrwl/schematics : Not Found
   @nrwl/storybook : 15.6.3
   @nrwl/web : 15.6.3
   @nrwl/webpack : 15.6.3
   @nrwl/workspace : 15.6.3
   @nrwl/vite : Not Found
   typescript : 4.9.3

PatrickMennen · 2023-05-31T06:23:14Z

For me temporarily switching node versions from 16 to 20 fixed it

ajwootto · 2023-06-08T17:03:17Z

same issue trying to upgrade from nx 14 to nx 16, using yarn 3.6:


 >  NX   Report complete - copy this into the issue template

   Node : 18.16.0
   OS   : darwin arm64
   yarn : 3.6.0

   nx : 14.7.11
   @nrwl/angular : Not Found
   @nrwl/cypress : 16.3.2
   @nrwl/detox : 16.3.2
   @nrwl/devkit : 16.3.2
   @nrwl/eslint-plugin-nx : 16.3.2
   @nrwl/expo : Not Found
   @nrwl/express : Not Found
   @nrwl/jest : 16.3.2
   @nrwl/js : 16.3.2
   @nrwl/linter : 16.3.2
   @nrwl/nest : Not Found
   @nrwl/next : 16.3.2
   @nrwl/node : 16.3.2
   @nrwl/nx-cloud : Not Found
   @nrwl/nx-plugin : Not Found
   @nrwl/react : 16.3.2
   @nrwl/react-native : 16.3.2
   @nrwl/schematics : Not Found
   @nrwl/storybook : Not Found
   @nrwl/web : 16.3.2
   @nrwl/workspace : 16.3.2
   typescript : 5.0.4
   ---------------------------------------
   Local workspace plugins:
   ---------------------------------------
   Community plugins:
   	 @nrwl/webpack: 16.3.2
   	 ```

mpellerin42 · 2023-06-12T09:42:58Z

Same issue for me:

 >  NX   Report complete - copy this into the issue template

   Node : 18.15.0
   OS   : darwin arm64
   yarn : 3.3.1
   
   nx                      : 15.9.2
   @nrwl/js                : 15.9.2
   @nrwl/jest              : 15.9.2
   @nrwl/linter            : 15.9.2
   @nrwl/workspace         : 15.9.2
   @nrwl/angular           : 15.9.2
   @nrwl/cli               : 15.9.2
   @nrwl/cypress           : 15.9.2
   @nrwl/devkit            : 15.9.2
   @nrwl/eslint-plugin-nx  : 15.9.2
   @nrwl/node              : 15.9.2
   @nrwl/tao               : 15.9.2
   @nrwl/webpack           : 15.9.2
   typescript              : 4.9.5
   ---------------------------------------
   Community plugins:
   @nxext/vite   : 15.1.0
   @nxext/svelte : 15.7.0
   ng-mocks      : 14.8.0
   nx-electron   : 15.0.0-alpha.0

ChrisSargent · 2023-06-13T07:58:41Z

For me temporarily switching node versions from 16 to 20 fixed it

unfortunately not for me. Still failing with:

> % yarn nx migrate latest                 
/bin/sh: /var/folders/gr/94gk9mr11w90b8q6jt1701sm0000gn/T/tmp-96357-ymYkluNVHP0f/node_modules/.bin/nx: No such file or directory
Command failed: /var/folders/gr/94gk9mr11w90b8q6jt1701sm0000gn/T/tmp-96357-ymYkluNVHP0f/node_modules/.bin/nx _migrate latest

Would be really nice to get some input from the NX team on this... but then they also have more than 500 open issues :-(

Also get this after manually updating packages to @nx16...

ChrisSargent · 2023-06-13T09:15:25Z

Was able to get this to work by directly running ./node_modules/.bin/nx _migrate latest twice from the repo root. First pass seemed to update the packages, second pass created the migrations file.

igilham · 2023-06-26T13:22:38Z

I have the same issue but the workaround works: ./node_modules/.bin/nx _migrate latest

yarn nx report

 >  NX   Report complete - copy this into the issue template

   Node   : 18.12.1
   OS     : darwin arm64
   yarn   : 3.5.1
   Hasher : Native
   
   nx             : 16.3.2
   @nx/workspace  : 16.3.2
   @nx/devkit     : 16.3.2
   @nrwl/tao      : 16.3.2
   nx-cloud       : 16.0.5
   typescript     : 4.9.5

noamfeldman · 2023-07-02T11:30:23Z

Same issue happened to me as well when running the following. I was trying to upgrade from "nx": "15.7.0" to the latest:

> % nx migrate latest                 
/bin/sh: /var/folders/gr/{some temp directories path}/node_modules/.bin/nx: No such file or directory
Command failed: /var/folders/gr/{some temp directories path}/node_modules/.bin/nx _migrate latest

I took #16338 (comment) as a workaround and it pass.
Please note that the 2nd step is the actual migration:
yarn nx migrate --run-migrations - it will fail as well with the same error above.
Instead I run: yarn nx _migrate --run-migrations for this step as well and it passed successfully.

  NX   Report complete - copy this into the issue template

   Node   : 18.12.1
   OS     : darwin-arm64
   yarn   : 3.3.0
   
   nx                 : 16.4.2
   @nx/js             : 16.4.2
   @nx/jest           : 16.4.2
   @nx/linter         : 16.4.2
   @nx/workspace      : 16.4.2
   @nx/cypress        : 16.4.2
   @nx/devkit         : 16.4.2
   @nx/eslint-plugin  : 16.4.2
   @nx/react          : 16.4.2
   @nx/rollup         : 16.4.2
   @nrwl/tao          : 16.4.2
   @nx/web            : 16.4.2
   @nx/webpack        : 16.4.2
   nx-cloud           : 16.0.5
   typescript         : 5.1.6

Our project is a micro-frontend mono-repo based application. It has a container with main menu and user preferences, for the logged in user - this is the "host". Each page is developed by a deferent team and is a "remote" application.
Before the upgrade we could share the host with the remote applications, for example - user selection of dark theme from the main menu affects all remote application (by using zustand as state management across the application).
After the upgrade to NX 16 the state is not shared anymore by default as with version 15.
To overcome it we changed the module-federation.config.js of the host application by adding the following:

  shared: (name, config) => {
    return { ...config, singleton: true };
  },

arvidboivie · 2023-08-21T07:55:49Z

There's another workaround that's setting YARN_NODE_LINKER="node-modules" for your command, i.e. YARN_NODE_LINKER="node-modules" yarn nx migrate latest.

I think the issue is that yarn defaults to pnp mode when used by nx, and it doesn't read your local .yarnrc file to check settings.

igilham · 2023-08-21T08:45:32Z

Interesting. I have the node linker mode set to node0-modules in my config but if it isn't being read then that won't help.

This suggests that the problem may be reproducible in Nx's automated testing in CI by running yarn with a few different configurations.

pongells · 2023-08-21T12:20:34Z

This fails with PNP enabled, as there is no node_modules/.bin/nx.

It is very annoying, but the "workaround" of falling back on node_modules via the YARN_NODE_LINKER="node-modules" environment variable seems to work..

YARN_NODE_LINKER="node-modules" yarn
YARN_NODE_LINKER="node-modules" yarn nx migrate latest
YARN_NODE_LINKER="node-modules" yarn nx migrate --run-migrations
yarn

(the last yarn just deletes all the node_modules that are not necessary because PNP)

ste7en · 2023-08-21T14:42:09Z

Still fails applying migrations with these workarounds. Yarn 3.4.1, Nx 16.7.2, Node 18.13.0.

Edit:
Running YARN_NODE_LINKER="node-modules" NX_MIGRATE_SKIP_INSTALL=1 yarn nx migrate --run-migrations solved my issue as the error comes when Nx runs yarn install before running migrations

nemonemi · 2023-09-05T15:04:51Z

@pongells' solution worked for me,
But this is far from ideal.

meeroslav · 2024-02-02T10:03:13Z

TL; DR; Workaround

For everyone affected by this problem, please follow the workaround proposed by @arvidboivie to use environment variable override. This is needed only for the nx migrate ... commands.

YARN_NODE_LINKER="node-modules" yarn nx migrate latest
yarn # no env var needed
YARN_NODE_LINKER="node-modules" yarn nx migrate --run-migrations

More details

When running nx migrate ... commands to upgrade from version A to version B, we need to run the migration command using the new version B. In order to do this, we generate a temp folder where we install version B and then run the nx migrate pointing to your repo. This works as expected for all cases apart from one. Sadly, that one affects you.

If you use Yarn Berry with nodeLinker set to node-modules everything works fine inside your repo, but that temp folder doesn't know about the nodeLinker settings (which is a bug we need to fix) so it uses the default, PnP installation.

Based on your repo, nx is still expecting to find binary in the node_modules\nx\.bin, but that doesn't exist for PnP, and you end up with that ugly error.

Solution

Now that we know the problem, we'll see the fix is implemented as soon as possible. Sorry for the delay.

Thank you all for your contribution to clarification of this issue.

github-actions · 2024-03-23T00:07:30Z

This issue has been closed for more than 30 days. If this issue is still occuring, please open a new issue with more recent context.

gremlin896 added the type: bug label Apr 17, 2023

AgentEnder added the scope: core core nx functionality label Apr 17, 2023

ajwootto mentioned this issue Jun 8, 2023

chore: update to latest nx DevCycleHQ/js-sdks#502

Merged

ajwootto mentioned this issue Jun 14, 2023

Multiple issues running @nx/js:node executor (entry file must be named main.ts) #17595

Closed

dtslvr mentioned this issue Nov 13, 2023

Upgrade to Nx 17.1 ghostfolio/ghostfolio#2635

Merged

meeroslav self-assigned this Feb 15, 2024

meeroslav mentioned this issue Feb 15, 2024

fix(core): ensure migrate works with yarn PnP #21824

Merged

FrozenPandaz closed this as completed in #21824 Feb 20, 2024

github-actions bot added the outdated label Mar 23, 2024

github-actions bot locked as resolved and limited conversation to collaborators Mar 23, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

nx migrate fails with Yarn berry #16338

nx migrate fails with Yarn berry #16338

gremlin896 commented Apr 17, 2023

pumano commented May 1, 2023

ChrisSargent commented May 18, 2023 •

edited

Loading

ciceropablo commented May 21, 2023

dlarroder commented May 24, 2023

PatrickMennen commented May 31, 2023

ajwootto commented Jun 8, 2023

mpellerin42 commented Jun 12, 2023

ChrisSargent commented Jun 13, 2023 •

edited

Loading

ChrisSargent commented Jun 13, 2023

igilham commented Jun 26, 2023

noamfeldman commented Jul 2, 2023 •

edited

Loading

arvidboivie commented Aug 21, 2023

igilham commented Aug 21, 2023

pongells commented Aug 21, 2023

ste7en commented Aug 21, 2023 •

edited

Loading

nemonemi commented Sep 5, 2023 •

edited

Loading

meeroslav commented Feb 2, 2024

github-actions bot commented Mar 23, 2024

nx migrate fails with Yarn berry #16338

nx migrate fails with Yarn berry #16338

Comments

gremlin896 commented Apr 17, 2023

Current Behavior

Expected Behavior

GitHub Repo

Steps to Reproduce

Nx Report

Failure Logs

Additional Information

pumano commented May 1, 2023

ChrisSargent commented May 18, 2023 • edited Loading

ciceropablo commented May 21, 2023

dlarroder commented May 24, 2023

PatrickMennen commented May 31, 2023

ajwootto commented Jun 8, 2023

mpellerin42 commented Jun 12, 2023

ChrisSargent commented Jun 13, 2023 • edited Loading

ChrisSargent commented Jun 13, 2023

igilham commented Jun 26, 2023

noamfeldman commented Jul 2, 2023 • edited Loading

arvidboivie commented Aug 21, 2023

igilham commented Aug 21, 2023

pongells commented Aug 21, 2023

ste7en commented Aug 21, 2023 • edited Loading

nemonemi commented Sep 5, 2023 • edited Loading

meeroslav commented Feb 2, 2024

TL; DR; Workaround

More details

Solution

github-actions bot commented Mar 23, 2024

ChrisSargent commented May 18, 2023 •

edited

Loading

ChrisSargent commented Jun 13, 2023 •

edited

Loading

noamfeldman commented Jul 2, 2023 •

edited

Loading

ste7en commented Aug 21, 2023 •

edited

Loading

nemonemi commented Sep 5, 2023 •

edited

Loading