Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[BUG] Error installing relative paths using workspaces #3847

Closed
1 task done
lfowlie opened this issue Oct 6, 2021 · 48 comments
Closed
1 task done

[BUG] Error installing relative paths using workspaces #3847

lfowlie opened this issue Oct 6, 2021 · 48 comments
Assignees
Labels
Bug thing that needs fixing Priority 1 high priority issue Release 8.x work is associated with a specific npm 8 release

Comments

@lfowlie
Copy link

lfowlie commented Oct 6, 2021

EDIT

See #3847 (comment) for minimum reproduction case. --@wraithgar


Is there an existing issue for this?

  • I have searched the existing issues

Current Behavior

Running npm install --workspace=<workspace> is resulting in npm exiting with an errorcode and the following stack trace:

290 timing command:install Completed in 2473ms
291 verbose stack TypeError: Cannot set property 'dev' of null
291 verbose stack     at calcDepFlagsStep (/usr/local/lib/node_modules/npm/node_modules/@npmcli/arborist/lib/calc-dep-flags.js:34:21)
291 verbose stack     at visit (/usr/local/lib/node_modules/npm/node_modules/@npmcli/arborist/lib/calc-dep-flags.js:12:20)
291 verbose stack     at visitNode (/usr/local/lib/node_modules/npm/node_modules/treeverse/lib/depth-descent.js:57:25)
291 verbose stack     at next (/usr/local/lib/node_modules/npm/node_modules/treeverse/lib/depth-descent.js:44:19)
291 verbose stack     at depth (/usr/local/lib/node_modules/npm/node_modules/treeverse/lib/depth-descent.js:82:10)
291 verbose stack     at depth (/usr/local/lib/node_modules/npm/node_modules/treeverse/lib/depth.js:27:12)
291 verbose stack     at calcDepFlags (/usr/local/lib/node_modules/npm/node_modules/@npmcli/arborist/lib/calc-dep-flags.js:10:15)
291 verbose stack     at Arborist.[copyIdealToActual] (/usr/local/lib/node_modules/npm/node_modules/@npmcli/arborist/lib/arborist/reify.js:1375:7)
291 verbose stack     at Arborist.reify (/usr/local/lib/node_modules/npm/node_modules/@npmcli/arborist/lib/arborist/reify.js:151:35)
291 verbose stack     at async Install.install (/usr/local/lib/node_modules/npm/lib/install.js:170:5)
292 verbose cwd /usr/src
293 verbose Linux 5.10.47-linuxkit
294 verbose argv "/usr/local/bin/node" "/usr/local/bin/npm" "install" "--workspace=scheduler"
295 verbose node v14.16.1
296 verbose npm  v7.24.2
297 error Cannot set property 'dev' of null
298 verbose exit 1

Dockerfile for container I'm running the command in:

FROM node:14.16.1
RUN npm install -g npm@7.24.2

RUN mkdir -p /usr/src/
WORKDIR /usr/src

COPY ./package.json /usr/src/
COPY ./scheduler/package.json /usr/src/scheduler/
RUN npm install --workspace=scheduler

node_modules seem to successfully install but npm exits with an error code

Expected Behavior

npm install completes without raising an error

Steps To Reproduce

  1. Setup an npm project with workspaces
  2. Create a docker container according to the above specifications, with one of the commands attempting to run npm install for a specific workspace
  3. Attempt to build the container
  4. npm installs node_modules successfully but exits with error

Environment

  • OS: Debian GNU/Linux 9 (stretch)
  • Node: 14.16.1
  • npm: 7.24.2
@lfowlie lfowlie added Bug thing that needs fixing Needs Triage needs review for next steps Release 7.x work is associated with a specific npm 7 release labels Oct 6, 2021
@niekcandaele
Copy link

Had the same issue but with npm ci --only=production --workspaces.

However, the command was working locally for me. It failed inside Docker. So I compared my host environment to the Docker one and noticed the NPM versions were slightly different.

FROM node:16-alpine # Uses NPM v7.24.0

Locally I was using v7.18.1.

FROM node:16-alpine as runner

RUN npm install -g npm@7.18.1

# < project specific stuff here >

RUN npm ci --only=production --workspaces

CMD [ "npm", "start" ]

Works :)

Hope this helps narrowing down the issue.

@Pitasi
Copy link

Pitasi commented Oct 19, 2021

I have this same problem using npm@7.24 and npm@8.

Works fine with npm@7.18.1.

@lfowlie
Copy link
Author

lfowlie commented Nov 1, 2021

Thanks for the comments @niekcandaele and @Pitasi. I'll try 7.18.1 and see if that works

@lfowlie
Copy link
Author

lfowlie commented Nov 4, 2021

Verified that npm@7.18.1 does work, thanks!! Leaving open as this should probably be fixed for newer versions

@mlaopane
Copy link

I had the same issue with npm@8.1.0 and the error disappeared with npm@7.18.1.
However I managed to make it work with npm@8.1.0 by fixing my Dockerfile.
The issue came from missing workspaces package.json which were dependencies of the package I was trying to install.

In my case, given that home-ui depends on core, I did the following fixes :

Before

FROM node:16-buster

COPY package*.json ./
COPY packages/home-ui/package.json ./packages/home-ui/

RUN npm ci -w packages/home-ui

After

FROM node:16-buster

COPY package*.json ./
COPY packages/core/package.json ./packages/core/
COPY packages/home-ui/package.json ./packages/home-ui/

RUN npm ci -w packages/home-ui

@chriskinsman
Copy link

chriskinsman commented Jan 27, 2022

Hitting this same issue using workspaces with npm@8.1.2 and npm@8.3.2

npm ERR! Cannot set properties of null (setting 'dev')

@Pavel-Husakouski
Copy link

npm@8.3.2 from time to time return this error on the package adding but the next attempt is always successful.

@dsuresh-ap
Copy link

Is there any workaround for this issue without downgrading to npm 7.18?

@winston0410
Copy link

+1 any workaround for this issue?

@orvn
Copy link

orvn commented Feb 9, 2022

This still seems to be an issue in npm 8, so I reverted to node 16.4.2, which was the last version to ship with npm 7.18.

An alternative when installing a workspace as a dependency of another workspace is to just write the directory path into the package.json of the workspace that wants the dependency. This works with npm 8, but I opted to stay on the safe side and keep 7.18 to avoid potential future problems.

{
  "dependencies": {
    "@my-project/another-workspace": "file:packages/another-workspace"
}

@Pavel-Husakouski
Copy link

Pavel-Husakouski commented Feb 11, 2022

Another similar issue with node 17.3 npm 8.3.2: Cannot set properties of null (setting 'peer')
The workaround is to delete the node_modules folder at the root package folder.

@domgenv
Copy link

domgenv commented Mar 4, 2022

I'm on node v16.13.2 and npm 8.1.2 and I ran into this problem too. I have a monorepo with three folders: common, web, and server. The web and server both use the common folder as a dependency. Project is structured like this...

/common
/web
/server
package.json

I had to update the package.json from this...

{
  "scripts": {
    "install-common:server": "npm install ./common --workspace server",
    "install-common:web": "npm install ./common --workspace web",
  },
  "workspaces": [
    "common",
    "server",
    "web"
  ]
}

To this...

{
  "scripts": {
    "install-common:server": "cd server && npm install --save ../common",
    "install-common:web": "cd web && npm install --save ../common",
  },
  "workspaces": [
    "common",
    "server",
    "web"
  ]
}

I'm new to using workspaces, but I think this likely defeats the purpose of using workspaces since each folder will now be using it's own version of common instead of using a shared folder referenced by symlinks. But this is my workaround for now.

@ljharb
Copy link
Contributor

ljharb commented Mar 4, 2022

@domgenv npm 8.1.2 is very old; can you try with v8.5.3?

@domgenv
Copy link

domgenv commented Mar 4, 2022

Thanks for the suggestion, @ljharb. I upgraded npm to 8.5.3 and it looks like it's still doing the same thing. I'm seeing the same thing in ovrn comment where the folder path is being removed, so the dependency is being added as file:common instead of file:../common.

@sergeyampo
Copy link

sergeyampo commented Mar 20, 2022

I hope there will be a solution soon because this error literally shows that npm workspaces doesn't work. Only thing working for me is to downgrade npm to 7.18.1 version.

@dztek
Copy link

dztek commented Mar 28, 2022

For me this works:
npm -w {ws} ci --install-peer-deps

then from root:
npm install

To make sure it's all wired up correctly:
npm ls

@evelant
Copy link

evelant commented Apr 20, 2022

Also running into this bug. npm 8.7.0 crashes with npm ERR! Cannot set properties of null (setting 'dev')

Like others have said npm install -g npm@7.18.1 reverts to a version that doesn't contain this bug

@sergeyampo
Copy link

Also running into this bug. npm 8.7.0 crashes with npm ERR! Cannot set properties of null (setting 'dev')

Like others have said npm install -g npm@7.18.1 reverts to a version that doesn't contain this bug

To keep using last node version with workspaces I decided to replace npm with yarn on the project

@mnapoli
Copy link

mnapoli commented May 2, 2022

I had the same problem with this setup:

package.json
foo/
  package.json
packages/
  bar/
    package.json

Switching to this layout solved the problem for me:

package.json
foo/
  package.json
bar/
  package.json

That also means that I changed foo/package.json:

  "dependencies": {
-    "@my-org/bar": "file:./packages/bar",
+    "@my-org/bar": "file:./bar",
    ...

I would conclude that NPM workspaces don't support packages that are not in the same directory.

@wraithgar
Copy link
Member

I'm having trouble finding any comments in here that show how to reproduce this. A lot reference package.json files but don't specify what's in them.

For starters, the package specs for workspaces should not be file they should reference the version that's in the workspace. npm will use the workspace automatically if the names match.

So as an example

$ npm init -y; npm init -y -w packages/workspace-a;npm init -y -w workspace-b
$ npm install
$ $ npm ls
ws@1.0.0 /Users/wraithgar/Development/npm/scratch/ws
├── workspace-a@1.0.0 -> ./packages/workspace-a
└── workspace-b@1.0.0 -> ./workspace-b
$ npm i workspace-a -w workspace-b
$ $ npm ls
ws@1.0.0 /Users/wraithgar/Development/npm/scratch/ws
├── workspace-a@1.0.0 -> ./packages/workspace-a
└─┬ workspace-b@1.0.0 -> ./workspace-b
  └── workspace-a@1.0.0 deduped -> ./packages/workspace-a
$ cat workspace-b/package.json|grep workspace-a
    "workspace-a": "^1.0.0"

This setup references workspaces at multiple directory levels, which rely one on another, without this error.

@wraithgar wraithgar added the Needs Discussion is pending a discussion label May 3, 2022
@hitesh-sourcefuse
Copy link

it is happening in the ".nvm/versions/node/v14.17.2/lib/node_modules/npm/node_modules/@npmcli/arborist/lib/calc-dep-flags.js" on line 34 as target is set to null
ArboristLink { name: '@services/common', packageName: 'common', location: 'a-service/node_modules/@service/common', path: '/Projects/app-api/services/a-service/node_modules/@services/common', realpath: '/Projects/app-api/services/pacakages/common', resolved: 'file:../../../pacakages/common', edgesIn: Set(1) { { a-service prod @services/common@file:../pacakages/common } }, target: null }

@shirshendubhowmick
Copy link

Problem exists with npm v8.7+ also.

What I have noticed is this happens while using npm workspaces and one of the dependency installed as file:...

@cowens-genvidtech
Copy link

@wraithgar If that is the case and workspace dependency file type references are disallowed, it should be documented. Nothing in the current NPM documentation suggests this setup would result in any kind of problem. If possible, it would be nice for a more useful error to be emitted by the NPM cli in this scenario. The current error is extremely misleading.

@mikemaccana
Copy link
Contributor

$ npm --version
8.10.0

$ npm i -ws
npm ERR! Cannot set property 'dev' of null

@xenobytezero
Copy link

Also happening for me, trying to set a dependency between two packages in a workspace

C:\Projects\Project>npm install @mypkg/common-node --save-dev -w @mypkg/components
npm ERR! Cannot set properties of null (setting 'dev')
  • Node 16.16
  • NPM 8.12.2

@kokokenada
Copy link

kokokenada commented Jul 21, 2022

Same problem:

npm: 8.15.0
v16.16.0

And thanks @shirshendubhowmick - removing all the file: dependencies from all the package.json's seems to work around the problem.

@alexFaunt
Copy link

alexFaunt commented Aug 18, 2022

Reproduction

I ran into this issue when I was trying to set up workspaces (as part of turborepo) without really knowing what I was doing

This is a small reproduction of what I did which produced this error https://github.com/alexFaunt/npm-cli-3847

Workaround

The manual fix i used, is explained in the readme, but essentially in your package-lock.json you'll find an entry similar to

    "apps/app-1/node_modules/package-1": {
      "resolved": "apps/app-1/packages/package-1",
      "link": true
    },

Notice that it's of the format {{APP_NAME}}/node_modules/{{PACKAGE_NAME}} - this entry was problematic (in my case).

Manually deleting it, and removing the dependency from app-1 manually stopped the errors occurring for me

Hope that helps someone!

@wraithgar wraithgar changed the title [BUG] TypeError: Cannot set property 'dev' of null when using --workspace flag [BUG] Error installing relative paths using workspaces Aug 18, 2022
@wraithgar
Copy link
Member

The bug is with installing relative paths using workspaces.

Minimum reproduction:

$ npm init -y -w workspaces/a -w workspaces/b --include-workspace-root
$ npm i ./workspaces/a -w b

Also breaks:

$ npm init -y -w workspaces/a -w workspaces/b --include-workspace-root
$ cd workspaces/b
$ npm i ../../workspaces/a

@wraithgar wraithgar added Priority 1 high priority issue Release 8.x work is associated with a specific npm 8 release and removed Needs Triage needs review for next steps Release 7.x work is associated with a specific npm 7 release Needs Discussion is pending a discussion labels Aug 18, 2022
@dclark
Copy link

dclark commented Aug 19, 2022

I also hit this error for a fresh workspace setup with no node_modules directories.

npm 8.17.0
node v16.16.0

Running

$npm run npm-install -ws

Where npm-install was a script in the workspaces to action npm install.

npm ERR! Cannot set properties of null (setting 'dev')

This was shown from several of the workspaces, re-running the command didn't produce the error.

However this is avoided by first running npm install in the workspace root.

$npm install

Then running

$npm run npm-install -ws

@wraithgar wraithgar self-assigned this Aug 23, 2022
@ThePlenkov
Copy link

Just having an exact issue with 8.12.1. I'm developing a devcontainer. And what I also see - the dependency link is created in node_modules. But I cannot navigate it:
image

@ThePlenkov
Copy link

ThePlenkov commented Aug 25, 2022

Well.. the issue is a little bit more strange as I expected. I've tried to create a symbolic link manually.. Here is what I noticed:

If I create a link like this

~/git_projects/bks/packages/order-search-graphql/node_modules/@finsys$ ln -s ./../../../order-search-client order-search-client

it works, but if I create a symlink like that, just going 1 step up

~/git_projects/bks/packages/order-search-graphql/node_modules$ ln -s ./../../order-search-client @finsys/order-search-client

It creates a symlink - but that link already doesn't lead to anywhere. What's the logic of creating a symlink during installation from the file?

P.S.: I have figured it out - when we create a symlink with a relative path. like ln -s {{target}} {{symlink}} - we may use nested symlink name like @finsys/order-search-client but.. target has to be relative to the location of symlink - not to the location of your pwd. Can it be the case for npm? That it just points to a wrong place? By the way - is not a devcontiner issue - in WSL is same.

@wraithgar
Copy link
Member

Fixed by #5350

@ThePlenkov
Copy link

By the way. May be not directly relates to an issue - but I was able to resolve my issue by actual removing the dependency and using

"workspaces": [
    "packages/*"
  ]

in the root package and "main":"src/index" (default in nx) in each of packages.
In case if ts-node is loaded - it may be also index.ts file which will be compiled on the fly.

So with this way I managed even not to use relative dependencies. ( in my case nx injects them later during build )

@leegee
Copy link

leegee commented Sep 1, 2022

Very strange - started getting this issue just now, having made no memorable change to the project structure.

Just upgraded npm to 8.19.0 on node v18.4.0 and still have the issue.

@wraithgar
Copy link
Member

@leegee The bug as described here was fixed by the linked issue. You'll need to create a new issue with steps to reproduce so that whatever bug you are experiencing can be fixed.

@yelinrachel
Copy link

When tried to install a specific workspace with internal dependencies, I stopped getting the above error (npm ERR! Cannot set properties of null (setting 'dev')) when installed also the "root" workspace.
I used the following command:
npm i --worspaces=false --include-workspace-root=true --workspace=<workspace-name>

@solsson
Copy link

solsson commented Dec 2, 2022

Still getting the issue, and based on #5350 (comment) saying

This was refactoring and cleanup trying to debug an issue, the bug still stands but this code allowed us to find where the bug starts, to be fixed later.

maybe the merge there should not have closed this issue?

I'm aware that comments above say the issue was fixed, but I can't find a PR that claims to be an actual fix.

Update: actually there seems two be two different error messages

My error is the latter. Might mean that #4064 shouldn't have been closed as a duplicate.

@kael-shipman
Copy link

Still getting this error. In my particular case, it happens when I bump the versions of my workspace packages to a pre-release version number, such as 8.0.0-rc1. That's literally the only difference between a working build and a broken one with this error.

@Nikoms
Copy link

Nikoms commented Feb 23, 2023

I solved this error by changing this:

"@yep/common": "file:packages/common",

into this:

"@yep/common": "^1.0.0",

But of course, the package-lock.json also needs to change, so I removed my package properly (npm r) and reinstalled it (npm i @yep/common -w @yep/app1). I don't know why, at first, it uses file: and now the version, but it feels more natural. I probably changed the config somewhere in the meantime, or used the package folder instead of the package name when I installed it the first time

heath-freenome added a commit to heath-freenome/react-jsonschema-form that referenced this issue Apr 13, 2023
- Sadly this causes an issue with npm install
npm/cli#3847
@tswaters
Copy link

tswaters commented Dec 7, 2023

I hit this same thing -- steps taken:

  • update "package": "file:/path/to/package" references to "package": "0.0.0"
  • update all version numbers to 0.0.0
  • rebuild package-lock by removing node_modules, package-lock and running npm i

Even then, it still did it... But only in docker. The reason was I was selectively copying only the packages a given service needed. Well, I missed one -- once it was added, the error went away.

Definitely agree with with @ariofrio says above, (btw, your post really helped to diagnose my own issue, thx!)

Maybe npm should check that the directory exists and give an appropriate error instead of the cryptic one it currently gives, helping people debug the problem more easily.

igrep added a commit to custard-lang/custard that referenced this issue Apr 18, 2024
The code before this change was a workaround for npm's bug (Ref. npm/cli#3847)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Bug thing that needs fixing Priority 1 high priority issue Release 8.x work is associated with a specific npm 8 release
Projects
None yet
Development

No branches or pull requests