Skip to content
This repository has been archived by the owner on Aug 11, 2022. It is now read-only.

npm prune is removing packages installed using git #19356

2 of 4 tasks
jsynowiec opened this issue Dec 8, 2017 · 16 comments
2 of 4 tasks

npm prune is removing packages installed using git #19356

jsynowiec opened this issue Dec 8, 2017 · 16 comments


Copy link

jsynowiec commented Dec 8, 2017

I'm opening this issue because:

  • npm is crashing.
  • npm is producing an incorrect install.
  • npm is doing something I don't understand.
  • Other (see below for feature requests):

What's going wrong?

Packages present under the dependencies in the package.json file that were installed using git/github package links, e.g. github:npm/npm#some-tag are being uninstalled when npm prune is executed.

How can the CLI team reproduce the problem?

  1. Install a package using standard syntax, save it as a dependency,
  2. Install another a package using git+<proto>:, git:// or github: format, save it as dependency.
  3. Run npm prune

Expected: Second package is present.
Actual: Second package is missing.

supporting information:

  • npm -v prints: 5.6.0
  • node -v prints: 8.9.1
  • npm config get registry prints:
  • Windows, OS X/macOS, or Linux?: macOS
  • Network issues (unrelated)

additional information:

Workaround - When a tarball url is used, the package is not removed on prune.

Copy link

iarna commented Dec 9, 2017

--only=production is not required to reproduce this.

Copy link

jsynowiec commented Dec 11, 2017

Don't know if this helps but I've noticed that this.idealTree in the Pruner instance lists git dependencies in the childs prop but also as missingDeps.
Also, shouldPrune, especially isExtraneous(child), returns true for those deps, so they end up in toPrune array.

Copy link

ValYouW commented Dec 18, 2017

For some reason that doesn't happen on windows... npm v5.5.1
Downgraded all the way to v4.6.1 to restore my linux builds...

Copy link

rankida commented Jan 4, 2018

I just encountered this too. Npm v5.6.0.
If my package is a regular semver (1.x.x) then it is correctly passed over by npm prune --production but if it is a git url it is removed.
I am running MacOS and node v8.9.3

@jsynowiec jsynowiec changed the title npm prune --only=production is removing packages installed with git npm prune is removing packages installed using git Jan 7, 2018
Copy link

I'm having this problem too

Copy link

@alphashuro @lucasklaassen (and others to follow) please use the 👍 one the issue #19356 (comment) to vote/confirm, rather than comment. You're not adding anything new and all of us are getting notifications about your comments. If you just want to subscribe for updates, use the subscribe button under Notifications that is available in the right column.

Copy link

Here's a partial output of running npm -ddd install somepackage when a package with git is already installed (eg: en-my-package-name):

npm sill currentTree +-- emojis-list@2.1.0
npm sill currentTree +-- en-my-package-name@1.0.5
npm sill currentTree +-- encodeurl@1.0.1

npm sill idealTree +-- emojis-list@2.1.0
npm sill idealTree +-- encodeurl@1.0.1

npm sill diffTrees remove en-my-package-name@1.0.5

npm sill unbuild en-my-package-name@1.0.5
npm info lifecycle en-my-package-name@1.0.5~preuninstall: en-my-package-name@1.0.5
npm info lifecycle en-my-package-name@1.0.5~uninstall: en-my-package-name@1.0.5
npm verb unbuild rmStuff en-my-package-name@1.0.5 from /path/to/my/repo/node_modules
npm info lifecycle en-my-package-name@1.0.5~postuninstall: en-my-package-name@1.0.5

npm sill remove /path/to/my/repo/node_modules/en-my-package-name

As you can see, en-my-package-name is not listed in the silly logs under the idealTree and subsequently removed.

Copy link

I've tried to debug this further and failed to come up with the underlying reason, hopefully some more information might help....

When returning true for my git installed package here -> I can get the prune to work with a git installed module.

if ( === 'my-git-installed-package') return true
Perhaps github are returning new meta data and now rawSpec isn't behaving correctly?

Also could be something related to a version bump from ?

Copy link

andyghiuta commented Jan 19, 2018

Indeed the childReq.rawSpec is not the same as requested.rawSpec, so the if on the line is skipped, although, they are very similar, eg: git+ssh:// vs git+ssh:// The difference is the commit hash.
It works if after line 56, this would be added:

if (childReq.rawSpec.slice(0, childReq.rawSpec.indexOf('#')) === requested.rawSpec.slice(0, requested.rawSpec.indexOf('#'))) return true

This solution is not ok as it breaks the update of the package, but I don't know enough about npm code to tell where childReq.rawSpec is coming from. I'm guessing from npm-package-arg

Copy link

I've taken a little browse around, this is something of a very simplified view of what may be happening within npm when we install from our github repos to produce the extended package.json files post-install:

const pacote = require('pacote')
var npa = require('npm-package-arg')

const spec = npa('github:some-organisation/some-private-repo#v0.0.0')
pacote.manifest(spec, { cache: './my-cache' }).then(modifiedPackageJson => {
  console.log(modifiedPackageJson) // this looks to be where all the extra attributes come from

I'm going to guess the bug here may be something obscure with how the npm <> npm-package-arg <> pacote packages are interacting?

The strange thing here is that my organisation have been using npm prune --production against our projects for several months with no issues, using the same container build image. Then, a few days ago, all our builds started failing due to our private github modules being pruned - private github modules that have been in our projects since the beginning. For reference, said container is built with:

    nvm install 6.11.5 && nvm install 8.9.0 && nvm alias default 8

I can't explain why our build processes would break overnight, it's possible I'm missing something on my end, but I'd suggest something may changed in a network/api response?

Copy link

shesek commented Jan 30, 2018

A simple temporary (and slow) workaround is to npm prune --production && npm install --production, which in my case results in:

removed 157 packages in 24.553s
added 1 package in 65.331s

That 1 package being added back is the one configured to be installed from git.

Copy link

sairathb commented Feb 5, 2018

Just wanted to add the ripple of this bug, I am using serverless to deploy my project to AWS and now since the prune is removing the git dependencies, my lambda code is failing, @shesek I will not be able to use this as serverless package command is doing it internally,

Is there a work around that I can use for now till this bug is fixed? Thanks

Copy link

@sairathb I'm in the same situation. Just install npm@5.5, clear your NPM cache, and serverless will work properly after that. The problem is only in version 5.6.

Copy link

I was able to reproduce this issue with node@8.9.4 and npm@5.5.1

I rolled back to npm@4.6.1 to resolve the issue.

Copy link

saboya commented Feb 14, 2018

A known workaround is to specify the commit hash in your package.json.

Copy link

Supposedly fixed in 857dab0 and included in npm@5.7 release

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
None yet
None yet

No branches or pull requests