Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Already on GitHub? Sign in to your account

Danger/breakage of npm shrinkwrap with de-duped dev+production dependencies #4036

Closed
aseemk opened this Issue Oct 24, 2013 · 4 comments

Comments

Projects
None yet
5 participants

aseemk commented Oct 24, 2013

Overview

It was hard to come up with a title for this, but the bug itself is straightforward:

If you install production dependency P and dev dependency D, where P happens to depend on D as well, shrinkwrapping and installing becomes unpredictable, non-deterministic, and in some cases, broken.

Example 1

Install dev dependency first, then production dependency:

npm init
npm install request@2.x --save-dev
npm install strong-agent --save
npm shrinkwrap

Result: missing needed nested dependency.

Details: https://gist.github.com/aseemk/7140369

Example 2

The opposite order:

npm init
npm install strong-agent --save
npm install request@2.x --save-dev
npm shrinkwrap

So far so good, but re-installing from this shrinkwrap then re-shrinkwrapping...

rm -rf node_modules/
npm install
npm shrinkwrap

Result: missing needed nested dependency.

Details: https://gist.github.com/aseemk/7140458

Reasons

There are other similar cases I've run into. I'd say the problem stems from a combination of:

  1. npm install goes breadth-first, i.e. it installs top-level dev dependencies before recursing into a production dependency and examining the shrinkwrap for it again.
  2. npm install de-dupes by default, which makes a given invocation unpredictable and non-deterministic, since it depends on the state of the environment at that time.

These two together don't play nicely with shrinkwrapping: dev dependencies get installed at higher levels, then they're de-duped at nested levels, even if they're needed as production dependencies.

Request

My request: can npm install not de-dupe by default? This seems like it would be so good from npm's philosophy of local dependencies as well: it'd prevent dependency X from altering the state of shared/de-duped dependency Y.

Contributor

luk- commented Oct 24, 2013

You can work around this by using npm shrinkwrap --dev

aseemk commented Oct 24, 2013

Indeed, I arrived at the same workaround — sorry for not mentioning that!

That feels like a band-aid though: e.g. if we uninstall dependency D later, again dependency P is broken — even with a --dev shrinkwrap. Re-shrinkwrapping won't work until the nested dependency D is met, and you won't be able to re-install dependency P without ignoring the shrinkwrap, but that'll ignore dependency P's other nested dependencies you had shrinkwrapped. What a mess. I've been in messes like this multiple times now, and it's not fun.

Contributor

luk- commented Oct 24, 2013

Yes, I agree it's frustrating and unintuitive.

Contributor

rlidwka commented Oct 29, 2013

looks like a duplicate of #3675

@domenic domenic closed this Apr 19, 2014

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment