[BUG] npm update --depth=N underperforms or crashes

[n1ngu]

I think there is a series of related issues regarding this
#1627
#944
#355

I create this issue with the hope that some additional information will help solving this situation

Current Behavior:

For a deep enough dependency tree and as the requested update depth is increased, the command will take increasing arbitrary time and memory to update until it crashes.

Also, it will produce a slightly different lock file as compared to a rm package-lock.json && npm install.

For reference, in a rather simple application I am working, the npm list command will yield about 4 thousand lines, and the deepest dependency is at level 11 due to the following dependency chain:

+-- nuxt@2.14.12
  +-- @nuxt/webpack@2.14.12
    +-- webpack@4.46.0
      +-- micromatch@3.1.10
        +-- snapdragon@0.8.2
          +-- base@0.11.2
            +-- cache-base@1.0.1
              +-- has-value@1.0.0
                +-- has-values@1.0.0
                  +-- is-number@3.0.0
                    +-- kind-of@3.2.2
                      +-- is-buffer@1.1.6

Also, for reference, this is the time progression for an increasing depth update until it crashed at 20:

Actual times are higher due to network communication with the registry, I guess. Sorry I haven't averaged the times with multiple runs, statistics for my setup would be too time-expensive. Note that

• depth 12 still takes longer that 11 even if there are no dependencies at that level
• depth >9 diverges from the linear tendency for lower depths
• depth 20 crashes
• locked final dependencies are different than as if I had run rm package-lock.json && npm install

Expected Behavior:

Given the pointers to use the magic --depth=9999 in the documentation, I'd expect this command:

1. not to crash
2. not to take arbitrary cputime and memory
3. stop taking more and more time and memory once a project's actual dependency depth is reached
4. to perform somewhat similar to rm package-lock.json && npm install?
5. to produce an exact lockfile as rm package-lock.json && npm install?

OR

I'd expect the documentation to advise against doing this? Yet, this would mean there is no actual way to update an only package plus its whole dependency subtree?

Steps To Reproduce:

My project requirements are as follows

  "dependencies": {
    "@ianwalter/vuex-reset": "x",
    "@nuxtjs/axios": "x",
    "@nuxtjs/dotenv": "x",
    "@nuxtjs/svg-sprite": "x",
    "@nuxtjs/toast": "x",
    "babel-helper-vue-jsx-merge-props": "x",
    "babel-plugin-syntax-jsx": "x",
    "babel-plugin-transform-vue-jsx": "x",
    "cypress": "x",
    "nuxt": "x",
    "start-server-and-test": "x",
    "stylelint-config-sass-guidelines": "x",
    "vue-infinite-loading": "x",
    "vue-signature-pad": "x",
    "vuex-persistedstate": "x"
  },
  "devDependencies": {
    "@babel/core": "x",
    "@babel/preset-env": "x",
    "@nuxtjs/eslint-config": "x",
    "@nuxtjs/eslint-module": "x",
    "@nuxtjs/stylelint-module": "x",
    "@nuxtjs/vuetify": "x",
    "@vue/test-utils": "x",
    "babel-core": "x",
    "babel-eslint": "x",
    "babel-loader": "x",
    "babel-preset-vue": "x",
    "eslint": "x",
    "eslint-loader": "x",
    "eslint-plugin-cypress": "x",
    "eslint-plugin-nuxt": "x",
    "husky": "x",
    "jest": "x",
    "jest-serializer-vue": "x",
    "lint-staged": "x",
    "node-sass": "x",
    "s3-deploy": "x",
    "sass": "x",
    "stylelint": "x",
    "vue-jest": "^4.0.0-0"
  },

Use them in a package.json and run npm install && npm update --depth=N

The different locking is hard to reproduce. I hope #944 (comment) succeeds

Environment:

• OS: Ubuntu 20.04
• Docker: version 20.10.1, build 831ebea
• Node: 13.10.1
• npm: 6.13.7

[ljharb]

Why are you using depth at all? like what happens if you run npm update by itself? What about with npm 7 (where i think the depth option is removed)?

[n1ngu]

@ljharb As the documentation says:

As of npm@2.6.1, the npm update will only inspect top-level packages. Prior versions of npm would also recursively inspect all dependencies. To get the old behavior, use npm --depth 9999 update.

And AFAIU there are no changes here for npm 7 https://docs.npmjs.com/cli/v7/commands/npm-update

That's why people is trying to use this flag, so that whole dependency (sub)trees get updates, not only first-order dependencies listed in packages.json!

[ljharb]

@n1ngu ah, thanks.

[darcyclarke]

npm v6 is no longer in active development; We will continue to push security releases to v6 at our team's discretion as-per our Support Policy.

If your bug is preproducible on v7, please re-file this issue using our new issue template.

If your issue was a feature request, please consider opening a new RRFC or RFC. If your issue was a question or other idea that was not CLI-specific, consider opening a discussion on our feedback repo

[n1ngu]

For the record, this is fixed in npm v7 because the --depth option of the update command has disappeared by virtue of RCF0019 https://github.com/npm/rfcs/blob/latest/implemented/0019-remove-update-depth-option.md

I am happy with the solution but it would have been nice to communicate that there was an ongoing effort to fix this.

Edit: of course @ljharb hinted this would happen, but some links to wherever this was being discussed would have saved some concerns in a couple of related issues.