[npm5]npm i with package moved lead to an incorrect package.json in node_modules, which cause Cannot find module

[yyjdelete]

I'm opening this issue because:

• npm is crashing.
• npm is producing an incorrect install.
• npm is doing something I don't understand.
• Other (see below for feature requests):

What's going wrong?

1. A lot attribute in package.json including main is lost after move, which makes it FAILED to be require(). "> " is inserted to description, _location is not changed.(Major)
2. Step 2 and 3 should get the extra same result like it in npm@4.(Maybe it's another issue with the same test-case)

https://gist.github.com/yyjdelete/6c3200abc8046852329dbea0003f0e26

How can the CLI team reproduce the problem?

>npm i babel-core@6 babel-preset-stage-0@6
+ babel-preset-stage-0@6.24.1
+ babel-core@6.25.0
added 84 packages in 10.828s
# Found babel-helpers@6 at `node_modules/babel-helpers`, everything is good.

>npm i babel-core@next babel-preset-stage-0@next
+ babel-preset-stage-0@7.0.0-alpha.12
+ babel-core@7.0.0-alpha.12
added 109 packages, removed 51 packages and updated 2 packages in 11.64s
# Found babel-helpers@7 at `node_modules/babel-core/node_modules/babel-helpers`, everything is good.

> npm i babel-core@next babel-preset-stage-0@next
+ babel-preset-stage-0@7.0.0-alpha.12
+ babel-core@7.0.0-alpha.12
added 4 packages, removed 14 packages, updated 2 packages and moved 40 packages in 6.284s
###
# Found babel-helpers@7 at `node_modules/babel-helpers`, `package.json` is strange.
###
---

# Remove node_modules and install again
> npm i
added 132 packages in 5.365s
# Found babel-helpers@7 at `node_modules/babel-helpers`, everything is good.

supporting information:

• npm -v prints:5.0.3/5.0.3-canary.12(npmc)
• node -v prints:v8.1.1
• npm config get registry prints:http://registry.npm.taobao.org/
• Windows, OS X/macOS, or Linux?:Windows
• Network issues:
  • Geographic location where npm was run:
  • I use a proxy to connect to the npm registry.
  • I use a proxy to connect to the web.
  • I use a proxy when downloading Git repos.
  • I access the npm registry via a VPN
  • I don't use a proxy, but have limited or unreliable internet access.
• Container:
  • I develop using Vagrant on Windows.
  • I develop using Vagrant on OS X or Linux.
  • I develop / deploy using Docker.
  • I deploy to a PaaS (Triton, Heroku).

[yyjdelete]

Not sure whether it's the reason for #16947

[yyjdelete]

UPDATE 20170630
This test case is INVAILD after 5.0.4-canary.9 or #17508, since now add&remove instead of move is used in this case, but the case in floor 1 still VAILD
added 1 package and removed 1 package in 3.165s

---

Also trigger in an simpler case(NOT EXTRA THE SAME).

1. npm init
2. npm i babel-runtime
3. mkdir babel-runtime/node_modules and move core-js to babel-runtime/node_modulesmaually
4. npm i and see moved 1 packages in 1.2345s

And found the same issue in package.json of core-js.
It's also strage if replace step 4 with npm dedupe, also an package move but everything works well.

Log of npm i

npm info it worked if it ends with ok
npm verb cli [ 'C:\\Program Files\\nodejs\\node.exe',
npm verb cli   'C:\\Users\\***\\AppData\\Roaming\\npm\\node_modules\\npm\\bin\\npm-cli.js',
npm verb cli   'i',
npm verb cli   '--loglevel=silly' ]
npm info using npm@5.0.3
npm info using node@v8.1.1
npm verb npm-session 2b18de3a06276e52
npm sill install runPreinstallTopLevelLifecycles
npm sill preinstall x1@1.0.0
npm info lifecycle x1@1.0.0~preinstall: x1@1.0.0
npm sill lifecycle x1@1.0.0~preinstall: no script for preinstall, continuing
npm sill install loadCurrentTree
npm sill install readLocalPackageData
npm sill install loadIdealTree
npm sill install cloneCurrentTreeToIdealTree
npm sill install loadShrinkwrap
npm sill install loadAllDepsIntoIdealTree
npm sill currentTree x1@1.0.0
npm sill currentTree +-- babel-runtime@6.23.0
npm sill currentTree | `-- core-js@2.4.1
npm sill currentTree +-- dedupe@2.1.0
npm sill currentTree +-- regenerator-runtime@0.10.5
npm sill currentTree `-- sigmund@1.0.1
npm sill idealTree x1@1.0.0
npm sill idealTree +-- babel-runtime@6.23.0
npm sill idealTree +-- core-js@2.4.1
npm sill idealTree +-- dedupe@2.1.0
npm sill idealTree +-- regenerator-runtime@0.10.5
npm sill idealTree `-- sigmund@1.0.1
npm sill install generateActionsToTake
npm sill diffTrees action count 1
npm sill diffTrees move core-js@2.4.1
npm sill decomposeActions action count 5
npm sill decomposeActions move core-js@2.4.1
npm sill decomposeActions build core-js@2.4.1
npm sill decomposeActions install core-js@2.4.1
npm sill decomposeActions postinstall core-js@2.4.1
npm sill decomposeActions refresh-package-json core-js@2.4.1
npm sill install executeActions
npm sill doSerial global-install 5
npm verb correctMkdir C:\Users\***\AppData\Roaming\npm-cache\_locks correctMkdir not in flight; initializing
npm verb lock using C:\Users\***\AppData\Roaming\npm-cache\_locks\staging-a69d5a555179e4f7.lock for C:\Users\***\Desktop\piao\x1\node_modules\.staging
npm sill doParallel extract 5
npm sill doReverseSerial remove 5
npm sill doSerial move 5
npm sill move C:\Users\***\Desktop\piao\x1\node_modules\babel-runtime\node_modules\core-js C:\Users\***\Desktop\piao\x1\node_modules\core-js
npm info lifecycle undefined~preuninstall: undefined
npm sill lifecycle undefined~preuninstall: no script for preuninstall, continuing
npm info lifecycle undefined~uninstall: undefined
npm sill lifecycle undefined~uninstall: no script for uninstall, continuing
npm verb unbuild rmStuff undefined 'from' 'C:\\Users\\***\\Desktop\\piao\\x1\\node_modules'
npm verb unbuild rmStuff in C:\Users\***\Desktop\piao\x1\node_modules\babel-runtime\node_modules
npm info lifecycle undefined~postuninstall: undefined
npm sill lifecycle undefined~postuninstall: no script for postuninstall, continuing
npm sill move move existing destination node_modules away C:\Users\***\Desktop\piao\x1\node_modules\core-js\node_modules
npm sill move remove existing destination C:\Users\***\Desktop\piao\x1\node_modules\core-js
npm sill move make sure destination parent exists C:\Users\***\Desktop\piao\x1\node_modules
npm sill move move source node_modules away C:\Users\***\Desktop\piao\x1\node_modules\babel-runtime\node_modules\core-js\node_modules
npm sill move move module dir to final dest C:\Users\***\Desktop\piao\x1\node_modules\babel-runtime\node_modules\core-js C:\Users\***\Desktop\piao\x1\node_modules\core-js
npm info lifecycle undefined~preinstall: undefined
npm sill lifecycle undefined~preinstall: no script for preinstall, continuing
npm sill doSerial finalize 5
npm sill doParallel refresh-package-json 5
npm sill refresh-package-json C:\Users\***\Desktop\piao\x1\node_modules\core-js
npm sill doParallel preinstall 5
npm sill doSerial build 5
npm sill build core-js@2.4.1
npm info linkStuff core-js@2.4.1
npm sill linkStuff core-js@2.4.1 has C:\Users\***\Desktop\piao\x1\node_modules as its parent node_modules
npm verb linkBins core-js@2.4.1
npm verb linkMans core-js@2.4.1
npm sill doSerial global-link 5
npm sill doParallel update-linked 5
npm sill doSerial install 5
npm sill install core-js@2.4.1
npm info lifecycle core-js@2.4.1~install: core-js@2.4.1
npm sill lifecycle core-js@2.4.1~install: no script for install, continuing
npm sill doSerial postinstall 5
npm sill postinstall core-js@2.4.1
npm info lifecycle core-js@2.4.1~postinstall: core-js@2.4.1
npm sill lifecycle core-js@2.4.1~postinstall: no script for postinstall, continuing
npm verb unlock done using C:\Users\***\AppData\Roaming\npm-cache\_locks\staging-a69d5a555179e4f7.lock for C:\Users\***\Desktop\piao\x1\node_modules\.staging
npm sill install runPostinstallTopLevelLifecycles
npm sill build x1@1.0.0
npm info linkStuff x1@1.0.0
npm sill linkStuff x1@1.0.0 has C:\Users\***\Desktop\piao as its parent node_modules
npm verb linkBins x1@1.0.0
npm verb linkMans x1@1.0.0
npm sill install x1@1.0.0
npm info lifecycle x1@1.0.0~install: x1@1.0.0
npm sill lifecycle x1@1.0.0~install: no script for install, continuing
npm sill postinstall x1@1.0.0
npm info lifecycle x1@1.0.0~postinstall: x1@1.0.0
npm sill lifecycle x1@1.0.0~postinstall: no script for postinstall, continuing
npm sill prepublish x1@1.0.0
npm info lifecycle x1@1.0.0~prepublish: x1@1.0.0
npm sill lifecycle x1@1.0.0~prepublish: no script for prepublish, continuing
npm info lifecycle x1@1.0.0~prepare: x1@1.0.0
npm sill lifecycle x1@1.0.0~prepare: no script for prepare, continuing
npm sill install saveToDependencies
npm verb saving []
npm info lifecycle undefined~preshrinkwrap: undefined
npm sill lifecycle undefined~preshrinkwrap: no script for preshrinkwrap, continuing
npm info lifecycle undefined~shrinkwrap: undefined
npm sill lifecycle undefined~shrinkwrap: no script for shrinkwrap, continuing
npm info lifecycle undefined~postshrinkwrap: undefined
npm sill lifecycle undefined~postshrinkwrap: no script for postshrinkwrap, continuing
npm WARN x1@1.0.0 No description
npm WARN x1@1.0.0 No repository field.

npm sill install printInstalled
npm verb exit [ 0, true ]
npm info ok

Log of npm dedupe

npm info it worked if it ends with ok
npm verb cli [ 'C:\\Program Files\\nodejs\\node.exe',
npm verb cli   'C:\\Users\\***\\AppData\\Roaming\\npm\\node_modules\\npm\\bin\\npm-cli.js',
npm verb cli   'dedupe',
npm verb cli   '--loglevel=silly' ]
npm info using npm@5.0.3
npm info using node@v8.1.1
npm verb npm-session 290f5e64b9461e19
npm sill install loadCurrentTree
npm sill install readLocalPackageData
npm sill install loadIdealTree
npm sill install cloneCurrentTreeToIdealTree
npm sill currentTree x1@1.0.0
npm sill currentTree +-- babel-runtime@6.23.0
npm sill currentTree | `-- core-js@2.4.1
npm sill currentTree +-- dedupe@2.1.0
npm sill currentTree +-- regenerator-runtime@0.10.5
npm sill currentTree `-- sigmund@1.0.1
npm sill idealTree x1@1.0.0
npm sill idealTree +-- babel-runtime@6.23.0
npm sill idealTree | `-- core-js@2.4.1
npm sill idealTree +-- dedupe@2.1.0
npm sill idealTree +-- regenerator-runtime@0.10.5
npm sill idealTree `-- sigmund@1.0.1
npm sill dedupe generateActionsToTake
npm sill diffTrees action count 1
npm sill diffTrees move core-js@2.4.1
npm sill decomposeActions action count 5
npm sill decomposeActions move core-js@2.4.1
npm sill decomposeActions build core-js@2.4.1
npm sill decomposeActions install core-js@2.4.1
npm sill decomposeActions postinstall core-js@2.4.1
npm sill decomposeActions refresh-package-json core-js@2.4.1
npm sill install executeActions
npm sill doSerial global-install 5
npm verb correctMkdir C:\Users\***\AppData\Roaming\npm-cache\_locks correctMkdir not in flight; initializing
npm verb lock using C:\Users\***\AppData\Roaming\npm-cache\_locks\staging-a69d5a555179e4f7.lock for C:\Users\***\Desktop\piao\x1\node_modules\.staging
npm sill doParallel extract 5
npm sill doReverseSerial remove 5
npm sill doSerial move 5
npm sill move C:\Users\***\Desktop\piao\x1\node_modules\babel-runtime\node_modules\core-js C:\Users\***\Desktop\piao\x1\node_modules\core-js
npm info lifecycle core-js@2.4.1~preuninstall: core-js@2.4.1
npm sill lifecycle core-js@2.4.1~preuninstall: no script for preuninstall, continuing
npm info lifecycle core-js@2.4.1~uninstall: core-js@2.4.1
npm sill lifecycle core-js@2.4.1~uninstall: no script for uninstall, continuing
npm verb unbuild rmStuff core-js@2.4.1 from C:\Users\***\Desktop\piao\x1\node_modules
npm verb unbuild rmStuff in C:\Users\***\Desktop\piao\x1\node_modules\babel-runtime\node_modules
npm info lifecycle core-js@2.4.1~postuninstall: core-js@2.4.1
npm sill lifecycle core-js@2.4.1~postuninstall: no script for postuninstall, continuing
npm sill move move existing destination node_modules away C:\Users\***\Desktop\piao\x1\node_modules\core-js\node_modules
npm sill move remove existing destination C:\Users\***\Desktop\piao\x1\node_modules\core-js
npm sill move make sure destination parent exists C:\Users\***\Desktop\piao\x1\node_modules
npm sill move move source node_modules away C:\Users\***\Desktop\piao\x1\node_modules\babel-runtime\node_modules\core-js\node_modules
npm sill move move module dir to final dest C:\Users\***\Desktop\piao\x1\node_modules\babel-runtime\node_modules\core-js C:\Users\***\Desktop\piao\x1\node_modules\core-js
npm info lifecycle core-js@2.4.1~preinstall: core-js@2.4.1
npm sill lifecycle core-js@2.4.1~preinstall: no script for preinstall, continuing
npm sill doSerial finalize 5
npm sill doParallel refresh-package-json 5
npm sill refresh-package-json C:\Users\***\Desktop\piao\x1\node_modules\babel-runtime\node_modules\core-js
npm sill doParallel preinstall 5
npm sill doSerial build 5
npm sill build core-js@2.4.1
npm info linkStuff core-js@2.4.1
npm sill linkStuff core-js@2.4.1 has C:\Users\***\Desktop\piao\x1\node_modules as its parent node_modules
npm verb linkBins core-js@2.4.1
npm verb linkMans core-js@2.4.1
npm sill doSerial global-link 5
npm sill doParallel update-linked 5
npm sill doSerial install 5
npm sill install core-js@2.4.1
npm info lifecycle core-js@2.4.1~install: core-js@2.4.1
npm sill lifecycle core-js@2.4.1~install: no script for install, continuing
npm sill doSerial postinstall 5
npm sill postinstall core-js@2.4.1
npm info lifecycle core-js@2.4.1~postinstall: core-js@2.4.1
npm sill lifecycle core-js@2.4.1~postinstall: no script for postinstall, continuing
npm verb unlock done using C:\Users\***\AppData\Roaming\npm-cache\_locks\staging-a69d5a555179e4f7.lock for C:\Users\***\Desktop\piao\x1\node_modules\.staging
npm sill install saveToDependencies
npm verb saving []
npm info lifecycle undefined~preshrinkwrap: undefined
npm sill lifecycle undefined~preshrinkwrap: no script for preshrinkwrap, continuing
npm info lifecycle undefined~shrinkwrap: undefined
npm sill lifecycle undefined~shrinkwrap: no script for shrinkwrap, continuing
npm info lifecycle undefined~postshrinkwrap: undefined
npm sill lifecycle undefined~postshrinkwrap: no script for postshrinkwrap, continuing
npm sill install printInstalled
moved 1 package in 0.283s
npm verb exit [ 0, true ]
npm info ok

The main difference is that a lot of undefined in the one with `npm i`, and the refresh-package-json's path.

---

But this one is not extra the same as the first one. This one doesn't need twice npm i to move, and _location is successfully updated in the case. And the log of first one doesn't contains undefined in move part.

[yyjdelete]

Do some debug, and found the reason is npm i only read simple metaData of modules from package-lock.json of root instead of full package.json, and use it to move package, and the simple metaData was overwrited the package.json.

https://github.com/npm/npm/blob/v5.0.3/lib/install/action/move.js#L29
An simple workaround is remove update-package-json from move, the data in package.json will still be updated by latter refresh-package-json, but not sure if it will cause other issues.

--

Still not sure why twice install with the same pkg&ver is needed to get the final result, maybe also related to package-lock.json, for with npm@4 only one install is needed.

[yyjdelete]

@kenany @zkat Also label this as npm5?

[yyjdelete]

Unable to reproducted in 5.2.0. Maybe fixed as a part of #17677.