npm just deleted my all files from node_modules

[shimondoodkin]

npm should not delete by itself files when installing new module. just install a new module

[gioragutt]

Can you provide reproduction steps?
The guys in npm can't really do anything to help you if they don't know what went wrong.
Can you reproduce it by installing/upgrading/removing certain packages?

[kenany]

Possibly a duplicate of #16853.

[shimondoodkin]

here is a test case (copy then paste to a terminal)

npm -v

mkdir test
cd test

# test passes:
mkdir install_normal_module__without_save_then_install_another_module
cd install_normal_module__without_save_then_install_another_module/
npm init -y
echo "console.log('test')">index.js
npm install underscore 
ls node_modules/ # the module is there.
npm install lodash --save
ls node_modules/  # the module is still there, ok.
cd ..

# test passes:
mkdir install_and_modify_normal_module_then_install_another_module
cd install_and_modify_normal_module_then_install_another_module/
npm init -y
echo "console.log('test')">index.js
npm install underscore --save
tail -n 1  node_modules/underscore/underscore.js # see no cahnges yet
echo "//modification" >>  node_modules/underscore/underscore.js
tail -n 2  node_modules/underscore/underscore.js # the changes are there
npm install lodash --save
tail -n 2  node_modules/underscore/underscore.js # see changes to node_modules/underscore/underscore.js preserved as expected
cd ..

#test fails:
mkdir install_github_module_then_install_another_module
cd install_github_module_then_install_another_module/
npm init -y
npm install shimondoodkin/chrome-headless-render-pdf --save
ls node_modules/ # the module is there. there are many modules.
npm install lodash --save
ls node_modules/ # folder chrome-headless-render-pdf is removed totally, even not reinstalled as supposed to. also removed all related modules.
cat package.json # still has entry for shimondoodkin/chrome-headless-render-pdf
cd ..


#test fails:
mkdir install_and_modify_github_module_then_install_another_module
cd install_and_modify_github_module_then_install_another_module/
npm init -y
echo "console.log('test')">index.js
npm install shimondoodkin/chrome-headless-render-pdf --save
tail -n 1  node_modules/chrome-headless-render-pdf/index.js
echo "//modification" >> node_modules/chrome-headless-render-pdf/index.js
tail -n 2  node_modules/chrome-headless-render-pdf/index.js
ls node_modules/
npm install lodash --save
ls node_modules/ # folder chrome-headless-render-pdf is removed totally, even not reinstalled as supposed to be
cat package.json # still has entry for  shimondoodkin/chrome-headless-render-pdf
tail -n 2  node_modules/chrome-headless-render-pdf/index.js # expected to see here '// modification', nothing to see, file missing
cd ..

#test fails:
mkdir create_new_module_then_install_another_module
cd create_new_module_then_install_another_module/
npm init -y
echo "console.log('test')">index.js
mkdir -p node_modules/newmodule
cd node_modules/newmodule/
npm init -y
echo "console.log('new module')">index.js
cd ../..
ls node_modules/ # newmodule is there 
npm install lodash --save
ls node_modules/ # the folder newmodule is removed, 
cd ..



cd ..
rm ./test -rf

output:

root@server2:~/app# npm -v
5.6.0
root@server2:~/app#
root@server2:~/app# mkdir test
root@server2:~/app# cd test
root@server2:~/app/test#
root@server2:~/app/test# # test passes:
root@server2:~/app/test# mkdir install_normal_module__without_save_then_install_another_module
root@server2:~/app/test# cd install_normal_module__without_save_then_install_another_module/
root@server2:~/app/test/install_normal_module__without_save_then_install_another_module# npm init -y
Wrote to /root/app/test/install_normal_module__without_save_then_install_another_module/package.json:

{
  "name": "install_normal_module__without_save_then_install_another_module",
  "version": "1.0.0",
  "description": "",
  "main": "index.js",
  "scripts": {
    "test": "echo \"Error: no test specified\" && exit 1"
  },
  "keywords": [],
  "author": "",
  "license": "ISC"
}


root@server2:~/app/test/install_normal_module__without_save_then_install_another_module# echo "console.log('test')">index.js
root@server2:~/app/test/install_normal_module__without_save_then_install_another_module# npm install underscore
npm notice created a lockfile as package-lock.json. You should commit this file.
npm WARN install_normal_module__without_save_then_install_another_module@1.0.0 No description
npm WARN install_normal_module__without_save_then_install_another_module@1.0.0 No repository field.

+ underscore@1.8.3
added 1 package in 0.853s
root@server2:~/app/test/install_normal_module__without_save_then_install_another_module# ls node_modules/ # the module is there.
underscore
root@server2:~/app/test/install_normal_module__without_save_then_install_another_module# npm install lodash --save
npm WARN install_normal_module__without_save_then_install_another_module@1.0.0 No description
npm WARN install_normal_module__without_save_then_install_another_module@1.0.0 No repository field.

+ lodash@4.17.4
added 1 package in 1.127s
root@server2:~/app/test/install_normal_module__without_save_then_install_another_module# ls node_modules/  # the module is still there, ok.
lodash  underscore
root@server2:~/app/test/install_normal_module__without_save_then_install_another_module# cd ..
root@server2:~/app/test#
root@server2:~/app/test# # test passes:
root@server2:~/app/test# mkdir install_and_modify_normal_module_then_install_another_module
root@server2:~/app/test# cd install_and_modify_normal_module_then_install_another_module/
root@server2:~/app/test/install_and_modify_normal_module_then_install_another_module# npm init -y
Wrote to /root/app/test/install_and_modify_normal_module_then_install_another_module/package.json:

{
  "name": "install_and_modify_normal_module_then_install_another_module",
  "version": "1.0.0",
  "description": "",
  "main": "index.js",
  "scripts": {
    "test": "echo \"Error: no test specified\" && exit 1"
  },
  "keywords": [],
  "author": "",
  "license": "ISC"
}


root@server2:~/app/test/install_and_modify_normal_module_then_install_another_module# echo "console.log('test')">index.js
root@server2:~/app/test/install_and_modify_normal_module_then_install_another_module# npm install underscore --save
npm notice created a lockfile as package-lock.json. You should commit this file.
npm WARN install_and_modify_normal_module_then_install_another_module@1.0.0 No description
npm WARN install_and_modify_normal_module_then_install_another_module@1.0.0 No repository field.

+ underscore@1.8.3
added 1 package in 0.65s
root@server2:~/app/test/install_and_modify_normal_module_then_install_another_module# tail -n 1  node_modules/underscore/underscore.js # see no cahnges yet
}.call(this));
root@server2:~/app/test/install_and_modify_normal_module_then_install_another_module# echo "//modification" >>  node_modules/underscore/underscore.js
root@server2:~/app/test/install_and_modify_normal_module_then_install_another_module# tail -n 2  node_modules/underscore/underscore.js # the changes are there
}.call(this));
//modification
root@server2:~/app/test/install_and_modify_normal_module_then_install_another_module# npm install lodash --save
npm WARN install_and_modify_normal_module_then_install_another_module@1.0.0 No description
npm WARN install_and_modify_normal_module_then_install_another_module@1.0.0 No repository field.

+ lodash@4.17.4
added 1 package in 1.438s
root@server2:~/app/test/install_and_modify_normal_module_then_install_another_module# tail -n 2  node_modules/underscore/underscore.js # see changes to node_modules/underscore/underscore.js preserved as expected
}.call(this));
//modification
root@server2:~/app/test/install_and_modify_normal_module_then_install_another_module# cd ..
root@server2:~/app/test#
root@server2:~/app/test# #test fails:
root@server2:~/app/test# mkdir install_github_module_then_install_another_module
root@server2:~/app/test# cd install_github_module_then_install_another_module/
root@server2:~/app/test/install_github_module_then_install_another_module# npm init -y
Wrote to /root/app/test/install_github_module_then_install_another_module/package.json:

{
  "name": "install_github_module_then_install_another_module",
  "version": "1.0.0",
  "description": "",
  "main": "index.js",
  "scripts": {
    "test": "echo \"Error: no test specified\" && exit 1"
  },
  "keywords": [],
  "author": "",
  "license": "ISC"
}


root@server2:~/app/test/install_github_module_then_install_another_module# npm install shimondoodkin/chrome-headless-render-pdf --save

> canvas-prebuilt@1.6.5-prerelease.1 install /root/app/test/install_github_module_then_install_another_module/node_modules/canvas-prebuilt
> node-pre-gyp install

[canvas-prebuilt] Success: "/root/app/test/install_github_module_then_install_another_module/node_modules/canvas-prebuilt/canvas/build/Release/canvas-prebuilt.node" is installed via remote
npm notice created a lockfile as package-lock.json. You should commit this file.
npm WARN install_github_module_then_install_another_module@1.0.0 No description
npm WARN install_github_module_then_install_another_module@1.0.0 No repository field.

+ chrome-headless-render-pdf@1.4.1
added 213 packages in 27.822s
root@server2:~/app/test/install_github_module_then_install_another_module# ls node_modules/ # the module is there. there are many modules.
abbrev               chrome-headless-render-pdf  css-system-font-keywords  graceful-fs              is-typedarray          osenv                 semver-diff          ultron
ajv                  chrome-remote-interface     dashdash                  har-schema               jsbn                   os-homedir            set-blocking         unique-string
ansi-align           cli-boxes                   debug                     har-validator            jsonify                os-tmpdir             shebang-command      units-css
ansi-regex           co                          deep-extend               has-flag                 json-schema            package-json          shebang-regex        unquote
ansi-styles          code-point-at               delayed-stream            has-unicode              json-stable-stringify  parse-css-font        signal-exit          unzip-response
aproba               color-convert               delegates                 hawk                     json-stringify-safe    path-is-absolute      sntp                 update-notifier
are-we-there-yet     color-name                  detect-libc               hoek                     jsprim                 path-is-inside        sshpk                url-parse-lax
asn1                 combined-stream             dot-prop                  http-signature           latest-version         path-key              string-argv          util-deprecate
assert-plus          commander                   duplexer3                 import-lazy              lowercase-keys         performance-now       string_decoder       uuid
asynckit             command-exists              ecc-jsbn                  imurmurhash              lru-cache              p-finally             stringstream         verror
aws4                 concat-map                  escape-string-regexp      inflight                 make-dir               pify                  string-width         viewport-dimensions
aws-sign2            configstore                 execa                     inherits                 mime-db                prepend-http          strip-ansi           which
babel-polyfill       console-control-strings     extend                    ini                      mime-types             process-nextick-args  strip-eof            wide-align
babel-runtime        core-js                     extsprintf                isarray                  minimatch              pseudomap             strip-json-comments  widest-line
balanced-match       core-util-is                forever-agent             isexe                    minimist               punycode              supports-color       wrappy
bcrypt-pbkdf         create-error-class          form-data                 is-fullwidth-code-point  mkdirp                 qs                    tar                  write-file-atomic
block-stream         cross-spawn                 fs.realpath               is-installed-globally    ms                     rc                    tar-pack             ws
boom                 cryptiles                   fstream                   is-npm                   node-pre-gyp           readable-stream       tcomb                xdg-basedir
boxen                crypto-random-string        fstream-ignore            isnumeric                nopt                   regenerator-runtime   term-size            yallist
brace-expansion      css-font-size-keywords      gauge                     is-obj                   npmlog                 registry-auth-token   timed-out
camelcase            css-font-stretch-keywords   getpass                   is-path-inside           npm-run-path           registry-url          tough-cookie
canvas-prebuilt      css-font-style-keywords     get-stream                is-redirect              number-is-nan          request               tunnel-agent
capture-stack-trace  css-font-weight-keywords    glob                      is-retry-allowed         oauth-sign             rimraf                tweetnacl
caseless             css-global-keywords         global-dirs               isstream                 object-assign          safe-buffer           @types
chalk                css-list-helpers            got                       is-stream                once                   semver                uid-number
root@server2:~/app/test/install_github_module_then_install_another_module# npm install lodash --save
npm WARN install_github_module_then_install_another_module@1.0.0 No description
npm WARN install_github_module_then_install_another_module@1.0.0 No repository field.

+ lodash@4.17.4
added 1 package and removed 213 packages in 4.169s
root@server2:~/app/test/install_github_module_then_install_another_module# ls node_modules/ # folder chrome-headless-render-pdf is removed totally, even not reinstalled as supposed to. also removed all related modules.
lodash
root@server2:~/app/test/install_github_module_then_install_another_module# cat package.json # still has entry for shimondoodkin/chrome-headless-render-pdf
{
  "name": "install_github_module_then_install_another_module",
  "version": "1.0.0",
  "description": "",
  "main": "index.js",
  "scripts": {
    "test": "echo \"Error: no test specified\" && exit 1"
  },
  "keywords": [],
  "author": "",
  "license": "ISC",
  "dependencies": {
    "chrome-headless-render-pdf": "github:shimondoodkin/chrome-headless-render-pdf",
    "lodash": "^4.17.4"
  }
}
root@server2:~/app/test/install_github_module_then_install_another_module# cd ..
root@server2:~/app/test#
root@server2:~/app/test#
root@server2:~/app/test# #test fails:
root@server2:~/app/test# mkdir install_and_modify_github_module_then_install_another_module
root@server2:~/app/test# cd install_and_modify_github_module_then_install_another_module/
root@server2:~/app/test/install_and_modify_github_module_then_install_another_module# npm init -y
Wrote to /root/app/test/install_and_modify_github_module_then_install_another_module/package.json:

{
  "name": "install_and_modify_github_module_then_install_another_module",
  "version": "1.0.0",
  "description": "",
  "main": "index.js",
  "scripts": {
    "test": "echo \"Error: no test specified\" && exit 1"
  },
  "keywords": [],
  "author": "",
  "license": "ISC"
}


root@server2:~/app/test/install_and_modify_github_module_then_install_another_module# echo "console.log('test')">index.js
root@server2:~/app/test/install_and_modify_github_module_then_install_another_module# npm install shimondoodkin/chrome-headless-render-pdf --save

> canvas-prebuilt@1.6.5-prerelease.1 install /root/app/test/install_and_modify_github_module_then_install_another_module/node_modules/canvas-prebuilt
> node-pre-gyp install

[canvas-prebuilt] Success: "/root/app/test/install_and_modify_github_module_then_install_another_module/node_modules/canvas-prebuilt/canvas/build/Release/canvas-prebuilt.node" is installed via remote
npm notice created a lockfile as package-lock.json. You should commit this file.
npm WARN install_and_modify_github_module_then_install_another_module@1.0.0 No description
npm WARN install_and_modify_github_module_then_install_another_module@1.0.0 No repository field.

+ chrome-headless-render-pdf@1.4.1
added 213 packages in 18.343s
root@server2:~/app/test/install_and_modify_github_module_then_install_another_module# tail -n 1  node_modules/chrome-headless-render-pdf/index.js
module.exports.default = RenderPDF;
root@server2:~/app/test/install_and_modify_github_module_then_install_another_module# echo "//modification" >> node_modules/chrome-headless-render-pdf/index.js
root@server2:~/app/test/install_and_modify_github_module_then_install_another_module# tail -n 2  node_modules/chrome-headless-render-pdf/index.js
module.exports.default = RenderPDF;
//modification
root@server2:~/app/test/install_and_modify_github_module_then_install_another_module# ls node_modules/
abbrev               chrome-headless-render-pdf  css-system-font-keywords  graceful-fs              is-typedarray          osenv                 semver-diff          ultron
ajv                  chrome-remote-interface     dashdash                  har-schema               jsbn                   os-homedir            set-blocking         unique-string
ansi-align           cli-boxes                   debug                     har-validator            jsonify                os-tmpdir             shebang-command      units-css
ansi-regex           co                          deep-extend               has-flag                 json-schema            package-json          shebang-regex        unquote
ansi-styles          code-point-at               delayed-stream            has-unicode              json-stable-stringify  parse-css-font        signal-exit          unzip-response
aproba               color-convert               delegates                 hawk                     json-stringify-safe    path-is-absolute      sntp                 update-notifier
are-we-there-yet     color-name                  detect-libc               hoek                     jsprim                 path-is-inside        sshpk                url-parse-lax
asn1                 combined-stream             dot-prop                  http-signature           latest-version         path-key              string-argv          util-deprecate
assert-plus          commander                   duplexer3                 import-lazy              lowercase-keys         performance-now       string_decoder       uuid
asynckit             command-exists              ecc-jsbn                  imurmurhash              lru-cache              p-finally             stringstream         verror
aws4                 concat-map                  escape-string-regexp      inflight                 make-dir               pify                  string-width         viewport-dimensions
aws-sign2            configstore                 execa                     inherits                 mime-db                prepend-http          strip-ansi           which
babel-polyfill       console-control-strings     extend                    ini                      mime-types             process-nextick-args  strip-eof            wide-align
babel-runtime        core-js                     extsprintf                isarray                  minimatch              pseudomap             strip-json-comments  widest-line
balanced-match       core-util-is                forever-agent             isexe                    minimist               punycode              supports-color       wrappy
bcrypt-pbkdf         create-error-class          form-data                 is-fullwidth-code-point  mkdirp                 qs                    tar                  write-file-atomic
block-stream         cross-spawn                 fs.realpath               is-installed-globally    ms                     rc                    tar-pack             ws
boom                 cryptiles                   fstream                   is-npm                   node-pre-gyp           readable-stream       tcomb                xdg-basedir
boxen                crypto-random-string        fstream-ignore            isnumeric                nopt                   regenerator-runtime   term-size            yallist
brace-expansion      css-font-size-keywords      gauge                     is-obj                   npmlog                 registry-auth-token   timed-out
camelcase            css-font-stretch-keywords   getpass                   is-path-inside           npm-run-path           registry-url          tough-cookie
canvas-prebuilt      css-font-style-keywords     get-stream                is-redirect              number-is-nan          request               tunnel-agent
capture-stack-trace  css-font-weight-keywords    glob                      is-retry-allowed         oauth-sign             rimraf                tweetnacl
caseless             css-global-keywords         global-dirs               isstream                 object-assign          safe-buffer           @types
chalk                css-list-helpers            got                       is-stream                once                   semver                uid-number
root@server2:~/app/test/install_and_modify_github_module_then_install_another_module# npm install lodash --save
npm WARN install_and_modify_github_module_then_install_another_module@1.0.0 No description
npm WARN install_and_modify_github_module_then_install_another_module@1.0.0 No repository field.

+ lodash@4.17.4
added 1 package and removed 213 packages in 3.787s
root@server2:~/app/test/install_and_modify_github_module_then_install_another_module# ls node_modules/ # folder chrome-headless-render-pdf is removed totally, even not reinstalled as supposed to be
lodash
root@server2:~/app/test/install_and_modify_github_module_then_install_another_module# cat package.json # still has entry for  shimondoodkin/chrome-headless-render-pdf
{
  "name": "install_and_modify_github_module_then_install_another_module",
  "version": "1.0.0",
  "description": "",
  "main": "index.js",
  "scripts": {
    "test": "echo \"Error: no test specified\" && exit 1"
  },
  "keywords": [],
  "author": "",
  "license": "ISC",
  "dependencies": {
    "chrome-headless-render-pdf": "github:shimondoodkin/chrome-headless-render-pdf",
    "lodash": "^4.17.4"
  }
}
root@server2:~/app/test/install_and_modify_github_module_then_install_another_module# tail -n 2  node_modules/chrome-headless-render-pdf/index.js # expected to see here '// modification', nothing to see, file missing
tail: cannot open 'node_modules/chrome-headless-render-pdf/index.js' for reading: No such file or directory
root@server2:~/app/test/install_and_modify_github_module_then_install_another_module# cd ..
root@server2:~/app/test#
root@server2:~/app/test# #test fails:
root@server2:~/app/test# mkdir create_new_module_then_install_another_module
root@server2:~/app/test# cd create_new_module_then_install_another_module/
root@server2:~/app/test/create_new_module_then_install_another_module# npm init -y
Wrote to /root/app/test/create_new_module_then_install_another_module/package.json:

{
  "name": "create_new_module_then_install_another_module",
  "version": "1.0.0",
  "description": "",
  "main": "index.js",
  "scripts": {
    "test": "echo \"Error: no test specified\" && exit 1"
  },
  "keywords": [],
  "author": "",
  "license": "ISC"
}


root@server2:~/app/test/create_new_module_then_install_another_module# echo "console.log('test')">index.js
root@server2:~/app/test/create_new_module_then_install_another_module# mkdir -p node_modules/newmodule
root@server2:~/app/test/create_new_module_then_install_another_module# cd node_modules/newmodule/
root@server2:~/app/test/create_new_module_then_install_another_module/node_modules/newmodule# npm init -y
Wrote to /root/app/test/create_new_module_then_install_another_module/node_modules/newmodule/package.json:

{
  "name": "newmodule",
  "version": "1.0.0",
  "description": "",
  "main": "index.js",
  "scripts": {
    "test": "echo \"Error: no test specified\" && exit 1"
  },
  "keywords": [],
  "author": "",
  "license": "ISC"
}


root@server2:~/app/test/create_new_module_then_install_another_module/node_modules/newmodule# echo "console.log('new module')">index.js
root@server2:~/app/test/create_new_module_then_install_another_module/node_modules/newmodule# cd ../..
root@server2:~/app/test/create_new_module_then_install_another_module# ls node_modules/ # newmodule is there
newmodule
root@server2:~/app/test/create_new_module_then_install_another_module# npm install lodash --save
npm notice created a lockfile as package-lock.json. You should commit this file.
npm WARN create_new_module_then_install_another_module@1.0.0 No description
npm WARN create_new_module_then_install_another_module@1.0.0 No repository field.

+ lodash@4.17.4
added 1 package and removed 1 package in 1.174s
root@server2:~/app/test/create_new_module_then_install_another_module# ls node_modules/ # the folder newmodule is removed,
lodash
root@server2:~/app/test/create_new_module_then_install_another_module# cd ..
root@server2:~/app/test#
root@server2:~/app/test#
root@server2:~/app/test#
root@server2:~/app/test# cd ..
root@server2:~/app# rm ./test -rf
root@server2:~/app#

[shimondoodkin]

expected behavior "
npm should not delete by itself files when installing new module. just install a new module

1. to not delete unknown repositories. that maybe been created by the user.
2. to not delete repositories installed from git (this is actually a bug, it just deletes git repository not installs it later Packages installed with git are removed on subsequent npm installs #19446 )
3. to not reinstall anything modified by the user, unless the name of the module is specified in the install command (not automatic submodules install).
4. to ask user if it removes or reinstalls anything. (if there is option 'yes to all', then print to user list of every module that will be removed before user can decide yes to delete all).

recently npm to me feels very shaky and unstable not sure why maybe many options have been developed and all the combinations of options usages times usage sequences are not tested.

[frank-dspeed]

@shimondoodkin i think the bug is on user side in this case you don't edited your package.json before running npm install maybe also mixed up lock file. i verified it if you install and save dependencies all gets honored right

[shimondoodkin]

@frank-dspeed not sure what you mean.
The following test it behaives not as expected with npm 5.

When you run
npm install module --save
It modifies the package json then installs the module.
But in this case . After installing two modules you end up with only one folder in node_modules. Not at last two as expected

#test fails:
mkdir install_github_module_then_install_another_module
cd install_github_module_then_install_another_module/
npm init -y
npm install shimondoodkin/chrome-headless-render-pdf --save
ls node_modules/ # the module is there. there are many modules.
npm install lodash --save
ls node_modules/ # folder chrome-headless-render-pdf is removed totally, even not reinstalled as supposed to. also removed all related modules.
cat package.json # still has entry for shimondoodkin/chrome-headless-render-pdf
cd ..

[junkert]

This is a duplicate of #16853

[lixuguang1988]

I use yarn instead of npm to avoid this