[BUG] .gitignore file missing in tmp clone when installing a Git dependency

[jaydenseric]

Current Behavior:

When npm v7 installs a Git dependency (e.g. "github:jaydenseric/extract-files") it creates a clone of the repo in a tmp directory to pack the package for installation, running the package prepare script, etc. Normally (in npm v6) a .gitignore file in the repo will be present in this clone, but it is missing for npm v7 installs. This cause cause problems if the package prepare script uses tools that utilize this file to ignore files or directories such as node_modules.

This is a follow up issue to #2084 , as the original error preventing installation of a Git dependency has now been fixed in npm v7.0.9 but now we are now able to progress to yet another error.

Expected Behavior:

Steps To Reproduce:

Try npm install with this package.json:

{
  "name": "npm-prepare-script-test",
  "private": true,
  "dependencies": {
    "extract-files": "github:jaydenseric/extract-files"
  }
}

Notice the error:

npm ERR! code 1
npm ERR! git dep preparation failed
npm ERR! command /Users/jaydenseric/.asdf/installs/nodejs/15.1.0/bin/node /Users/jaydenseric/.asdf/installs/nodejs/15.1.0/.npm/lib/node_modules/npm/bin/npm-cli.js install --cache=/Users/jaydenseric/.npm/_cacache --prefer-offline=false --prefer-online=false --offline=false --no-progress --no-save --no-audit
npm ERR! > extract-files@9.0.0 prepare
npm ERR! > npm run prepare:clean && npm run prepare:babel && npm run prepare:jsdoc && npm run prepare:prettier
npm ERR! 
npm ERR! 
npm ERR! > extract-files@9.0.0 prepare:clean
npm ERR! > rm -rf public
npm ERR! 
npm ERR! 
npm ERR! > extract-files@9.0.0 prepare:babel
npm ERR! > babel src -d . --keep-file-extension
npm ERR! 
npm ERR! Successfully compiled 5 files with Babel (437ms).
npm ERR! 
npm ERR! > extract-files@9.0.0 prepare:jsdoc
npm ERR! > jsdoc-md
npm ERR! npm WARN deprecated urix@0.1.0: Please see https://github.com/lydell/urix#deprecated
npm ERR! npm WARN deprecated resolve-url@0.2.1: https://github.com/lydell/resolve-url#deprecated
npm ERR! npm WARN deprecated fsevents@1.2.13: fsevents 1 will break on node v14+ and could be using insecure binaries. Upgrade to fsevents 2.
npm ERR! npm WARN deprecated chokidar@2.1.8: Chokidar 2 will break on node v14+. Upgrade to chokidar 3 with 15x less dependencies.
npm ERR! npm WARN deprecated fsevents@1.2.13: fsevents 1 will break on node v14+ and could be using insecure binaries. Upgrade to fsevents 2.
npm ERR! npm WARN deprecated chokidar@2.1.8: Chokidar 2 will break on node v14+. Upgrade to chokidar 3 with 15x less dependencies.
npm ERR! /Users/jaydenseric/.npm/_cacache/tmp/git-clone-a61aefcd/node_modules/jsdoc-md/private/jsdocToMember.js:43
npm ERR!   if (getJsdocAstTag(jsdocAst.tags, 'ignore')) return;
npm ERR!                               ^
npm ERR! 
npm ERR! TypeError: Cannot read property 'tags' of undefined
npm ERR!     at jsdocToMember (/Users/jaydenseric/.npm/_cacache/tmp/git-clone-a61aefcd/node_modules/jsdoc-md/private/jsdocToMember.js:43:31)
npm ERR!     at jsdocMd (/Users/jaydenseric/.npm/_cacache/tmp/git-clone-a61aefcd/node_modules/jsdoc-md/public/jsdocMd.js:61:22)
npm ERR!     at Object.<anonymous> (/Users/jaydenseric/.npm/_cacache/tmp/git-clone-a61aefcd/node_modules/jsdoc-md/cli/jsdoc-md.js:21:1)
npm ERR!     at Module._compile (node:internal/modules/cjs/loader:1083:30)
npm ERR!     at Object.Module._extensions..js (node:internal/modules/cjs/loader:1112:10)
npm ERR!     at Module.load (node:internal/modules/cjs/loader:948:32)
npm ERR!     at Function.Module._load (node:internal/modules/cjs/loader:789:14)
npm ERR!     at Function.executeUserEntryPoint [as runMain] (node:internal/modules/run_main:72:12)
npm ERR!     at node:internal/main/run_main_module:17:47
npm ERR! npm ERR! code 1
npm ERR! npm ERR! path /Users/jaydenseric/.npm/_cacache/tmp/git-clone-a61aefcd
npm ERR! npm ERR! command failed
npm ERR! npm ERR! command sh -c jsdoc-md
npm ERR! 
npm ERR! npm ERR! A complete log of this run can be found in:
npm ERR! npm ERR!     /Users/jaydenseric/.npm/_cacache/_logs/2020-11-07T21_57_24_059Z-debug.log
npm ERR! npm ERR! code 1
npm ERR! npm ERR! path /Users/jaydenseric/.npm/_cacache/tmp/git-clone-a61aefcd
npm ERR! npm ERR! command failed
npm ERR! npm ERR! command sh -c npm run prepare:clean && npm run prepare:babel && npm run prepare:jsdoc && npm run prepare:prettier
npm ERR! 
npm ERR! npm ERR! A complete log of this run can be found in:
npm ERR! npm ERR!     /Users/jaydenseric/.npm/_cacache/_logs/2020-11-07T21_57_24_124Z-debug.log

npm ERR! A complete log of this run can be found in:
npm ERR!     /Users/jaydenseric/.npm/_logs/2020-11-07T21_57_26_754Z-debug.log

The reason for the error, is that the jsdoc-md command used in the package prepare script usually uses the project .gitignore file to ignore node_modules when scanning for JSDoc when generating markdown docs under the API heading in readme.md. For some reason, npm v7 clones most of the project files, but not .gitignore - causing the jsdoc-md command to error on some bad JSDoc in node_modules.

This is a serious breaking change, and is surely an npm v7 bug. Lots of tools that might be used in a package prepare script reference the project's .gitignore file.

Environment:

• OS: macOS v10.15.7
• Node.js: v15.1.0
• npm: v7.0.9

[isaacs]

Took a bit of digging to see exactly what was going on here, but yes, this is a bug. Fixed by npm/pacote#59.

[isaacs]

This will be fixed when we pull pacote 11.2.2 into the cli. Thanks for reporting :)