Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[BUG] npm install doesn't create the .bin directory with a v6 package-lock #2147

Closed
closingin opened this issue Nov 8, 2020 · 15 comments
Closed
Labels
Bug thing that needs fixing Release 7.x work is associated with a specific npm 7 release

Comments

@closingin
Copy link

closingin commented Nov 8, 2020

Hello there, here's a little (but really annoying) bug I just found out !

Current Behavior:

I just upgraded to node@15 and npm@7.

When deleting my node_modules and reinstalling using npm i, npm doesn't create the .bin directory, which ends up in me having no working scripts at all (eg: npm run build, supposed to run next build, but it can't find it obviously).

If I delete my current package-lock.json and reinstall, the .bin directory is correctly re-created, but I can't do that as it breaks my app.

npm bin correctly returns [redacted]/node_modules/.bin.

From what I can see in my git diff, my old package-lock.json does not have the "bin": {} property on packages that have binaries, while the fresh one has it.

Expected Behavior:

npm install should create and populate the .bin directory with older package-lock.json files, and upgrade the lock file.

Steps To Reproduce:

Having a v6 package-lock I guess ? I don't know what's causing this, but I can give you my package-lock.json and package.json if needed.

Environment:

  • OS: Arch Linux
  • Node: 15.1.0
  • npm: 7.0.8 (the bug also exists in previous versions)
@closingin closingin added Bug thing that needs fixing Needs Triage needs review for next steps Release 7.x work is associated with a specific npm 7 release labels Nov 8, 2020
@jaydenseric
Copy link

We've noticed a few weird things relating to package scripts, npm v6/7, and lockfile v1/2 interoperability:

https://twitter.com/jaydenseric/status/1318321502098345985?s=20

Each time we tried to remember the exact reproduction steps but ended up confused.

@isaacs
Copy link
Contributor

isaacs commented Nov 11, 2020

Can you check with 7.0.10? I believe this was likely fixed by a recent arborist update.

@nornagon
Copy link

I also encountered this (on macOS, also 7.0.8). Upgrading to 7.0.10 did not fix the issue.

dr-js added a commit to dr-js/imagemin-min that referenced this issue Nov 13, 2020
The `package-lock.json` is bad and will not create `node_modules/.bin/` on wither `npm i` or `npm ci`
The `package-lock-good-recreated-with-npm@7.0.8.json` is the supposed correct version
Related issue: npm/cli#2147

--original commit message--
imagemin-min@0.1.1
notable change:
- fix: add bin `#!` for `bin/test-boot.js`
@dr-js
Copy link
Contributor

dr-js commented Nov 13, 2020

Got the bug, too.
Don't know how this happens, but here is a preserved "crash site":

And a typical diff looks like: (good file will add bin|engines|funding fields)

        "table": "^5.2.3",
        "text-table": "^0.2.0",
        "v8-compile-cache": "^2.0.3"
+      },
+      "bin": {
+        "eslint": "bin/eslint.js"
+      },
+      "engines": {
+        "node": "^10.12.0 || >=12.0.0"
+      },
+      "funding": {
+        "url": "https://opencollective.com/eslint"
      }
    },

With the bad lockfile, npm@7.0.10 will not create node_modules/.bin/ with either npm i or npm ci.

@closingin
Copy link
Author

Hello @isaacs , I just updated my npm to 7.0.10 to test, but npm still doesn't create the .bin directory.

@christianhaller3000
Copy link

christianhaller3000 commented Nov 14, 2020

@closingin what package are your installing?

@closingin
Copy link
Author

@closingin what package are your installing?

Uh, I'm just doing a standard npm install. It's not related to a specific package, the bug occurs when you have a "broken" package-lock.json, alongside any package that has a binary.

@isaacs I think I just found how to work around the bug. It may help you in finding the culprit.

When you upgrade to node 15 / npm 7 and do any operation that would update your package-lock.json to "lockfileVersion": 2, npm actually doesn't migrate the file correctly, as it does not add the "bin", "engines", and other properties. I guess those are now required. To work around it, you just need to manually set lockfileVersion to 1, and do a new npm i.

My bold prediction is that npm works correctly when installing, based on what it has in the lockfile. The problem surely comes from the first operation that updates the lockfile to version 2, I think it should either let the lockfileVersion to 1, or migrate the lockfile completely.

@dr-js
Copy link
Contributor

dr-js commented Nov 14, 2020

Might be related, I think the bad lockfile is created after apt updated nodejs@15.2.0 deb from nodesource, and I think the process also reset npm to npm@7.0.8.
Before that I used nodejs@15.1.x with manually downgraded npm@6.14.8, so maybe the bad lockfile is created by npm@7.0.8 in a repo when node_modules and v1 lockfile state match, and later with npm@7.0.10 and do install again will not fix the already bad lockfile.

@dr-js
Copy link
Contributor

dr-js commented Nov 15, 2020

A test to reproduce the bad lockfile from the itself: (with nodejs@15.2 and npm@7.0.10)
The related test files: test-package-json.zip

mkdir "test-lock/"
cd "test-lock/"
# copy the `package.json` and `package-lock.json-npm708-bad` file here

cp "package-lock.json-npm708-bad" "package-lock.json"

npx --yes npm@6 install
ls "node_modules/.bin/" # there's file
npx --yes npm@6 ci
ls "node_modules/.bin/" # there's file
cp "package-lock.json" "package-lock.json-npm6-good"

npx --yes npm@7.0.10 install
ls "node_modules/.bin/" # there's file because npm do not reset them yet
npx --yes npm@7.0.10 ci
ls "node_modules/.bin/" || echo "no file" # there's no file
cp "package-lock.json" "package-lock.json-npm710-bad-not-fixed"

@dr-js
Copy link
Contributor

dr-js commented Nov 17, 2020

Just tested above process with npm@7.0.11 and this should be fixed
A similar issue: #1957
The fix commit: c3e7aa31

@closingin
Copy link
Author

closingin commented Nov 18, 2020

I can confirm that npm@7.0.11 fixed this issue!

@closingin
Copy link
Author

closingin commented Nov 18, 2020

Apologies: I wrote too fast, issue not fixed. I did my test with @dr-js 's snippet, but the problem still exists on my personal project, using npm@7.0.12.

@closingin closingin reopened this Nov 18, 2020
@ruyadorno
Copy link
Collaborator

Hi @closingin do you happen to have a package-lock.json file that was migrated to v2 before updating to npm@7.0.12 ?

As I mentioned over in #1957 if that's the case you might already have a broken file and you would need to force the migration again in order to fix the missing bins issue. If that's the case you can try this trick:

# Make sure you got npm@7.0.11 or newer:
$ npm install -g npm@7

# npm@6 install:
$ npx npm@6 install

# npm@7 install to migrate package-lock file again:
$ npm install

# You should have a fixed-up package-lock.json file now :)

hail2u added a commit to hail2u/hail2u.net that referenced this issue Nov 25, 2020
@darcyclarke darcyclarke removed the Needs Triage needs review for next steps label Nov 30, 2020
@closingin
Copy link
Author

closingin commented Dec 1, 2020

Alright, looks like it worked @ruyadorno, thanks! Sorry that I didn't find the existing issue and recreated one btw.

@ianef
Copy link

ianef commented Dec 2, 2020

I've had very similar problems too. I couldn't get it working properly on 6.14.8 and was considering doing some tests with earlier versions.

However I decided to test out the process in a new Debian docker container, copied the basics of the package.json in my project and it built correctly using npm 7.0.14 with a.bin folder. I was confused! I copied the whole of my project package.json and it failed trying to load fortawesome-pro. I copied in my .npmrc and the build completed but there was no .bin folder.

In my .npmrc there was this, I have no idea why!:

bin-links=false

Removing this line solved the problem and the .bin folder was created correctly.

notlee added a commit to Financial-Times/o-brand that referenced this issue Jan 15, 2021
viktor-yakubiv added a commit to oacore/search that referenced this issue Jan 19, 2021
1. Fixes package-lock.json using the method suggested by @ruyadorno:
   npm/cli#2147 (comment)

   The issue was causing missing binaries in the `/node_modules/.bin`.

2. Restores version to the package.json based on the latest
   git tag.
viktor-yakubiv added a commit to oacore/search that referenced this issue Jan 19, 2021
1. Fixes package-lock.json using the method suggested by @ruyadorno:
   npm/cli#2147 (comment)

   The issue was causing missing binaries in the `/node_modules/.bin`.

2. Restores version to the package.json based on the latest
   git tag.
viktor-yakubiv added a commit to oacore/search that referenced this issue Jan 19, 2021
1. Fixes package-lock.json using the method suggested by @ruyadorno:
   npm/cli#2147 (comment)

   The issue was causing missing binaries in the `/node_modules/.bin`.

2. Restores version to the package.json based on the latest
   git tag.
andersk added a commit to andersk/flatpickr that referenced this issue Jun 10, 2021
package-lock.json was migrated to "lockfileVersion": 2 by an old buggy
version of npm@7, and is therefore missing important information such
as the "bin" records that npm@7 needs for new installs.

    $ npm i; npm run build
    […]
    > flatpickr@4.6.9 build
    > run-s build:pre build:build build:esm build:types build:post

    sh: line 1: run-s: command not found

Fix it by un-migrating with npm@6 and re-migrating with a current
npm@7.

npm/cli#2147 (comment)

Signed-off-by: Anders Kaseorg <andersk@mit.edu>
Minnozz added a commit to Minnozz/rescript-lang.org that referenced this issue Jun 12, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Bug thing that needs fixing Release 7.x work is associated with a specific npm 7 release
Projects
None yet
Development

No branches or pull requests

9 participants