Skip to content
This repository has been archived by the owner. It is now read-only.

package-lock.json file not updated after package.json file is changed #16866

Closed
vdclouis opened this issue Jun 1, 2017 · 112 comments

Comments

Projects
None yet
@vdclouis
Copy link

commented Jun 1, 2017

I'm opening this issue because:

  • npm is crashing.
  • npm is producing an incorrect install.
  • npm is doing something I don't understand.
  • Other (see below for feature requests):

What's going wrong?

clean project with some deps in package.json.
you run npm i
modules are installed and package-lock.json file is created.
say you update module A in package.json file.
you run npm i. I would expect this updates the package-lock.json file but it doesn't. which results in module A not being updated.

How can the CLI team reproduce the problem?

any project. see description above.

supporting information:

  • npm -v prints: 5.x.x
  • node -v prints: 7.x.x
  • npm config get registry prints: private one
  • Windows, OS X/macOS, or Linux?: macOS
  • Network issues:
    • Geographic location where npm was run:
    • I use a proxy to connect to the npm registry.
    • I use a proxy to connect to the web.
    • I use a proxy when downloading Git repos.
    • I access the npm registry via a VPN
    • I don't use a proxy, but have limited or unreliable internet access.
  • Container:
    • I develop using Vagrant on Windows.
    • I develop using Vagrant on OS X or Linux.
    • I develop / deploy using Docker.
    • I deploy to a PaaS (Triton, Heroku).

@vdclouis vdclouis changed the title package-lock.json file not updated after package.json file is updated package-lock.json file not updated after package.json file is changed Jun 1, 2017

@JamesMessinger

This comment has been minimized.

Copy link

commented Jun 1, 2017

I also find this behavior problematic. For now I'm working around it by changing my npm install command to rm -f package-lock.json && npm install. Obviously, that's not an elegant solution, and somewhat defeats the purpose of having a lockfile in the first place.

@yomansk8

This comment has been minimized.

Copy link

commented Jun 1, 2017

Same problem here with npm 5.0.0 and node 7.3.0 on macOS.
Each time I add a dependency in my package.json file, I run npm i and I've got this :

$ npm i 
removed 3 packages in 5.953s

My new dependencies are not installed after this. I had to delete the package-lock.json and re-run npm i and this time my new dependencies are correctly installed.

@gyszalai

This comment has been minimized.

Copy link

commented Jun 2, 2017

If you run npm update package-lock.json gets updated and the new dependency is installed.

@falsandtru

This comment has been minimized.

Copy link

commented Jun 3, 2017

I think adding npm update command's new option to update only package-lock.json will be a better solution. And then npm install would be better to throw an error or show alerts of the conflicts of the two files.

@Maziar-Fotouhi

This comment has been minimized.

Copy link

commented Jun 8, 2017

npm update does not solve it for me.
For some reason it keeps thinking that some package versions are invalid when I try to use npm update.
This is super annoying and a huge downside to npm@5. I'm really disappointed at npm!
npm should not be that hard to use. It was much better when a simple npm install did the job.
Now I have a good reason to switch to yarn!

@Zone-Ghost

This comment has been minimized.

Copy link

commented Jun 9, 2017

Yea, this has been causing me serious grief the last couple of days. The "speed up" isn't worth the constant requirement to delete my package-lock file every time I need to update a dependency... Changing my package.json never updates my package-lock, and npm update after changing the version doesn't update it either.

@hyposimon

This comment has been minimized.

Copy link

commented Jun 9, 2017

@Zone-Ghost thats exactly my point

we are updating our dependencies automated every moring ... since two days we have a up-to-date package.json file ... but we dont get the new npm cuz the package-lock.json points to the wront version.

to delete the package-lock.json is an absolute useless step ... but fix it.

is there a way to update the package-lock.json??

@jakeNiemiec

This comment has been minimized.

Copy link

commented Jun 9, 2017

Put "preinstall": "npm config set package-lock false" in your scripts to disable package-lock.json.

If we wanted yarn, we would have used that. This half-hearted attempt helps no one.

Edit: I merely wanted to highlight that npm should not try and be yarn at the expense of npm. They are both solid package managers built to solve different problems.

@legodude17

This comment has been minimized.

Copy link
Contributor

commented Jun 9, 2017

@jakeNiemiec It helps the people who had their builds sped up by whole minutes due to package-lock.json

@fxck

This comment has been minimized.

Copy link

commented Jun 12, 2017

I don't get it. So what are you supposed to do after you update your package.json? Run npm update? I'm getting the same "invalid" messages as @Maziar-Fotouhi does when I try that.

@chrisui

This comment has been minimized.

Copy link

commented Jun 12, 2017

Do we just need a npm lock command like we had npm shrinkwrap?

This just generates the shrinkwrap with current dependencies (so has the effect of updating the lock if you've added a new dependency to package.json).

@Maziar-Fotouhi

This comment has been minimized.

Copy link

commented Jun 12, 2017

@fxck after reading @BigstickCarpet 's comment, I added this to my package.json in the scripts section:
"freshtall": "rm -rf node_modules && rm -rf package-lock.json && npm install",
Now I just need to run npm run freshtall and I will get a fresh install of every package.
P.S: "freshtall" was just a cheesy name I came up with, you can name yours whatever else you want if you want to change it!
But I am using yarn right now since I am only using npm for installing packages.
@jakeNiemiec This really depends on the project that you are working on but what is the point if you don't use the lock file?! This is just sweeping everything under the carpet. I prefer to keep an unbiased opinion about this issue and use yarn (which by the way easily utilizes its lock file and is much faster), until this is fixed in npm.

@jakeNiemiec

This comment has been minimized.

Copy link

commented Jun 12, 2017

@Maziar-Fotouhi don't forget a npm cache clean --force in there too.

what is the point if you don't use the lock file

I only do this because updating 'individual packages' was removed. I can't: npm update @me/myCustomDep anymore.

@Maziar-Fotouhi

This comment has been minimized.

Copy link

commented Jun 12, 2017

@jakeNiemiec That makes sense. I change the version numbers directly in package.json. Maybe that can help.
Thanks for the hint! Why do we need to clean the cache though? The npm documentation says:

it should never be necessary to clear the cache for any reason other than reclaiming disk space, thus why clean now requires --force to run.

@jakeNiemiec

This comment has been minimized.

Copy link

commented Jun 12, 2017

@Maziar-Fotouhi Thanks for the hint! Why do we need to clean the cache though?

#16665 crash installing: "Error: Argument #2: Expected string or string but got null"

This nasty error that intertwines with this issue, at least for me. I have found that removing node_modules can have weird aftereffects with npm5.

@Maziar-Fotouhi

This comment has been minimized.

Copy link

commented Jun 12, 2017

@jakeNiemiec Thank you for sharing that! Sounds like you had a lot of fun with this issue!
So here is the script that I have added to the scripts section in package.json and using it through npm run freshtall:
"freshtall": "npm cache clean --force && rm -rf node_modules && rm -f package-lock.json && npm install",

@ldrick

This comment has been minimized.

Copy link

commented Jun 13, 2017

@jakeNiemiec Thanks, following your advice, i simply added this line to my .npmrc file.
package-lock = false

@raineorshine

This comment has been minimized.

Copy link

commented Jun 13, 2017

Removing node_modules is good as a workaround but it is overkill in my opinion. It would also not be viable for large projects that have 5+ min install times due to native build steps.

Deleting package-lock.json each time seems kludgy and creates problems for automated tools like npm-check-updates, which I maintain. Users of npm-check-updates as well as users who hand edit their package.json need a way to regenerate the package-lock.json file without having to reinstall all modules.

@fullstackhacker

This comment has been minimized.

Copy link

commented Jun 13, 2017

Isn't the purpose of package-lock.json to block changes in package.json. It makes sense that the node_modules don't update until you wipe/invalidate package-lock.json when making edits to package.json.

I do agree that this can be bothersome if you're not expecting this behavior by default. I wouldn't, however, say that this is a bug or anything.

@jakeNiemiec

This comment has been minimized.

Copy link

commented Jun 13, 2017

@Ketamin I initially did this, but this won't work for anyone else using your package (in my case: co-workers). I was hoping it was temporary, but after seeing #16991 it's probably better to downgrade to npm4 and wait for npm5 to stabilize.

@kapadiamush I would expect that package.json is treated as the prime source of truth...package-lock.json must validate against package.json. If found to be invalid, update only the needed libs and reflect those updates in package-lock.json.

Use case:
package.json

  • myCoolLib@~1.0.0
  • reactOrAngularSuperLib@~2.2.0

package-lock.json

  • myCoolLib@1.0.4
  • ReactOrAngularSuperLib@2.2.13

I want to update myCoolLib to 1.0.5. My choices:

  • In package.json set myCoolLib@~1.0.5 and run npm install. You would expect package-lock.json to update the new locked version.

  • npm update myCoolLib This worked in npm4, it would be ideal since it only updates the item I want. I would expect the entry in package.lock to updated as well.

  • npm update This will update myCoolLib along with everything else (very bad for large libs). This is OK, but it negates the usefulness of the lock file since it doesn't change hence this issue.

  • rm -f package-lock.json && npm install Same intention as above except package-lock.json gets updated.

At least, thats how I imagined everything should work from personally working with other package locking systems.

@fullstackhacker

This comment has been minimized.

Copy link

commented Jun 13, 2017

@jakeNiemiec Yeah, it's the other way around.. sorta. It's my understanding that the package-lock.json is meant to prevent exactly what you're trying to do.

You don't need to remove node_modules to update the package-lock.json -- rm package-lock.json && npm install should do the trick (and be significantly faster).

@raineorshine

This comment has been minimized.

Copy link

commented Jun 14, 2017

@kapadiamush I disagree with your interpretation... there are multiple installs possible for a given package.json. package-log.json is meant to lock node_modules to a particular installation for a given package.json. package-log.json is not meant to lock the package.json itself, which is obviously in source control and not changing arbitrarily.

The problem arises from the fact that the package.json can be changed before npm install is run, leaving the package-lock.json in an invalid state. Since they are separate files, this is somewhat inevitable, but there should be a way to update the lock file. If this can be done without re-installing all modules, even better.

@fullstackhacker

This comment has been minimized.

Copy link

commented Jun 14, 2017

@raineorshine That actually makes a lot more sense regarding the package-lock.json locking the node_modules in place.

You should still be able to update the package-lock.json without completely removing all of the node_modules. From what I've seen while playing around with it, removing package-lock.json and running npm install should "update" the package-lock.json without reinstalling all the node_modules.

@cmdelatorre

This comment has been minimized.

Copy link

commented Jun 14, 2017

I have over 750MB of dependencies, so removing node_modules is not a good idea.
Now, what is package-lock.json? well, it is not intuitive. I kinda got an idea now that I read about it, but new team members will not know about it.
What are the improvements of npm@5? I'm not sure. yarn worked well from scratch (and it is really faster and more intuitive) but I don't really need it... so I'm not gonna use it.

For the time being, I'm going back to npm@4 at least until version 5 becomes the de-facto standard in the industry.

@wulfsberg

This comment has been minimized.

Copy link

commented Jun 19, 2017

I am absolutely on board with @jakeNiemiec 's interpretation. package.json should be the source of truth, and the lock file should be validated against that. If I update my package.json, it is very unintuitive that npm refuses to install what is in that file.

@kidd3

This comment has been minimized.

Copy link

commented Aug 7, 2017

@wulfsberg I am glad someone else is understanding the problem now, npm install is causing the newest version of a package to be installed, ignoring the lock file. In my eyes this completely defeats the point of having a lock file, and I might as well not include it as it's not locking anything.
@Maziar-Fotouhi I am fully aware of how semver works, I am including the range in my package.json file for when I actually want to do an npm update it will get the newest version of the package based on the range. But as I say, this would be WHEN I choose to do an npm update. Not an npm install which ignores the lock file. Including a range shouldn't matter IF the lock file was actually locking the version specified.
As @wulfsberg put it, what is the difference between install and update if they are both ignoring the lock file and installing the newest version every time?
My original question and still the source of all my issues is, how can I create a stable environment for production testing when every time npm install is run for a fresh install it installs the newest versions of packages?

@Maziar-Fotouhi

This comment has been minimized.

Copy link

commented Aug 7, 2017

@wulfsberg is the npm install command (without any other options), changing your package.json content?! If so, that is a bug and very weird!
@kidd3 You are absolutely right. In your case, npm install and npm update are doing the exact same thing. That's why I personally never use a range for the version numbers.

All in all, I think you have a valid point as right now there is no way (at least that I know of!) to just read from the lock files and not the package.json. Maybe a new option can be added to the npm install command in future (E.g. --lock or --read-only) to just read the lock file and disregard the package.json.

But this is not the right thread for that. I believe a new feature request must be created for that.

For now, you can do what I am doing. Only use exact version numbers. This means another thing to do for you as you will have to update your packages manually, but it also gives you much more control over the details which is always good. Another command that can be useful if you decide to do this, is npm outdated.

@wulfsberg

This comment has been minimized.

Copy link

commented Aug 7, 2017

@Maziar-Fotouhi No, it is not. I was referring to (and disagreeing with) @albertosantini's #16866 (comment) case 1 and #16866 (comment) which state it should.

@kidd3

This comment has been minimized.

Copy link

commented Aug 8, 2017

@Maziar-Fotouhi thank you for your suggestion. Having a way to create a static and repeatable installation is essential for testing as currently with version 5.3.0 I can not do this. A new flag such as --lock or --read-only would allow npm install to ONLY install from the lock file, and would then give a repeatable installation every time. How can I request this? It is something I feel is incredibly important as currently version 5.3.0 is completely unusable in this situation.

@Maziar-Fotouhi

This comment has been minimized.

Copy link

commented Aug 8, 2017

@kidd3 I agree with you. Let's continue this discussion in #17979 that you created.

@jakeNiemiec

This comment has been minimized.

Copy link

commented Aug 8, 2017

Glad we were able to sort that out, but I feel really bad for those coming from Google and seeing this mess of (mostly true) information on package locks. @zkat or another maintainer might consider locking this issue.

@scottmas

This comment has been minimized.

Copy link

commented Sep 27, 2017

It seems to me that there's a lot of differing use cases and personal preferences for the package lock file. Generally people like it, but the default behavior doesn't work for everyone. I'm highly doubtful we'll ever be able to find a default behavior that works for everyone.

So why not make the package lock configurable and manually controllable?

For example, why not have a configurable field in .npmrc that lets users disable package-lock? Or another option that you require explicit updates to package-lock.json? (this latter approach is my personal preference)

And then provide devs tools to manually run the package-lock algorithm. Like

npm update-lock mycoolmodule

Those of us who opt non-default behavior will still need a way to configure our package-lock to our liking.

What do you think of this two pronged proposal?

  1. Make package-lock configurable via .npmrc or some other way
  2. Provide low level commands to update and set package-lock
@jakeNiemiec

This comment has been minimized.

Copy link

commented Sep 27, 2017

@scottmas

This comment has been minimized.

Copy link

commented Sep 28, 2017

Oh, good point that package-lock.json can already be turned off in .npmrc. But that is the only configuration possible. On or off. It needs to be more tunable. For instance, a setting to make package-lock only update when explicitly updated:

package-lock=explicit-only

And then commands to tune the package lock, such as:

npm update-lock mycoolmodule

Tldr, the fact that .npmrc lets us turn package-lock on or off doesn't get us nearly close enough to making the package-lock workflow customizable.

@jakeNiemiec

This comment has been minimized.

Copy link

commented Sep 28, 2017

@scottmas please see this comment #17979 (comment) by @iarna.

@vikrantsingh47

This comment has been minimized.

Copy link

commented Oct 2, 2017

should we turn off the package-lock in .npmrc or not to avoid install errors?

@NathanCH

This comment has been minimized.

Copy link

commented Oct 5, 2017

Has there been any consensus here? I participated in this thread months ago but today got completely ass-bitten by package-lock. Are there any behaviour changes between npm 5.0.x and 5.5.x?

@jakeNiemiec

This comment has been minimized.

Copy link

commented Oct 6, 2017

@NathanCH It looked like things were fixed in 5.4.x

What happened?

@alexkreidler

This comment has been minimized.

Copy link

commented Oct 6, 2017

See #18286 for the addition of a --from-lock-file (or similar) flag that would install only from the package-lock.json without modifying it.

@FrancescoBorzi

This comment has been minimized.

Copy link

commented Oct 26, 2017

I can confirm that clearing the package lock file, the issue is solved

@aervin

This comment has been minimized.

Copy link

commented Nov 24, 2017

If anyone needs to support Windows, this appears to work for our team:

{
  "scripts": {
    "clean": "rm -Rf node_modules/ && rm -f ./package-lock.json && npm cache clean -f",
    "clean_windows": "IF EXIST node_modules rd /s /q node_modules && IF EXIST package-lock.json DEL package-lock.json && npm cache clean -f",
    "rebuild": "npm run clean && npm i",
    "rebuild_windows": "npm run clean_windows && npm i"
  }
}

After running npm i <some-package>, those on Windows machines use npm run rebuild_windows to ensure the application's dependencies are actually updated.

@smithaitufe

This comment has been minimized.

Copy link

commented Dec 15, 2017

I understand that manually updating package.json won't update package-lock.json.

It is for issues like this that we need package-lock.json removed. Yarn uses yarn.lock and won't stand in our way if we modified package.json

@zkat

This comment has been minimized.

Copy link
Member

commented Dec 16, 2017

Manually updating a package.json should be updating package-lock.json as of several versions ago. If you're not seeing these updates, please first update to the latest npm version before submitting a new issue with specifics. In the meantime, I'm going to lock this thread because I think it'll only be confusing to others if there's folks saying otherwise due to misunderstanding or older versions.

@npm npm locked as resolved and limited conversation to collaborators Dec 16, 2017

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
You can’t perform that action at this time.