npm ci and CI cache

[ArmorDarks]

I'm opening this issue because:

• npm is crashing.
• npm is producing an incorrect install.
• npm is doing something I don't understand.
• npm is producing incorrect or undesirable behavior.
• Other (see below for feature requests):

What's going wrong?

npm ci remove current node_modules, thus making CI's caching usage impossible.

How can the CLI team reproduce the problem?

Restore node_modules from cache on any CI, then run npm ci

What's the feature?

npm ci flag which preserves node_modules before running installation

What problem is the feature intended to solve?

In large projects usually it's quicker to restore node_modules from cache of previous build and run npm install based on it.

Is the absence of this feature blocking you or your team? If so, how?

No, it doesn't. It just hurts.

Is this feature similar to an existing feature in another tool?

No

Is this a feature you're prepared to implement, with support from the npm CLI team?

I'm not sure I'm capable to do it

[legodude17]

npm ci only works when there is no node_modules. What do you want to happen here?

[ArmorDarks]

npm ci actually removes node_modules if it exists.

Basically, we'd want npm install which follows stricly package-lock.json, does not update it and errors if there is no lockfile. In other words, everything npm ci does, except removing of the node_modules.

Purpose — to use it on CI services like Circle CI.

We found that npm ci is great option for CI, because it eliminates issues with sporadically updated locfkiles by npm install (which in turn leading to caching issues in some cases), but it is a major drawback for very large projects that npm ci completely eliminates possibility to leverage caching of node_modules.

I think the issue could be resolved the other way around — by adding ability for NPM to specify offline modules storage path, like it can do yarn with offline mirror. Than CIs could cache that mirror. However, it would introduce it own pitfalls...

[legodude17]

Are you concerned about network access or speed? You could tell your CI to cache npm's cache dir, and then use --prefer-offline.

[ArmorDarks]

Are you concerned about network access or speed?

About speed. npm ci without cache turned out to be 3 times slower than npm install with a restored cache (npm install time measurement includes cache saving and restoration time).

You could tell your CI to cache npm's cache dir, and then use --prefer-offline.

Hm, didn't know about -prefer-offline flag. We'll try it, alongside with ~/.npm caching.

[Poyoman39]

I agree with @ArmorDarks we have the same performace issues with npm ci. It's a great feature since it uses package-lock for install, but currently npm install --no-save is going way faster than npm ci because it does not clean node-modules at first.