Skip to content

Loading…

feature request: force install from cache. #2568

Open
dominictarr opened this Issue · 37 comments
@dominictarr
npm install module --cache

would behave as if there is not network connection.

this would be very useful when the connection is slow,
and you know the modules you are installing well.

it would also create the possibility for a background process that tails npm
and keeps your cache up to date. making installs super fast!

@isaacs
npm member

Hm. This used to work as npm install --no-registry, but apparently that's broken now.

@isaacs
npm member

Oh, you can also set the cache-min config value to something greater than 0, and then nothing younger than that number of seconds will be fetched, ever.

npm install --cache-min 999999 would effectively be what you're asking for.

@dominictarr

thanks!

@robashton

Is no-registry going to be fixed? cache-min still appears to ping the registry to ask what the times are and that's undesirable?

@isaacs
npm member

Ack, indeed, this was broken with the npm-registry-client refactor. Fixed on 81fa33a.

@robashton

Cool thanks :-)

@isaacs
npm member

Also, --no-registry will be fixed on the next npm version (included in node 0.8.3 and 0.9.0)

@felixrabe

There is no mention of --no-registry in npm help install.

@luk- luk- reopened this
@luk-

This is still broken.

@digitalmaster

Any word on the status of this flag?

@dominictarr

by the way, this works with npmd if you use npmd install foo --offline

@grahamlyons

I've seen this fail using a command like this:

npm install --no-registry path/to/package.tgz

But this succeeds:

npm install --no-registry --cache path/to/cache/dir

That's perfect for my use case but I don't know if it's how it's intended to work.

This was with version 1.2.18.

@dominictarr

@grahamlyons you are using quite an old version of npm. yes, installing from a file path needs to point to a directory.

@mikermcneil

@grahamlyons in npm v1.3.17, the following fails now:
npm install ejs --no-registry --cache /Users/mike/.npm

As does:
npm install ejs --no-registry
#3691 (comment)

@domenic
npm member

This is in my notes from JSConf EU as something we are worse at than bower. We should make it more reliable to install while offline. But, probably as part of the big ol' cache refactor.

@mikermcneil

@domenic I'll help if I can :) (I'm trying to refactor sails new to use what npm already provides and avoid reinventing the wheel- just ran into this along the way. Checked out npmd and looks really promising- can't add it to our deps though :\ )

@isaacs Btw-- cache-min works great in 1.3.17, both from the CLI and programmatically (it's a little slower than doing copies, but presumably that's because it's properly checking on descendant dependencies in the modules)

So, for our friends who might be googling this in the future:

Install npm dependencies from cache (i.e. offline, no connection)

Tested in npm v1.3.17

CLI
npm install ejs sails-disk optimist grunt --cache-min 999999999
Programmatically
var npm = require('npm');

var modules = ['ejs', 'grunt', 'sails-disk', 'optimist'];

npm.load({
    loglevel: 'silent',
    'cache-min': 999999999999
}, function(err) {
    if (err) throw err;

    console.time('npm install');
    npm.commands.install(modules, function(err, data) {
        if (err) throw err;
        console.timeEnd('npm install');
    });
});
@mikermcneil

posting this here in case it helps anyone else

So ran into one more problem-- including npm as a dependency is pretty expensive as far as weight and install-time for your module. What would be great is to use a user's local npm which is already installed-- which could probably be achieved through some sort of npm trickery I couldn't figure out-- but for the short term, I put this together:

https://npmjs.org/package/enpeem

It's a very thin wrapper around using require('child-process').exec to access the globally installed npm on the system. It tries doing an npm -v first to make sure something version-esque gets printed to stdout, and if it doesn't it fails with a somewhat helpful message explaining that npm isn't available, and so the dependencies will need to be manually installed.

Of course, the jackpot would be if npm could be require()'d without having to npm install itself-- but in the meantime, this gets the job done.

@timoxley
npm member

Would like this feature for npm search as well npm/npm-registry-client#35

@ruyadorno

An --offline and/or --cache flag should definitively be a thing, these are much more descriptive and intuitive than this hacky --cache-min 9999999 solution.

@svnlto svnlto referenced this issue in hoodiehq/hoodie-cli
Closed

force install from cache #74

@majgis

In my case, setting cache-min had no effect. A locally installed karma was causing npm install --no-registry to fail. Installing karma globally fixed the problem.

@addyosmani

I've tried to advocate the use of --cache-min 9999999 but due to this not being that trivial to commit to memory/use often, I still see value in a proper --offline flag being baked in. --no-registry is still broken afaik.

@aramk

It should reduce a lot of time if npm checked the cache before making so many network requests when installing libraries. Adding --offline would be nice.

@mikermcneil

It should reduce a lot of time if npm checked the cache before making so many network requests when installing libraries. Adding --offline would be nice.

@aramk it does, I believe. @isaacs or @domenic can correct me if I'm wrong here, but I think the reason it may seem kind of "grabby" about network requests is to resolve ^x.y.z-style dependencies. I'm not sure the cache-min setting currently applies for "is there a new version?" types of lookups. I'll have a peek around the source when I have a moment sometime and see if I can't confirm or invalidate that hypothesis.

@halt-hammerzeit

Why isn't this simple issue still resolved?

@domenic
npm member

Because you haven't written a pull request for it?

@halt-hammerzeit

Wrong.

Work harder, comrade.

@othiym23

@kuchumovn

  1. This is not a simple issue, as the npm cache is integral to npm's job as an installer, and it currently has pretty tightly coupled code. If you believe otherwise, you can prove it by submitting a fix in a PR.
  2. We are working on a fix, by factoring the caching code out into an independent npm-cache module with its own API. This is one of many projects the npm team has under way, so progress is slow but steady.
  3. Being rude to npm contributors will not make the work happen any faster.
@othiym23 othiym23 added this to the cache rewrite milestone
@ipuustin

It would be great to get an update to this issue. Problem with --cache-min 9999999 is that installing dependencies in a tree does not work: it expects packages to be installed in a flat hierarchy. In a perfect world I could add several packages in the cache and then install only one; the rest would be automatically used as dependencies to the top-level package.

@bseng bseng referenced this issue in kmees/MSBuild.NodeTools
Open

Make use of Npm cache on default #29

@despairblue

Using --cache-min 999999 will also fail npm install whenever you install a package that depends on a newer version of a package you installed in the passed. Took me a while to figure out why that happened:

screen shot 2015-12-18 at 23 05 38

@timoxley
npm member

@despairblue Yep it's painful. Tracking that issue here #8581

@cowboy cowboy referenced this issue in bocoup/deployment-workflow
Merged

only npm install if you need to #67

@nicgirault

Does someone understand what is the main purpose of the npm cache directory if running twice 'npm install package' generate two http requests?

@othiym23

@nicgirault When packages are in the cache, the initial metadata request sends the cached etag for the package, and in the vast majority of cases, the registry will return a 304 and the package tarball won't need to be downloaded again.

@nicgirault

@othiym23 thank you, your answer is clear.

@zachborboa

This may now be relevant.
strongloop/loopback#1079

@othiym23

@zachborboa That's strictly an issue with StrongLoop's packages, and is unrelated to this issue.

@zachborboa

@othiym23 I still think it is related. If the optional dependency were truly cached, they would only ever receive a single (analytics) ping for the specified version.

@othiym23

That's not how the cache works, @zachborboa – it's always going to fire off a request to see if it has the latest version of a resource. Having an "offline" or "use the cache only" mode would prevent those network checks, and, barring what changes would come from making the cache content-addressable / read-through by fetching tarballs from the cache by their shasums, not the default mode for the cache, now or in the planned future. Discussion about how StrongLoop uses optionalDependencies belongs over there, and is not a part of this discussion.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Something went wrong with that request. Please try again.