Join GitHub today
GitHub is home to over 28 million developers working together to host and review code, manage projects, and build software together.Sign up
Bundle/Include Dependencies #4210
Create a means within npm of generating a
This involves two separate and distinct aspects:
This is a rewrite of the closed ticket #3429.
Reason for Feature Request
The reason for this feature request is to provide a means for npm to be used to install a given package in an offline environment and without relying on the npm registry being available online. By using a bundled npm package tarball that contains the package and all necessary dependencies, npm is not forced to go out to the internet and find the dependencies required. This functionality is desirable for use in isolated/off-the-grid systems and for usage by package owners whom wish to distribute their package as a complete unit.
Consider this simple problem case: Install
(In the process described below "LIVE" indicates a system with an internet connection and "OFFLINE" indicates a system without an internet connection.)
Your first attempt might go something like this:
The problem with this is that this did not actually install
Your next attempt might be to get the
npm is attempting to go out and get the dependencies from http://registry.npmjs.org which clearly will not work on our remote offline system.
Okay, so you can just get and install all the dependencies. Except there isn't a way to list the dependencies of an npm package from npm. The best way to get the dependency tree in node is to actually install the npm package, so let's do that.
Well, this is the same problem as before. Basically, there is not an easy way to tell npm to use all these
npm does cache it's files and we can set that cache to somewhere, so maybe we can use the cache functionality.
Success! A little convoluted and a bit of extra work, but a success. But I think we can do better...
Goals for the Feature
Specifically, there are two aspects to this feature request: Bundling a Package into a
In this aspect of the feature the objective is to create a
For a given package...
Suggestions and additional things to keep in mind...
Install from Bundle
In this aspect of the feature the objective is to take a previously bundled
From a given
Suggestions and additional things to keep in mind...
This ticket was initially written up in #3429. I believe that #3429 was closed due to misunderstanding of the desired feature and/or the problem space. This rewrite is an attempt to clarify that misunderstanding.
npmbox is an attempt to provide an example of the desired functionality described in this feature request in a third party tool. However, npmbox is not a final solution because it depends on npmbox being first installed on the target system and thus we have a chicken and egg problem.
pac is somewhat related, but also not really the solution desired. It works by examining your package.json file and then downloading the tarball of any package listed there as a dependency into the .modules folder. While this might seem like a similar answer, it does not download the entire dependency tree nor does it provide a means to install the package or its dependencies.
I think I have something that will help with this: https://gist.github.com/jackgill/7687308
It's a script that installs a package, rewrites its package.json to copy dependencies to bundleDependencies, and then packs it. This creates a .tgz file which can be installed offline using npm. You would use it like this:
The script is a rough cut, but it should be the right idea -- let me know if I'm missing something about your use case.
It would be nice to have a feature like this built into npm, since rewriting package.json for 3rd party modules seems sketchy.
This was referenced
Dec 17, 2013
I would like to +1 this feature request. I've been using npm for different applications, but recently I've tried to install JSHint support for Sublime Text Editor on a virtual machine without internet access. Unfortunately, JSHint is only available through npm. I finally ran into npmbox, but I think it would make a great standard component of npm.
Likewise, the same might be applicable when running Grunt for your workflow on such a disconnected development machine.
I don't think it's really needed.
Firstly if you need project to be installable offline, you should just install it, pack it with all it's dependencies and distribute it that way. Afterwards all that is needed to install, is to unpack and run
I have many projects that I need to have to totally network independent, and have no issues with above workflow.
Just one thing that npm misses, is ability to install project but without compilation. Currently after install you need to make sure to exclude all build folders, to have clean bundle.
So, basically, what you want is a
Also, just for convenience, it should probably automatically ignore anything that looks like
This is a good and useful idea, and probably not that hard to work in, though it's not trivial. (See the code of
@isaacs What I'm looking for is
I personally like the idea of just adding a switch to
I played around with something like this a bit - basically pointing the cache at a subfolder then installing. To install only off of the local cache, run install with the registry url unset so only the cache is used. While this generally works, one of the downsides is the cache structure is pretty verbose (both the expanded and tar'd versions of the libraries are there)
It basically looks like this:
Perhaps this is an approach that could work with minimal changes - either a replacement cache that just does tarballs or modifying the existing cache logic to store/work with a bit less data. Then add a nice command line option(s) to package all this logic up.
This wouldn't provide what @arei was looking for exactly (a module with its dependencies inside a single .tgz) but would result in a local only install option.
Looks like we're going to finally get some resolution, or at least the tools to build the resoltuion. http://blog.npmjs.org/post/91303926460/npm-cli-roadmap-a-periodic-update
+1 This is a very good idea. I just want to add that it should not be necessary to include a package's dependency tree inside the package's tarball. An alternative would be to download the package and its dependencies all as individual tarball files and point the installer program to a directory of these tarballs.
In the Python world we have this feature in Buildout and Virtualenv. For example I can run Buildout online and tell it to download and install my packages, but at the same time also keep copies of the source tarballs in a directory. Then I can initiate an offline install of all the packages by running this command:
python bootstrap.py --find-links=./third_party/dist/
Another nice feature is if your Buildout config file does not list a particular dependency of a listed package, Buildout will print to the console these unlisted dependencies and you can copy and paste the list into your config file to make your listing complete.
changed the title from
[Feature] Bundle/Include Dependencies
Sep 20, 2014
I'm surprised this has been such a topic of debate. In many organizations there are closed intranets that can greatly benefit from this kind of functionality.
Python's package manager makes this incredibly easy:
On the online machine:
@arei Thank you for your ongoing crusade in lobbying for this!
I created the following package to behave like npm-pack, but it accepts the additional arguments of npm-install and includes dependencies in the output:
I propose that any official npm-bundle feature consider including the same arguments as
One interesting issue I found when creating this package is that
referenced this issue
Apr 3, 2016
P.S node-pac is another option, but it can't deal with the packages which still need downloading something for installation.
This is something that is highly relevant to @seldo's interests – having a better story for deployment is something that concerns a lot of the people at npm, Inc. thinking about deployment as a product. That said, it's not that clear where this fits on the product roadmap either for the company or for the CLI team, so I don't know when we'll work on this. There quite a few subtleties to getting this set up right (e.g. do we deal with cross-platform support for native modules? if so, how? or do we just deal with cross-compilation? or nothing at all and force an
Doesn't `node-gyp` come with `node`?…
On Mon, Dec 19, 2016 at 4:21 PM, jdmairs ***@***.***> wrote: I could not get npmbox to install node-gyp successfully...everything else has worked great. Comments would be appreciated! — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#4210 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AKr56Jn2YuIPkebo226nr4KKU31jesz9ks5rJx-IgaJpZM4BQca1> .
referenced this issue
Mar 21, 2017
This was referenced
Apr 18, 2017
We're closing this issue as it has gone thirty days without activity. In our experience if an issue has gone thirty days without any activity then it's unlikely to be addressed. In the case of bug reports, often the underlying issue will be addressed but finding related issues is quite difficult and often incomplete.
If this was a bug report and it is still relevant then we encourage you to open it again as a new issue. If this was a feature request then you should feel free to open it again, or even better open a PR.
For more information about our new issue aging policies and why we've instituted them please see our blog post.
I think for now it may be worth just researching the alternatives for a solution? Which do you think is the most reliable alternative? And which ones wouldn't require the same node-version on both machines? ( I think we can assume at least Node V6 for everything)
I'm actually going to argue against reopening this issue.
npm5 significantly changed the landscape with regards to potential-use cases for this feature, and I think that it makes the most sense to restart this conversation with that context in mind.
My $0.02 is that
If you have a
@schmod Interesting... what about if you move that node_modules folder to another computer that requires a rebuild of bindings. Would a "npm rebuild" suffice do you think?
I've been using npmbox / npmunbox to move the node_modules. I wonder the differences between that and doing a tarball. Thanks.