Allow subdirectories within git repos in npm install

[SLaks]

I would like to be able to run npm install for a package.json file in a subdirectory of a git repo.

This would be useful for projects that use releases systems (such as nzakas/parserlib), and for git repos that contain multiple npm packages .

Can this be added?

[guybrush]

cd subdirectory && npm install :D

[strk]

Same problem I have with https://github.com/nmrugg/LZMA-JS - the actual module (package.json) is in a src/ subdirectory, but I don't find a way to reference that subdir from the package.json of a module that depends on it.
Is that possible ? (can't script a cd && npm install in package.json, can I?)

[guybrush]

you could try something like {"scripts":{"install":"cd src && npm install"}} in your package.json.

but that is just not how its meant to be done

but I don't find a way to reference that subdir from the package.json of a module that depends on it

when your module depends on another module that other module will be installed in node_modules/<module-name> and in that directory there will be its package.json. thats not only how npm does it - thats how the node-modules work, see http://nodejs.org/api/modules.html#modules_all_together

so i am pretty sure, the way how it works now will not change (its set in stone)

[isaacs]

@SLaks There's no plans to do this. If npm installs a git repo, it assumes that the git repo is the package. I don't really know how we could specify a sub-path easily, since all parts of the git url are already used for other things. For example, git://x.com/path/to/whatever#some-branch. Where would you put the subpath in there, such that it can be deterministically differentiated from the path to the repo in all cases? I don't see any way to do it without resorting to some awful hacks.

How would you express this?

[strk]

What about a space ?

git://x.com/path/to/whatever#some-branch some/sub/dir

[djMax]

+1 for this use case. For me, it's about including tests from other components (Java components). We use mocha to test these things, and service A wants to import service B's tests but we can't use npm because it doesn't have the concept of branches. Since the tests will be against some branch of service A, AND because the tests are part of the service rather than their own repo, we would benefit from subtree installs in GIT. Maybe another argument like --nodedir?

[dbkaplun]

This would be great. I'm using highlight.js which stores its source, including package.json, in src/.

[bitdivine]

This would indeed be great. As it is I will have to make a whole bunch of dummy projects. I hope that I can simply have each main project as a git submodule and put a link called package.json to the real package.json. That might break though...

[lukaslehmann]

any news on that? or is there a workaround for that case?

[gaboom]

+1 Unfortunately not all npm git repos can store their contained npm module in their root's!

git://user@example.com/path/to/resource//path/to/directory#commit-ref

[SLaks]

@gaboom: That's actually a breaking change, in the (admittedly unlikely) case of // appearing in a git URL path.

[gaboom]

Hmm,
git://user@example.com/path/to/resource/[path/to/directory]#commit-ref
brackets are only legal as IPv6 literal hostnames, needs to be escaped otherwise.

[fwestrom]

I could definitely use this feature too.

We have a Git repository with several of our product's subsystems, and I want to be able to use npm to install individual micro-services as packages from their subdirectories (services/{package-name}/package.json).

[banks]

Another +1, I agree URL syntax is ugly but apache/thrift is a good example of a nodejs project embedded in a bigger repo and trying to work with a non-tagged release is made very hard without this feature.

[nevcos]

Another +1. Need to import other module that's not in the root and don't want to create a specific project for that module.

[kirill-konshin]

An example: https://github.com/pubnub/javascript/tree/develop/node.js - as you see, the actual code for NPM is located in node.js directory, not in the root of the repo. So if I need to install develop branch, the only way is to locally do npm link.

[JuHwon]

+1. would require it in a fork of the breeze node repo https://github.com/JuHwon/breeze.server.node

[xogeny]

+1 for me as well. The issue I face is that I want to keep code in a private GitHub repository. If I have one module per repository, I'll end up with tons of private repos which would cost a fortune.

[febeling]

There are the really big companies (FB, Google) which make the decision to keep software in monolithic repositories. And that's not a decision made out of ignorance, I believe. It would make a lot of sense to support this model of working for others, too.

[mightyiam]

I feel much like @piranna.

Also, I'm an avid Greenkeeper user and you can forget about using it with monorepos.

[ghost]

@piranna
This debate is basically prescriptivism versus descriptivism. You're saying that people shouldn't be using monorepos. We're saying that they are using monorepos, without regard to whether they should be, and therefore we (consumers of npm packages; not necessarily producers) need support for it.

[ptitjes]

So there is no solution ? Are we stuck with making manual installs ?

But isn't this just a change directory to do after the cloning ? Please npm folks !

[ptitjes]

@othiym23 The following are the parts that would need to be modified to make this work, am I right ?

https://github.com/npm/npm-package-arg/blob/master/npa.js#L218
https://github.com/zkat/pacote/blob/latest/lib/fetchers/git.js#L73

If that is the case, I might like to do a PR.

[jacksonrayhamilton]

@othiym23 will you please re-open this issue so we can open bounties for it on bountysource.com?

[bjankord]

Hi @zkat, @othiym23

There has been a lot of discussion on this issue and it seems the community has a need for this. A handful of ideas have been proposed from npm users which, IMO, are worth more input from the npm team.

From using a separator which branch names cannot use like git://github.com/babel/babel.git#master::babel/packages/babel-core/ or using a query param syntax like git://github.com/babel/babel.git#master?path=babel/packages/babel-core/

It would be nice to get some more input from the npm team on these 😀

[ch1ll0ut1]

Please :(

[subpardaemon]

so this issue is now closed after

There has been a lot of discussion on this issue and it seems the community has a need for this.

...without an actual solution?

seriously?

no wonder npm is considered a cussword in most programming circles.

[jacksonrayhamilton]

It’s pretty sad. There seem to be plenty of people who would even write improvements for npm themselves (or we could pay them to do it), but the maintainers will not even respond to us.

[subpardaemon]

i mean now i have to scrub my entire repo and create a separate repo for all the separable packages of my project. if there was even a tiny goddamn HINT in the official "how to contribute packages" doc about npm's inability to handle monorepos then i'd have designed my repos accordingly.

it's still going to be a nightmare, a different project for each package in netbeans, and my system so far has about 15 packages... thanks a lot indeed.

[ptitjes]

@subpardaemon @jacksonrayhamilton If I understand the code correctly the main contribution and new tests are to be made on https://github.com/zkat/pacote and https://github.com/npm/npm-package-arg. Then there is not much to do on npm itself (maybe add a test case).

This would require to make a PR on both sides: pacote does the fetching of the package (in our case, from a git repository) given a manifest that is built by npm-package-arg from the parsing of a git url.

The difficulty is to convince both parties that this is a feature to have. Apart from @othiym23, we could also lobby @zkat (creator of pacote) and @iarna (that also contributed both the files I mentioned in #2974 (comment)). Again, I'd be glad to contribute that.

[Brantron]

It would be nice if NPM would look at this as a potential win for their project vs a nuisance.

[matijs]

How would npm know about versions of individual packages inside a monorepo? Installing @latest might not always be what you want.

[lukescott]

How would npm know about versions of individual packages inside a monorepo? Installing @latest might not always be what you want.

Since this is about installing individual monorepo packages within a Git repo, you would target a branch tag or commit.

90% of the time you want to install the individual packages off npm, which is what we do now. 10% of the time there is a bug fix for one of those packages and they haven't done a release yet. I'm experiencing this right now with babel-plugin-istanbul and the arrow function coverage bug w/ Babel 7.

For a non-monorepo package you can simply point directly to the Github repo. This case is similar, but you want to scope it just to a single package within the repo.

For those that make monorepos they don't necessarily need this feature. It's for those that use projects that use monorepos. Telling them to not organize their projects into monorepos doesn't help people who make use of these projects.

[bjankord]

@othiym23 mentioned back in Jan 2016

monorepo support is not something that's on the CLI team's roadmap for the next 6-12 months. I'm interested in seeing better support for it within the CLI, but the team isn't going to have the time and attention to do it ourselves. For that reason, specific proposals (perhaps in the form of (prototype) code) are going to be more useful than generic requests for improved monorepo support.

Its been about 25 months now. People are interested in contributing PRs to get this working. Some additional guidance from the CLI team would be helpful.

If this issue seems to be making no traction with npm's CLI team, I'd encourage people interested in this feature to look at yarn instead.

There are a few issues on yarns repo about this feature:

• Allow subdirectories within git repos in yarn install yarnpkg/yarn#4725
• Allow to install packages from git monorepos yarnpkg/yarn#1570

Yarn also has an RFC process which may offer a better discussion platform compared to this GitHub issue.

[trusktr]

Haha, entertaining. :)

Still though, this would be useful.