Join GitHub today
GitHub is home to over 28 million developers working together to host and review code, manage projects, and build software together.Sign up
NPM Namespaces or Sub Projects #798
The NPM namespace has only one level, which means that eventually, all the good names are going to be taken. This is going to happen quickly, as programmers use the obvious names to describe their projects. (i.e. parser, cms, mail, email, store, etc.)
One of the nice things about the GitHub is that people can create a small project, to capture a small idea, and give it the obvious name, since that project will be in the context of their user name. So, you can create a small library, like
I'd like to be able to create a sub project of packet called
I believe the
I'd propose either allowing installations of sub modules, so that you could have
Every namespace is a single-layer namespace. What you're suggesting is enforcing a username prefix onto package names, and that's not a good idea, imo.
If you really want this, no one is stopping you. Name your packages
I see no reason to change.
Exactly. So get to hacking! ;)
If someone else builds a binding to something that you actually own (ie, they publish a package called "jquery", and you're John Resig) and you are unable to work it out with the publisher, then let me know. Otherwise, I encourage module authors to work things out between themselves. The land-grab for good names is a way to encourage people to get cooking rather than make reservations.
If you want it, build it. If someone else beats you to it, then good news! you probably just found someone else who is willing to work on this project with you! Go make friends.
+1 I agree with @bigeasy. It would certainly be nice if periods(or some other character) where reserved for future proofing the whole npmjs system.
Whether we like it or not people do often take small prejudices towards shorter and more logical names. I'm not saying we have to change the system now, but future proofing for github
In the future
For example, there could be folders inside of
You could then simply link to
Another issue not mentioned above, is names that are simply short but provide no meaning (and potentially little value).
Considering people can link variables to any package name, it could be valuable to limit package names to 3 characters or more. Hopefully people would start going for more meaningful package and not simply the length of the package name, but that is another issue entirely.
I understand you've said this was not going to happen, but I just wanted to add my voice in favor of namespacing modules. @limeblack has given many of the details that make the lack of it very annoying. And this can only get worse.
It is not a matter of whether one can do it or not, it is about the fact that the system itself should enforce it. I guess no one can deny there are tons of noise when doing
rather than the (current) ridiculous:
…at least I know they are competitors to the same functionality rather than competitors to a sexy name.
Blaming it on the users would be akin to Node having the top-level scope default to the global scope, and blaming it on the users for not scoping properly themselves. Namespaces were invented for a reason, Java enforces them for a reason, GitHub enforces them for a reason, and the lack of them is against NPM’s job.
Indeed, considering NPM’s job is to:
Having to read through 8 different packages with a difference so slight in their name one does not even know whether this is a typo or there's an actual difference is against 1.
As it has already been said: this can only get worse. I don't believe the “gold rush” on names is an incentive to “get hacking” but rather an incentive to be childish and deliver quantity fast rather than focus on quality, just so one can have his one-letter or canonically-named module. And, considering how fast Node evolves and the seemingly high turnover of weekend hackers (which I totally understand), we're just bringing involuntary cybersquatting to a whole new level. Can't wait for having to choose between
Do you know that, though? How do you know if they're not just that the authors didn't realize there was already something by that name?
Now, which of those share DNA (ie, are forks of one another), and which are simply resulting from the fact that the user didn't realize there was already a markdown lib on npm, and so published their own? Which ones are bugfixes waiting to be merged in?
npm makes it very easy to find the github repo of a project, and very easy to depend on your own git fork instead of the official one, while awaiting a patch to be merged in.
Furthermore, we have a policy for handling actual name disputes, whether arising from trademark or author abandonment.
There's no excuse for
npm isn't github, and the Java language is not even remotely relevant to this discussion. (Unless you're talking about maven or something? I don't know, I've used Java only when absolutely necessary.)
Github doesn't "enforce namespaces", per se. Github puts the username in the project URL. That's because github is specifically designed to encourage forking, and in a fork-friendly environment, my "markdown" project is different from yours. Github also tracks where each project was forked from, and shows the code, and is a source code repository, not a package manager registry.
npm is not designed to encourage forking in the registry. npm is, in fact, designed to discourage that. It's designed to get people to either decide on a single thing called
In other words, it makes a lot of sense for github to have this, and for npm not to, because github and npm are profoundly different in their goal, their scope, and their use.
I don't believe that this makes finding packages any easier. It certainly does not make managing dependencies more efficient.
Let's clarify what the request really is here. You're asking for the package naming rules to be changed so that all packages must have a single
And would I now have to do
I maintain that having to read through 8 different packages with the exact same name would be even worse. A "typo" in the package name would indicate that it's not actually the same thing (it's not as if you type it each time -- it goes in package.json once, and that's it.)
The maintainers field is quite clear, both in the search output, the list output, and explicitly via the
And you can also depend on git urls (including a sha), or urls to tarballs, or bundle your dependencies.
The gloom-mongering isn't helpful. I believe that one way it could get worse would be to add namespacing.
If you have specific problems that you'd like to see addressed, then bring those up. It's not useful to make vague prophesies about things "getting worse" because we don't have "namespaces" (which is itself a very vague feature request that has been explored numerous times in the past and rejected).
If you see packages like that, then please contact the author, and me, and we'll get them removed.
Again, how is that different from
If you want a registry that works differently, make your own registry, and write a tool that uses that. The source is open.
@isaacs You’re right. There was a confusion between “adding namespaces” and “making it easier to find something in the public registry”. Namespacing is so common in other packaging solutions, that this issue felt like a good place to present frustration regarding the search part. There would be no benefit in implementing namespaces in the package manager.
I still believe we’ll need to find a way out of the current situation, and that pushing for username namespacing rather than stupid puns would be much more efficient. But it’s a matter of recommendation, just like the current “avoid js and node in names” recommendation, not a matter of code.
Without arguing on a feature request, here are some more details regarding the “it can only get worse” part, and why it is not simply “gloom-mongering”. Once again, this is not directly related to namespacing anymore, I am simply describing the problem I believe most people here encountered, and why some (or at least I) thought namespacing could be a partial solution to it.
Oh, I never said I’d know whether there was a relationship between them! There would be no way to know which are forks. I only said:
To rephrase with the example you gave: it is easier to make my choice between
…than it is between
Indeed, an author not realizing there was already a lib for his stuff in the public registry wouldn’t be saved by default namespacing. But at least he wouldn’t come up, after an
I’m afraid it’s only a question of time. Once all shorter and canonical names for a given feature are taken, creative ones are used. Once in a while, a completely unrelated name is chosen. And people who don’t feel creative when wanting to publish simply start breaking the rules (or the recommendations, as you prefer): they add the dreaded
You can say there’s no excuse for adding
This is a self-sustaining problem: the harder it is to find quality modules in the public registry, the less likely it is developers will take the time needed to find the one they want, and the more they will start their own, adding yet another duplicate. I don’t believe it is gloom-mongering to simply describe a positive feedback entropy loop.
Added to the FAQ on f7d681f.
+## Why no namespaces? + +Please see this discussion: <https://github.com/isaacs/npm/issues/798> + +tl;dr - It doesn't actually make things better, and can make them worse. + +If you want to namespace your own packages, you may: simply use the +`-` character to separate the names. npm is a mostly anarchic system. +There is not sufficient need to impose namespace rules on everyone.
referenced this issue
Oct 17, 2012
For all the reasons stated by @unscriptable, @bigeasy, and @MattiSG, I'm putting in my vote for real namespacing support. This current "dispute or make up a nonsensical name" model isn't sustainable. Neither is the ugly dichotomy it creates between AMD and Node environments. IMHO, it is much more valuable to the web ecosystem for us to be advancing dead-simple reuse of modules across JS environments--ways that don't involve artificial things like "foo/amd" or keeping two copies of the code on your filesystem.
-1. I think namespacing is a premature optimization for a problem that doesn't exist.
Single namespace (canonical naming) seems to work just fine for both Ruby (https://rubygems.org/gems) and Python (http://pypi.python.org/pypi?%3Aaction=index). What's different about the Node community (ignoring AMD, as that's not really a Node problem) that this would be different?
I agree that there's a nonzero risk that it will be "unsustainable", in practice this hasn't really happened in either community, and philosophically speaking I'm not sure why Node would be any different. In rubygem land, sure, there was a proliferation of named gems, but they tend to weed themselves out over time (I think rubygems has a sort of LRU cache that will kick out unmaintained, aged gems, but I'm not sure). I expect npm to suffer the same short-term growing pains. Long term, it will naturally work itself out.
The gains to be able to mention a name in common conversation, e.g. "capybara", "selenium", etc. and know which one you're getting is huge. The lack of fragmentation in packages is huge. That goes away when you have to remember to namespace "Foobar/package", not "Bazquxx/package". If a package really "wins" then everyone will know the name; who cares what it's called?
Actually, namespacing doesn't solve the core issue. Sure, it helps eliminate the nonsensical names. However, we really need package aliasing if we're going to be able to use libraries across environments. Here's why:
There are currently thousands of AMD and/or CommonJS projects in github. These projects are already namespaced so there are no conflicts on github. Many of these projects use other projects as dependencies. Take cujojs/wire, for instance. wire.js uses when.js and several other libs.
Internally, wire.js uses when.js via
When node.js started becoming popular, we decided to make our libs node-compatible. Yay, users will be able to use our projects in the browser and in node! Unfortunately, somebody already grabbed the names "when", "curl", and "poly" in npm before we did. If we publish our libs under alternate names in npm, such as "cujojs-poly", then anybody using poly.js via npm has to
So, we're left with two choices:
Neither of these is a real solution, imho, so we're left telling our users not to use npm.
FWIW, AMD and other CommonJS environments (like PINF) allow aliasing and/or package mapping. So, for instance, we can install cujojs/poly (via cpm, volo, plain old git, etc.) in any folder we like and map references from "poly/whatevs" to that folder via package mappings. If npm (or node?) allowed the same, we could give simple aliasing instructions to users.
A simpler solution might be a program that could detect and automatically
not that hard if you use a parser like http://esprima.org/
On Thu, Oct 25, 2012 at 12:23 AM, John Hann email@example.com:
Hey @dominictarr, this isn't a formatting issue. We're already using a simple UMD format that let's us use the same code in AMD, or a node-like environment (we like the node extensions to CJS Modules/1.1).
The problem is that the combination of npm + node_modules doesn't let us rename or alias packages. Ideally, code will be located in multiple places (npm, github, cpm, bower, etc.). If we aren't lucky enough to grab our favorite name in all those locations, then that sucks. Namespacing would help, here. But it seems that people who have already grabbed their favorite names -- or are starting a node-only project from scratch -- don't seem to think there's a problem.
However, what's worse is that that npm+node_modules won't let us change the name once the package is installed. For instance, if we're forced to name our project "cujojs-poly" in npm (and "poly" everwhere else), then code that uses poly.js has to be written differently if it's going to be retrieved with npm.
That's not a maintainable solution.
Workflow to find a config file loader:
Clearly that's too much for my brain to compute, so I'll do a first sort:
Already here, I missed
Hey cool, there's
Oh, but this canonical name is not a config loader.
And then, see them all coming:
…all of them claiming to be the best of the breed, or rather, the simplest solution ever made. (And this is most likely why
Awesome. Now I have to
Admittedly, namespacing wouldn't help in everything, especially not in the quality comparison part. I agree namespacing is just a subset of the solutions that could be implemented in improving the search experience. I also think it would be most useful as a public registry recommendation, perhaps more than a core package management feature. I am also not saying it would solve the multiple environments problem described by @unscriptable .
However, here is how it would help, with our previous example:
Of course, most of these issues are more social than technical. Of course the package manager is a technical system. But the public registry is a social system, and the technical design, no matter how simple / well-thought it is, is offering a very poor base for the social system. If we are to solve the cognitive overload issues that are making working with NPM-PR painful, we need to fix the system as a whole.
Yes, this specific solution means adding technical complexity to favor social simplicity. I understand it may be frustrating from the technical point of view. But, whether you like it or not, as soon as humans come in, there is responsibility of a social system.
I'm sure there could be better solutions and, as I already said a few months ago, I am not arguing for namespacing as a core feature request. But please, don't come up saying the current system is “simple”. This is true only in a very limited meaning of what the “system” is.
Ah, I think everyone agrees that module discovery has room for improvement.
modules do have tags, but only the authors can add them.
and of course, this is an open source project,
On Thu, Oct 25, 2012 at 7:22 AM, Matti Schneider
But again, as @unscriptable argued, it's not just discovery, it's how you use these package names in your code. I honestly don't care if a package has the longest name ever when I do an npm install, because I'll do that once, but I'll write those require calls many times, so it would be cool if I could alias them to shorter names in my package.json.
but then, when I am reading your code it says
anyway, you can do that;
#then in your code
The best of both worlds! now I know that you have done something
Of course, you should use a very simple script to create these aliases, if
On Thu, Oct 25, 2012 at 11:16 AM, Tiago Rodrigues
mmm...good point. And now that you mention it, a lot of people in the node community do something similar when they create index.js files in directories with easier paths for modules inside that directory. Yes, maybe a tool to help with that would be a good idea!
If you want "namespaced" modules then use urls. As long as the url refers to a package, and the package.json in that package has the same name that you used as the key in your package.json's
Anything else, all the other features suggested and argued for in this thread, and in many others, and 3 years ago when we were all debating about the "mappings" addition to the package.json spec, back when CommonJS was a thing, all these features are actively harmful. They add complexity, and zero benefit. No. They're not ever going to be in npm. The success of npm is proof that these features are not important. If you believe that they are essential for a package manager to have these features, then go build one. You can steal all my code. Surely, with the benefit of namespaces, you'll take over as the go-to node package manager in no time, and I'll get to retire. Nothing would make me happier, though it would be a huge surprise.
@isaacs I think the shorthand GitHub URLs for namespacing definitely fixes the major issue here.
However, I think it introduces another issue: you end up with dozens of GitHub repos for a single project. For a real-world example, look at VoxelJS or PrimoJS:
This is especially problematic for companies with private GitHub repos -- often you will have limits on the number of repositories you can have, so this approach is not reasonable.
It would be ideal if you could have a single repository for multiple modules. This way, your documentation, wiki, issue tracking, release builds, SEO, commit histories, module discovery, etc. is all contained in a single github repository.
Allowing to checkout just a sub-folder might be one solution. See here:
@mattdesl: You can technically store any number of discrete repositories in one top-level "repository" by keeping each module's repository as a branch. Then, for your packages, refer to the different module-branches via the commit-ish part:
One point of complexity would be that you'll have to namespace the modules' tags. I think you can actually do it with just straight-up subfolders:
The process to create such a franken-repo is left as an exercise for the reader.
In practice this doesn't sound like such a good solution. Say you are developing your "core" module and need to add something to the "utils" package. This leads to a lot of commits/checkouts (or stashes) just for a very small change.
It gets worse when you have a team, and a bugfix (which would be done in a new branch) needs to span multiple modules. Sounds like a disaster.
Now that private packages are in the works, I am curious how the namespacing will go. Keeping the current format will lead in a messy management problem.
Currently we have this option (as far as I know it):
I've seen a situation where a project had around 32 private packages, hosted on bitbucket and used a custom download script
Relying on creativity for naming and remembering 32 private packages is very hard, for me at least.
Regarding GitHub urls, I'm having issues with installing them as global packages and then updating them with
There are lots of yeoman generators with names like "simple", "express-simple", "simpleapp" that are fundamentally meaningless and ambiguous. They all still have a hyphenated namespace (generator-) which doesn't help anyway. Please consider adding a username namespace like Github does