Myself and @substack were talking in IRC and came up with an interesting idea around discoverabiity.
What if we allow the community to create arbitrary indexes. This would allow all users including module authors to add an arbitrary string tag to a module.
This would effectively allow creation of sub communities. We could start tagging modules with "works_browserify" or "passes_testling" or "angular_js_friendly".
The steps needed to make this useful would be:
Optional things that might be nice:
My inspiration for this is http://stackoverflow.com/ which has a very good tagging system driven by users that causes user maintained sub communities within stackoverflow.
Don't keywords already provide this? They are tags after all.
What would be nice if keyword discoverability and discoverability via keywords (2 different things) would be improved.
One option would be that a user that did a search gives feedback about which module actually did what he needed and that gets stored in npm somehow.
The other option is for users to indicate which keyword best describes the module, i.e. via an upvote.
Browse by keywords could then be added to find the top modules for just one keyword. Also keywords that are commonly found together for lots of modules could be grouped together (group would contain those keywords as children).
Just some ideas here, but I feel like we should improve the features that npm currently provides to improve discoverability instead of complicating searches further by adding more features.
The idea of "voting" (i.e.: this number of users support this particular classification of this module) is interesting for building a "trust" system that's like closed curated communities, without the "closed" part. Helps if one could see the people who classified the module, in the same way as "npm star," though I'm not sure how the user interface would adapt to this case.
@thlorenz keywords provide this.
The problem with keywords is that only module authors can edit or create them. This puts a very high burden on module authors when really we want to spread the burden in a distributed fashion across all users of NPM.
keywords would work just fine if they could be edited by non-owners.
That is true, but maybe we can allow users to suggest keywords for a module, the same you suggest for tags?
The author could then be pushed to add them via an automatic PR. This way it's protected from some users misunderstanding what the module does since the author has the last say.
Also authors need to become better about keywords and/or PRs need to be made. I feel like once keywords are used heavily to help search (like I suggested above), authors will improve on this since they want their modules to be found.
Adding tags just categorizes each module twice:
That could get confusing rather quickly.
Oh, by reading @thlorenz I actually realise this problem is similar to what tagged file systems propose to fix wrt discoverability in hierarchical file systems. There are a few papers on this that address the discoverability of files within a million of irrelevant files, without the need to enforce extensive tagging.
The most interesting I've read used things like keyword suggestion from natural language analysis to decrease the burden of thinking about keywords, and weight linking of keywords (i.e.: if a user searches for X, but clicks on a possibly related Y keyword, the weight of the link between X and Y is increased, and Y gets suggested as an alternative search when looking for X).
The thing about social tagging is, imho, more relevant than burdening the author with maintaining the keyword list, though. It's similar to what Delicious and other tag-based systems use for improving searches.
@thlorenz we could allow them to suggest keywords.
However having to go through an "automatic PR" with the github integration and having the author vet it still makes a centralized process bottle necked by the author.
If I wanted to tag something as "not_tap_friendly_test_framework" then the author might not like me doing that. The tagging system would become less free and less distributed.
The main problem I'm worried with though is the fact that "tagging something" requires github integration and pull requests, meaning it breaks for non github projects and it breaks for users who don't have a github account.
The relevant research I cited is: http://works.bepress.com/cgi/viewcontent.cgi?article=1017&context=lucemia
Has the idea of having a preset number of topics to tag with for consistency been poopoo-ed? Thinking of something like pypi as an example. As a less-experienced package publisher, the number of topics can be intimidating but at least gives users a base to start from.
I know as a newer node user, finding packages that work with the flavor of node I'm working in has been difficult(such as things that play well with browserify).
You might not want me, as a user, tagging others' packages. I do dig the idea of peer review being the means to maintain the quality of the tag, however. That would minimize my lack of exposure damaging the tag quality.
This would be way easier to implement if you built it as a side project. A separate module not a pull request to npm - although if successful it could be added to npm by default, like npm is now part of the default node installation.
I wish that npm star (and for that matter, github star) incorporated a short message, how do you know if a "star" means "heh, cool idea. keep at it kid" or does it mean "we use this in production at . rock solid"?
Open source is a social thing, but it could be more social.
👍 as a seperate module / system to prototype it.
Although a seperate system would need a bit of "advertisement" to pick up some tagging momentum.
@hackygolucky I feel that tags serve the same purposes as categories, but yeah, the difficult part with tags is that they're too open-ended, which is one of the reasons that suggesting keywords for module authors/users lessens this pain a bit. Could be done by tracking how a user got to a module (which keywords did they use, which modules did they visit before) :(
Maybe it's ok if tags are open ended. The tags that get used on a lot modules or get a lot of votes will be considered the "high quality" tags.
@Raynos that is easy, just get stackvm using it - a bit of twitter, etc...
also it's way easier to get crazy indexes working with leveldb than with couch
Of course, there is no easy solution to indexing anything as freeform a a node module.
user defined tags could definately be helpful though. But maybe most helpful is search + weighting by what people actually click on (as @robotlolita describes) although I also like @substack's idea of a irc bot that posts messages when some one types in a search query, and the results are what people reply to it (combined with cached responses from previous queries)
This is going to be handled by ecosystems and collections on the new npm user experience. As such, it's probably not going to come to the npm CLI. We've added stars and that's been a decidedly mixed bag. But keep an eye on newww, and if it looks like it would be useful to get at that stuff via the CLI, we can probably add some endpoints and client code to manipulate them.