# Sanctioned way to mark packages as browser-(in)compatible #4321

Closed
opened this Issue Dec 16, 2013 · 62 comments

### 24 participants

npm member

I am (finally) logging a series of feature requests I discussed with @addyosmani at JSConf EU, as to what would make npm the best JavaScript package manager, and help it crush all competition.

The idea would be that we use and endorse some package.json field as "the way" to mark a package as browser-ready. Then, we would provide dedicated interfaces to searching npmjs.org for browser-specific packages.

Some of these ideas have been prototyped by @substack in his http://browserify.org/search, although they are a bit testling-specific. (Also, they don't seem to work anymore?)

I think we'd want to avoid drawing a bright line between browser-ready and server-specific components, i.e. not segregate the entire npmjs.org browsing experience, but instead just have a "rank browser packages higher in this search" type of checkbox.

There are various concrete possibilities here: @defunctzombie's package.json browser field (maybe encourage browser: true and browser: false, in addition to its use for mapping); a specific tag; or even the engines field.

It's possible to scope-creep this into various other levels of support (e.g., which browsers do you support), but that's probably not a good idea.

npm member

If anything it kinda goes the other way around, some modules are browser only because of their use of browser globals.

This is interesting; it suggests some kind of tri-value situation of "browser only", "server only", and "both".

Also, I found another note I had scratched down: we should ideally have some way of crowdsourcing the does-it-work-in-browserness. That is, something on npmjs.org that you click to say "yes, this works in the browser," so that even if maintainers are not willing to add yet another package.json field, you can still browse and find that package as something many people have had success with in the browser.

I think that an approach that could express "browser only", "server only", and "both" would be better than just browser=true and browser=false. It is important to know whenever the code can run on the server.

This could be taken a step further to allow for other js engines like rhino, nashorn and even browser specific versions:

{ "engines" : { "node" : ">=0.10.3 <0.12" , "chrome": "31", "rhino": "1.7"} }


Although, that would probably be complicating it too much.

If no browser support field is found, the package is considered for-node-only? I think it should.

yes to engines, so then tools can do interesting things around browser support, like when you install a module that doesn't support the browser's your app does.

referenced this issue in sammyt/nap Jan 14, 2014
Merged

#### add a requirejs example of using nap #5

Yes to both the tri-state and the engines meta-data, although the engines data would only be useful as an excluder.
The lack of the engines value means nothing while it's presence can tell you that this package may not work in your JS runtime if your version is less than the value in the package.json.

Could this just be handled by the tags property that already exists? Add a browser tag if it is specific to browser use, a server tag if it is specific to server use?

@addyosmani That makes a lot of sense. This would mean that existing packages would need to add this if they are browser compatible. Seems like a great way to drum up pull requests and help out!

There is no tag or tags property that I know of (except for git tags). Do you mean the keywords property?

Anyway, that does seem like a good and easy to implement idea. Additionally we could have server (or server-only) and server-browser keywords. As I said before, I think it is important to be able to distinguish server only from packages that can work in both environments.

+1 to using the keywords field. Assuming server compatibility seems like a safe bet, so maybe just browser and browser-only are sufficient? FWIW, including browserify-transform in keywords for Browserify transform packages is now recommended, so this sort of approach has some existing proponents.

npm member

I don't think server compat is a given; I've published many packages that depend on the presence of window and document globals.

Right, so in those cases you would add browser-only. Though sometimes even those modules can be run in Node with something like jsdom.

My main point though was that server and server-only seem unnecessary, as those would likely be the assumption in the absence of info/keywords stating otherwise.

We could add a test to the system that is run on publish that determines the compatibility of a module. This seems to me to be the lowest friction option for module publishers.

It bums me out thinking of using npm as a browser package manager and having my use-case be "second-class", but browser and browser-only would work well.

Otherwise we could write a module that users can run locally that will determine compatibility and give feedback as to why a module might not be compatible for the server or browser. Was considering a similar test to determine if a module's output is suitable for a piped transform ( to help with the browserify ecosystem ).

npm member

It bums me out thinking of using npm as a browser package manager and having my use-case be "second-class", but browser and browser-only would work well.

I agree. I think positioning npm as server-first is wrongheaded.

I would think a more realistic approach is that two keywords like browser and server can be added, which signify "I've tested that this works on the browser" or "I've tested that this works on the server." Otherwise, you just don't know---it might work on one, or the other, or both, or even neither.

@rpflorence let's stay positive :). There are no second class modules. This is only for people like @addyosmani who would like it to be easier to discover all the amazing browser compatible modules.

@kristoferjoseph I am "people like @addyosmani" and there are second class modules on npm: anything that isn't designed for node and doesn't use cjs. npm, after all, installs things to a folder named "node_modules".

I would think a more realistic approach is that two keywords like browser and server

^ I support this whole-heartedly

I certainly wasn't implying browser modules are "second class"; heck, it's basically all I write/use. I was just trying to imagine a system that would minimize the amount of work for existing modules. We're not really expecting/proposing that the 55,000 existing modules will be updated, are we? Right now browser modules are the minority, and this proposal is largely about how to change that fact, so my thinking was that the extra "burden" would be put on the modules that are mostly yet to come.

That being said, I have nothing against browser and server, other than the fact it means a larger portion of the repository will be in "non-compliance" from day one.

npm member

Right now browser modules are the minority

I don't think this is accurate, or at least, not to the degree you're imagining. With browserify's shim layer in place a large number of existing modules will "just work" in both places. Whether they've been tested extensively enough to feel confident identifying as "browser" or "server" or both is unclear, but that's the default state for everyone.

Good point. So browser and server then? :)

Relatedly, is there a way to npm publish just the metadata for a package without using --force? If there isn't now, maybe that should be added to make it easier for things like keywords and READMEs to be updated?

I would be more than happy with browser and server keywords with the majority of the registry being considered non-compli from the start. The intention of my initial suggestion around assuming server-first was primarily aimed at minimizing the investment effort required by the community. If it makes more sense to explicitly go for two keywords, by all means..let's do it :)

+1 on minimizing friction for existing modules.

@addyosmani I think its reassuring to folks like me wondering how seriously npm is taking the browser use-case, rather than the often heard "you can put anything on npm! just do whatever you want with package.json and tags"

+1. It's incredibly encouraging that the browser use-case is being seriously considered. It's fantastic we're able to have this discussion at a practical level beyond "anything goes" :)

Would this change the way that node_modules gets built? Right now if you have 3 browser modules that all depend on jquery, they'll each get their own sub node_modules folder with jquery in it.

npm member

@robdodson not if you use peerDependencies

edit: I'm not sure how/if browserify handles them though

@timoxley I followed your advice with peerDependencies for the customelements packages I published to npm but I've been told not to use peerDependencies for such a use case and to do a npm dedupe instead.

It's clear that client side developers would prefer an automatic npm dedupe approach which you can do with the postinstall script value. Or even a flat dependency tree, it's one of the main reasons bower is choosen over npm.

I've already asked for a new name for node_modules in issue #4524 and it seems to have gained no interest. If npm wants to be best of the best then it should make a clear statement that it's not wedded to node and is aiming to be a generic package manager for any use and platform.

The peerDependencies discussion probably deserves its own issue if it's one we're going to have.

Regarding the original purpose of this issue though, it sounds like we've reached consensus that packages compatible for use in the browser should include a browser keyword, packages compatible for use on the server should include a server keyword, and packages compatible with both environments should include both keywords.

Assuming everyone is in agreement, how do we "make it so"? Simply go forth and spread the word and update our own packages, or is there more to it?

npm member

Perhaps a "is this a server or browser package? (server)" question could be put into the default config for npm init

referenced this issue Jan 28, 2014
Closed

#### There should be a way to install browser packages as siblings #4561

I started a new thread at #4561 to continue the discussion. Could someone label it please?

Perhaps a "is this a server or browser package? (server)" question could be put into the default config for npm init

Seems a good approach.

npm member

@substack pointed out that https://www.npmjs.org/browse/keyword/browser is pretty full. So I think we should settle on that. I will open another issue for npm init upgrades.

Ember is interested in buying into such an ecosystem and I'll recommend them using the browser keyword.

npm member

I will leave this bug open for discussion of other ways we can make this more official, e.g. npm-www filters, ... what else?

referenced this issue Apr 19, 2014
Closed

#### Add prompt for browser keyword to npm init #5106

npm member

I think that's a great idea. The npm inc. people are usually around in IRC more during weekdays, so I will be sure to bug them about such a possibility then.

I think Component(1) is a better comparison than Bower. Does npm have a position on how it plans to cater for browser UI components (or multiple asset types)?

npm member

@necolas npm just delivers whatever files are in the bundle, it's up to whatever is consuming the files to handle the files. There's nothing for npm to cater for.

npm member

@addyosmani it's a bit different story for npm, since everyone is already on npm (or if they aren't, people can feel free to publish their own forks as needed). There's no need to fork to a different org just to add a bower.json type thing.

That said, I think some kind of way to add "community sourced" keywords, probably through npm-www, would be generally valuable. It's a harder engineering challenge though (you probably want upvotes, downvotes, thresholds, etc.)

referenced this issue in jquery/jquery-release Apr 21, 2014
Closed

#### Support keywords for npm publishing #63

Making keyword search + query more intuitive would solve most of these issues. Most of the time I want to do a query like:

search npm for packages with keyword browser and includes router in name or keywords

Something like npm search -k browser router. Would this not solve this issue and future issues?

Also using the search on npmjs.org is a bit unintuitive, there's no way to filter by keywords after or during your initial query (afaik)

Keywords/tags solve everything, we just need better querying capabilities.

@domenic what exactly does it mean for a module to work in a browser? To me there are currently three types of packages I commonly see on npm:

A. sets a global if you <script src="node_modules/package/file.js"></script>
B. requireable and is browserifyable / [insert your bundler here]
C. requireable but depends on a node api that is not browserifyable

Does adding the browser keyword imply A and B, or just B for this initiative? It sure is more useful when everything is requireable!

To be clear, I am not an advocate of A on npm, but I have run into them more often than I would have expected. Therefore, I vote B... but maybe that's what's completely implied by this gh issue in the first place?

I think we should have more environment agnostic approach and design for API's and not for specific engines. Doing browser/server distinction, we're making same mistake as in old days where instead of doing feature detection we ported specific functionalities just to e.g. Firefox browser

It's not difficult today to bring browser API's to Node.js (I use DOM on server myself), or Node.js API's to certain browser's.

I host many cross env packages (some depend strictly on API's provided natively only in browsers) and don't see a value in browser and/or server tags, they just cement the wall, we don't need.

@addyosmani still I think it goes down to API's.

CSS is purely visual, so logically it's mainly about engines with visual interface. Naturally we think just browsers (front-end), but what if we want to generate PDF's out of DOM on server side, and want to use existing web components that were labelled browser-only (?)

On my side I use web components, which are structured so CSS and DOM (js) modules are separate files. In engine with visual interface (browser) I include both, On server-side, if I just want to generate DOM, I use just DOM module, if CSS module is required, it has no effect it's ignored.

Of course Node by default won't work with CSS files as by default there are no interfaces that can deal with that, but it doesn't mean it's server: false, same as we can use CJS bundler (like Browserify) to port CSS modules to browser, same way we can provide CSS modules on server side, if we have prepared interfaces for that

npm member

@kirbysayshi it means the author intends their package to be used by developers who are building web applications, as well as/instead of developers who are building Node.js applications.

@domenic but does that include a module transport for JS?

When evaluating a new package for client-side use, I spend the most time looking through its modules determining if:

• it can be easily requireed in a browser environment (a.k.a. doesn't use globals)
• it uses node-specific stuff

If adding a browser tag just means it uses window, document, and stuff, then what does that really say about the package? It still requires the consumer to go through and figure out if/how they can actually consume the modules, as @medikoo mentions.

To me, browser means "if the package contains JS, it's CJS bundler-compatible".

npm member

@kirbysayshi I've intentionally given a minimal definition, and I think it's the one we should stick to since it's most broadly useful to client-side developers looking for a package manager, even if they haven't bought into any specific toolchain such as browserify. We can use further tags or mechanisms to narrow it down if you like specific toolchains.

FWIW my 2 cents

the quantum browser property name for package.json seems the most straight forward and easy to implement in a backward compatible way.

• browser: false or "false" to mark it as server side only, as default since npm means Node Package Manager and I expect modules to work with node first
• browser: "only" to mark as not compatible with node.js even if in npm
• browser: "too", "OK", 2, "as well" … anything else that is not falsy or the string "only" to explicitly mark it as suitable for both browsers and server

#### the side problem

there are modules that might be named same way and offer different files if the env is either the server, node.js, or the browser. In such case I'd expect the main script specified in package.json to be specific too in case the browser property is "as well"

The easiest way that I can think of right now is to allow main property as an object or string, where if object will point to {browser: "file/name.js"} and if not {default: "server/name.js"}

Last, but not least, thanks for exploring any possibility and considering this in first place.

npm member

@WebReflection note that "browser": "file/name.js" is exactly how browserify currently uses of the browser package.json property to specify browser-specific overrides: https://github.com/substack/node-browserify#browser-field

referenced this issue in medikoo/modules-webmake Apr 24, 2014
Closed

#### Bower package? #40

This discussion is aligned with what some people call "isomorphic" code (code that can run in multiple runtimes), and for that, we have been using the term affinity to specify one, few or all supported runtime. Javascript keeps growing, and it is very likely that new runtimes become popular in the future, in which case, having something that allow us to expand to other runtimes will be beneficial, and on top of that, it is a term that is not polluted at all: https://www.npmjs.org/browse/keyword/affinity

While I'd love to see NPM get more usage for the front-end, marking packages as browser (in)compatible requires a lot of buy-in from the community, and is very error-prone. Just to think outside of the box: couldn't (most of?) this be achieved through static analysis, with better accuracy? Check if the code in the package makes any calls to browser-only or server-only APIs (that can't be swapped out by things like Browserify) itself or through its dependencies, then presenting the platforms it supports. It could even go as far as generating a compatibility table, like what you'd see on http://caniuse.com. Added bonus: support isn't necessarily required in NPM, or even the NPM site. Could be a standalone thing.

referenced this issue in bootstrap-tagsinput/bootstrap-tagsinput Aug 1, 2014
Closed

#### Use bower for development #164

I like the idea of dangling more information off the engines property. However, front-end developers have been trained for years to stop sniffing the user agent, and instead use feature-detection. So, how about using dom and html to hint that the code requires the DOM? Similarly, how about having ecmascript?

For example, my package uses Canvas, Array#forEach and document.addEventListener, so my package.json reads:

{
"engines": {
"dom": ">=2",
"ecmascript": ">=5",
"html": ">=5"
}
}

This way, when my package is included in a project that specifies dom: 1 and html: 4, the build tools for that project might be able to automatically suggest the necessary poly-fills (e.g. html5shiv and es5-shim, etc).

@addyosmani @mzgol many of what what we assume as browser API's have node dedicated implementations in user land. There are also many packages that stand on those API's but are meant to be used in either environment. Why they should be marked as targeted just for browser?

What about other, not as popular environments like e.. Adobe Photoshop? Should they use browser or node dedicated packages?

It's clearly about API's that are provided in given environment, and developers should write utilities for API's not environments, we've already learned that lesson when we developed for different browsers, why to repeat the history?

I'm trying to figure out what to put in my package.json. What is the TL;DR?

👍 for { engines: { browser: es6 } }

referenced this issue in chaijs/chai-docs Nov 9, 2015
Closed

#### Grunt-like plugin listing #34

referenced this issue in mathjax/MathJax Dec 10, 2015
Closed

#### simplifying npm workflows #1328

added the front-end label Feb 3, 2016

Sorry, for digging out this old issue, but I still believe the distinction "browser" vs "node" is not sufficient. There's loads of fairly popular platforms which are V8 based and allow for including npm packages, but do not offer node APIs. Examples: https://www.nativescript.org or https://www.arangodb.com

Both of which provide a facade for some typical node modules such as 'http', but not for e.g. filesystem.

So I like the "affinity" idea. Currently weeding out packages that have a dependency on a very low level is an absolute pain and mostly based on intuition, grepping and trial & error.

Quite a bit has happened over the last two years! It feels uncontroversial to now say that npm is a repository for tools for Node, the browser, and many other environments, and I think the number of packages on the registry that only work with Node is now in the minority. Also, the community has done a lot of work to improve the experience and workflows of non-Node-based packages. This proposal points to a few things we’d like npm to do better:

• simplify discovery of packages intended for specific purposes
• ease the process of assembling and working with applications based on non-Node deployment environments
• make it easier to visualize and discover how front-end tools are meant to be used (see also front-end issues)

These are actually pretty basic (and interesting!) product areas for a package manager like npm to handle. They’re much larger than the scope of a single issue! As such, while this is a really valuable discussion and source of ideas, it’s been outstripped by reality a bit: ”browser” already exists in the ecosystem and is heavily used (thanks, browserify!), and ”engines” is still in the process of becoming whatever it’s going to be when it grows up. So I’m going to close this issue! Thanks to all for the very valuable discussion, and I’d expect to see more happening around this as the team continues to work on improving npm’s support for front-end (and other non-Node) workflows.

closed this Jun 2, 2016