Sanctioned way to mark packages as browser-(in)compatible

[domenic]

I am (finally) logging a series of feature requests I discussed with @addyosmani at JSConf EU, as to what would make npm the best JavaScript package manager, and help it crush all competition.

The idea would be that we use and endorse some package.json field as "the way" to mark a package as browser-ready. Then, we would provide dedicated interfaces to searching npmjs.org for browser-specific packages.

Some of these ideas have been prototyped by @substack in his http://browserify.org/search, although they are a bit testling-specific. (Also, they don't seem to work anymore?)

I think we'd want to avoid drawing a bright line between browser-ready and server-specific components, i.e. not segregate the entire npmjs.org browsing experience, but instead just have a "rank browser packages higher in this search" type of checkbox.

There are various concrete possibilities here: @defunctzombie's package.json browser field (maybe encourage browser: true and browser: false, in addition to its use for mapping); a specific tag; or even the engines field.

It's possible to scope-creep this into various other levels of support (e.g., which browsers do you support), but that's probably not a good idea.

[defunctzombie]

I like the browser field approach. Taking it a step further you could
actually say that only packages with a browser field are browser capable to
avoid the false value requirement. Tho I also think that more modules can
work in a browser setting as long as the underlying APIs are shimmed.

If anything it kinda goes the other way around, some modules are browser
only because of their use of browser globals.
On Dec 15, 2013 7:16 PM, "Domenic Denicola" notifications@github.com
wrote:

I am (finally) logging a series of feature requests I discussed with
@addyosmani https://github.com/addyosmani at JSConf EU, as to what
would make npm the best JavaScript package manager, and help it crush all
competition.

The idea would be that we use and endorse some package.json field as
"the way" to mark a package as browser-ready. Then, we would provide
dedicated interfaces to searching npmjs.org for browser-specific packages.

Some of these ideas have been prototyped by @substackhttps://github.com/substackin his
http://browserify.org/search, although they are a bit testling-specific.
(Also, they don't seem to work anymore?)

I think we'd want to avoid drawing a bright line between browser-ready and
server-specific components, i.e. not segregate the entire npmjs.orgbrowsing experience, but instead just have a "rank browser packages higher
in this search" type of checkbox.

There are various concrete possibilities here: @defunctzombiehttps://github.com/defunctzombie's
package.json browser field (maybe encourage browser: true and browser:
false, in addition to its use for mapping); a specific tag; or even the
engines field.

It's possible to scope-creep this into various other levels of support
(e.g., which browsers do you support), but that's probably not a good idea.

—
Reply to this email directly or view it on GitHubhttps://github.com/isaacs/npm/issues/4321
.

[domenic]

If anything it kinda goes the other way around, some modules are browser only because of their use of browser globals.

This is interesting; it suggests some kind of tri-value situation of "browser only", "server only", and "both".

---

Also, I found another note I had scratched down: we should ideally have some way of crowdsourcing the does-it-work-in-browserness. That is, something on npmjs.org that you click to say "yes, this works in the browser," so that even if maintainers are not willing to add yet another package.json field, you can still browse and find that package as something many people have had success with in the browser.

[lfac-pt]

I think that an approach that could express "browser only", "server only", and "both" would be better than just browser=true and browser=false. It is important to know whenever the code can run on the server.

This could be taken a step further to allow for other js engines like rhino, nashorn and even browser specific versions:

{ "engines" : { "node" : ">=0.10.3 <0.12" , "chrome": "31", "rhino": "1.7"} }

Although, that would probably be complicating it too much.

If no browser support field is found, the package is considered for-node-only? I think it should.

[ryanflorence]

yes to engines, so then tools can do interesting things around browser support, like when you install a module that doesn't support the browser's your app does.

[briandipalma]

Yes to both the tri-state and the engines meta-data, although the engines data would only be useful as an excluder.
The lack of the engines value means nothing while it's presence can tell you that this package may not work in your JS runtime if your version is less than the value in the package.json.

[kristoferjoseph]

Could this just be handled by the tags property that already exists? Add a browser tag if it is specific to browser use, a server tag if it is specific to server use?

[addyosmani]

I was also initially thinking a tagging based implementation for packages could offer a cheap solution to this problem. A simpler take could be just having a 'browser' tag/keyword for packages explicitly for the browser and assume by default everything else is for the server.

[kristoferjoseph]

@addyosmani That makes a lot of sense. This would mean that existing packages would need to add this if they are browser compatible. Seems like a great way to drum up pull requests and help out!

[lfac-pt]

There is no tag or tags property that I know of (except for git tags). Do you mean the keywords property?

Anyway, that does seem like a good and easy to implement idea. Additionally we could have server (or server-only) and server-browser keywords. As I said before, I think it is important to be able to distinguish server only from packages that can work in both environments.

[addyosmani]

In the above we were indeed referring to the keywords property rather than tags :) It offers a relatively straight-forward path to making this a reality (I think). One benefit of introducing a single keyword to classify something as just being for the browser is that the community could help us bootstrap quite a lot of the 'tagging' effort though PRs.

Two keywords (browser, server) could also work but I'm curious what people think about that idea.

I also wonder what updates could be proposed for npmjs.org search to better enable finding browser-only packages. Could just re-use existing search with pre-populated search term for the keyword.

[bclinkinbeard]

+1 to using the keywords field. Assuming server compatibility seems like a safe bet, so maybe just browser and browser-only are sufficient? FWIW, including browserify-transform in keywords for Browserify transform packages is now recommended, so this sort of approach has some existing proponents.

[domenic]

I don't think server compat is a given; I've published many packages that depend on the presence of window and document globals.

[bclinkinbeard]

Right, so in those cases you would add browser-only. Though sometimes even those modules can be run in Node with something like jsdom.

My main point though was that server and server-only seem unnecessary, as those would likely be the assumption in the absence of info/keywords stating otherwise.

[kristoferjoseph]

We could add a test to the system that is run on publish that determines the compatibility of a module. This seems to me to be the lowest friction option for module publishers.

[ryanflorence]

It bums me out thinking of using npm as a browser package manager and having my use-case be "second-class", but browser and browser-only would work well.

[kristoferjoseph]

Otherwise we could write a module that users can run locally that will determine compatibility and give feedback as to why a module might not be compatible for the server or browser. Was considering a similar test to determine if a module's output is suitable for a piped transform ( to help with the browserify ecosystem ).

[domenic]

It bums me out thinking of using npm as a browser package manager and having my use-case be "second-class", but browser and browser-only would work well.

I agree. I think positioning npm as server-first is wrongheaded.

I would think a more realistic approach is that two keywords like browser and server can be added, which signify "I've tested that this works on the browser" or "I've tested that this works on the server." Otherwise, you just don't know---it might work on one, or the other, or both, or even neither.

[kristoferjoseph]

@rpflorence let's stay positive :). There are no second class modules. This is only for people like @addyosmani who would like it to be easier to discover all the amazing browser compatible modules.

[ryanflorence]

@kristoferjoseph I am "people like @addyosmani" and there are second class modules on npm: anything that isn't designed for node and doesn't use cjs. npm, after all, installs things to a folder named "node_modules".

I would think a more realistic approach is that two keywords like browser and server

^ I support this whole-heartedly

[bclinkinbeard]

I certainly wasn't implying browser modules are "second class"; heck, it's basically all I write/use. I was just trying to imagine a system that would minimize the amount of work for existing modules. We're not really expecting/proposing that the 55,000 existing modules will be updated, are we? Right now browser modules are the minority, and this proposal is largely about how to change that fact, so my thinking was that the extra "burden" would be put on the modules that are mostly yet to come.

That being said, I have nothing against browser and server, other than the fact it means a larger portion of the repository will be in "non-compliance" from day one.

[domenic]

Right now browser modules are the minority

I don't think this is accurate, or at least, not to the degree you're imagining. With browserify's shim layer in place a large number of existing modules will "just work" in both places. Whether they've been tested extensively enough to feel confident identifying as "browser" or "server" or both is unclear, but that's the default state for everyone.

[bclinkinbeard]

Good point. So browser and server then? :)

Relatedly, is there a way to npm publish just the metadata for a package without using --force? If there isn't now, maybe that should be added to make it easier for things like keywords and READMEs to be updated?

[addyosmani]

I would be more than happy with browser and server keywords with the majority of the registry being considered non-compli from the start. The intention of my initial suggestion around assuming server-first was primarily aimed at minimizing the investment effort required by the community. If it makes more sense to explicitly go for two keywords, by all means..let's do it :)

[kristoferjoseph]

+1 on minimizing friction for existing modules.

[ryanflorence]

@addyosmani I think its reassuring to folks like me wondering how seriously npm is taking the browser use-case, rather than the often heard "you can put anything on npm! just do whatever you want with package.json and tags"

[addyosmani]

+1. It's incredibly encouraging that the browser use-case is being seriously considered. It's fantastic we're able to have this discussion at a practical level beyond "anything goes" :)

[robdodson]

Would this change the way that node_modules gets built? Right now if you have 3 browser modules that all depend on jquery, they'll each get their own sub node_modules folder with jquery in it.

[timoxley]

@robdodson not if you use peerDependencies

edit: I'm not sure how/if browserify handles them though

[briandipalma]

@timoxley I followed your advice with peerDependencies for the customelements packages I published to npm but I've been told not to use peerDependencies for such a use case and to do a npm dedupe instead.

It's clear that client side developers would prefer an automatic npm dedupe approach which you can do with the postinstall script value. Or even a flat dependency tree, it's one of the main reasons bower is choosen over npm.

I've already asked for a new name for node_modules in issue #4524 and it seems to have gained no interest. If npm wants to be best of the best then it should make a clear statement that it's not wedded to node and is aiming to be a generic package manager for any use and platform.

[jb55]

Making keyword search + query more intuitive would solve most of these issues. Most of the time I want to do a query like:

search npm for packages with keyword browser and includes router in name or keywords

Something like npm search -k browser router. Would this not solve this issue and future issues?

Also using the search on npmjs.org is a bit unintuitive, there's no way to filter by keywords after or during your initial query (afaik)

Keywords/tags solve everything, we just need better querying capabilities.

[kirbysayshi]

@domenic what exactly does it mean for a module to work in a browser? To me there are currently three types of packages I commonly see on npm:

A. sets a global if you <script src="node_modules/package/file.js"></script>
B. requireable and is browserifyable / [insert your bundler here]
C. requireable but depends on a node api that is not browserifyable

Does adding the browser keyword imply A and B, or just B for this initiative? It sure is more useful when everything is requireable!

To be clear, I am not an advocate of A on npm, but I have run into them more often than I would have expected. Therefore, I vote B... but maybe that's what's completely implied by this gh issue in the first place?

[medikoo]

I think we should have more environment agnostic approach and design for API's and not for specific engines. Doing browser/server distinction, we're making same mistake as in old days where instead of doing feature detection we ported specific functionalities just to e.g. Firefox browser

It's not difficult today to bring browser API's to Node.js (I use DOM on server myself), or Node.js API's to certain browser's.

I host many cross env packages (some depend strictly on API's provided natively only in browsers) and don't see a value in browser and/or server tags, they just cement the wall, we don't need.

[addyosmani]

The distinction allows us to more easily identify packages primarily
containing frontend assets (HTML/CSS/ maybe some JS etc). Note that I don't
strictly call out JS as the main content. If that's all a package could
contain, browserify solves that today.

My use cases:
CSS libraries
Web Components
Themes etc.
On 22 Apr 2014 08:58, "Mariusz Nowak" notifications@github.com wrote:

I think we should have more environment agnostic approach and design for
API's and not for specific engines. Doing browser/server distinction,
we're making same mistake as in old days where instead of doing feature
detection we ported specific functionalities just to e.g. Firefox browser

It's not difficult today to bring browser API's to Node.js (I use DOM on
server myself), or Node.js API's to certain browser's.

I host many cross env packages (some depend strictly on API's provided
natively only in browsers) and don't see a value in browser and/or
server tags, they just cement the wall, we don't need.

—
Reply to this email directly or view it on GitHubhttps://github.com//issues/4321#issuecomment-41013223
.

[medikoo]

@addyosmani still I think it goes down to API's.

CSS is purely visual, so logically it's mainly about engines with visual interface. Naturally we think just browsers (front-end), but what if we want to generate PDF's out of DOM on server side, and want to use existing web components that were labelled browser-only (?)

On my side I use web components, which are structured so CSS and DOM (js) modules are separate files. In engine with visual interface (browser) I include both, On server-side, if I just want to generate DOM, I use just DOM module, if CSS module is required, it has no effect it's ignored.

Of course Node by default won't work with CSS files as by default there are no interfaces that can deal with that, but it doesn't mean it's server: false, same as we can use CJS bundler (like Browserify) to port CSS modules to browser, same way we can provide CSS modules on server side, if we have prepared interfaces for that

[domenic]

@kirbysayshi it means the author intends their package to be used by developers who are building web applications, as well as/instead of developers who are building Node.js applications.

[kirbysayshi]

@domenic but does that include a module transport for JS?

When evaluating a new package for client-side use, I spend the most time looking through its modules determining if:

• it can be easily requireed in a browser environment (a.k.a. doesn't use globals)
• it uses node-specific stuff

If adding a browser tag just means it uses window, document, and stuff, then what does that really say about the package? It still requires the consumer to go through and figure out if/how they can actually consume the modules, as @medikoo mentions.

To me, browser means "if the package contains JS, it's CJS bundler-compatible".

[domenic]

@kirbysayshi I've intentionally given a minimal definition, and I think it's the one we should stick to since it's most broadly useful to client-side developers looking for a package manager, even if they haven't bought into any specific toolchain such as browserify. We can use further tags or mechanisms to narrow it down if you like specific toolchains.

[WebReflection]

FWIW my 2 cents

the quantum browser property name for package.json seems the most straight forward and easy to implement in a backward compatible way.

• browser: false or "false" to mark it as server side only, as default since npm means Node Package Manager and I expect modules to work with node first
• browser: "only" to mark as not compatible with node.js even if in npm
• browser: "too", "OK", 2, "as well" … anything else that is not falsy or the string "only" to explicitly mark it as suitable for both browsers and server

the side problem

there are modules that might be named same way and offer different files if the env is either the server, node.js, or the browser. In such case I'd expect the main script specified in package.json to be specific too in case the browser property is "as well"

The easiest way that I can think of right now is to allow main property as an object or string, where if object will point to {browser: "file/name.js"} and if not {default: "server/name.js"}

Last, but not least, thanks for exploring any possibility and considering this in first place.

[timoxley]

@WebReflection note that "browser": "file/name.js" is exactly how browserify currently uses of the browser package.json property to specify browser-specific overrides: https://github.com/substack/node-browserify#browser-field

[caridy]

This discussion is aligned with what some people call "isomorphic" code (code that can run in multiple runtimes), and for that, we have been using the term affinity to specify one, few or all supported runtime. Javascript keeps growing, and it is very likely that new runtimes become popular in the future, in which case, having something that allow us to expand to other runtimes will be beneficial, and on top of that, it is a term that is not polluted at all: https://www.npmjs.org/browse/keyword/affinity

[afeld]

While I'd love to see NPM get more usage for the front-end, marking packages as browser (in)compatible requires a lot of buy-in from the community, and is very error-prone. Just to think outside of the box: couldn't (most of?) this be achieved through static analysis, with better accuracy? Check if the code in the package makes any calls to browser-only or server-only APIs (that can't be swapped out by things like Browserify) itself or through its dependencies, then presenting the platforms it supports. It could even go as far as generating a compatibility table, like what you'd see on http://caniuse.com. Added bonus: support isn't necessarily required in NPM, or even the NPM site. Could be a standalone thing.

[jokeyrhyme]

I like the idea of dangling more information off the engines property. However, front-end developers have been trained for years to stop sniffing the user agent, and instead use feature-detection. So, how about using dom and html to hint that the code requires the DOM? Similarly, how about having ecmascript?

For example, my package uses Canvas, Array#forEach and document.addEventListener, so my package.json reads:

{
  "engines": {
    "dom": ">=2",
    "ecmascript": ">=5",
    "html": ">=5"
  }
}

This way, when my package is included in a project that specifies dom: 1 and html: 4, the build tools for that project might be able to automatically suggest the necessary poly-fills (e.g. html5shiv and es5-shim, etc).

[mgol]

HTML isn't versioned so that approach wouldn't work in most scenarios.

Node & a browser are really two very distinct environments; all the
browsers are much more similar than any of them and Node. That's why here
it seems OK to target just Node or just a browser.

[addyosmani]

+9001 once again for just targeting Node and the browser. These are clear
enough environments for the majority of package needs to be sufficiently
met.
On 16 Sep 2014 23:02, "Michał Gołębiowski" notifications@github.com wrote:

HTML isn't versioned so that approach wouldn't work in most scenarios.

Node & a browser are really two very distinct environments; all the
browsers are much more similar than any of them and Node. That's why here
it seems OK to target just Node or just a browser.

—
Reply to this email directly or view it on GitHub
#4321 (comment).

[medikoo]

@addyosmani @mzgol many of what what we assume as browser API's have node dedicated implementations in user land. There are also many packages that stand on those API's but are meant to be used in either environment. Why they should be marked as targeted just for browser?

What about other, not as popular environments like e.. Adobe Photoshop? Should they use browser or node dedicated packages?

It's clearly about API's that are provided in given environment, and developers should write utilities for API's not environments, we've already learned that lesson when we developed for different browsers, why to repeat the history?

[jehoshua02]

I'm trying to figure out what to put in my package.json. What is the TL;DR?

[qfox]

👍 for { engines: { browser: es6 } }

[steinerj]

Sorry, for digging out this old issue, but I still believe the distinction "browser" vs "node" is not sufficient. There's loads of fairly popular platforms which are V8 based and allow for including npm packages, but do not offer node APIs. Examples: https://www.nativescript.org or https://www.arangodb.com

Both of which provide a facade for some typical node modules such as 'http', but not for e.g. filesystem.

So I like the "affinity" idea. Currently weeding out packages that have a dependency on a very low level is an absolute pain and mostly based on intuition, grepping and trial & error.

[othiym23]

Quite a bit has happened over the last two years! It feels uncontroversial to now say that npm is a repository for tools for Node, the browser, and many other environments, and I think the number of packages on the registry that only work with Node is now in the minority. Also, the community has done a lot of work to improve the experience and workflows of non-Node-based packages. This proposal points to a few things we’d like npm to do better:

• simplify discovery of packages intended for specific purposes
• ease the process of assembling and working with applications based on non-Node deployment environments
• make it easier to visualize and discover how front-end tools are meant to be used (see also front-end issues)

These are actually pretty basic (and interesting!) product areas for a package manager like npm to handle. They’re much larger than the scope of a single issue! As such, while this is a really valuable discussion and source of ideas, it’s been outstripped by reality a bit: ”browser” already exists in the ecosystem and is heavily used (thanks, browserify!), and ”engines” is still in the process of becoming whatever it’s going to be when it grows up. So I’m going to close this issue! Thanks to all for the very valuable discussion, and I’d expect to see more happening around this as the team continues to work on improving npm’s support for front-end (and other non-Node) workflows.

[jcrben]

@othiym23 it's great to hear that it's on the radar, but at the end of the day we don't seem to have an official sanctioned way to record this metadata. It might also be helpful to link to a place we can watch for future updates on this specific topic.