Skip to content

HTTPS clone URL

Subversion checkout URL

You can clone with HTTPS or Subversion.

Download ZIP

Loading…

Sanctioned way to mark packages as browser-(in)compatible #4321

Open
domenic opened this Issue · 60 comments

21 participants

Domenic Denicola Roman Shtylman Luís Cardoso Ryan Florence Brian Di Palma Kristofer Joseph Addy Osmani Ben Clinkinbeard Rob Dodson Tim Oxley Nicolas Gallagher William Casarin Andrew Petersen Mariusz Nowak Andrea Giammarchi Caridy Patiño Aidan Feldman Ron Waldon Michał Gołębiowski Stoutie Alexej Yaroshevich
Domenic Denicola
Collaborator

I am (finally) logging a series of feature requests I discussed with @addyosmani at JSConf EU, as to what would make npm the best JavaScript package manager, and help it crush all competition.

The idea would be that we use and endorse some package.json field as "the way" to mark a package as browser-ready. Then, we would provide dedicated interfaces to searching npmjs.org for browser-specific packages.

Some of these ideas have been prototyped by @substack in his http://browserify.org/search, although they are a bit testling-specific. (Also, they don't seem to work anymore?)

I think we'd want to avoid drawing a bright line between browser-ready and server-specific components, i.e. not segregate the entire npmjs.org browsing experience, but instead just have a "rank browser packages higher in this search" type of checkbox.

There are various concrete possibilities here: @defunctzombie's package.json browser field (maybe encourage browser: true and browser: false, in addition to its use for mapping); a specific tag; or even the engines field.

It's possible to scope-creep this into various other levels of support (e.g., which browsers do you support), but that's probably not a good idea.

Roman Shtylman
Domenic Denicola
Collaborator

If anything it kinda goes the other way around, some modules are browser only because of their use of browser globals.

This is interesting; it suggests some kind of tri-value situation of "browser only", "server only", and "both".


Also, I found another note I had scratched down: we should ideally have some way of crowdsourcing the does-it-work-in-browserness. That is, something on npmjs.org that you click to say "yes, this works in the browser," so that even if maintainers are not willing to add yet another package.json field, you can still browse and find that package as something many people have had success with in the browser.

Luís Cardoso

I think that an approach that could express "browser only", "server only", and "both" would be better than just browser=true and browser=false. It is important to know whenever the code can run on the server.

This could be taken a step further to allow for other js engines like rhino, nashorn and even browser specific versions:

{ "engines" : { "node" : ">=0.10.3 <0.12" , "chrome": "31", "rhino": "1.7"} }

Although, that would probably be complicating it too much.

If no browser support field is found, the package is considered for-node-only? I think it should.

Ryan Florence

yes to engines, so then tools can do interesting things around browser support, like when you install a module that doesn't support the browser's your app does.

Marcus Stade mstade referenced this issue in sammyt/nap
Merged

add a requirejs example of using nap #5

Brian Di Palma

Yes to both the tri-state and the engines meta-data, although the engines data would only be useful as an excluder.
The lack of the engines value means nothing while it's presence can tell you that this package may not work in your JS runtime if your version is less than the value in the package.json.

Kristofer Joseph

Could this just be handled by the tags property that already exists? Add a browser tag if it is specific to browser use, a server tag if it is specific to server use?

Addy Osmani
Kristofer Joseph

@addyosmani That makes a lot of sense. This would mean that existing packages would need to add this if they are browser compatible. Seems like a great way to drum up pull requests and help out!

Luís Cardoso

There is no tag or tags property that I know of (except for git tags). Do you mean the keywords property?

Anyway, that does seem like a good and easy to implement idea. Additionally we could have server (or server-only) and server-browser keywords. As I said before, I think it is important to be able to distinguish server only from packages that can work in both environments.

Addy Osmani
Ben Clinkinbeard

+1 to using the keywords field. Assuming server compatibility seems like a safe bet, so maybe just browser and browser-only are sufficient? FWIW, including browserify-transform in keywords for Browserify transform packages is now recommended, so this sort of approach has some existing proponents.

Domenic Denicola
Collaborator

I don't think server compat is a given; I've published many packages that depend on the presence of window and document globals.

Ben Clinkinbeard

Right, so in those cases you would add browser-only. Though sometimes even those modules can be run in Node with something like jsdom.

My main point though was that server and server-only seem unnecessary, as those would likely be the assumption in the absence of info/keywords stating otherwise.

Kristofer Joseph

We could add a test to the system that is run on publish that determines the compatibility of a module. This seems to me to be the lowest friction option for module publishers.

Ryan Florence

It bums me out thinking of using npm as a browser package manager and having my use-case be "second-class", but browser and browser-only would work well.

Kristofer Joseph

Otherwise we could write a module that users can run locally that will determine compatibility and give feedback as to why a module might not be compatible for the server or browser. Was considering a similar test to determine if a module's output is suitable for a piped transform ( to help with the browserify ecosystem ).

Domenic Denicola
Collaborator

It bums me out thinking of using npm as a browser package manager and having my use-case be "second-class", but browser and browser-only would work well.

I agree. I think positioning npm as server-first is wrongheaded.

I would think a more realistic approach is that two keywords like browser and server can be added, which signify "I've tested that this works on the browser" or "I've tested that this works on the server." Otherwise, you just don't know---it might work on one, or the other, or both, or even neither.

Kristofer Joseph

@rpflorence let's stay positive :). There are no second class modules. This is only for people like @addyosmani who would like it to be easier to discover all the amazing browser compatible modules.

Ryan Florence

@kristoferjoseph I am "people like @addyosmani" and there are second class modules on npm: anything that isn't designed for node and doesn't use cjs. npm, after all, installs things to a folder named "node_modules".

I would think a more realistic approach is that two keywords like browser and server

^ I support this whole-heartedly

Ben Clinkinbeard

I certainly wasn't implying browser modules are "second class"; heck, it's basically all I write/use. I was just trying to imagine a system that would minimize the amount of work for existing modules. We're not really expecting/proposing that the 55,000 existing modules will be updated, are we? Right now browser modules are the minority, and this proposal is largely about how to change that fact, so my thinking was that the extra "burden" would be put on the modules that are mostly yet to come.

That being said, I have nothing against browser and server, other than the fact it means a larger portion of the repository will be in "non-compliance" from day one.

Domenic Denicola
Collaborator

Right now browser modules are the minority

I don't think this is accurate, or at least, not to the degree you're imagining. With browserify's shim layer in place a large number of existing modules will "just work" in both places. Whether they've been tested extensively enough to feel confident identifying as "browser" or "server" or both is unclear, but that's the default state for everyone.

Ben Clinkinbeard

Good point. So browser and server then? :)

Relatedly, is there a way to npm publish just the metadata for a package without using --force? If there isn't now, maybe that should be added to make it easier for things like keywords and READMEs to be updated?

Addy Osmani

I would be more than happy with browser and server keywords with the majority of the registry being considered non-compli from the start. The intention of my initial suggestion around assuming server-first was primarily aimed at minimizing the investment effort required by the community. If it makes more sense to explicitly go for two keywords, by all means..let's do it :)

Kristofer Joseph

+1 on minimizing friction for existing modules.

Ryan Florence

@addyosmani I think its reassuring to folks like me wondering how seriously npm is taking the browser use-case, rather than the often heard "you can put anything on npm! just do whatever you want with package.json and tags"

Addy Osmani

+1. It's incredibly encouraging that the browser use-case is being seriously considered. It's fantastic we're able to have this discussion at a practical level beyond "anything goes" :)

Rob Dodson

Would this change the way that node_modules gets built? Right now if you have 3 browser modules that all depend on jquery, they'll each get their own sub node_modules folder with jquery in it.

Tim Oxley
Collaborator

@robdodson not if you use peerDependencies

edit: I'm not sure how/if browserify handles them though

Brian Di Palma

@timoxley I followed your advice with peerDependencies for the customelements packages I published to npm but I've been told not to use peerDependencies for such a use case and to do a npm dedupe instead.

It's clear that client side developers would prefer an automatic npm dedupe approach which you can do with the postinstall script value. Or even a flat dependency tree, it's one of the main reasons bower is choosen over npm.

I've already asked for a new name for node_modules in issue #4524 and it seems to have gained no interest. If npm wants to be best of the best then it should make a clear statement that it's not wedded to node and is aiming to be a generic package manager for any use and platform.

Ben Clinkinbeard

The peerDependencies discussion probably deserves its own issue if it's one we're going to have.

Regarding the original purpose of this issue though, it sounds like we've reached consensus that packages compatible for use in the browser should include a browser keyword, packages compatible for use on the server should include a server keyword, and packages compatible with both environments should include both keywords.

Assuming everyone is in agreement, how do we "make it so"? Simply go forth and spread the word and update our own packages, or is there more to it?

Roman Shtylman
Tim Oxley
Collaborator

Perhaps a "is this a server or browser package? (server)" question could be put into the default config for npm init

Rob Dodson

I started a new thread at #4561 to continue the discussion. Could someone label it please?

Brian Di Palma

Perhaps a "is this a server or browser package? (server)" question could be put into the default config for npm init

Seems a good approach.

Domenic Denicola
Collaborator

@substack pointed out that https://www.npmjs.org/browse/keyword/browser is pretty full. So I think we should settle on that. I will open another issue for npm init upgrades.

Ember is interested in buying into such an ecosystem and I'll recommend them using the browser keyword.

Domenic Denicola
Collaborator

I will leave this bug open for discussion of other ways we can make this more official, e.g. npm-www filters, ... what else?

Domenic Denicola domenic added the ember-asks label
Addy Osmani
Domenic Denicola
Collaborator

I think that's a great idea. The npm inc. people are usually around in IRC more during weekdays, so I will be sure to bug them about such a possibility then.

Nicolas Gallagher

I think Component(1) is a better comparison than Bower. Does npm have a position on how it plans to cater for browser UI components (or multiple asset types)?

Tim Oxley
Collaborator

@necolas npm just delivers whatever files are in the bundle, it's up to whatever is consuming the files to handle the files. There's nothing for npm to cater for.

Addy Osmani
Domenic Denicola
Collaborator

@addyosmani it's a bit different story for npm, since everyone is already on npm (or if they aren't, people can feel free to publish their own forks as needed). There's no need to fork to a different org just to add a bower.json type thing.

That said, I think some kind of way to add "community sourced" keywords, probably through npm-www, would be generally valuable. It's a harder engineering challenge though (you probably want upvotes, downvotes, thresholds, etc.)

Dave Methvin dmethvin referenced this issue in jquery/jquery-release
Closed

Support keywords for npm publishing #63

William Casarin

Making keyword search + query more intuitive would solve most of these issues. Most of the time I want to do a query like:

search npm for packages with keyword browser and includes router in name or keywords

Something like npm search -k browser router. Would this not solve this issue and future issues?

Also using the search on npmjs.org is a bit unintuitive, there's no way to filter by keywords after or during your initial query (afaik)

Keywords/tags solve everything, we just need better querying capabilities.

Andrew Petersen

@domenic what exactly does it mean for a module to work in a browser? To me there are currently three types of packages I commonly see on npm:

A. sets a global if you <script src="node_modules/package/file.js"></script>
B. requireable and is browserifyable / [insert your bundler here]
C. requireable but depends on a node api that is not browserifyable

Does adding the browser keyword imply A and B, or just B for this initiative? It sure is more useful when everything is requireable!

To be clear, I am not an advocate of A on npm, but I have run into them more often than I would have expected. Therefore, I vote B... but maybe that's what's completely implied by this gh issue in the first place?

Mariusz Nowak

I think we should have more environment agnostic approach and design for API's and not for specific engines. Doing browser/server distinction, we're making same mistake as in old days where instead of doing feature detection we ported specific functionalities just to e.g. Firefox browser

It's not difficult today to bring browser API's to Node.js (I use DOM on server myself), or Node.js API's to certain browser's.

I host many cross env packages (some depend strictly on API's provided natively only in browsers) and don't see a value in browser and/or server tags, they just cement the wall, we don't need.

Addy Osmani
Mariusz Nowak

@addyosmani still I think it goes down to API's.

CSS is purely visual, so logically it's mainly about engines with visual interface. Naturally we think just browsers (front-end), but what if we want to generate PDF's out of DOM on server side, and want to use existing web components that were labelled browser-only (?)

On my side I use web components, which are structured so CSS and DOM (js) modules are separate files. In engine with visual interface (browser) I include both, On server-side, if I just want to generate DOM, I use just DOM module, if CSS module is required, it has no effect it's ignored.

Of course Node by default won't work with CSS files as by default there are no interfaces that can deal with that, but it doesn't mean it's server: false, same as we can use CJS bundler (like Browserify) to port CSS modules to browser, same way we can provide CSS modules on server side, if we have prepared interfaces for that

Domenic Denicola
Collaborator

@kirbysayshi it means the author intends their package to be used by developers who are building web applications, as well as/instead of developers who are building Node.js applications.

Andrew Petersen

@domenic but does that include a module transport for JS?

When evaluating a new package for client-side use, I spend the most time looking through its modules determining if:

  • it can be easily requireed in a browser environment (a.k.a. doesn't use globals)
  • it uses node-specific stuff

If adding a browser tag just means it uses window, document, and stuff, then what does that really say about the package? It still requires the consumer to go through and figure out if/how they can actually consume the modules, as @medikoo mentions.

To me, browser means "if the package contains JS, it's CJS bundler-compatible".

Domenic Denicola
Collaborator

@kirbysayshi I've intentionally given a minimal definition, and I think it's the one we should stick to since it's most broadly useful to client-side developers looking for a package manager, even if they haven't bought into any specific toolchain such as browserify. We can use further tags or mechanisms to narrow it down if you like specific toolchains.

Andrea Giammarchi

FWIW my 2 cents

the quantum browser property name for package.json seems the most straight forward and easy to implement in a backward compatible way.

  • browser: false or "false" to mark it as server side only, as default since npm means Node Package Manager and I expect modules to work with node first
  • browser: "only" to mark as not compatible with node.js even if in npm
  • browser: "too", "OK", 2, "as well" … anything else that is not falsy or the string "only" to explicitly mark it as suitable for both browsers and server

the side problem

there are modules that might be named same way and offer different files if the env is either the server, node.js, or the browser. In such case I'd expect the main script specified in package.json to be specific too in case the browser property is "as well"

The easiest way that I can think of right now is to allow main property as an object or string, where if object will point to {browser: "file/name.js"} and if not {default: "server/name.js"}

Last, but not least, thanks for exploring any possibility and considering this in first place.

Tim Oxley
Collaborator

@WebReflection note that "browser": "file/name.js" is exactly how browserify currently uses of the browser package.json property to specify browser-specific overrides: https://github.com/substack/node-browserify#browser-field

Mariusz Nowak medikoo referenced this issue in medikoo/modules-webmake
Closed

Bower package? #40

Caridy Patiño

This discussion is aligned with what some people call "isomorphic" code (code that can run in multiple runtimes), and for that, we have been using the term affinity to specify one, few or all supported runtime. Javascript keeps growing, and it is very likely that new runtimes become popular in the future, in which case, having something that allow us to expand to other runtimes will be beneficial, and on top of that, it is a term that is not polluted at all: https://www.npmjs.org/browse/keyword/affinity

Aidan Feldman

While I'd love to see NPM get more usage for the front-end, marking packages as browser (in)compatible requires a lot of buy-in from the community, and is very error-prone. Just to think outside of the box: couldn't (most of?) this be achieved through static analysis, with better accuracy? Check if the code in the package makes any calls to browser-only or server-only APIs (that can't be swapped out by things like Browserify) itself or through its dependencies, then presenting the platforms it supports. It could even go as far as generating a compatibility table, like what you'd see on http://caniuse.com. Added bonus: support isn't necessarily required in NPM, or even the NPM site. Could be a standalone thing.

Alex Kocharin rlidwka referenced this issue in TimSchlechter/bootstrap-tagsinput
Closed

Use bower for development #164

Ron Waldon

I like the idea of dangling more information off the engines property. However, front-end developers have been trained for years to stop sniffing the user agent, and instead use feature-detection. So, how about using dom and html to hint that the code requires the DOM? Similarly, how about having ecmascript?

For example, my package uses Canvas, Array#forEach and document.addEventListener, so my package.json reads:

{
  "engines": {
    "dom": ">=2",
    "ecmascript": ">=5",
    "html": ">=5"
  }
}

This way, when my package is included in a project that specifies dom: 1 and html: 4, the build tools for that project might be able to automatically suggest the necessary poly-fills (e.g. html5shiv and es5-shim, etc).

Michał Gołębiowski
Addy Osmani
Mariusz Nowak

@addyosmani @mzgol many of what what we assume as browser API's have node dedicated implementations in user land. There are also many packages that stand on those API's but are meant to be used in either environment. Why they should be marked as targeted just for browser?

What about other, not as popular environments like e.. Adobe Photoshop? Should they use browser or node dedicated packages?

It's clearly about API's that are provided in given environment, and developers should write utilities for API's not environments, we've already learned that lesson when we developed for different browsers, why to repeat the history?

Stoutie

I'm trying to figure out what to put in my package.json. What is the TL;DR?

Alexej Yaroshevich

:+1: for { engines: { browser: es6 } }

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Something went wrong with that request. Please try again.