Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

When will browserify be done? [v4 discussion] #636

Closed
substack opened this issue Feb 7, 2014 · 20 comments

Comments

Projects
None yet
@substack
Copy link
Collaborator

commented Feb 7, 2014

I think browserify has been going pretty well at achieving the goal of making npm modules viable in the browser, but what would it look like for browserify to be "done"? Not "done" in terms of "dead", but "done" in terms of the vision is clearly defined and settled, with only patches and dependency upgrades necessary, with other depended-upon modules taking up the bulk of the experimental work. I want browserify to become a boring tool that hardly ever changes like sed or awk that you can just take for granted.

Here are some hurdles to being "done":

  • es6 modules. These already somewhat work with userland transforms and can be turned into a default global transform once a stable version of node ships with es6 modules, perhaps slightly before. However, es6 modules have all kinds of fancy new runtime loading mechanisms that we will need to hack around and ignore to make everything play nicely together.
  • addressing the use-cases for multiple bundles and different kinds of output
  • feature additions to modules in node core

For v4 I want to heavily start pushing for module authors to depend on explicit npm versions of modules instead of whatever gets bundled in with the core shims, echoing what @rvagg says in this blog post about readable-stream. Explicit versioning will go a long ways toward future-proofing modules so that we can depend on them and they will just work, even far in the future when node may have changed out from under these modules. Part of making this easy will be working with module authors to get approximate 1:1 parity with node core module names like path published to npm so that module authors can require('path/') and put path in their package.json with explicit versioning.

A new command, browserify --warn will be introduced to point out where implicit globals and core modules are being depended on instead of explicit versions.

The support for multiple bundle output in will also be improved in v4. Much of that work has already been done in factor-bundle; it's just a matter of deciding whether to bring this use-case properly into core or to integrate this into a new plugin system that has been proposed.

For v4 I'm punting on es6 modules for now because there aren't enough concrete manifestations of them for it to be a problem for browserify yet. However all of browserify's AST machinery should now work with es6 syntax.

@rauchg

This comment has been minimized.

Copy link

commented Feb 7, 2014

+1 on not bundling node modules automatically, and introducing --warn

@substack

This comment has been minimized.

Copy link
Collaborator Author

commented Feb 7, 2014

Here's what the upgrade path for node modules could look like:

  • v4 - introduce --warn
  • v5 - turn warn on by default, suppress with --quiet
  • v6 - hide resolving node modules behind an option???? This would depend on how the uptake in v4-v5 goes and how much of a vibrant ecosystem we can get rallying around maintaining userland versions of core modules.
  • v7 - remove resolving node core modules altogether???? I can dream. Probably this will never happen.

@substack substack referenced this issue Feb 7, 2014

Closed

fixes for <=ie8 #5

@feross

This comment has been minimized.

Copy link
Member

commented Feb 7, 2014

I'm maintaining buffer on npm now: https://npmjs.org/package/buffer

@substack

This comment has been minimized.

Copy link
Collaborator Author

commented Feb 7, 2014

As far as making 1:1 parity of node core with npm goes, here's the status:

  • assert - published
  • buffer - published
  • child_process - available, limited applicability to browserify
  • cluster - taken, limited applicability to browserify
  • console - taken, somewhat similar, better version currently at console-browserify
  • constants - taken, browserify version at constants-browserify. Rare to need this in the browser.
  • crypto - taken, browserify version at crypto-browserify
  • dram - available, limited applicability to browserify
  • dns - available, limited applicability to browserify
  • domain - taken, limited applicability to browserify
  • events - published
  • fs - available, limited applicability to browserify
  • http - taken
  • https - available
  • module - taken, limited applicability and rare to see in any code
  • net - taken, limited applicability to browserify
  • os - available
  • path - published, but stale. browserify uses path-browserify
  • punycode - published
  • querystring - published, open browser compat issues
  • readline - taken, limited applicability to browserify
  • repl - taken, limited applicability to browserify
  • stream - published but stale. It would be good to cross-publish readable-stream to this package name
  • _stream_duplex - available, rare to use this package name explicitly
  • _stream_passthrough - available, rare to use this package name explicitly
  • _stream_readable - available, rare to use this package name explicitly
  • _stream_transform - available, rare to use this package name explicitly
  • _stream_writable - available, rare to use this package name explicitly
  • string_decoder - published
  • sys - taken, rare to see anymore
  • timers - taken, browserify version is at timers-browserify
  • tty - available, limite applicability to browserify
  • url - published
  • util - published
  • vm - taken, browserify version at vm-browserify
  • zlib - taken, browserify version at zlib-browserify

Packages with limited applicability to browserify are still useful for the reasons listed in rvagg's post. If node shifts out from under your feet, everything will still work.

@substack

This comment has been minimized.

Copy link
Collaborator Author

commented Feb 7, 2014

So here's a priority queue of what we can do on the 1:1 front.

Publish these packages (available, applicable to browserify):

  • https
  • os

Convince owners to publish modules under these names to have parity with node core:

  • crypto
  • http
  • timers
  • vm
  • zlib

and perhaps publish these packages (available, but rarely used):

  • _stream_duplex
  • _stream_passthrough
  • _stream_readable
  • _stream_transform
  • _stream_writable
  • tty - more important to just have a userland version for node code than browser code
  • fs - more important to have a userland version for node code than browser code

and these modules would be nice to have (taken and applicable but rare):

  • console
  • constants
  • module

Where possible, these modules should work in node AND the browser, similar to how readable-stream is doing things so we can get explicit versioning in both node and browser code without having to worry about the platform shifting under our feet. Dependent modules should defer to other userland modules instead of core where possible.

It would probably be a good idea to get a similar project organized around node core libraries, but I am personally focusing on the browser stuff. We should probably update the browser shims to be compatible with node that aren't just purely algorithmic with a main/browser field split.

@heapwolf

This comment has been minimized.

Copy link

commented Feb 7, 2014

/cc @Gozala (crypto), @yuanyan (http), @popomore (timers), @fool2fish (vm), @kkaefer (zlib)

@kkaefer

This comment has been minimized.

Copy link

commented Feb 7, 2014

i.e. the npm zlib module should become a JS-only module that provides the same API as node core? Currently, the zlib module is a C++ module that provides synchronous(!) zlib functionality. There are a few users of the module, but given that it's not compatible with anything newer than 0.8, this may not be such a big case. I did a short review of the depending modules, and there are only two that we can't exclude by bumping the version:

3vot-cli           ~1.0.5
alexandria         ~1.0.5
cloudpub           >=1.0.5
deluge-add         ~1.0.5
get-down           1.0.5
gitteh             ~1.0.x
gulp-htmlmin       ~1.0.5
jqbuild            ~1.0.2
livedb-dynamodb    ~1.0.5
mimosa-s3-deployer ~1.0.5
passport-saml      1.0.x
tarzan             ~1.0.5
tgz                ~1.0.5
tinsmod            ~1.0.5
vim-plugger        *
virtualenv         1.x

We'd have to work with these developers to update their package.json to restrict the versions to 1.x.

It's unlikely that I will develop a JS-only zlib module, but I'm willing to share or give up the namespace for a browserify-compatible zlib module.

@terinjokes

This comment has been minimized.

Copy link
Contributor

commented Feb 7, 2014

I was originally hesitant to this idea, but after reflection and seeing the timeline here, I'm warmed up to the idea.

Some of the core modules are closely tied to the event loop and libuv, how would you suggest handling that with cross-node userland modules? Better and stable C bindings?

@tilgovi

This comment has been minimized.

Copy link
Contributor

commented Feb 7, 2014

⛵️

@substack

This comment has been minimized.

Copy link
Collaborator Author

commented Feb 7, 2014

@kkaefer It's ok for the zlib module to use c++ (but problematic in node when the node core api for bindings changes out from under us) so long as the browser version is pure js. You can do this with the browser field:

{
  "main": "index.js",
  "browser": "browser.js"
}
@rvagg

This comment has been minimized.

Copy link

commented Feb 7, 2014

@substack @kkaefer the problem is that the compile will still have to happen on npm install even if it's just being downloaded for browser use. Imagine all the issue reports from Windows users who are trying to use it just for browserify ...

@jquense

This comment has been minimized.

Copy link

commented Feb 7, 2014

@substack ditto @rvagg here, us on Windows have more then a few issues with Binary module ease, one reason I use the node core modules over user and ones is that they just work, no windows sdk, specific version of visual studio, or adding special flags to npm install. Gyp is always complaining

@defunctzombie

This comment has been minimized.

Copy link
Contributor

commented Feb 7, 2014

  • --warn doesn't need to be a major bump since it causes no change by default
  • url and path modules I have access to on npm and will migrate over the latest versions
@defunctzombie

This comment has been minimized.

Copy link
Contributor

commented Feb 7, 2014

I think that v4 should use consider using this module instead of detective in module-deps: https://github.com/creationix/mine This will help with some perf.

@jquense

This comment has been minimized.

Copy link

commented Feb 7, 2014

@pkrumins they aren't insurmountable problems, they are just unnecessary ones when you are building for browsers. I shouldn't have to figure out how to get binaries to build when i won't be using any of them. Not everyone (lots of people) aren't going to realize that all those npm install issues probably don't matter to you because you don't need the binary.

@substack

This comment has been minimized.

Copy link
Collaborator Author

commented Feb 9, 2014

@defunctzombie That was already merged upstream in module-deps yesterday. Hopefully we can port the rest of @creationix's mad science parsing experiments for insert-module-globals (until we can drop it into userland entirely) and derequire.

@tjwebb

This comment has been minimized.

Copy link

commented Apr 22, 2014

Docs should be somehow clearer about which Browserify versions support which node versions. For example, what version do I grab if I want es6/harmony support? Maybe some version of browserify is already feature-complete for node 0.8.26, but that same version maybe isn't even functional with node 0.11.10. This is hand-waving, but maybe you get my point.

@substack

This comment has been minimized.

Copy link
Collaborator Author

commented Apr 22, 2014

@tjwebb All browserify versions support 0.8 and 0.10 and I would imagine that everything should work on 0.11 too. You can just look at the .travis.yml to see which versions of node browserify is continuously being currently tested against. As for harmony versions, browserify and all dependent modules are using esprima-fb, which is es6 compatible.

@tjwebb

This comment has been minimized.

Copy link

commented Apr 22, 2014

Yea I guess when I say node 0.11, that includes --harmony invocations; that is, after all, a feature of 0.11.

Thanks for the info.

@substack

This comment has been minimized.

Copy link
Collaborator Author

commented May 10, 2014

v4 is out and now uses readable-stream. Still, modules should switch to require('readable-stream') instead of require('stream') in their own code. Closing for now.

@substack substack closed this May 10, 2014

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.