Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Make a browserify recipe #369

Closed
contra opened this issue Mar 23, 2014 · 114 comments

Comments

Projects
None yet
@contra
Copy link
Member

commented Mar 23, 2014

Watchify is not needed, you can use browserify just fine.

This is an example of me bundling a browserify project into a standalone file:

var buffer = require('vinyl-buffer');
var source = require('vinyl-source-stream');
var browserify = require('browserify');
var uglify = require('gulp-uglify');

var bundler = browserify('./js/index.js');

gulp.task('compile', function(){
  return bundler.bundle({standalone: 'noscope'})
    .pipe(source('noscope.js'))
    .pipe(buffer())
    .pipe(uglify())
    .pipe(gulp.dest('./dist'));
});

I think this could be a perfectly fine plugin as well:

var uglify = require('gulp-uglify');
var browserify = require('gulp-browserify');

gulp.task('compile', function(){
  return browserify('./js/index.js', {standalone: 'noscope'})
    .pipe(uglify())
    .pipe(gulp.dest('./dist'));
});
@phated

This comment has been minimized.

Copy link
Member

commented Mar 23, 2014

I disagree that it should be a plugin. Seems to go against node paradigms.

@contra

This comment has been minimized.

Copy link
Member Author

commented Mar 23, 2014

@phated What does?

@contra

This comment has been minimized.

Copy link
Member Author

commented Mar 23, 2014

Plugin source code:

module.exports = function(opt, fileName){
  var bundler = browserify(opt);
  return bundler.bundle()
    .pipe(source(fileName))
    .pipe(buffer());
};
@phated

This comment has been minimized.

Copy link
Member

commented Mar 23, 2014

@contra combining those 3 plugins into 1 that is coupled so tightly to gulp

@contra

This comment has been minimized.

Copy link
Member Author

commented Mar 23, 2014

@phated Combing things to make things easier and more logically complete is not against node paradigms FYI

@phated

This comment has been minimized.

Copy link
Member

commented Mar 23, 2014

I thought the concept was to reduce coupling, not make it even worse

@phated

This comment has been minimized.

Copy link
Member

commented Mar 23, 2014

Reasons to not make a gulp-browserify

  • The first example isn't that complex and allows for flexibility.
  • The plugin would remove the flexibility of doing extra requires on your bundle or other browserify functionality.
  • Browserify has an ecosystem of transforms that should be used instead of gulp plugins. If you want to use gulp plugins, vinyl-source-stream is simple enough to transform the stream.
  • Having plugins for things that already deal in streams goes against gulp's supposed ideology of "when everything deals in streams, gulp will no longer need to exist" because it feels like you are saying that browserify doesn't deal in streams well enough.
  • The proposed plugin doesn't deal in globs, which is what most of the people looking for gulp-browserify want.
@hughsk

This comment has been minimized.

Copy link
Contributor

commented Mar 23, 2014

@contra the only reason I'd suggest not making it a plugin is that browserify has its own API outside of just the bundle method, e.g. transform, require, external etc. The plugin would then have to be responsible for exposing those and keeping everything up to date, unless it's intended to only cover the basic use case. If gulp-browserify had links to the relevant recipes I think that would point people in the right direction :)

@contra

This comment has been minimized.

Copy link
Member Author

commented Mar 23, 2014

I'm just trying to ease the pain of beginners. vinyl-source-stream is great but no plugins work with streams and having to require another module to convert it to a buffer requires that you understand the difference between the two in the first place

@sinan

This comment has been minimized.

Copy link

commented Mar 23, 2014

To have those both concepts in a wrapper would be nice in some shape, but I agree as a plugin it may cause more problems in the future. For the time being it causes some code repetition and as @contra pointed out it's a pain when you're new to things. I think, until the best way is found, some samples should be around, when I first tried that combination last month, it wasn't the easiest thing to get them all work nicely.

What we ended up doing was a gulpBrowserify function, because as @hughsk says, you need those transforms, shimming etc.:

gulpBrowserify = (options, bundleOptions) ->
  options.extensions or= ['.coffee']
  bundleOptions or= {}
  b = browserify options
  b.transform coffeeify
  b.bundle bundleOptions

and it is used as this:

source = require 'vinyl-source-stream'
buffer = require 'gulp-buffer'
gulp.task 'coffee', ->
  stream = gulpBrowserify
      entries : entryPath
    .pipe source entryPath
    .pipe buffer()
    .pipe ...

putting this here, in case anyone needs it.

@laurelnaiad

This comment has been minimized.

Copy link
Contributor

commented Mar 27, 2014

I'm just trying to ease the pain of beginners. vinyl-source-stream is great but no plugins work with streams and having to require another module to convert it to a buffer requires that you understand the difference between the two in the first place

Regardless of whether browserify should be a plugin, I think more (and more consistent), well-explained examples of working with streams in the gulp context, and vinyl-source-stream in particular, might help here. I think it's unfortunate that there are two streams -- the stream of files in a pipeline, and the streams that may or may not be the files' contents. So making a big bright line between the two probably needs some emphasis for beginners.

@contra

This comment has been minimized.

Copy link
Member Author

commented Mar 27, 2014

@stu-salsbury We had discussed removing file.contents as a stream before and I'm still open to it. While the performance benefits are huge, I think it's hard to implement on the plugin side and really almost no plugins support it at this point. Plugins that do support it usually don't do it right (it is tricky) and the perf is actually worse than just going buffered

@laurelnaiad

This comment has been minimized.

Copy link
Contributor

commented Mar 27, 2014

Yes, I'm aware of that. I like having content streams available. Perhaps more could be done to help auto-mediate between plugins that can deal in streams and those that can't? I'm sure you've already thought about that a lot more than I have.

@contra

This comment has been minimized.

Copy link
Member Author

commented Mar 27, 2014

@stu-salsbury Plugins can buffer the files themselves or users can insert something like vinyl-buffer into the pipeline but buffering the stream makes the performance worse than if you had not done streaming at all. The only time you get the benefits of file.contents as a stream is if

  1. it is never buffered
  2. plugins follow the guidelines
@contra

This comment has been minimized.

Copy link
Member Author

commented May 2, 2014

How is everybody feeling about this? Plugin or more recipes?

@simenbrekken

This comment has been minimized.

Copy link

commented May 2, 2014

Watchify becomes a necessity when builds get large, in my current project it cuts down compile time from 3s to 250ms.

Recipe:

gulp.task('watch-scripts', function() {
  var watchify = require('watchify')
  var util = require('gulp-util')
  var source = require('vinyl-source-stream')

  var watcher = watchify('./src/index.js')

  var bundle = function() {
    return watcher
      .bundle({debug: true})
      .on('error', util.log)
      .pipe(source('client.js'))
      .pipe(gulp.test('./build'))
  }

  watcher.on('update', bundle)

  return bundle()
})
@kevinSuttle

This comment has been minimized.

Copy link

commented May 2, 2014

I just want to use Browserify with Gulp. I'm new to Browserify, and stumbled onto this thread. What's the recommendation?

@jasonrhodes

This comment has been minimized.

Copy link

commented May 2, 2014

Every example I've seen that uses Browserify or Watchify directly only
sources one single file. I want to use a glob like everywhere else in gulp
a la gulp.src -- if that's not possible or easy then gulp-browserify and
gulp-faster-browserify and whatever else is out there will keep living on
forever, I think.

On Fri, May 2, 2014 at 3:14 PM, Kevin Suttle notifications@github.comwrote:

I just want to use Browserify with Gulp. I'm new to Browserify, and
stumbled onto this thread. What's the recommendation?


Reply to this email directly or view it on GitHubhttps://github.com//issues/369#issuecomment-42067467
.

@phated

This comment has been minimized.

Copy link
Member

commented May 3, 2014

I'm currently doing watchify stuff with multiple entry points. There is absolutely no reason to be using gulp.src to glob the files, just include glob as a dependency and use it in your code.

@contra

This comment has been minimized.

Copy link
Member Author

commented May 3, 2014

@phated A browserify plugin done right would act as a stream head that outputs vinyl files, and wouldn't have anything to do with gulp.src. It's purpose purely to encapsulate complexity of dealing with vinyl-source-stream and vinyl-buffer

@RnbWd

This comment has been minimized.

Copy link
Contributor

commented May 4, 2014

I ran into an issue today trying to integrate gulp - browserfy - reactify (this should apply equally to all transforms). I realized that I need to put the .on('error' handler) right after .bundle(), not after the vinyl-source-stream. Reactify was emitting errors that I couldn't catch. @contra - Today's my first day using gulp, and I didn't try vinyl-buffer, but if that error is repeatable then outputting a stream could lead to major issues with transforms

@contra

This comment has been minimized.

Copy link
Member Author

commented May 4, 2014

@RnbWd Can you post your code?

@RnbWd

This comment has been minimized.

Copy link
Contributor

commented May 4, 2014

var browserify = require('browserify');
var source = require('vinyl-source-stream');
var reactify = require('reactify');
var gutil = require('gulp-util');
var livereload = require('gulp-livereload');

gulp.task('scripts', function() {
  var b = browserify({
    entries: ['./src/js/main.jsx'],
    extensions: ['.jsx']
  }).transform(reactify); //adding transform in package.json seems to work identical
   b.bundle({debug: true})
   .on('error', handleError())   //error handler for reactify
   .pipe(source('main.js'))
   .pipe(gulp.dest('./build'))
   .pipe(livereload());
});

@contra If I place the error handler after .pipe(source('main.js')) or before .bundle(), gulp crashes and I get the following response:

stream.js:94
      throw er; // Unhandled stream error in pipe.
            ^
ReactifyError: /Users/Rainbow/Registration/src/js/signin.jsx: Parse Error: Line 8: Unexpected identifier

(etc etc)

With the error handler placed after .bundle() ReactifyError is logged properly. Also I noticed that if I returned browserify instead of declaring it as a var, the system wouldn't update if I fixed an error.

@contra

This comment has been minimized.

Copy link
Member Author

commented May 4, 2014

@RnbWd Have you tried placing the error handler after the transform statement instead of bundle?

@RnbWd

This comment has been minimized.

Copy link
Contributor

commented May 4, 2014

@contra Yep, I just tried it and didn't work..
I haven't tried adding var buffer = require('vinyl-buffer'), not sure if it make a difference.

@RnbWd

This comment has been minimized.

Copy link
Contributor

commented May 4, 2014

outputting a stream could lead to major issues with transforms

Sorry I misspoke, there's no issues with the stream. I was referring to the plugin source code. Adding an event handler after .bundle() fixed my issues with the reactify transform.


On another note, my recipe for improving performance of large browserify builds is to separate the static resources into a separate task with global window exposed in the file. see here

//Instead of this
//window.React = require('react/addons'); ~~
//react is huge

//I'm using this:
gulp.task('libs', function () {
  return browserify()
    .require('react/addons')
    .bundle()
    .on('error', handleError())
    .pipe(source('libs.js'))
    .pipe(gulp.dest('./build/'));
});

//adding .external('react/addons') to the scripts task and including necessary files in html

Updating my files via gulp.watch(./**, ['scripts']) (global React) was actually much faster than using watchify (local React), and the current watchify recipe would stop working after reactify errors, even though it handles them.

There are other ways to do this in gulp, I'm not sure how to implement.

@contra

This comment has been minimized.

Copy link
Member Author

commented May 4, 2014

@RnbWd You can do something like bundler.external('react/addons'); which will exclude it from the bundle

@suprMax

This comment has been minimized.

Copy link

commented Sep 4, 2014

@ruimarinho yep, very same problem. I trimmed it down:

JS_TRANSFORMS = ['coffeeify', 'jadeify']
compileJavascripts = (src, options)->
  bundler = browserify(entries: src, extensions: ['.coffee', '.jade'], debug: config.source_maps)
  bundler.transform(transform) for transform in JS_TRANSFORMS

  bundler.bundle()
    .pipe(source(options.name))
    .pipe(gulp.dest(options.dest))

And getting:

events.js:72
        throw er; // Unhandled 'error' event
              ^
Error: Parsing file /Users/max/Projects/mp.dev-star/node_modules/core/javascripts/client/modules/overlay_module.coffee: Line 6: Unexpected identifier
  at Deps.parseDeps (/Users/max/Projects/mp.dev-star/node_modules/browserify/node_modules/module-deps/index.js:360:28)
  at fromSource (/Users/max/Projects/mp.dev-star/node_modules/browserify/node_modules/module-deps/index.js:303:48)
  at /Users/max/Projects/mp.dev-star/node_modules/browserify/node_modules/module-deps/index.js:298:17
  at ConcatStream.<anonymous> (/Users/max/Projects/mp.dev-star/node_modules/browserify/node_modules/concat-stream/index.js:32:43)
  at ConcatStream.emit (events.js:117:20)
  at finishMaybe (/Users/max/Projects/mp.dev-star/node_modules/browserify/node_modules/concat-stream/node_modules/readable-stream/lib/_stream_writable.js:460:14)
  at endWritable (/Users/max/Projects/mp.dev-star/node_modules/browserify/node_modules/concat-stream/node_modules/readable-stream/lib/_stream_writable.js:469:3)
  at ConcatStream.Writable.end (/Users/max/Projects/mp.dev-star/node_modules/browserify/node_modules/concat-stream/node_modules/readable-stream/lib/_stream_writable.js:436:5)
  at DuplexWrapper.onend (/Users/max/Projects/mp.dev-star/node_modules/browserify/node_modules/duplexer2/node_modules/readable-stream/lib/_stream_readable.js:537:10)
  at DuplexWrapper.g (events.js:180:16)
  at DuplexWrapper.emit (events.js:117:20)
  at /Users/max/Projects/mp.dev-star/node_modules/browserify/node_modules/duplexer2/node_modules/readable-stream/lib/_stream_readable.js:934:16
  at process._tickCallback (node.js:419:13)
@ruimarinho

This comment has been minimized.

Copy link

commented Sep 4, 2014

Seems like a fairly straightforward stream. I don't spot any issue with the code - have you tried with other transforms?

@suprMax

This comment has been minimized.

Copy link

commented Sep 4, 2014

Looks like there's a problem with either browserify 5.x itself, or coffeify transform. Used to work perfectly well with 4.2.3. I should probably try it out on a very simple coffee file with only coffeify transform and then see how that goes. I haven't tried any other transforms, I need to get coffee compiling :)

@baer

This comment has been minimized.

Copy link
Contributor

commented Sep 4, 2014

@contra - Can this be closed? There are now two working Browserify recipes. They should probably be combined but it seems like the intention of this issue has been resolved.

Browserify + Uglify2 with Sourcemaps
Watchify

@contra

This comment has been minimized.

Copy link
Member Author

commented Sep 5, 2014

@baer I'll close this when a good browserify plugin exists - I'm not sure who wants to take ownership and actually make it

@baer

This comment has been minimized.

Copy link
Contributor

commented Sep 5, 2014

@contra Fair enough - maybe I'll take a crack at it. Thanks for all your awesome work!

@barneycarroll

This comment has been minimized.

Copy link
Contributor

commented Nov 10, 2014

I'm loathe to further dilute the subject matter in this huge thread, but it still seems like the most pertinent place to ask: the canonical recipes (as highlighted by @baer) show how you can vinyl-stream Browserify' output, but has anyone has any luck piping streams into Browserify? I'm running some minor preprocessing on my JS based on CLI arguments (ie gulp dev --search_endpoint http://dev02.example.com/search), but my inclination is to believe that this would have to be a Browserify 'transform'. Any thoughts?

@tnguyen14

This comment has been minimized.

Copy link

commented Nov 10, 2014

@barneycarroll I ran into the exact same issue recently and spent hours trying to figure out an answer as well. I found a solution that worked using vinyl-transform (https://medium.com/@sogko/gulp-browserify-the-gulp-y-way-bb359b3f9623). However, that module seems to be rather outdated, so I'm not sure if it is still the correct way to do this.

@barneycarroll

This comment has been minimized.

Copy link
Contributor

commented Nov 10, 2014

@tnguyen14 that's an excellent write-up either way – next time someone asks me about the gulp-browserify controversy I'll point them there instead of this thread – thanks!

@contra

This comment has been minimized.

Copy link
Member Author

commented Nov 11, 2014

@barneycarroll AFAIK Browserify doesn't accept any input besides a file path, since it does it's own dependency resolution and I/O internally

@jedmao

This comment has been minimized.

Copy link
Contributor

commented Nov 11, 2014

@contra are you suggesting that Browserify would need to accept a vinyl File as input in order to be considered gulp-friendly? I understand that it would definitely need to return a vinyl File, but does it matter what it accepts as input to bear the gulpfriendly tag?

@contra

This comment has been minimized.

Copy link
Member Author

commented Nov 11, 2014

@jedmao No I wasn't suggesting that, just responding to @barneycarroll's question. gulpfriendly doesn't really have anything to do with vinyl, it just describes modules that were made to be used with gulp. See the docs for a definition of each keyword.

@jedmao

This comment has been minimized.

Copy link
Contributor

commented Nov 11, 2014

@contra that seems inconsistent w/ something you said earlier:

Browserify doesn't return vinyl files so it is not "gulp-friendly"

As for the docs, the only place I've seen an explanation for gulpfriendly is at the bottom of Writing a Plugin, which only says:

Other libraries that are not file manipulating through streams but are made for use with gulp are tagged with the gulpfriendly keyword on npm.

I guess it's up to interpretation what "made for use with gulp" means. That's fine. I'm just trying to define these terms before explaining them to others.

@contra

This comment has been minimized.

Copy link
Member Author

commented Nov 11, 2014

@jedmao Browserify was created before gulp existed, and doesn't care about working with gulp in any way. Write now we have layers of stuff we have to wrap around it to make it work within gulp. I would expect substack not to put "gulpfriendly" in the keywords of his package.json. If he (using my example from earlier) added support for returning vinyl files or taking in vinyl files, or exported a gulp task or something, then it would be "gulpfriendly" because there was some thought put into making it work with gulp. There aren't any specific things you need to hit to be "gulpfriendly", it's extremely loose

@jedmao

This comment has been minimized.

Copy link
Contributor

commented Nov 11, 2014

Thanks for the explanation.

@barneycarroll

This comment has been minimized.

Copy link
Contributor

commented Nov 11, 2014

Let's be clear: Browserify is inconvenient to integrate with a Gulp-oriented environment because it sets its own precedents for input/output which are inconsistent with Gulp: Gulp uses the relatively standard streams, whereas Browserify invented it's own API. You can bundle Browserify's output to produce something consumable by Gulp, but there's no idiomatic way of feeding Gulp streams into Browserify. In much the same way that Gulp has engendered a panoply of gulp-{packageX} packages, which wrap packageXs API in order to work as streams, so Browserify has {packageX}ify, which do the same for Browserify's 'transform' input API. The crux of the matter, if you want to use both, is that any process you wish to perform before Browserify must be available on the filesystem or in terms that integrate directly with Browserify's own API.

If we want to talk about who should budge to accommodate the other, then the onus should fall on Browserify: as much as there are a million gulp-{packageX} packages out there, these are ultimately just stream-friendly adapters for esoteric APIs – despite the fact all of these are developed and named for Gulp compatibility, they're adopting a wider standard. On the other hand, Browserify deals with things the Browserify way and doesn't justify its API based on any other convention or authority.

Of course, you might say that this isn't a problem when dealing with other build tools such as Grunt: but the problem with Grunt (and other tools like it) is that they require each process to be aware of the one before it, and more often than not write new files to the filesystem for consumption by the next step. So in fact every transformation needs to be configured to work with the one before and the one after. The beauty of streams, as used by Gulp, is that a well-written transformation function can do very complex things while remaining completely agnostic as to where it sits in any given context.

@tnguyen14

This comment has been minimized.

Copy link

commented Nov 11, 2014

That's an awesome writeup @barneycarroll.

I am still a little confused. @contra said that Browserify doesn't accept anything except file paths, so how does the vinyl-transform work? It takes gulp-src and return file paths instead for Browserify to consume?

@RnbWd

This comment has been minimized.

Copy link
Contributor

commented Nov 11, 2014

This section of the browserify-handbook may be relevant. I've tried pulling apart browserify, gulp, and putting them back together again, and there's no clean way to do it. I shared a few patterns earlier in this thread, one involved tapping into the gulp stream, pulling out the file name (or stream), plugging that into browserify, and then pushing the browserify results back into gulp stream. Another pattern I used involved utilizing some of the dependencies of browserify within a gulp stream for a minimalistic (and fast) build without using browserify itself.

In the end, I just use the simple pattern that everyone else does. Gulp and browserify don't don't work well together. Although it's possible to make browersify 'gulp friendly', I felt like every attempt I tried just limited what I could do with browersify and gulp separately, together.

@sogko

This comment has been minimized.

Copy link

commented Nov 12, 2014

@tnguyen14
Firstly, browserify accepts both file paths and regular file streams (refer to API here).

Second, the way vinyl-transform works is that it takes in and gives you file paths (strings) from incoming gulp pipe for you to work with, and expects you to return a regular text stream (which you can get from browserify.bundle()).

vinyl-transform then takes that regular text stream and turn it into a vinyl transform stream, which you can then use downstream.

It's pretty simple but extremely useful library; @hughsk has more libraries like it.

@hughsk

This comment has been minimized.

Copy link
Contributor

commented Nov 12, 2014

@tnguyen14 nice writeup, glad you guys have found those packages useful :)

@barneycarroll browserify uses node streams, which are a part of node core. Gulp's vinyl streams are actually far less idiomatic in node land, though they're now definitely a common standard of sorts and stand well on their own. You're going to have an incredibly difficult time trying to convince substack and company to budge from what's in place now. While gulp has a ton of plugins, browserify also has a massive ecosystem of its own and was around long before gulp.

The reason I wrote vinyl-transform, vinyl-source-stream, etc. was so that the two could work together without a great deal of friction or debate – I'm curious as to where people are still having issues that aren't already solved with the recipes in the gulp docs? Do we just need a thing which takes gulp pipelines and turns them into browserify transforms? Because that's not difficult for me to do :)

@sogko

This comment has been minimized.

Copy link

commented Nov 12, 2014

Do we just need a thing which takes gulp pipelines and turns them into browserify transforms? Because that's not difficult for me to do :)

@hughsk That totally would be neat thing to do lol

Though frankly, I'm not convinced if there is a need to do so. I've been using gulp and browserify for quite sometime now and have documented several notes on my experience. And I agree vinyl-transform etc has been sufficient. (I've listed some useful gulp glues here; mostly just your libraries lol).

My question would be: what are the use-cases for having to pipe something into browserify streams, that are not already handled with browserify existing transforms and plugins?

It would be nice to others to share their use-cases for this? (Maybe on a separate thread? this thread is getting too long to be useful).

Personally, browserify has always been on the front of my pipelines, I have yet to encounter a case where I need a transformation done on a gulp upstream before it, that I don't already have a browserify transform for (reactify for JSX, deamdify to transform AMD to CommonJS, brfs for inline requires etc).

@jmm

This comment has been minimized.

Copy link
Contributor

commented Jan 7, 2015

@sogko Your recipe using vinyl-transform is broken as of browserify 8.0.2 (see browserify/browserify#1044), at least when using browserify.transform(). I think that wasn't a great match in the first place as vinyl-transform is designed to pipe to the return value of transformFn (browserify.bundle() in this case), isn't it? Which makes no sense with browserify. I think it only worked in a sort of accidental way. That being said, there may be a way to adapt the recipe to continue to work using vinyl-transform (see the browserify issue).

@jmm

This comment has been minimized.

Copy link
Contributor

commented Jan 7, 2015

@contra It's confusing trying to reconcile a few of your comments:

I think this could be a perfectly fine plugin as well:

I agree it shouldn't be a plugin.

I don't think there has ever been an argument about if there should be a plugin or not. I absolutely think there should be one...

Maybe this thread would more accurately express your opinion with edits to some of your comments clarifying what you mean by "plugin". From the example in your first post here and your comment on a different issue it seems that here maybe you're using the term plugin in a different sense than the commonly understood meaning of it in the context of gulp.

@phated

This comment has been minimized.

Copy link
Member

commented Mar 22, 2015

We are satisfied with our latest browserify recipe at https://github.com/gulpjs/gulp/blob/master/docs/recipes/browserify-uglify-sourcemap.md. Closing this

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.