[Team] Review Gulp

[addyosmani]

A new streaming build system which tries to offer a build pipeline alternative to Grunt. Has some nice advantages like being faster and requiring a lot less configuration (both in terms of the Gulp file (Gruntfile equiv)) and during task writing.

I don't think it's perfect, but may be worth a look just to be aware of what is being worked on.

• http://slid.es/contra/gulp
• https://github.com/wearefractal/gulp

[sindresorhus]

Would be interesting if someone tried converting the webapp generator Gruntfile to Gulp.

[jdalton]

\cc @phated

[yocontra]

Author of gulp here. I'll be lurking for feedback.

[Munter]

From my point of view Gulp solves some of the things that Grunt is already does well, albeit slowly, but doesn't address the things that Grunt doesn't do well.

The things that most task runners do really well are operations on individual files. Grunt tasks address this by doing one at a time and serializing to disk. Gulp by streaming. But both of these approaches still only deal with individual files.

One of the really complex problems to solve is manipulating a a 'web', a collection of files of different types with relations to each other. This is incidentally also one of the primary sought feature of Grunt. Things like bundling, cache renaming, cache manifest generation and similar tasks that require a knowledge of file relations and not just individual files.

I certainly see a lot of benefit in using Gulp for the individual file processing. In frontend development that is limited to preprocessing though. There needs to be a developer friendly format to live develop in, which ideally is the raw unminified and unbundled file to serve to the browser, in order to keep the development complexity low and debuggability high.

The next step is production building, which is where the notion if individual files is not as useful.

So from my point of view, which already deems the current Grunt workflow wrong, Gulp cannot really improve on the things where the Grunt workflow hurts the most.

If people like to keep the existing workflow and use Gulp for streaming instead of saving to disk in .tmp there are probably some performance benefits to be gained.

[yocontra]

@Munter gulp plugins aren't limited to being one file in, one file out. A plugin can wait for X number of files to be passed to it, do something with them, and pass one file (or two or three) back out to the next processor. gulp-concat is a good example of this. As far as I can tell this should cover the specific cases you mentioned.

[Munter]

In that case I'd say it's an improvement over Grunt. Are there existing Gulp plugins that implement a complete web app minification flow that I can take a look at? I might be interested in contributing to that

[yocontra]

@Munter since every plugin is its own stream they have total control over everything - what makes these plugins gulp specific is just a set of conventions and a File object they take in and put out. It's pretty great.

As for the complete web app gulp plugin that goes against the ideology of gulp which is to keep plugins small. Plugins that use other plugins are certainly common though so you could make an aggregate plugin that expresses your own opinions.

[Munter]

In that case I'll stick to promoting assetgraph for that type of complete web application build step.

[sindresorhus]

@contra thanks for chiming in. I took a good look at Gulp yesterday and I do like what I'm seeing. It's pretty close to how I would have created a task runner and it's very Nodenic (bad pun on Pythonic). I do agree with @Munter though. Even though it's fast and explicit, it still leaves solving the hard problems up to the users. I guess some of it can be solved with best practises and plugins, but you can also make Grunt superfast with grunt-newer (which imho should have been built-in from the start). I personally prefer code over config. But the benefit with reusable tasks and standardized config (even though it's super bloated) is that it's actually possible to use it without knowing JS. That is super powerful. I've talked to designers and other non-coders that have never touched JS, but still being able to set up Grunt.

I also like your rules about Gulp plugins. Grunt plugins has really become the jQuery plugins of the Node world. Not a good thing.

Too be honest though. I wish neither Grunt or Gulp were necessary. I miss the days where I could just code something up and upload with FTP, refresh the browser and view source...

[yocontra]

@lazd has totally redone the README so if anything was unclear before it should be easier to understand now.

https://github.com/wearefractal/gulp#gulp---

We also started a wiki at https://github.com/wearefractal/gulp/wiki

[addyosmani]

To give you an indication of where we see Gulp fitting into the Yeoman ecosystem long-term, we imagine that (depending on community uptake), we will end up supporting:

• Gulp for more experienced developers
• Grunt for beginners - intermediates. It remains a solution with a very low barrier of entry and is quite easy to get to grips with or write wrapper tasks for.

+1 on the comments about rules for Gulp plugins. There's a great opportunity there to improve on the situation we currently see with Grunt tasks - everyone is writing them and not to a consistently high standard.

[tomByrer]

Perhaps gulp-grunt can provide a solution?

[nmn]

I actually think that the MAIN benefit of Gulp is that it is much easier to pick up than Grunt.
The speed benefits are just an added bonus. So I would say Gulp is better for beginners.

[jbcpollak]

It seems to me that Gulp is to Grunt the same way Maven is to Ant in the Java world. One of my complaints with Grunt (I am admittedly new to it) is the lack of lifecycle stages and convention-over-configuration. Regarding experienced vs new users, I think convention-over-config is huge for beginners.

[jviotti]

Agree that Gulp is much faster to learn and to get up and running that Grunt. I've been always delaying learning Grunt in depth as the tons of configuration needed discourage me completly. I was able to get up and running with Gulp in a day, and also was able to write a plugin that I needed. Gulp simply "made sense" to me, and has been my choice in all my recent projects, so naturally I'll love to be able to use it with Yeoman, at least as an alternative.

[cmalven]

👍 for Gulp being much easier to wrap your head around than Grunt. I'm not a beginner, so I can't say with much authority, but I'd be surprised if a novice in both didn't have a much easier time understanding and modifying a Gulpfile than a Gruntfile running roughly equivalent tasks. The biggest difference being that a Gulpfile actually reads like a series of actions. I don't have a great understanding of how streams work, and I was able to create a Gulpfile that worked as I expected within minutes.

[addyosmani]

I've begun work on an initial Yeoman generator with support for Gulp. Follow along at https://github.com/yeoman/generator-gulp-webapp. Early days :)

[kaiquewdev]

I'm just playing with yo https://github.com/fth-ship/yo/blob/feature/migration-to-gulp/gulpfile.js but this error appears in the final of all tests pass https://gist.github.com/kaiquewdev/8797704
what is that?

[SBoudrias]

@kaiquewdev That's the runned generator run() method throwing. Method should be a function on the generator prototype, any chance you delete a method dynamically somewhere along the way?

[kaiquewdev]

@SBoudrias guess not for now, because I have not modified anything beyond gulpfile.js, this may mean that the tests are not covering this case? or my implementation is wrong!

[kaiquewdev]

@addyosmani and @SBoudrias any considerations about that implementation https://github.com/fth-ship/yo/blob/feature/migration-to-gulp/gulpfile.js ?

[kaiquewdev]

And now passing of course.

[SBoudrias]

@kaiquewdev You should send a PR and we'll see how the team is feeling about it.

[kaiquewdev]

@SBoudrias thanks, PR sended!

[mishu-]

Any ETA when this will be available? Thanks!
+1 on Gulp being easier to pick up. Since I only used Gulp so far, this is the one issue why I haven't started working with Yeoman so far.

[addyosmani]

The Yeoman team have already started using Gulp for some of our own projects on the org and we also released https://github.com/yeoman/generator-gulp-webapp, which we'll continue to maintain. In the future, I see us exploring how Gulp can be used with more of our official generators, perhaps once we've landed our next large feature - composability.

[addyosmani]

Closing this issue as we've now explored Gulp and are happy with it. The takeaway is: it's awesome, we love it, the ecosystem is still young but we see it going places. We'll continue to support both Grunt and Gulp.

[mishu-]

On that note, here is another build system :) http://www.solitr.com/blog/2014/02/broccoli-first-release/index.html
Ah, the frustration of having too many similar tools available and not knowing which one to choose!

[kevva]

Choose the one you like best. Personally, I like gulps use of streams.

[yocontra]

FYI there is some misinformation in that blog post.

Points to clarify:

1. Streams can handle more than one item at a time
2. Streams can buffer N items before operating, thus allowing them to build in-memory trees
3. gulp has multiple caching plugins that enable you to do incremental builds

The author did not do their research. Not a fan.

[sindresorhus]

@joliss

[joliss]

@contra I'm happy to add your points on the blog post. Will ping you on IM so we can sync up. (Update: We synced up & I made the gulp section on the blog post more precise. Thanks @contra!)

[yisibl]

+1

[norfish]

Stream is good, but I like gulp than grunt not becase stream but it's one plugin do one thing. It makes my task more readable.
To understand better of stream, you can read this https://github.com/substack/stream-handbook

[arthurvr]

In the future, I see us exploring how Gulp can be used with more of our official generators

Just wondering, but are there any plans on converting more generators to gulp? The ecosystem is getting better and better (even better than grunt in some ways). I'm sure that I'm not the only one who wants to use more generators with gulp.

[SBoudrias]

Gulp is harder to compose between differents generators. I think this is the main reason we'll probably keep using Grunt as the primary option.

On a side note, my personal take on it, Gulp is still pretty immature and is kinda working but not as good as I'd like (just error handling is pretty bad). Grunt on the other hand is pretty simple and have solid interfaces.

Most concerns about Grunt are the growing complexity of the setup, but IMO, the real issue in these cases is the lack of code organization where it is hard to build a dependency graph and optimize it from there. There's tools out there to help anyone build it up (Requirejs, assetgraph, webpack, etc) - and if people can't optimize the build based on graphs, then no matter the choice (Gulp or Grunt), the build system is going to be horrible.

[kevva]

grunt isn't that actively maintained anymore either. The plugins, yes. But there isn't any super activity going on in the core.

[arthurvr]

The gulp vs grunt discussion is probably a never-ending one, but some thoughts:

• As @kevva pointed out, grunt isn't actively maintained anymore.
• People say gulp is more nodey, but I personally think it's just more javascript likely. Looking at a gruntfile just feels like looking at a big object definition. I personally think many front-end developers, which are using javascript daily, will feel more familiar with gulp.
• Looking at many discussions on various github projects, people actually seem to like gulp more.
• It seems like the gulp community has, in quite a short time, catched up with its plugins offer. No real missing plugins anymore.

Maybe we should just give people the choice. Would it be difficult to maintain a gruntfile and a gulpfile in a generator?

[SBoudrias]

@arthurvr It would to a point prevent composition.

[Munter]

What are the things missing in Grunt that makes 'not actively maintained' a factor? If projects have small enough scope they will eventually be finished, not requiring any active maintenance.

Personally I think the build chains I've seen with both grunt and gulp become to complex to understand and maintain. But I have been tooting the horn of dependency graph based build systems for years, so I'm biased on that point

[kevva]

https://github.com/gruntjs/grunt/issues?q=is:open. I wouldn't call a project with that many issues open "finished" ;Pp.

[yocontra]

@SBoudrias In what way is grunt better than gulp for composition?

[SBoudrias]

@contra I mean composition in the Yeoman context. It is fairly easy to reason about composing a series of configurations inside a Gruntfile - see http://yeoman.io/authoring/gruntfile.html

For Gulp this kind of high level API to edit the build file is at the very least harder to design. I haven't come up with a solution yet.

Until we have a good solution, it'll prevent generator providing a Gulp option from being composed with other one needing to provide partial build system parts.

[yocontra]

@SBoudrias Never used yeoman before so I'm not 100% on how it works, but couldn't you do something like this?

this.gulpfile.insert(function(){
  var whatever = 123;
  gulp.task('js', function(){

  });
});

Where gulpfile.insert just takes the source of the function and inserts it into the gulpfile as a self-executing function

So the resulting file ends up looking like

// inserted by x generator
(function(){
  var whatever = 123;
  gulp.task('js', function(){
    // whatever
  });
})();

// inserted by z generator
(function(){
  gulp.task('css', function(){
    // whatever
  });
})();

The problem here seems to be more about programmatically generating a gulpfile, not composition.

[yocontra]

If that feels too loose, you could only allow specifying tasks instead of arbitrary code.

this.gulpfile.insertTask('js', function(){
  // code here
});

would yield

gulp.task('js', function(){
  // code here
});

[SBoudrias]

Yeah, that is not so hard, the harder part is if we need to apply multiple transform stream on a single resource type, ordering (although that's not really fixed for Gruntfile either) and combining these tasks in commons ones that the user can just run out of the box.

I think the main part of this answer remains in simplifying how a build is handled with smarter assets graph. If our builds are relatively simple and provide mostly management tasks (running tests, coverage, deployment), then generalizing concepts for most build system shouldn't be that hard.

[Munter]

What your gulp composition examples hint at is that there is a very limited set of best ways to create a build pipeline, where it's mostly about filling in blanks, but not moving the steps around in any big way.

So why even bother having the user assemble this final pipeline when we could just as well simply build the entire pipeline and offer configuration on whether to run each step and how each step is configured?

[yocontra]

@Munter That's something for a higher level tool to do, not something we are interested in having in gulp. If somebody wanted to write a tool on top of gulp that assembles the pipelines based on configuration that would be cool and I'm all for that. gulp is a lower level build tool that can power other higher level tools which expose more intuitive or domain specific interfaces.

[Munter]

The problem that Simon is trying to express is that if you want composition, then this higher level tool will have to be written by the consumer of the multiple composed generators, leaving the most difficult work in scaffolding undone when the generators have run. Not a very desirable outcome.

[arthurvr]

Maybe time for such an issue, but for broccoli? Early-days, but why not get the ball rolling?

[sindresorhus]

@arthurvr Nah. It has yet to see enough adoption for us to consider it.

[dancancro]

ember-cli uses Broccoli. That seems like a decent endorsement to me.

Excellent beard, Sindre!

[sindresorhus]

I'm aware, but it hasn't outside of that ecosystem. Not saying we ever won't, but it's a big investment we're not convinced is worth it yet.