Task Running

[ericelliott]

Choosing a Task Runner for the Cloverfield Boilerplate Scaffold

Cloverfield aims to create a next generation JavaScript project boilerplate. That means we'll use the tools that coders in-the-know will be using over the next 1 - 3 years. We're starting with the 2016 edition. Read more.

The JS community has been splintering when it comes to task runner consensus. For a while, Grunt was the clear winner, but that isn't the case anymore. We need to make a decision on one of these options, because many of our generators are going to produce some sort of task runner config. Please do some research and weigh in by answering the questions below.

Gulp and Brocolli have entered the community radar, and there is a growing trend to lean more heavily on Unix pipes instead of Node streams or the heavy file i/o typical of Grunt setups.

And the long-time reigning champion that's been a part of Unix since the time of the ancient grey beards is starting to make inroads. See Building JavaScript Projects with Make.

Oh, and npm has a built-in task runner that lets you easily leverage Unix pipes. Substack thinks that's the way to go, and he's not alone.

This is a pretty confusing landscape for those who haven't investigated all of these possible solutions, so lets try to lay out the pros and cons of each.

Questions

• What's your pick?
• What's the main reason you use it?
• What are the other advantages?
• What are the disadvantages?

Greener fields

• If you could change one thing, what would it be?
• If you could only keep one thing, what would it be?

[benlesh]

Having worked with Grunt, Gulp and Broccoli, my current go to is Broccoli. Mostly because I'm always setting up front-end build processes, testing and dev workflows and Broccoli was engineered specifically for that problem space. It's faster to deal with during dev builds than grunt, and there's generally less noodling with the Brocfile than I'd have to do with the others.

The downside is it's not as general purpose as the other two. But that's also the upside got me at least.

Grunt has the most mature community, but is dirt-slow.

Gulp is faster than grunt (usually), but you end up doing a lot more work to set up your workflow than you will with Broccoli because of Broccoli's built in server and watching. At least if your developing UI.

[benlesh]

If I were doing something more general purpose, I suspect I'd use Gulp ATM.

[clohr]

+1 for npm's task runner.

I have found that Grunt files become unwieldy on large projects and is slower than Gulp. Having used Gulp on recent projects, I found that there are some cool ways to better organize your tasks, but a number of plugins tend to violate the Gulp guidelines and you need to research the good modules to create optimal Gulp recipes. Broccoli looks cool, but if you are looking for something that works equally well on *Nix and Windows, it doesn't appear to be quite ready for Windows.

For me, each of these task runner tools create an extra layer that can be solved simpler by just utilizing npm run per Substack. Sticking with npm eliminates the need for additional dependencies which will reduce the overall boilerplate footprint and create a low barrier of adoption for those already familiar with Node.

[jesstelford]

Makefiles

I've said it before; Using a "flavor of the month" tool for a project that is meant to last a long time is a bad idea.

Everything you can do in grunt/gulp/taskjs you can do in Makefiles, with the added benefit of decades of backing, knowledge, tutorials, and examples already out there. Given it's cross-project and cross-language, it allows others who may not be 100% familiar with JS build systems to get up and running quickly - one less tool to learn (I'm assuming most decent developers know Make already).

If not make, then npm task runners would be my second choice. Because these can be simple calls out to a bash script that does what a Makefiles would have done for you otherwise. (And besides, Makefiles often call out to external scripts anyway - it's the Unix Philosophy!)

[benlesh]

@jesstelford

"(I'm assuming most decent developers know Make already)."

You assume wrong. The majority of developers, particularly web developers, either don't have a build process or have their build processes handed to them by IDEs like Visual Studio. More importantly, they have absolutely no reason to know what a make file is, let alone how to write one. That doesn't make them bad developers.

I see nothing wrong with Make, and I applaud the spirit of the suggestion, people are too happy for shiny things. But using Make with JavaScript projects is more "flavor of the month" (ala hipster) than anything else these days. It's pretty rare to see a JS project using Make, in my experience. If you're looking for "tried and true" in JavaScript-land, Grunt or simple npm run scripts are most likely your gotos.

[jesstelford]

@Blesh Thanks for calling me on that statement. I have a bias toward Make since I come from a C/C++ background.

have their build processes handed to them by IDEs like Visual Studio.

Even more reason for using Makefiles; VS has a Makefile Project Wizard built in, whereas the others require a plugin to be runable from within that IDE.

Although, but implying the majority of web developers use VS disregards the large numbers of devs who work in *nix environments (Linux, OSX, etc).

It's pretty rare to see a JS project using Make

I was curious, so I did some basic searching on Github;

• 99,548 references to .js inside a Makefile
• 436,468 references to grunt in either grunt.js or gruntfile.js
• 72,380 references to gulp in either gulp.js or gulpfile.js

If my search-fu is correct (it probably isn't), it shows Grunt as by far and away the most popular build tool for Javascript, but Make still comes in before Gulp.

[ericelliott]

Make has been around for decades and is still in constant use. There is a TON of tooling for make. Most IDEs either know how to assist you with make files, or have plugins that do. It's true that few JavaScript projects use it, but some do, and it may be worthwhile to encourage Make to spread in the JavaScript ecosystem.

Contrast with Grunt, Gulp and Brocolli, the only one with a large ecosystem is Grunt, and it's quickly going out of fashion. Gulp and Brocolli are new things to learn for most JS developers. As long as you have to learn something new, shouldn't you have some assurance that it will still be useful in 2017?

I think make and npm are the only safe bets available today...

That said, make is something new to learn for many JavaScript developers.

Maybe we should stick with npm as the standard for minimal projects, and generate a makefile with our scaffolding. That way, digging into make is opt-in, and developers who don't want to can use simple commands like npm run build.

Thoughts?

[getify]

Just gonna leave this here: https://gist.github.com/substack/8313379

[cswenor]

My vote is for Gulp. It is something new to learn, but it is very intuitive. A Gulp File is very easy to read an understand what is going on. I have to agree with @clohr that you have to do research to find the good plugins/scripts because the community is still small.

I do like the sound of make though. I personally never considered using it with JS. I'm going to have to look at a few examples.

[jesstelford]

@ericelliott

make is something new to learn

Arguably, so is Grunt, Gulp, etc - their syntax can be (almost) as opaque as make at times. I whole heartedly agree with it may be worthwhile to encourage Make to spread in the JavaScript ecosystem.

generate a makefile with our scaffolding

Now you're talking about implementing autoconf / cmake. I'm not so sure that's a good route to go down (it's the equivalent of implementing a gruntfile-builder package that autogenerates your gruntfile.js based on gruntfilelists.txt :P)

It's always possible to alias npm tasks to make targets:

// package.json
{
  "scripts": {
    "build": "make build"
  }
}

That way, it feels familiar to a JS user who might not have seen make before, but doesn't otherwise care about what make / Makefiles actually are.

@cswenor

Check out Building javascript projects with make for a great intro ;)

[cswenor]

One thought I have about using make is that it is a new language.

If I imagine myself giving a Junior Developer that has started learning with JavaScript and they see this and see it is a completely different syntax they might be spooked a bit.

Just a thought.

[getify]

I think we may be missing an important point... build processes are not necessarily appropriate for beginning developers to be _task_ed (pun intended) with writing. Build processes are more a "devops" thing than a "junior javascript developer just joing the team" kind of thing.

That's not to be elitist and say that non-devops devs shouldn't be able to understand and even help manage these things. I'm just saying in the spectrum of important influences on choice of build systems, I wouldn't say "immediate familiarity to a junior dev" is high on my list.

[jesstelford]

@getify

I wouldn't say "immediate familiarity to a junior dev" is high on my list.

The caveat to that is; We may encounter non-junior / intermediate / senior JS devs who have never used Make before, but want to use the cloverfield project (in a way which requires altering the build system). In that case, I'd argue that forcing them to use/learn Make is a good thing, but it's almost guaranteed to turn some people off. The same way that using Grunt over Gulp might turn some people off, etc.

[getify]

I'd say "ability to use/execute the build system" and "ability to configure/customize/write the build system components" are two separate and almost orthagonal concerns.

From my limited perspective, it seems the former is more what's at concern here with sentiments like "almost guaranteed to turn some people off". So I think the more important question there is what is the interface to the system, and how accessible is it to those with less experience. I can't imagine npm can be beaten in that respect.

What it runs on under the covers should be debated separately, and the focus should be IMO less on "ease of familiarity" and more on "right fit". I linked to "task.js" earlier because for me, writing JS code to build my JS code is the most natural fit. If you're at a PHP shop, though, I bet PHP is the more natural fit. Same with a perl shop.

I personally don't feel like make is a natural fit in such environments, but I also wouldn't turn to bash scripting (anymore, now that I have node) so I recognize if you're purely coming from the sysadmin/devops PoV make and bash are pretty natural.