Make the kickstarter fast again for it's main purpose: development

[krnlde]

I experience huge loads each time I'm downloading, installing, starting and running the kickstarter project. Most of the tasks that run for development aren't per se necessary at the beginning of a project. Sure we need reports, sure we need imagemin and all that sorts of stuff. But, as the name quotes, a kick-ass start would be the biggest gainer :)

What do you think @mischah ?

[mischah]

Thanks for your feedback. Good idea to reconsider and perhaps restructure the tasks …
Gonna use this issues to talk about that soon 🐔

[mischah]

@krnlde
First of all: I don’t feel your pain entirely. But I’m open to changes affecting our performance because perf matters in tooling too 😄

But I don’t address your complaints against download and npm install time in this ticket. Because that are one time »problems« you have with every project.

1. dev task

The dev task takes approx. 3 seconds (in a naked project).
I’m firing this task once a day (or even less) for every project via the default task.
After firing it once the watch tasks will take care of the rest.

before:

Execution Time (2015-09-21 21:57:48 UTC)
less:dev          625ms  ▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇ 20%
autoprefixer:dev  261ms  ▇▇▇▇▇▇▇▇▇▇▇▇ 8%
copy:server       140ms  ▇▇▇▇▇▇▇ 5%
generator:dev      34ms  ▇▇ 1%
plato:dist        119ms  ▇▇▇▇▇▇ 4%
jsdoc:dist        310ms  ▇▇▇▇▇▇▇▇▇▇▇▇▇▇ 10%
htmllint:all       1.1s  ▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇ 36%
bootlint:files    202ms  ▇▇▇▇▇▇▇▇▇▇ 7%
eslint:target     274ms  ▇▇▇▇▇▇▇▇▇▇▇▇▇ 9%
Total 3.1s

After getting rid of plato and »jsdoc« (which are obviously pretty useless in the dev task) we get down to a little less than 3 seconds.

after:

Execution Time (2015-09-21 22:13:44 UTC)
less:dev          624ms  ▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇ 23%
autoprefixer:dev  265ms  ▇▇▇▇▇▇▇▇▇▇▇▇▇▇ 10%
copy:server       126ms  ▇▇▇▇▇▇▇ 5%
generator:dev      35ms  ▇▇ 1%
htmllint:all       1.1s  ▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇ 42%
bootlint:files    198ms  ▇▇▇▇▇▇▇▇▇▇▇ 7%
eslint:target     291ms  ▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇ 11%
Total 2.7s

Not a big win 😞

But hej, it’s almost a half of a second. So I did the change with 39150d3.

I don’t see any other enhancement possibilities within the dev task. htmllint is of course a huge painpoint (performance wise). But I don’t want to get rid of it in the dev task.

[mischah]

2. build task

The build task takes around. 8.5 seconds (in a naked project).
I’m firing this task between zero and many times a day depending of the project and where we are with the project.

Execution Time (2015-09-21 22:22:10 UTC)
htmllint:all        1.1s  ▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇ 13%
bootlint:files     195ms  ▇▇▇▇ 2%
eslint:target      283ms  ▇▇▇▇▇ 3%
less:dev           643ms  ▇▇▇▇▇▇▇▇▇▇▇ 8%
autoprefixer:dev   269ms  ▇▇▇▇▇ 3%
uncss:dist          1.2s  ▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇ 14%
cssmin:assets      311ms  ▇▇▇▇▇▇ 4%
imagemin:dist      986ms  ▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇ 12%
copy:server        115ms  ▇▇ 1%
bower_concat:dist  378ms  ▇▇▇▇▇▇▇ 4%
uglify:bower        2.4s  ▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇ 28%
plato:dist         147ms  ▇▇▇ 2%
jsdoc:dist         384ms  ▇▇▇▇▇▇▇ 4%
Total 8.5s

Are there any task we could remove here but keep them in the release tasks?

I guess the following are valid candidates:

uncss:dist         1.2s
imagemin:dist     986ms
plato:dist        147ms
jsdoc:dist        384ms
---------------------------------
total              2.7s

Which would take the build task from 8.5 seconds down to 5.8 seconds.

BUT that means that when people don’t use the release tasks they won’t have optimized images.
Is that worth the 2.7 seconds?

This isn’t a problem for me personally. Because I do use the release tasks. But I’d like to get feedback from @krnlde @mfroehner @uschmidt and @revrng

[krnlde]

Thanks for breaking down the times spent on each task. I think not each task itself is a problem (still it is good that plato and jsdoc are removed for dev), no I think the problem is that the tasks are executed sequentially despite not having any dependencies on each other, except for less->autoprefixer, and htmllint->bootlint. This would in the best case reduce the whole time to 1.1 + .198 seconds (htmllint + bootlint). I don't know if there is a possibility to run tasks in parallel in grunt. Any ideas?

Edit: I think uncss and imagemin are unnecessary loads in dev. Because it has nothing to do with developing products, but rolling out builds or releases.
I read that wrong, in a build these are necessary. Also a build is not the problem I was talking about. Just the dev stuff. The startup and reload times.

[mischah]

Ahh. Good point 👌

Yes there is something called grunt-concurrent which I’ve never used. But I’ll give it a try 😏

[krnlde]

With gulp it would be really really easy and bulletproof at the same time 💨

[mischah]

😆

Without a doubt.

But You would need some extra effort to run tasks synchronously which are depending on each other less → autoprefixer → copy for example. This would increase complexity of your Gulp setup.

And I don’t want this discussion to drift into the direction Gulp vs. Grunt vs. Broccoli vs. Brunch vs. Webpack.

Although I’m open to switch to Gulp – in case it makes sense!
But there are a few things to consider to answer this Question.
I still like to check Gulp when there are no open issues left. But you (or anyone else) could also adapt our workflow with Gulp and I would be happy to see if there is any (real) benefit from that switch.

But like I mentioned before this is not the issue to talk about switching from Grunt to Gulp. Feel free to open another issue if you like to discuss this

[krnlde]

run-sequence is your friend anywhere you need sequencial pipes.

Sure anyone can rebuild that, but - speaking for myself - I currently don't have the time to build the exact same pipeline in gulp you built in grunt. Because it is really time-consuming copycatting the exact options and look for differences in every task rather than building a detached (new) one just focusing on getting the best out of pipes and parallelism.

That being said, I really would love to see how the grunt-concurrent task works out, because it uses real cores for every concurrent task, which should be a HUGE impact in terms of performance. Also the API looks nice and understandable.

[mischah]

Back to topic …

That being said, I really would love to see how the grunt-concurrent task works out, because it uses real cores for every concurrent task, which should be a HUGE impact in terms of performance.

Using grunt-concurrent makes things worse.

I changed the dev task to run htmllint, bootlint and eslint concurrently which had a negativ impact on performance.

before:

Execution Time (2015-09-21 22:13:44 UTC)
less:dev          624ms  ▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇ 23%
autoprefixer:dev  265ms  ▇▇▇▇▇▇▇▇▇▇▇▇▇▇ 10%
copy:server       126ms  ▇▇▇▇▇▇▇ 5%
generator:dev      35ms  ▇▇ 1%
htmllint:all       1.1s  ▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇ 42%
bootlint:files    198ms  ▇▇▇▇▇▇▇▇▇▇▇ 7%
eslint:target     291ms  ▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇ 11%
Total 2.7s

after:

Execution Time (2015-09-22 19:43:05 UTC)
less:dev          605ms  ▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇ 12%
autoprefixer:dev  270ms  ▇▇▇▇▇▇▇▇ 5%
copy:server       134ms  ▇▇▇▇ 3%
concurrent:dev     4.1s  ▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇ 79%
Total 5.2s

😱

It felt to me that the concurrent task hat something like an initial »boot time«.

@sindresorhus describes the problem over here:

Why is in my case concurrent slowing down the grunt tasks?

The task spawns child processes of grunt that runs concurrently. Spawning child processes generally comes with some overhead and launching grunt is very slow, especially on Windows and if you're not on a SSD harddrive. It's not a magic bullet and works best when you have heavy tasks with many files. In your case this task is overkill.

Seems to be a architectural problem with Grunt – compared to Gulp 😦

[krnlde]

👎 child_processes don't need that much time (like seconds) to start, aaand they should speed up the computation to run everything in parallel despite having a slower start :-/

Mhm... I guess we stick to the pipeline as it is right now then...

[mischah]

Jep. I have no clue what’s going on there 😥

[mischah]

Closing this for now.

[krnlde]

😿

[sindresorhus]

The main problem is not spawning child processes, but that grunt is incredible slow to start up.

[mischah]

@sindresorhus Thanks for your feedback :unicorn_face: