Performance, round 2

[MoOx]

Now that perf are back to "normal" (limited by webpack speed), we need to find tricks to make webpack build faster
Maybe

• https://webpack.github.io/docs/list-of-plugins.html#dllplugin
• https://github.com/amireh/happypack
• https://github.com/markfinger/unfort

An interesting issue: webpack/webpack#250

Also: http://www.slideshare.net/trueter/how-to-make-your-webpack-builds-10x-faster

[thangngoc89]

I tried happypack with phenomic before. Only gain 1-2s. I tried it again soon

[MoOx]

Just an idea: find a way to detect if there is any file change (or not) to decide if we need to compile via webpack, or just use previously compiled files. This way we could skip webpack build step (eg: if no files in web_modules has been changed + node_modules tree untouched). This may be convenient when you only write content.

[thangngoc89]

It sounds like a good idea. But I don't know how hard would it be to implementing such a feature like that.

[MoOx]

Another idea: think about removing webpack (or making it optional, eg: if you just have JS(X), you can just use babel and a "simpler"(faster?) tool like browserify). That can definitely take a thorn out of our side.

[MoOx]

The webpack dll thing should help a lot. Will try to work on that soon as it should not be complicated.

Also https://github.com/FrendEr/webpack-optimize-example

[MoOx]

https://www.npmjs.com/package/hard-source-webpack-plugin

webpack/webpack#250 (comment)

I haven't been able to test on a large project yet, but testing on two smaller personal projects I've seen times for a ~5s project drop down to ~500ms and a ~8s project drop down to ~1s. Those projects respectively have ~600 and ~400 modules.

OMG. We have to try this.

[thangngoc89]

There is 1 place we can improve: disable Uglify plugin for node build (whether production or not)

[thangngoc89]

I gave hard-source-webpack-plugin a try and it's ridiculous fast

D:\web\phenomic\docs (hard-source-wp)                             
λ npm run build                                                   

> Phenomic@ build D:\web\phenomic\docs                            
> phenomic build                                                  

  phenomic:builder ✓ Static build: assets copy completed +0ms     
  phenomic:builder ✓ Static build: client build completed +27s    
  phenomic:builder ✓ Static build: static build completed +5s     
  phenomic:static ✓ Static html files: 30 files written. +2s      
  phenomic:static ✓ .nojekyll created. +2ms                       
  phenomic:static ✓ CNAME is phenomic.io. +1ms                    
  phenomic:static ✓ Build successful. +1ms                        

D:\web\phenomic\docs (hard-source-wp)                             
λ npm run build                                                   

> Phenomic@ build D:\web\phenomic\docs                            
> phenomic build                                                  

  phenomic:builder ✓ Static build: assets copy completed +0ms     
  phenomic:builder ✓ Static build: client build completed +2s     
  phenomic:builder ✓ Static build: static build completed +9s     
  phenomic:static ✓ Static html files: 30 files written. +2s      
  phenomic:static ✓ .nojekyll created. +3ms                       
  phenomic:static ✓ CNAME is phenomic.io. +1ms                    
  phenomic:static ✓ Build successful. +1ms                        

Issues:

• only work with browser build, node build throw random error
• no manifest.appcache
• doesn't work with uglifyJS (need a simplify version before send out an issue)
• doesn't work with extract text plugin (missing css file output from extract-text-webpack-plugin mzgoddard/hard-source-webpack-plugin#1)

[mzgoddard]

@thangngoc89 landed a fix for hard-source that gets extract text and uglify working. Please file a issue if that didn't fix those. (I'm not aware enough about phenomic to know if your other stated issues are caused by hard-source or how to build a test to see if it is.)

[MoOx]

@mzgoddard thanks for you work. Phenomic is just a "classic" webpack consumer, nothing fancy. We will take a deeper look and get back to you if needed!

[thangngoc89]

@mzgoddard Thanks for your quick response. I pushed my work here: https://github.com/MoOx/phenomic/tree/hard-source-wp

It doesn't work with UglifyJSPlugin.

Step to reproduce:

git clone https://github.com/MoOx/phenomic/tree/hard-source-wp
npm install && npm run docs

# Trigger a build 
cd docs && npm run build

The second time you trigger the build, you'll receive some error like this:

Error: phenomic.browser.bff0132abecff214e404.js from UglifyJs
TypeError: Cannot read property 'sections' of undefined
    at new SourceMapConsumer (/home/khoa/web/phenomic/node_modules/source-map/lib/source-map-consumer.js:20:19)
    at RawSource.CacheModule.source.node (/home/khoa/web/phenomic/node_modules/hard-source-webpack-plugin/index.js:79:7)
    at PrefixSource.node (/home/khoa/web/phenomic/node_modules/webpack-core/lib/PrefixSource.js:27:26)
    at /home/khoa/web/phenomic/node_modules/webpack-core/lib/ConcatSource.js:38:49
    at Array.map (native)
    at ConcatSource.node (/home/khoa/web/phenomic/node_modules/webpack-core/lib/ConcatSource.js:37:60)
    at /home/khoa/web/phenomic/node_modules/webpack-core/lib/ConcatSource.js:38:49
    at Array.map (native)
    at ConcatSource.node (/home/khoa/web/phenomic/node_modules/webpack-core/lib/ConcatSource.js:37:60)
    at /home/khoa/web/phenomic/node_modules/webpack-core/lib/ConcatSource.js:38:49
    at /home/khoa/web/phenomic/node_modules/webpack/lib/optimize/UglifyJsPlugin.js:135:31
    at Array.forEach (native)
    at Compilation.<anonymous> (/home/khoa/web/phenomic/node_modules/webpack/lib/optimize/UglifyJsPlugin.js:44:10)
    at Compilation.applyPluginsAsync (/home/khoa/web/phenomic/node_modules/tapable/lib/Tapable.js:71:13)
    at Compilation.<anonymous> (/home/khoa/web/phenomic/node_modules/webpack/lib/Compilation.js:567:9)
    at Compilation.next (/home/khoa/web/phenomic/node_modules/tapable/lib/Tapable.js:67:11)
    at ExtractTextPlugin.<anonymous> (/home/khoa/web/phenomic/docs/node_modules/extract-text-webpack-plugin/index.js:303:4)
    at Compilation.applyPluginsAsync (/home/khoa/web/phenomic/node_modules/tapable/lib/Tapable.js:71:13)
    at Compilation.<anonymous> (/home/khoa/web/phenomic/node_modules/webpack/lib/Compilation.js:563:8)
    at Compilation.next (/home/khoa/web/phenomic/node_modules/tapable/lib/Tapable.js:67:11) +2s

If you remove UglifyJSPlugin from the plugin stack (by removing the last commit), the error goes away, everything works at expected (as for as I know) and THE BUILD TIME SPEEDS UP 200% (20sec vs 40sec)

[MoOx]

Doing "cd docs && npm install" is incorrect. According to our specific case (package using deps in a folder above itself), we need the some symlink. Just do "npm i && npm run docs" to get the correct setup to run docs properly.

[thangngoc89]

@MoOx edited my comment above. Does it seem correct to you ?

[MoOx]

Yep! :)

[MoOx]

Closing as it's pretty nice in current master branch.