Incremental builds are slow

[csvan]

• I understand that GitHub issues are not for tech support, but for questions specific to this generator, bug reports, and feature requests.

Item	Version
generator-angular-fullstack	4.1
Node	6.9.1
npm	3.10.8
Operating System	Windows 7 Enterprise

Item	Answer
Transpiler	TypeScript
Markup	HTML
CSS	LESS
Router	ui-router
Client Tests	Jasmine
DB	None
Auth	N

While I understand that incremental build time (e.g. during gulp serve) is not fixed, it seems to scale pretty poorly with project size.

In our case, we work on a mid-sized project (approx 40k+ lines JS, 2k+ LESS, not including deps). Changing a source triggers an incremental builds which takes 5-10 seconds, and really hits the CPU hard (I have a Dell M4800, never had the fans run so high before).

I am not a webpack guru, but is there no way to tweak the chunk system to make this more efficient? Our modules are relatively small (about 500-1k JS each). Right now it feels as if the entire code base is being recompiled rather than the part which changed.

[csvan]

Relevant: webpack/webpack#1574

[Azayzel]

Looks like we'd have to go with HMR. Webpack does a Hot Module Replacement but, it requires code to be inserted seemingly in every new component.

[csvan]

FWIW, I managed to reduce our incremental build times drastically (5-10s down to about 1-2) with the following:

1. Force latest version of postcss-discard-duplicates (npm install --save-dev postcss-discard-duplicates). There was a severe performance regression which was fixed just a day ago (writing this), and cssnano does not seem to bundle it yet.
2. Set config.devtool = 'inline-eval-cheap-source-map';. This alone shaved a lot of time off builds, though I am not sure if it negatively affects the maps themselves (have not seen any problems while debugging yet).
3. For the Babel loader, pass cacheDirectory=true i.e. loader: 'babel-loader?cacheDirectory=true'
4. Add babel-preset-node6 and babel-plugin-transform-runtime. NPM install both and then make sure your .babelrc looks like:

{
  "presets": [
    "es2015",
    "node6"
  ],
  "plugins": [
    "transform-class-properties",
    "transform-runtime"
  ]
}

1. Use HappyPack (https://github.com/amireh/happypack) for parallel buils. For example, for the Babel loader:

let HappyPack = require('happypack');

config.plugins = [
    ..........
    new HappyPack({
      id: 'babel',
      loaders: ['babel-loader?cacheDirectory=true']
    })
    ..........
  ];

config.module = {
   .............
   loaders: [
      {
        // JS LOADER
        // Reference: https://github.com/babel/babel-loader
        // Transpile .js files using babel-loader
        // Compiles ES6 and ES7 into ES5 code
        test: /\.js$/,
        loader: 'happypack/loader?id=babel',
        include: [
          path.resolve(__dirname, 'client/'),
          path.resolve(__dirname, 'node_modules/lodash-es/')
        ]
      }
      .............
  }

[csvan]

@Azayzel HMR seems like the way to go. Despite the improvements above (mostly JS-specific anyway) LESS rebuilds still remain a significant bottleneck. Mostly, this seems to be due to bootstrap.less being recompiled every time a reload is triggered.

It also seems like ExtractTextPlugin may have something to do with it. webpack/less-loader#88

[Azayzel]

I'll start looking into it. Still unsure if that code snippet needs to live in everything or just the components. I'd assume everything to pick up on CSS changes and such.

[csvan]

@Azayzel I started looking into it as well, but it is pretty daunting stuff.

https://webpack.github.io/docs/hot-module-replacement.html

[csvan]

Meanwhile, for people having trouble with LESS loading, I found that including the CSS of bootstrap/font-awesome rather than the LESS shaves a good 2-3 seconds off incremental builds:

@import (css) '../../node_modules/font-awesome/css/font-awesome.css';
@import (css)'../../node_modules/bootstrap/dist/css/bootstrap.css';

It seems like the less-loader does not do any caching, so both libs are completely recompiled during incremental builds otherwise.

If you want to do this but need some of the bootstrap LESS variables/mixins in your own styles, you can import them directly, e.g.

@import (css)'../../node_modules/bootstrap/dist/css/bootstrap.css';
@import '../../node_modules/bootstrap/less/variables';

There certainly is some bloat involved in this, but I am not sure just how much.

[stephengardner]

@csvan thansk for writing that up.

I have an 11 second build right now, and trying to speed it up... So I have done just about everything you mentioned.

postcss-discard-duplicates -- my build wasn't built depending on that nor css-nano so that must be specific to your build (perhaps adding less? -- I added sass)

babel-loader and cacheDirectory=true have been put in place. FWIW I'm using typescript on the server side, not babel. But this loader is still present in my build.

I have set my babelrc to be as suggested

I added happypack for what I could. Babel, and my Styles. (there is an open PR due to it not being usable with awesome-typescript-loader.

Still taking 11 seconds to build (haha), perhaps 400ms improvement.

[stephengardner]

Update on my trial by fire:
On a Brand New build of Yo Angular Fullstack, I used Webpack DLLs and all the other suggested methods and my build time is ~3 seconds. Time To Refresh is about 6 seconds due to everything else.

Version: webpack 1.13.3
Time: 3077ms
               Asset     Size  Chunks             Chunk Names
       app.bundle.js   674 kB       0  [emitted]  app
../client/index.html  1.31 kB          [emitted]

On a fresh build you can even completely remove the babel loader, so I have.

I've spent hours on this and am baffled at how it takes this long. (~6 seconds total to refresh after updating any file in the /server directory)

Any thoughts appreciated

[csvan]

@stephengardner did you try changing the sourcemap settings? I have found that the default (config.devtool = 'source-map';) seems to add 2-3 seconds to the builds all by itself.

[stephengardner]

@csvan certainly have. I've changed that to what you specified, inline-eval-cheap-source-map.

I've literally spent an all nighter trying to figure this out, to no avail.

If you spin up a new yo angular-fullstack and get even that to build in ~2 seconds, I will be extremely curious how. It's been a tough friday night haha

Also -
I've gotten my client-side refreshes extremely fast!
This is, again, when I make any edit to any server side code, this triggers nodemon to re-start and all turtles break loose. Obviously this is all how the generator works, nothing I did.

[Azayzel]

This article on HMR is worth noting

[stephengardner]

This is a problem that isn't fixed. And HMR -- while great -- is not a viable solution simply because the angular-fullstack framework is not set up properly. It rebuilds on every server change.

I posted a reasonable solution on this isue, which involves slightly altering the way angular-fullstack is set up.