Build times very slow with NX 12

[jon-zu]

Current Behavior

Since the upgrade to NX 12.0.6 the build on my dev machine(3700x, 32gb Ram) takes ~30 minutes. It took like 2 minute max. When I run with --progress=true It gets stuck while showing '30% building x/n modules 50 active'. I noticed that only one core will be utilized, while the rest stays idle and the memory usage reached the default threshold(2 GB) and then drops to ~700 MB multiple times. Once the cache is build the build times become fast again.

Expected Behavior

Build time should be on par with 11.4.

Steps to Reproduce

I'm unable to do this right now. Our project:

• uses SCSS
• has one .ts file with 5 MB(which contains our auto-generated api services)

When I edit this file and retry to compile It seems It will take ages to build again. So It definitely seems to be a problem with some babel/webpack plugin and larger code files.

Failure Logs

Environment

yarn run v1.22.5$ nx report

>  NX  Report complete - copy this into the issue template

  Node : 14.16.0
  OS   : linux x64
  yarn : 1.22.5
  
  nx : Not Found
  @nrwl/angular : Not Found
  @nrwl/cli : 12.0.6
  @nrwl/cypress : 12.0.6
  @nrwl/devkit : 12.0.6
  @nrwl/eslint-plugin-nx : 12.0.6
  @nrwl/express : Not Found
  @nrwl/jest : 12.0.6
  @nrwl/linter : 12.0.6
  @nrwl/nest : Not Found
  @nrwl/next : Not Found
  @nrwl/node : 12.0.6
  @nrwl/react : 12.0.6
  @nrwl/schematics : Not Found
  @nrwl/tao : 12.0.6
  @nrwl/web : 12.0.6
  @nrwl/workspace : 12.0.6
  @nrwl/storybook : 12.0.6
  @nrwl/gatsby : Not Found
  typescript : 4.1.5

[kenptr]

I also noticed that nx adds 3 terser minimizers:

[
  TerserPlugin {
    options: {
      test: /\.m?js(\?.*)?$/i,
      chunkFilter: [Function: chunkFilter],
      warningsFilter: [Function: warningsFilter],
      extractComments: true,
      sourceMap: true,
      cache: true,
      cacheKeys: [Function: cacheKeys],
      parallel: true,
      include: undefined,
      exclude: undefined,
      minify: undefined,
      terserOptions: [Object]
    }
  },
  HashedModuleIdsPlugin {
    options: {
      context: null,
      hashFunction: 'md4',
      hashDigest: 'base64',
      hashDigestLength: 4
    }
  },
  BundleBudgetPlugin { options: { budgets: [Array] } },
  CleanCssWebpackPlugin {
    _options: { sourceMap: true, test: [Function: test] }
  },
  TerserPlugin {
    options: {
      test: /\.m?js(\?.*)?$/i,
      chunkFilter: [Function: chunkFilter],
      warningsFilter: [Function: warningsFilter],
      extractComments: true,
      sourceMap: true,
      cache: true,
      cacheKeys: [Function: cacheKeys],
      parallel: true,
      include: undefined,
      exclude: undefined,
      minify: undefined,
      terserOptions: [Object]
    }
  },
  TerserPlugin {
    options: {
      test: /\.m?js(\?.*)?$/i,
      chunkFilter: [Function: chunkFilter],
      warningsFilter: [Function: warningsFilter],
      extractComments: true,
      sourceMap: true,
      cache: true,
      cacheKeys: [Function: cacheKeys],
      parallel: true,
      include: undefined,
      exclude: undefined,
      minify: undefined,
      terserOptions: [Object]
    }
  }
]

I got it by adding following to ./tools/webpack/myreact.js and using it in workspace.json/"webpackConfig": "./tools/webpack/myreact.js".

const nxWebpack = require("@nrwl/react/plugins/webpack");
const fs = require("fs");

module.exports = (config, context) => {
  const newConfig = nxWebpack(config, context);
  console.log("newConfig.plugins");
  fs.writeFileSync("/tmp/a.json", JSON.stringify(newConfig));
  console.log(newConfig.optimization.minimizer)
  return newConfig;
};

[kenptr]

I did a small experiment. Looks like 2 extra terser plugins reduce final bundle size from 2.0M to 1.9M, but add extra 188s (5x times slow).

Packages: react, antd, firebase, apollo, and some more

Command: time nx run admin:build:production --verbose --skip-nx-cache

Custom webpack configuration, optimization=true, 2 terser plugins are removed

Time: 45.76s

❯ ls -lah dist/apps/admin
total 22960
drwxr-xr-x   5 ken  staff   160B Apr 18 22:25 assets
-rw-r--r--   1 ken  staff   593B Apr 18 22:25 index.html
-rw-r--r--   1 ken  staff   2.0M Apr 18 22:25 main.1db5a5f5072b44caec84.esm.js
-rw-r--r--   1 ken  staff   7.8M Apr 18 22:25 main.1db5a5f5072b44caec84.esm.js.map
-rw-r--r--   1 ken  staff    79K Apr 18 22:25 polyfills.4c7aec88bcb295b780aa.esm.js
-rw-r--r--   1 ken  staff   389K Apr 18 22:25 polyfills.4c7aec88bcb295b780aa.esm.js.map
-rw-r--r--   1 ken  staff   1.5K Apr 18 22:25 runtime.7b550c4ecb2a593f9f1c.js
-rw-r--r--   1 ken  staff   8.1K Apr 18 22:25 runtime.7b550c4ecb2a593f9f1c.js.map
-rw-r--r--   1 ken  staff   489K Apr 18 22:25 styles.11dfca914847fa168d29.css
-rw-r--r--   1 ken  staff   177K Apr 18 22:25 styles.11dfca914847fa168d29.css.map

Default webpack configuration, optimization=true, @nrwl/react/plugins/webpack

Time: 233.86s

❯ ls -lah dist/apps/admin
total 22784
drwxr-xr-x   5 ken  staff   160B Apr 18 22:30 assets
-rw-r--r--   1 ken  staff   593B Apr 18 22:30 index.html
-rw-r--r--   1 ken  staff   1.9M Apr 18 22:30 main.7138825ca38755728d13.esm.js
-rw-r--r--   1 ken  staff   7.7M Apr 18 22:30 main.7138825ca38755728d13.esm.js.map
-rw-r--r--   1 ken  staff    79K Apr 18 22:30 polyfills.af4b9a1dcbf09abb404f.esm.js
-rw-r--r--   1 ken  staff   388K Apr 18 22:30 polyfills.af4b9a1dcbf09abb404f.esm.js.map
-rw-r--r--   1 ken  staff   1.5K Apr 18 22:30 runtime.365178a7a0f8f038a0f1.js
-rw-r--r--   1 ken  staff   8.0K Apr 18 22:30 runtime.365178a7a0f8f038a0f1.js.map
-rw-r--r--   1 ken  staff   489K Apr 18 22:30 styles.11dfca914847fa168d29.css
-rw-r--r--   1 ken  staff   177K Apr 18 22:30 styles.11dfca914847fa168d29.css.map

[jon-zu]

So I did some more digging with the SpeedMeasurePlugin and It seems sass-loader is the main culprit for us:

General output time took 27 mins, 5.91 secs

 SMP  ⏱  Plugins
TerserPlugin took 1 min, 24.13 secs
LicenseWebpackPlugin took 1.12 secs
MiniCssExtractPlugin took 0.352 secs
ForkTsCheckerWebpackPlugin took 0.297 secs
(unable to deduce plugin name) took 0.237 secs
HashedModuleIdsPlugin took 0.048 secs
CleanCssWebpackPlugin took 0.034 secs
BundleBudgetPlugin took 0.027 secs
CopyPlugin took 0.023 secs
SuppressExtractedTextChunksWebpackPlugin took 0.01 secs
ContextReplacementPlugin took 0.008 secs
DefinePlugin took 0.007 secs

 SMP  ⏱  Loaders
@nrwl/web, and 
postcss-loader, and 
sass-loader took 25 mins, 37.11 secs
  module count = 27
modules with no loaders took 25 mins, 30.98 secs
  module count = 1065
@nrwl/web took 25 mins, 29.89 secs
  module count = 186
@nrwl/web, and 
postcss-loader took 7.76 secs
style-loader, and 
@nrwl/web, and 
postcss-loader took 0.017 secs
  module count = 12

Any ideas how to fix this? What was changed on the scss part on version 12?

[HaveF]

Off topic.
hi, @JonasZ95 , could you tell me how to use SpeedMeasurePlugin in nx? Just in webpack.config.js? add line

const smp = new SpeedMeasurePlugin()

I cannot make it works. Thanks!

[joshwiens]

There is a comment early in the referenced angular/cli thread about the terser plugin, it's lack of file system caching using webpack v5 as well as performance hits in sass.

The performance issues you are having have to do with cli v12 and the issues mentioned above, this isn't a problem that NX can solve. So you either have to go back to Angular 11.x and wait or contend with the performance degradation for the time being unfortunately.

[jon-zu]

@HaveF as kenptr pointed out you set the webpack config path in your workspace.json. Then you can use something like this:

const nxWebpack = require('@nrwl/react/plugins/webpack');
const SpeedMeasurePlugin = require('speed-measure-webpack-plugin');
const webpack = require('webpack');

const logConfig = false;
const profileWebpack = false;
const measureSpeed = true;

module.exports = (config, context) => {
  let newConfig = nxWebpack(config, context);

  if(measureSpeed) {
    const smp = new SpeedMeasurePlugin();
    newConfig = smp.wrap(newConfig);
  }

  if(profileWebpack) {
    const profilePlugin = new webpack.debug.ProfilingPlugin();
    newConfig.plugins.push(profilePlugin)
  }

  if(logConfig)
    console.log(JSON.stringify(newConfig.module, null, 4));

  return newConfig;
};

With profiling I found the error actually It seems that 'babel-plugin-transform-typescript-metadata' is the problem here and takes 30 minutes to detect that my 5mb client.ts file is too large to process. I disabled 'emitDecoratorMetadata', which is enabled by default, for now which works as a workaround.

Is there an option to use NX without Babel? Because I would prefer to keep out as much build tools written in pure JS as possible, because in general I think they are low quality and hard to debug.

[ghost]

any news regarding this one?

[soer7022]

Anyone have any ideas, I'm experiencing 1hour+ build times

[jon-zu]

@soer7022 I don't think we can solve this problem. I think as of now nx needs better ways to debug and profile the build process, because we just need to accept the fact that a lot of the JS build tools are kinda awful IMO.
I suggest we add options to create a profile via the webpack ProfilingPlugin and use the SpeedMeasurePlugin to get a better overview about the build times.

[github-actions]

This issue has been automatically marked as stale because it hasn't had any recent activity. It will be closed in 14 days if no further activity occurs.
If we missed this issue please reply to keep it active.
Thanks for being a part of the Nx community! 🙏

[github-actions]

This issue has been closed for more than 30 days. If this issue is still occuring, please open a new issue with more recent context.