[v1.0.0-beta.2] parallelization isn't working

[stevefan1999-personal]

Well I was hacking next.js and tried to play with the new toy in town

I attempted to remove the old uglifyJs from webpack and installed the beta version back into the plugins config in wepback:

removeUglifyJsPlugin () {
  // create a clone of config.plugins without UglifyJsPlugin
  delete this.config.plugins.filter(p => p.constructor.name === 'UglifyJsPlugin')[0]
  this.config.plugins = this.config.plugins.filter(p => p.constructor.name !== 'UglifyJsPlugin')

  return this
}

addUglifyEsPlugin (opts) {
  const plugin = new UglifyJSPlugin({
    parallel: true,
    sourceMap: opts.sourceMap,
    uglifyOptions: { // UglifyJS config for every workers
      ...opts, 
      ecma: 5
    }
  })

  this.config.plugins.push(plugin)
  return this
}

So then I built several times

$ next build
> Using external babel configuration
> Location: "/mnt/d/Seafile/power-next/.babelrc"
> Using "webpack" config function defined in next.config.js.
Done in 23.60s.

$ next build
> Using external babel configuration
> Location: "/mnt/d/Seafile/power-next/.babelrc"
> Using "webpack" config function defined in next.config.js.
Done in 24.89s.

$ next build
> Using external babel configuration
> Location: "/mnt/d/Seafile/power-next/.babelrc"
> Using "webpack" config function defined in next.config.js.
Done in 21.28s.

~22s, this is pretty slow...
I can validate uglify-es is working by checking the output(which is originally ES6+)

So I decided to turn off parallelization:

addUglifyEsPlugin (opts) {
  const plugin = new UglifyJSPlugin({
    parallel: false,
    sourceMap: opts.sourceMap,
    uglifyOptions: { // UglifyJS config for every workers
      ...opts, 
      ecma: 5
    }
  })

  this.config.plugins.push(plugin)
  return this
}

Shockingly the build time is almost the same with parallel option off!

$ next build
> Using external babel configuration
> Location: "/mnt/d/Seafile/power-next/.babelrc"
> Using "webpack" config function defined in next.config.js.
Done in 22.37s.

$ next build
> Using external babel configuration
> Location: "/mnt/d/Seafile/power-next/.babelrc"
> Using "webpack" config function defined in next.config.js.
Done in 22.63s.

$ next build
> Using external babel configuration
> Location: "/mnt/d/Seafile/power-next/.babelrc"
> Using "webpack" config function defined in next.config.js.
Done in 23.42s.

What. The. Hell. I never realized there's no parallelization!
But even without parallelization there the cache mechanism should have left in effect. I turned back on the parallel option.
Strangely the cache folder is created:

steve@~/power-next$ cd node_modules/.cache/
steve@~/power-next/node_modules/.cache$ ls
babel-loader  uglifyjs-webpack-plugin
steve@~/power-next/node_modules/.cache$

But it doesn't turn the game up whatsoever...Even after post-cache-build...

Interesting, but what in the world just happened?

I have a beast 16-cores Xeon E5 high performance workstation and it definitely kills parallel jobs!

steve@~/power-next/node_modules/.cache$ cat /proc/cpuinfo
processor       : 0
vendor_id       : GenuineIntel
cpu family      : 6
model           : 45
model name      :        Intel(R) Xeon(R) CPU E5-2660 0 @ 2.20GHz
stepping        : 7
microcode       : 0xffffffff
cpu MHz         : 2201.000
cache size      : 256 KB
physical id     : 0
siblings        : 16
core id         : 0
cpu cores       : 8
apicid          : 0
initial apicid  : 0
fpu             : yes
fpu_exception   : yes
cpuid level     : 6
wp              : yes
flags           : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm pni pclmulqdq dtes64 monitor ds_cpl vmx s
mx est tm2 ssse3 cx16 xtpr pdcm pcid dca sse4_1 sse4_2 x2apic popcnt tsc_deadline_timer aes xsave osxsave avx
bogomips        : 4402.00
clflush size    : 64
cache_alignment : 64
address sizes   : 36 bits physical, 48 bits virtual
power management:

processor       : 15
vendor_id       : GenuineIntel
cpu family      : 6
model           : 45
model name      :        Intel(R) Xeon(R) CPU E5-2660 0 @ 2.20GHz
stepping        : 7
microcode       : 0xffffffff
cpu MHz         : 2201.000
cache size      : 256 KB
physical id     : 0
siblings        : 16
core id         : 0
cpu cores       : 8
apicid          : 0
initial apicid  : 0
fpu             : yes
fpu_exception   : yes
cpuid level     : 6
wp              : yes
flags           : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm pni pclmulqdq dtes64 monitor ds_cpl vmx s
mx est tm2 ssse3 cx16 xtpr pdcm pcid dca sse4_1 sse4_2 x2apic popcnt tsc_deadline_timer aes xsave osxsave avx
bogomips        : 4402.00
clflush size    : 64
cache_alignment : 64
address sizes   : 36 bits physical, 48 bits virtual
power management:

And I don't think I have wrongfully configured UglifyJSPlugin...

I ran fine with webpack-parallel-uglify-plugin and it really speeds up by a factor of 3_(~22s -> ~8s, close to raw compilation time without any minification)_...Not in this case though.

You may say why shouldn't I adapt to the webpack-parallel-uglify-plugin I used to but because I'm preparing for the future.

I can simply switch back, but it definitely kills my intent...

WELL ANYWAY PLZ FIX

I'm sorry if my wordings are kinda ambiguous, but English is not my mother tongue, heh

[stevefan1999-personal]

"uglifyjs-webpack-plugin": "v1.0.0-beta.2",

in package.json if you're interested

[stevefan1999-personal]

So I think maybe the worker farm configuration was off?
I checked my process list and did find multiple Node processes with worker-farm/lib/child/index.js during building phase, but only one of them is burning with high CPU usage(~13%), the same behavior with parallelization off, and the other processes are pretty much idling in ~0.5%, where as webpack-parallel-uglify-plugin they are amortized. Hmm...

[ahumblenerd]

I have also observed something similar. It is creating two workers and one of them is peaking and the other one is idling.

The package @stevefan1999 suggested did provide me performance boosts as well.

[michael-ciniawsky]

@stevefan1999 @ahumblenerd I need your more info about your setups please (webpack.config.js), webpack wrappers (Next.js, Angular CLI) etc.

[stevefan1999-personal]

@michael-ciniawsky Sorry for responding lately, I have other works to do.

Here it is my config (actually next.config.js, not webpack.config.js):

import UglifyJSPlugin from 'uglifyjs-webpack-plugin'

export default {
  getUglifyJsPluginOptions (config) {
    return config.plugins.find(p => p.constructor.name === 'UglifyJsPlugin').options // there's one and only UglifyJsPlugin
  },
  removeUglifyJsPlugin (config) {
    // create a clone of config.plugins without UglifyJsPlugin
    delete config.plugins.filter(p => p.constructor.name === 'UglifyJsPlugin')[0]
    config.plugins = config.plugins.filter(p => p.constructor.name !== 'UglifyJsPlugin')
  },
  addUglifyEsPlugin (config, opts) {
    const plugin = new UglifyJSPlugin({
      parallel: true,
      sourceMap: opts.sourceMap,
      uglifyOptions: { // UglifyJS config for every workers
        ...opts,
        ecma: 5
      }
    })

    config.plugins.push(plugin)
  },

  webpack (config, { dev }) {
    if (!dev) {
      const opts = this.getUglifyJsPluginOptions(config)
      this.removeUglifyJsPlugin(config)
      this.addUglifyEsPlugin(config, opts)
    }

    return config
  }
}

I didn't see any issues there

[alexander-akait]

@stevefan1999 found problems when ending #108 PR, parallelization just not works, before merge this PR i recommend disable parallelization, thanks!

[michael-ciniawsky]

@stevefan1999 Could you try the latest beta uglifyjs-webpack-plugin@1.0.0-beta.3 and test if this fixes the issue ?

[mdstaff]

I'm seeing an error when passing an object to parallel:

    parallel: {
        cache: true,
        workers: 2,
    },

$
...
> webpack-dev-server --env.dev --progress --color --watch

Running: webpack.config.js
about to uglify
Exception { [Error: Validation Error

UglifyJs Plugin Invalid Options

options.parallel should be boolean
options.parallel should be integer
options.parallel should match exactly one schema in oneOf

[michael-ciniawsky]

@mdstaff #141 (comment)

[zbjornson]

1.0.0-beta.3 shows the same behavior that stevefan1999 described with one process doing 99% of the work, for me.

[alexander-akait]

@zbjornson can you create minimum reproducible test repo with latest beta version?