Improve performance of hashRegExp lookup

[ryanwilsonperkin]

Fixes #16758
For applications with a very large number of assets, the cost of invoking a single regular expression with many many values in a group becomes very high.

By changing to a list of regular expressions (with helper methods for maintaining the original design) we can get a large performance improvement.

A simple way to view the performance difference between these two approaches is with the following short scripts:

// benchmark-regex-group.js
const crypto = require('node:crypto');

function makeRandomString(length) {
  return crypto.randomBytes(length).toString('hex');
}

const name = makeRandomString(40);
const keys = Array.from(Array(3000), () => makeRandomString(20));
const regex = new RegExp(keys.join("|"), "g");

name.replace(regex, found => 'replaced');

const crypto = require('node:crypto');

function makeRandomString(length) {
  return crypto.randomBytes(length).toString('hex');
}

const name = makeRandomString(40);
const keys = Array.from(Array(3000), () => makeRandomString(20));
const regexes = keys.map(key => new RegExp(key, "g"));

let result = name;
for (const regex of regexes) {
  result = result.replace(regex, found => 'replaced');
}

Benchmarking these against each other I get the following results:

❯ hyperfine 'node benchmark-regex-many.js' 'node benchmark-regex-group.js'
Benchmark 1: node benchmark-regex-many.js
  Time (mean ± σ):      87.2 ms ± 143.6 ms    [User: 39.5 ms, System: 4.9 ms]
  Range (min … max):    39.3 ms … 495.8 ms    10 runs

Benchmark 2: node benchmark-regex-group.js
  Time (mean ± σ):     18.669 s ±  0.101 s    [User: 18.564 s, System: 0.054 s]
  Range (min … max):   18.556 s … 18.840 s    10 runs

Summary
  'node benchmark-regex-many.js' ran
  214.02 ± 352.29 times faster than 'node benchmark-regex-group.js'

What kind of change does this PR introduce?

Bugfix, performance

Did you add tests for your changes?

No, given that this is a performance improvement I'm not sure that the test suite is setup to test the difference.

Does this PR introduce a breaking change?

No, hashes remain the same

What needs to be documented once your changes are merged?
N/A

[webpack-bot]

For maintainers only:

• This needs to be documented (issue in webpack/webpack.js.org will be filed when merged)
• This needs to be backported to webpack 4 (issue will be created when merged)

[ryanwilsonperkin]

One thing that I was unsure of was whether or not a given asset could be expected to have multiple asset hashes in it's filename. To be defensive against that case, I've designed hashReplace and hashMatch to always iterate through the entire array of regular expressions, but if there's a known invariant that only one can match then we can speed this up even more by breaking on the first match/replacement.

[devvsakib]

LGTM

[TheLarkInn]

@alexander-akait when you are back and available will you look at this?

[TheLarkInn]

Re-running failed jobs which look to be flakey and then auto-merge.

[ryanwilsonperkin]

@TheLarkInn looks like one last test still flaky: https://github.com/webpack/webpack/actions/runs/4266358978/jobs/7635349344

[lvivski]

@ryanwilsonperkin thanks for this PR! I was thinking of a similar change, while working on #16491, and was very excited to see you making it :)
Unfortunately, at least for our scenario (~2M LOC, 10k hashes to replace per runtime, 5+ runtimes, 5000 output js files) this change has degraded performance quite a bit. Previously our builds would spend around 600 seconds in RealContentHash, and 90s after #16491, now they timeout.

Even generating multiple RegExp's is slower than having a single huge one for us. Do you see the same improvements, you've mentioned in PR, after upgrading to the latest version? I'm trying to understand why it would be faster for your scenario, which sounds relatively similar to ours in terms of size and complexity, but much slower for us.

[ryanwilsonperkin]

Hey @lvivski thanks for bringing it to my attention. Could you include some information about the format of the hashes that you're replacing and the number and size of assets that you're replacing them in? And are you able to isolate the timeout to this part of the code?

[lvivski]

@ryanwilsonperkin

Could you include some information about the format of the hashes

we're using xxxhash64 8 chars.

the number and size of assets that you're replacing them in

~5 runtime assets
~10k hashes to replace. 5k from content hash and 5k through the hook with webpack-subresource-integrity. I've also tried disabling SRI and it didn't help.

And are you able to isolate the timeout to this part of the code?

Yes, after creating a custom webpack build with this change reverted, it works fine.

I have just ran your stress-test build from https://github.com/ryanwilsonperkin/webpack-16758

CPU: 2.4 GHz 8-Core Intel Core i9
RAM: 64 GB 2667 MHz DDR4
OS: Mac OS 13.2.1
NodeJS: v19.7.0

v5.76.0

Run 1: 50487 ms asset processing > RealContentHashPlugin
Run 2: 51002 ms asset processing > RealContentHashPlugin
Run 3: 51650 ms asset processing > RealContentHashPlugin

v5.76.0 w/ changes from this PR reverted

Run 1: 9812 ms asset processing > RealContentHashPlugin
Run 2: 10482 ms asset processing > RealContentHashPlugin
Run 3: 10761 ms asset processing > RealContentHashPlugin

Around 5x decrease in processing time if I revert this change.

[ryanwilsonperkin]

I'm getting very different results than you on that stress test build:

CPU: Apple M1 Pro
RAM: 16 GB
OS: Mac OS 13.2.1
NodeJS: v19.4.0

v5.75.0

Run 1: 46028 ms asset processing > RealContentHashPlugin
Run 2: 46286 ms asset processing > RealContentHashPlugin
Run 3: 45108 ms asset processing > RealContentHashPlugin

v5.76.0

Run 1: 25504 ms asset processing > RealContentHashPlugin
Run 2: 26187 ms asset processing > RealContentHashPlugin
Run 3: 27294 ms asset processing > RealContentHashPlugin

Your machine appears to be more powerful, but it seems odd to me that when running the test from https://github.com/ryanwilsonperkin/webpack-16758 that your results would be twice as slow against 5.76.0 but four times as fast against 5.75.0.

the number and size of assets that you're replacing them in

I should clarify this further, I meant the size (in bytes) of the assets that you are replacing the hashes in, and how many of those assets you're processing.

[lvivski]

Sorry, I should have clarified that I wasn't running 5.7_6_ against 5.7_5_ on your test setup . I was comparing 5.7_6_ and then manually removing changes from this PR, but keeping all other changes from 5.7_6_.

I wonder if this is M1 vs Intel difference. Do you use any CI? If so, do you also see similar improvement numbers there?

I meant the size (in bytes) of the assets that you are replacing the hashes in

This is generally hard to tell, ranging between several Kb to low hundreds, with about 150Mb generated JS total.

But yeah, I think that running multiple small RexExp's is less efficient than a single big one, in our case, because of the total number and size of the files. While initialization time might be longer for a big regex, subsequent comparisons are faster with only a single pass through the each file/string.

When I was investigating perf for #16491 I wasn't able to create a test setup that would be able to re-create the same level of memory pressure, that our production build has.

[lvivski]

I've created a setup with random strings and maybe there will be a difference on M1 because I'm seeing the same 5x change. https://gist.github.com/lvivski/b3245a494be6a862b63dc4e00012eb1b

v5.7*5*.0

Run 1: 79448 ms asset processing > RealContentHashPlugin
Run 2: 80579 ms asset processing > RealContentHashPlugin

v5.7*6*.0

Run 1: 399699 ms asset processing > RealContentHashPlugin
Run 2: 418393 ms asset processing > RealContentHashPlugin

v5.7*6*.0 w/ this PR reverted

Run 1: 77584 ms asset processing > RealContentHashPlugin
Run 2: 78269 ms asset processing > RealContentHashPlugin

So it seems that the root cause is this change.

[vankop]

@alexander-akait @TheLarkInn could we test this one more time.. problem here is that if you have small amount of chunks (not a 3000 as in @ryanwilsonperkin example 😀 ) => it is fine to concat them + matching on huge chunk content would be better. So what this changes does in reality, if you have 90kb chunk and 20 refs ( hashes ) => webpack creates 20 regexps and run them 20 times on 90kb chunk.

---

maybe if you have vice versa setup => a lot of small chunks, this give some performance benefits.

---

Also I guess this is very Node.js performance related thing, so it could really depends on Node.js version

[ryanwilsonperkin]

I've got access to a powerful Intel-based machine through work and can confirm what you're seeing:

CPU: Intel Xeon (32) @ 1.999GHz
RAM: ~200Gb
OS: Ubuntu 22.04.2 LTS x86_64
Node: v19.7.0

5.75.0

Run 1: 15341 ms asset processing > RealContentHashPlugin
Run 2: 15118 ms asset processing > RealContentHashPlugin
Run 3: 16097 ms asset processing > RealContentHashPlugin



5.76.0

Run 1: 70324 ms asset processing > RealContentHashPlugin
Run 2: 67705 ms asset processing > RealContentHashPlugin
Run 3: 70182 ms asset processing > RealContentHashPlugin

So perhaps this comes down to architecture set or processor speed. I don't think that webpack should focus on optimizing for M1s at the expense of other architectures since they're more likely to be the ones performing these large builds, so I'd agree that this should be considered a regression and reverted.

#16803