`ng build` for production on ng16 (webpack builder) has significantly higher memory pressure

[jpike88]

Command

build

Is this a regression?

• Yes, this behavior used to work in the previous version

The previous version in which this bug was not present was

15.1.5 (typescript version 4.9.5)

Description

After upgrading to ng16.0.0-RC0 (typescript 5.0.3), I've noticed a much higher load on my build runners, to the point of an out of memory error.

I run with NODE_OPTIONS="--max-old-space-size=8192" at all times.

Originally, builds would work fine on
buildjet-4vcpu-ubuntu-2204-arm, but with 16-RC0, I had to try buildjet-8vcpu-ubuntu-2204-arm, then buildjet-16vcpu-ubuntu-2204-arm until the memory crash didn't occur.

Below are the stats, to show where I started and where I ended up.
vCPU RAM YAML runner tag
4 vCPU 6 GB buildjet-4vcpu-ubuntu-2204-arm
8 vCPU 12 GB buildjet-8vcpu-ubuntu-2204-arm
16 vCPU 24 GB buildjet-16vcpu-ubuntu-2204-arm

So I went from just 6GB to needing somewhere between 12GB-24GB on my runners to not have an out of memory error. This cannot be easily explained by pointing at memory possibly used by other services.

Attached are two cpu profiles, named appropriately. These were taken on an M1 Max with 32GB RAM. Obviously the pipeline seemed to be quite different, so it's hard to tell where the memory bloat might be happening.

cpuProfiles.zip

Minimal Reproduction

Do a production build (including optimizations etc) with 15.1.5, then try with 16-RC0, compare memory usage

Exception or Error

No response

Your Environment

n/a because comparing two seperate versions of ng and typescript

Anything else relevant?

No response

[alan-agius4]

Hi @jpike88, can you please provide memory snapshots which can be generated using the Chrome dev tool?

[jpike88]

heap snapshot links below:
https://drive.google.com/file/d/1-gvzK62UnaRTiVxM84l78_aZwuMyv12v/view?usp=sharing
https://drive.google.com/file/d/1GGTCwCkn8zebaHUfVwR7X2iDbSXRbHCO/view?usp=sharing
https://drive.google.com/file/d/1HjzbaUhSko4jfJtd4GSWcNc7JagABLir/view?usp=sharing
https://drive.google.com/file/d/1_U-kqfky5bFz2IQaf1Dywbevh8Mh4pMF/view?usp=sharing
https://drive.google.com/file/d/1qV9rhLZDdCc8jsO9fK1kZfz5uB06uJX1/view?usp=sharing

[jpike88]

More profiling of the runner itself. ng build process begins at 05:21:45 and terminates near the end of the graph. Memory usage of the process rises to 2GB... then appears to spike, on it's way to 5GB it seems.

[jpike88]

Memory profile of a much larger runner (The one that we're running with for now). Build begins just after the 05:40 mark, so that means the total build used over 6GB of memory... that's pretty excessive...

[alan-agius4]

@jpike88, 6Gb usage in this case might be because of max-old-space-size=8192 setting, which will cause V8 to run the garbage collector less frequent as the process memory usage is far from approaching the configured heap limit.

Can you remove the usage of max-old-space-size and see if this is still a problem? My understanding is that max-old-space-size should no longer be needed since version Node.js 16+.

[jpike88]

Minor improvement, looks like peak of just over 4gb. Still too much for the runner we were using with ng15 however, which is weird because the runner says it has 6 GB... ah well I guess this will have to do.