ng build causes out of memory

[vicatcu]

🐞 Bug report

Command (mark with an x)

• new
• build
• serve
• test
• e2e
• generate
• add
• update
• lint
• extract-i18n
• run
• config
• help
• version
• doc

Is this a regression?

Yes, this started happening when I migrated our server from Angular 9 to Angular 11, and from Node 10 to Node 14.

Description

When I run ng build on my server, during the Generating browser application bundles (phase: building)... message, eventually I get an out of memory crash. This actually happens on my server, which has 2GB or RAM, as well as on my development machine, which has 16GB of RAM.

🔥 Exception or Error

<--- Last few GCs --->

[23183:0x6323910]   114992 ms: Scavenge (reduce) 2029.0 (2046.3) -> 2028.2 (2047.5) MB, 5.6 / 0.0 ms  (average mu = 0.259, current mu = 0.247) allocation failure 
[23183:0x6323910]   115058 ms: Scavenge (reduce) 2029.3 (2046.8) -> 2028.5 (2047.8) MB, 5.2 / 0.0 ms  (average mu = 0.259, current mu = 0.247) allocation failure 
[23183:0x6323910]   115100 ms: Scavenge (reduce) 2029.6 (2046.8) -> 2028.8 (2047.8) MB, 5.1 / 0.0 ms  (average mu = 0.259, current mu = 0.247) allocation failure 


<--- JS stacktrace --->

FATAL ERROR: Ineffective mark-compacts near heap limit Allocation failed - JavaScript heap out of memory
 1: 0xa03530 node::Abort() [ng build]
 2: 0x94e471 node::FatalError(char const*, char const*) [ng build]
 3: 0xb7773e v8::Utils::ReportOOMFailure(v8::internal::Isolate*, char const*, bool) [ng build]
 4: 0xb77ab7 v8::internal::V8::FatalProcessOutOfMemory(v8::internal::Isolate*, char const*, bool) [ng build]
 5: 0xd32345  [ng build]
 6: 0xd32ecf  [ng build]
 7: 0xd40f5b v8::internal::Heap::CollectGarbage(v8::internal::AllocationSpace, v8::internal::GarbageCollectionReason, v8::GCCallbackFlags) [ng build]
 8: 0xd44b1c v8::internal::Heap::AllocateRawWithRetryOrFailSlowPath(int, v8::internal::AllocationType, v8::internal::AllocationOrigin, v8::internal::AllocationAlignment) [ng build]
 9: 0xd131fb v8::internal::Factory::NewFillerObject(int, bool, v8::internal::AllocationType, v8::internal::AllocationOrigin) [ng build]
10: 0x10592f0 v8::internal::Runtime_AllocateInOldGeneration(int, unsigned long*, v8::internal::Isolate*) [ng build]
11: 0x13ff179  [ng build]
Aborted

🌍 Your Environment

Angular CLI: 11.0.2
Node: 14.15.1
OS: linux x64

Angular: 11.0.2
... animations, cli, common, compiler, compiler-cli, core, forms
... language-service, platform-browser, platform-browser-dynamic
... router
Ivy Workspace: Yes

Package                         Version
---------------------------------------------------------
@angular-devkit/architect       0.1100.2
@angular-devkit/build-angular   0.1100.2
@angular-devkit/core            11.0.2
@angular-devkit/schematics      11.0.2
@angular/cdk                    11.0.1
@angular/material               11.0.1
@schematics/angular             9.1.7
@schematics/update              0.1100.2
rxjs                            6.5.3
typescript                      4.0.5

Anything else relevant?
Possibly related to #18398, but I'm not sure how to confirm. I don't think I'm using i18n but I think I am targeting es5.

It doesn't crash if I do node --max_old_space_size=4096 ./node_modules/@angular/cli/bin/ng build

[alan-agius4]

Hi @vicatcu,

I looks like this issue is not related to #18398 since the OOM error is occurring during the building phase. unfortunately, without either a reproduction or sharing the problematic project (even privately) it would be hard for us to investigate what is causing the increase in memory usage in your project.

[vicatcu]

I would be happy to share the project privately @alan-agius4

[alan-agius4]

Got the invite for the private repo, thanks

[alan-agius4]

Hi @vicatcu,

I tried to had a look at the reproduction you provided however I am unable to start it due to compilation errors.

When I run ng build I get the below

Error: node_modules/vega-lite/build/src/index.d.ts:1:25 - error TS2732: Cannot find module '../package.json'. Consider using '--resolveJsonModule' to import module with '.json' extension

1 import { version } from '../package.json';
                          ~~~~~~~~~~~~~~~~~
src/app/core/server-api.service.ts:7:24 - error TS2307: Cannot find module '../../../../epi-env' or its corresponding type declarations.

7 import * as __env from '../../../../epi-env';
                         ~~~~~~~~~~~~~~~~~~~~~
src/app/shared/html-conversion/html-conversion.service.ts:6:24 - error TS2307: Cannot find module '../../../../../epi-env' or its corresponding type declarations.

6 import * as __env from '../../../../../epi-env';
                         ~~~~~~~~~~~~~~~~~~~~~~~~
src/app/shared/vega/vega.component.ts:228:53 - error TS2345: Argument of type 'import("/node_modules/vega-typings/types/spec/config").Config'.
  Type 'Config<SignalRef | ExprRef>' is not assignable to type 'Config'.
                                                        ~~~~~

[vicatcu]

@alan-agius4 right, sorry I forgot you need a file just above the project root called epi-env.js and it can simply contain: module.exports='dev'; and that should suffice to compile after an npm install. Happy to zoom with you or something if that helps, just shoot me an email to victor.aprea@wickeddevice.com.

[alan-agius4]

So I started looking at this today and I did a couple of things;

1. I went back to the branch and downgraded to use version 9, that did build without an OOM and using /usr/bin/time -l ng build I saw a peek memory usage of 1 552.22835 megabytes
2. I updated to version 10 and that started showing OOM errors at a peek of 2 435.44064 megabytes, one of the biggest changes in v10 was the removal of FESM 5 from NPM packages, and downlevelling of node_modules to ES5 which now happens during application build using Babel, the new pipeline will try to downlevel all node packages. This should indeed consume more memory but not as much.
3. So I changed the tsconfig target to es2015 from es5 and added a .browserslistrc under the root folder containing last 1 Chrome version to disable differential loading and that did build without OOM with a peek memory usage of 1 778.09818 megabytes.
4. It seems that the usage of @grapecity/spread-sheets is causing the problem, this package is huge to say the least, it is 2.56 MB minified, and when trying to downlevel this package the minified code will need to be transformed to AST which causes drastic increase in memory usage.

Side notes you should also consider:

• Create a .browserslistrc and specify which browsers versions you want to support. see: https://github.com/angular/angular-cli/blob/master/packages/schematics/angular/application/files/.browserslistrc.template. This ensures that the right JavaScript and CSS versions are generated based on the supported browsers.
• Consider enabling differential loading so that you don't penalize users not using IE11 by slower and biggest ES5 bundles.

At this point I think using --max_old_space_size is the sensible workaround and solution. Adding large packages as a dependency of the application will inevitable add the requirement of needing more memory and other resources on the server when building the application.

[alan-agius4]

Another observations, removing all files from the compilation and adding import '@grapecity/spread-sheets'; in main.ts, will cause the build peek memory to be 1 722.81856 megabytes removing this import will bring down the peek memory usage to 394.768384 megabytes.

[vicatcu]

@alan-agius4 thank you so much for looking into this and for your suggestions. I'll look into adding browserlistrc later today. I'm not sure I understand the last comment, as I can't remove '@grapecity/spread-sheets' as a dependency of my project. You are just saying it is implicated on its own as contributing a huge increase in build memory demand?

As an aside it's the a way to pass --max_old_space_size to ng directly instead of invoking ng with node as I did in the OP?

Also just read https://angular.io/guide/deployment re: differential loading and found it illuminating, thanks for the heads up!