Angular v10 - Excessive memory consumption at build

[qortex]

Since upgrading to Angular 10, the build process takes an excessive amount of RAM, making it impossible to deploy on Heroku (build fails with OOM error).

For the same code, just upgraded:

• Angular 9 memory consumption at prod build: ~2 to 2.5 Gb.
• Angular 10 memory consumption at prod build: ~8 to 9 Gb.

Heroku limit is 2.5 Gb, so it fails with Angular 10.

On my local computer, with no such memory limit, the build works ok.

🐞 bug report

Is this a regression?

Yes

Description

The memory spike seems to occur after ngcc has finished its pre-compilation steps es2015 to esm2015 and starts actual compilation of the code.

Angular Version:

     _                      _                 ____ _     ___
    / \   _ __   __ _ _   _| | __ _ _ __     / ___| |   |_ _|
   / △ \ | '_ \ / _` | | | | |/ _` | '__|   | |   | |    | |
  / ___ \| | | | (_| | |_| | | (_| | |      | |___| |___ | |
 /_/   \_\_| |_|\__, |\__,_|_|\__,_|_|       \____|_____|___|
                |___/
    

Angular CLI: 10.0.0
Node: 14.4.0
OS: linux x64

Angular: 10.0.1
... animations, cdk, common, compiler, compiler-cli, core, forms
... language-service, localize, material, platform-browser
... platform-browser-dynamic, router
Ivy Workspace: Yes

Package                           Version
-----------------------------------------------------------
@angular-devkit/architect         0.1000.0
@angular-devkit/build-angular     0.1000.0
@angular-devkit/build-optimizer   0.1000.0
@angular-devkit/build-webpack     0.1000.0
@angular-devkit/core              10.0.0
@angular-devkit/schematics        10.0.0
@angular/cli                      10.0.0
@angular/flex-layout              10.0.0-beta.32
@ngtools/webpack                  10.0.0
@schematics/angular               10.0.0
@schematics/update                0.1000.0
rxjs                              6.5.5
typescript                        3.9.5
webpack                           4.43.0

[charanseeram]

please unsubscribe me

…

On Tue, Jun 30, 2020 at 1:00 PM Michael Monerau ***@***.***> wrote: Since upgrading to Angular 10, the build process takes an excessive amount of RAM, making it impossible to deploy on Heroku (build faits with OOM error). For the same code, just upgraded: - Angular 9 memory consumption at prod build: *~2 to 2.5 Gb*. - Angular 10 memory consumption at prod build: *~8 to 9 Gb*. Heroku limit is 2.5 Gb, so it fails with Angular 10. On my local computer, with no such memory limit, the build works ok. 🐞 bug report Is this a regression? Yes Description The memory spike seems to occur after ngcc has finished its pre-compilation steps es2015 to esm2015 and starts actual compilation of the code. *Angular Version:* _ _ ____ _ ___ / \ _ __ __ _ _ _| | __ _ _ __ / ___| | |_ _| / △ \ | '_ \ / _` | | | | |/ _` | '__| | | | | | | / ___ \| | | | (_| | |_| | | (_| | | | |___| |___ | | /_/ \_\_| |_|\__, |\__,_|_|\__,_|_| \____|_____|___| |___/ Angular CLI: 10.0.0 Node: 14.4.0 OS: linux x64 Angular: 10.0.1 ... animations, cdk, common, compiler, compiler-cli, core, forms ... language-service, localize, material, platform-browser ... platform-browser-dynamic, router Ivy Workspace: Yes Package Version ----------------------------------------------------------- @angular-devkit/architect 0.1000.0 @angular-devkit/build-angular 0.1000.0 @angular-devkit/build-optimizer 0.1000.0 @angular-devkit/build-webpack 0.1000.0 @angular-devkit/core 10.0.0 @angular-devkit/schematics 10.0.0 @angular/cli 10.0.0 @angular/flex-layout 10.0.0-beta.32 @ngtools/webpack 10.0.0 @schematics/angular 10.0.0 @schematics/update 0.1000.0 rxjs 6.5.5 typescript 3.9.5 webpack 4.43.0 — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <https://github.com/angular/angular/issues/37838>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AESUZ7MPMWDHGSIESBQMRB3RZGIABANCNFSM4OL6ACSA> .

[JoostK]

There was a memory leak in v10 with incremental compilations which we have since tracked down and fixed (see #18034) but yours appears to be a single compilation. As such, I suspect it's a different cause but we'd need more information to be able to investigate.

[qortex]

Ah yes.
What kind of information would be useful? I'm not familiar with debugging memory issues in a javascript setting.

Fyi, I use a custom builder to pre-compile *.graphql files, but I don't think updating to angular v10 would change much on that side. Is there a way to check (hard to disable completely since I need the files to be pre-compiled)?

[JoostK]

I can have a look if you are able to share a repo (privately is okay). Otherwise, the easiest way to debug memory issues is to attach a debugger (such as ndb) to get insight into the memory consumption of the processes, from which you can also create heaps dumps. For very large heaps it may be problematic to inspect the dumps, as it takes quite a bit of memory. From a single heap dump it's not always clear whether there's an actual leak (as it's just a single snapshot and spotting leaks may require monitoring over time) but it should already give quite a bit of insight.

[qortex]

Thanks that's awesome, just invited you to the private repo. Let's follow up the discussion there.

[JoostK]

Thank you, I will have a look later today.

[JoostK]

Alright, I found the culprit., and it was not what I would have ever expected.

With the migration to Angular 10, the .browserlistrc was changed in a way where differential loading is no longer enabled (as none of the browsers that require ES5 are included in the browsers list). When differential loading is enabled, the CLI uses a pipeline that works a bit differently in certain areas, one of which is how it deals with asset files. With differential loading disabled, copying assets is done using the copy-webpack-plugin which is currently affected by a bug that's been fixed in webpack-contrib/copy-webpack-plugin#507, but that's not yet been released. Without the bug fix, the plugin is invoked for each Webpack compilation, of which there are 121 in the reproduction (I don't know why it's that many). Each of those loads ~58MB of asset files into memory, accounting for some 7 GB of memory usage 😱

I applied the patch in webpack-contrib/copy-webpack-plugin#507 locally and it reduces memory usage by 120 times 58 MB, bringing it down to the levels pre-upgrade.

Given that there's no copy-webpack-plugin release yet, the simplest workaround for now would be to revert the changes to the .browserlistrc file to regain 7GB of RAM. And yes, I also find that hard to believe!

Angular 9 is also affected here since #17867 (released 3 weeks ago in CLI 9.1.8), as the upgrade to v6 of copy-webpack-plugin introduces the regression. It's only because the browserlist configuration triggered differential loading that it didn't result in memory issues since 9.1.8.