Using new NG_PERSISTENT_BUILD_CACHE results in js heap memory error

[yharaskrik]

Bug Report

Affected Package

The issue is caused by package @angular/....

Probably @angular/compiler-cli, @angular/compiler and/or @angular/cli

Is this a regression?

Yes, the previous version in which this bug was not present was: ....

Not present in the same version when not using the flag

Description

A clear and concise description of the problem...

When enabling the NG_PERSISTENT_BUILD_CACHE flag with optimization, aot and buildOptimizer enabled on a large project:

[9613:0xf60cd90]   390146 ms: Mark-sweep 4057.7 (4126.0) -> 4048.0 (4129.2) MB, 2318.9 / 0.1 ms  (average mu = 0.099, current mu = 0.007) allocation failure scavenge might not succeed
[9613:0xf60cd90]   392529 ms: Mark-sweep 4060.6 (4129.2) -> 4050.7 (4132.5) MB, 2365.1 / 0.1 ms  (average mu = 0.055, current mu = 0.008) allocation failure scavenge might not succeed


<--- JS stacktrace --->

==== JS stack trace =========================================

    0: ExitFrame [pc: 0x1389a0c]
    1: StubFrame [pc: 0x1369280]
Security context: 0xffff941408d1 <JSObject>
    2: split [0xffff9414cd51](this=0xfffe7b9c1d41 <String[54]: /node_modules/node-vibrant/node_modules/lodash/_Set.js>,0xfffe7b9c1da1 <JSRegExp <String[#3]: \/+>>)
    3: /* anonymous */ [0xfffde63be589] [/home/circleci/project/repo/node_modules/@*******-devkit/build-*******/node_modules/webpack/node_modules/source-map/lib/source-...

FATAL ERROR: Ineffective mark-compacts near heap limit Allocation failed - JavaScript heap out of memory
 1: 0xa146c0 node::Abort() [/opt/circleci/.nvm/versions/node/v12.20.2/bin/node]
 2: 0xa14a80 node::OnFatalError(char const*, char const*) [/opt/circleci/.nvm/versions/node/v12.20.2/bin/node]
 3: 0xb7d4f4 v8::Utils::ReportOOMFailure(v8::internal::Isolate*, char const*, bool) [/opt/circleci/.nvm/versions/node/v12.20.2/bin/node]
 4: 0xb7d6f4 v8::internal::V8::FatalProcessOutOfMemory(v8::internal::Isolate*, char const*, bool) [/opt/circleci/.nvm/versions/node/v12.20.2/bin/node]
 5: 0xd284a8 v8::internal::Heap::EnsureFromSpaceIsCommitted() [/opt/circleci/.nvm/versions/node/v12.20.2/bin/node]
 6: 0xd289b0 v8::internal::Heap::RecomputeLimits(v8::internal::GarbageCollector) [/opt/circleci/.nvm/versions/node/v12.20.2/bin/node]
 7: 0xd355c0 v8::internal::Heap::PerformGarbageCollection(v8::internal::GarbageCollector, v8::GCCallbackFlags) [/opt/circleci/.nvm/versions/node/v12.20.2/bin/node]
 8: 0xd36314 v8::internal::Heap::CollectGarbage(v8::internal::AllocationSpace, v8::internal::GarbageCollectionReason, v8::GCCallbackFlags) [/opt/circleci/.nvm/versions/node/v12.20.2/bin/node]
 9: 0xd38884 v8::internal::Heap::AllocateRawWithRetryOrFail(int, v8::internal::AllocationType, v8::internal::AllocationOrigin, v8::internal::AllocationAlignment) [/opt/circleci/.nvm/versions/node/v12.20.2/bin/node]
10: 0xd00e30 v8::internal::Factory::NewFillerObject(int, bool, v8::internal::AllocationType, v8::internal::AllocationOrigin) [/opt/circleci/.nvm/versions/node/v12.20.2/bin/node]
11: 0x1019e30 v8::internal::Runtime_AllocateInYoungGeneration(int, unsigned long*, v8::internal::Isolate*) [/opt/circleci/.nvm/versions/node/v12.20.2/bin/node]
12: 0x1389a0c  [/opt/circleci/.nvm/versions/node/v12.20.2/bin/node]

This does not happen when building the same project with the same configuration without the flag

Your Environment

Angular Version:

Angular CLI: 12.1.0
Node: 12.22.1
Package Manager: yarn 1.22.10
OS: darwin x64

Angular: 12.1.0
... animations, cdk, cli, common, compiler, compiler-cli, core
... forms, language-service, material, platform-browser
... platform-browser-dynamic, platform-server, router
... service-worker, youtube-player

Package                         Version
---------------------------------------------------------
@angular-devkit/architect       0.1201.0
@angular-devkit/build-angular   12.1.0
@angular-devkit/core            12.1.0
@angular-devkit/schematics      12.0.2
@angular/fire                   6.1.4
@angular/flex-layout            12.0.0-beta.34
@nguniversal/builders           12.0.1
@nguniversal/common             12.0.1
@nguniversal/express-engine     12.0.1
@schematics/angular             12.1.0
ng-packagr                      12.0.2
rxjs                            6.6.3
typescript                      4.3.4
webpack                         4.46.0

Anything else relevant?

ARM, Ubuntu on CircleCI

[alan-agius4]

@yharaskrik, it is expected that when caching is enabled the memory usage is increased.

What's the heap memory usage when caching is disabled versus when it is enabled?

Also, typically you don't want to enable caching on CI, since the result will be stored in node_modules which will not or typically shouldn't be persisted between runs. Since typically you'd have a workflow where you cache the package managers cache and re-run npm ci/i / yarn install on every run.

[yharaskrik]

Hey Alan,

Not quite sure how to measure the heap usage but I have the max node space size set to 4096 as a flag when running the command. So it's more than that with caching and less otherwise.

We use an Nx repo with multiple apps being built in parallel (I reproduced this by running a single app though as well) and thought that the persistent cache may be useful as they all share a lot of code, thus can all benefit from a shared persistent cache (from the same CI run of course) was this an incorrect assumption?

[alan-agius4]

Not quite sure how to measure the heap usage but I have the max node space size set to 4096 as a flag when running the command. So it's more than that with caching and less otherwise.
It might be the case that you were on the limit before. Considering that you set node space size set to 4096 before, it's very likely that you will need to increase this when caching is enabled. To measure the heap you can use the below.

• node --inspect-brk node_modules/@angular/cli/lib/init.js b --configuration production
• Open Chrome and type chrome://inspect in the address bar
• Click on the inspect button from one of your applications in the Remote Target section.
• Go to Chrome DevTools > Memory
• Take a number of heap snapshots during the build.

We use an Nx repo with multiple apps being built in parallel (I reproduced this by running a single app though as well) and thought that the persistent cache may be useful as they all share a lot of code, thus can all benefit from a shared persistent cache (from the same CI run of course) was this an incorrect assumption?

Unless you re-run the same exact build/test suit/e2e suit (IE: same inputs and same options) you will not have any performance improvements from using NG_PERSISTENT_BUILD_CACHE. Nb: NG_PERSISTENT_BUILD_CACHE only effects the Angular CLI builders that use Webpack under the hood.

[pavankjadda]

Also, typically you don't want to enable caching on CI, since the result will be stored in node_modules which will not or typically shouldn't be persisted between runs. Since typically you'd have a workflow where you cache the package managers cache and re-run npm ci/i / yarn install on every run.

@alan-agius4 what's the typical use case for this? as you said we don't persist node_modules between runs in Jenkins and typically use npm ci. Is this for local serve or build?

NG_PERSISTENT_BUILD_CACHE=1 ng serve --watch --hmr

or

NG_PERSISTENT_BUILD_CACHE=1 ng build

[alan-agius4]

Typically this cache is more beneficial in local environments during development.

[alan-agius4]

Closing since increase in memory usage is expected and memory profiles have not been provided to investigate this any further.

[angular-automatic-lock-bot]

This issue has been automatically locked due to inactivity.
Please file a new issue if you are encountering a similar or related problem.

Read more about our automatic conversation locking policy.

_{This action has been performed automatically by a bot.}