yarn add netlify-cli fails with "JavaScript heap out of memory" #3788

coelmay · 2021-12-10T04:10:15Z

Describe the bug

FATAL ERROR: Ineffective mark-compacts near heap limit Allocation failed - JavaScript heap out of memory occurs when adding netlify-cli with yarn.

Originally identified on the forums https://answers.netlify.com/t/error-adding-netlify-cli-with-yarn/48579

To Reproduce

Steps to reproduce the behavior:

Run yarn add netlify-cli

Configuration

% npx envinfo --system --binaries --npmPackages netlify-cli --npmGlobalPackages netlify-cli

  System:
    OS: macOS 12.0.1
    CPU: (12) x64 Intel(R) Core(TM) i5-10600 CPU @ 3.30GHz
    Memory: 1.76 GB / 16.00 GB
    Shell: 5.8 - /bin/zsh
  Binaries:
    Node: 16.13.1 - /usr/local/bin/node
    Yarn: 1.22.17 - /usr/local/bin/yarn
    npm: 8.1.2 - /usr/local/bin/npm
  npmPackages:
    netlify-cli: ^8.0.19 => 8.0.19
  npmGlobalPackages:
    netlify-cli: 8.0.19

Expected behavior

Install to not error

CLI Output

% yarn add netlify-cli
yarn add v1.22.17
info No lockfile found.
[1/4] 🔍  Resolving packages...
warning netlify-cli > @netlify/plugin-edge-handlers > rollup-plugin-node-polyfills > rollup-plugin-inject@3.0.2: This package has been deprecated and is no longer maintained. Please use @rollup/plugin-inject.
warning netlify-cli > @netlify/zip-it-and-ship-it > @vercel/nft > node-pre-gyp@0.13.0: Please upgrade to @mapbox/node-pre-gyp: the non-scoped node-pre-gyp package is deprecated and only the @mapbox scoped package will recieve updates in the future
warning netlify-cli > @netlify/zip-it-and-ship-it > precinct > detective-postcss > postcss-values-parser > flatten@1.0.3: flatten is deprecated in favor of utility frameworks such as lodash.
warning netlify-cli > copy-template-dir > readdirp > micromatch > snapdragon > source-map-resolve > resolve-url@0.2.1: https://github.com/lydell/resolve-url#deprecated
warning netlify-cli > copy-template-dir > readdirp > micromatch > snapdragon > source-map-resolve > urix@0.1.0: Please see https://github.com/lydell/urix#deprecated
[2/4] 🚚  Fetching packages...
[3/4] 🔗  Linking dependencies...
warning "netlify-cli > @netlify/build > ts-node@10.4.0" has unmet peer dependency "@types/node@*".
warning "netlify-cli > @netlify/build > ts-node@10.4.0" has unmet peer dependency "typescript@>=2.7".

<--- Last few GCs --->

[2325:0x7fe856200000]    99973 ms: Scavenge 4013.0 (4122.8) -> 4006.9 (4123.8) MB, 21.9 / 0.0 ms  (average mu = 0.126, current mu = 0.054) allocation failure
[2325:0x7fe856200000]   100005 ms: Scavenge 4016.2 (4123.8) -> 4010.8 (4128.8) MB, 8.5 / 0.0 ms  (average mu = 0.126, current mu = 0.054) allocation failure
[2325:0x7fe856200000]   100694 ms: Mark-sweep 4021.5 (4128.8) -> 4011.8 (4135.6) MB, 666.7 / 0.0 ms  (average mu = 0.185, current mu = 0.231) allocation failure scavenge might not succeed


<--- JS stacktrace --->

FATAL ERROR: Ineffective mark-compacts near heap limit Allocation failed - JavaScript heap out of memory
 1: 0x10e767225 node::Abort() [/usr/local/bin/node]
 2: 0x10e7673a8 node::OnFatalError(char const*, char const*) [/usr/local/bin/node]
 3: 0x10e8de9c7 v8::Utils::ReportOOMFailure(v8::internal::Isolate*, char const*, bool) [/usr/local/bin/node]
 4: 0x10e8de963 v8::internal::V8::FatalProcessOutOfMemory(v8::internal::Isolate*, char const*, bool) [/usr/local/bin/node]
 5: 0x10ea7cfe5 v8::internal::Heap::FatalProcessOutOfMemory(char const*) [/usr/local/bin/node]
 6: 0x10ea8100b v8::internal::Heap::RecomputeLimits(v8::internal::GarbageCollector) [/usr/local/bin/node]
 7: 0x10ea7d8ec v8::internal::Heap::PerformGarbageCollection(v8::internal::GarbageCollector, v8::GCCallbackFlags) [/usr/local/bin/node]
 8: 0x10ea7ad9a v8::internal::Heap::CollectGarbage(v8::internal::AllocationSpace, v8::internal::GarbageCollectionReason, v8::GCCallbackFlags) [/usr/local/bin/node]
 9: 0x10ea880f0 v8::internal::Heap::AllocateRawWithLightRetrySlowPath(int, v8::internal::AllocationType, v8::internal::AllocationOrigin, v8::internal::AllocationAlignment) [/usr/local/bin/node]
10: 0x10ea88171 v8::internal::Heap::AllocateRawWithRetryOrFailSlowPath(int, v8::internal::AllocationType, v8::internal::AllocationOrigin, v8::internal::AllocationAlignment) [/usr/local/bin/node]
11: 0x10ea550c7 v8::internal::Factory::NewFillerObject(int, bool, v8::internal::AllocationType, v8::internal::AllocationOrigin) [/usr/local/bin/node]
12: 0x10ee010ae v8::internal::Runtime_AllocateInYoungGeneration(int, unsigned long*, v8::internal::Isolate*) [/usr/local/bin/node]
13: 0x10f1a30b9 Builtins_CEntry_Return1_DontSaveFPRegs_ArgvOnStack_NoBuiltinExit [/usr/local/bin/node]
14: 0x10f1b9056 Builtins_ArrayPrototypeJoin [/usr/local/bin/node]
zsh: abort      yarn add netlify-cli

Additional context

Works with npm add netlify-cli

The text was updated successfully, but these errors were encountered:

overlordofmu · 2021-12-10T06:29:38Z

This is currently being escalated as it is causing deployment failures for builds which require netlify-cli in many CI/CD pipelines (including GitHub Actions and Netlify itself). We will an update about this very soon.

overlordofmu · 2021-12-10T07:44:12Z

It turns out that this is an issue in yarn itself:

yarnpkg/yarn#8734

I have tested in an Ubuntu VM with 4 GiB of memory and I can consistently reproduce this error with yarn install. I used the same package JSON for both yarn and npm and all tests used the node version in this report - 16.13.1.

Yarn always returns the JavaScript heap out of memory error while npm install is successful. This happens even with a netlify-cli release from eight days ago (version 8.0.10).

To summarize, this appears to be a yarn bug and not a netlify-cli bug.

overlordofmu · 2021-12-10T07:59:31Z

I tried rolling back to older yarn versions like 1.22.5 and I still get the JavaScript heap out of memory error. Meanwhile npm install is working for the same netlify-cli and node versions. This looks like a regression in some yarn dependency and not in yarn itself based on that behavior.

The only workaround I can find is to use npm and not yarn.

knupska · 2021-12-10T08:32:15Z

@overlordofmu I found the same as you when I investigated. Even using versions of netlify-cli and yarn up to one year old still resulted in the same error. The version of node used 12, 14, 16 also had no impact.

Installing @netlify/build directly with yarn doesn't encounter the same issue.

kirillgroshkov · 2021-12-10T10:51:58Z

I wonder if we can trace and find the exact change/sub-dependency of netlify-cli that is causing yarn to fail now?

erezrokah · 2021-12-10T10:59:33Z

Hello all 👋 We're actively looking into this. The current workaround is using npm, and if you're using a global install this shouldn't be a big change.

@kirillgroshkov this is happening on very old yarn + CLI versions too, and with other packages (see yarnpkg/yarn#8734), so not sure this was caused by a CLI change.

erezrokah · 2021-12-10T11:23:30Z

You can also track this via our status page https://www.netlifystatus.com/.

We're currently trying to figure out if there's a workaround that doesn't involve people changing a package manager

erezrokah · 2021-12-10T12:02:41Z

Hi all 👋 For those who are installing the CLI locally, can you try adding the following to your package.json:

"resolutions": {
    "@oclif/plugin-help": "3.2.14"
  },

This will pin one of the CLI's dependencies when installing with yarn and should work around the issue.
We're working on publishing a new version of netlify-cli with this fix as well.

knupska · 2021-12-10T12:14:05Z

@erezrokah I've tested locally with an empty folder and fresh package.json.

I confirm that the install completes successfully with the provided resolution, (and that it still fails if I remove it).

erezrokah · 2021-12-10T12:22:48Z

Thank you for confirming @knupska ❤️

erezrokah · 2021-12-10T13:26:02Z

We're working on publishing a new version of netlify-cli with this fix as well.

The timeline for this is next week, as we're already very close to removing oclif from the codebase, see #3717

RodEsp · 2021-12-10T16:05:35Z

FYI we are tracking this at the oclif level here: oclif/oclif#758

RodEsp · 2021-12-10T17:32:47Z

There is a temporary fix out there for this now, please see oclif/oclif#758 (comment) for more details.

erezrokah · 2021-12-10T17:36:49Z

Thanks for the follow up @RodEsp! I can confirm the installation goes through now with yarn add netlify-cli.

Closing this issue. Please comment if this issue has not been resolved for you.

stevepepple · 2021-12-10T19:19:21Z

Hi all 👋 For those who are installing the CLI locally, can you try adding the following to your package.json:
"resolutions": {
    "@oclif/plugin-help": "3.2.14"
  },
This will pin one of the CLI's dependencies when installing with yarn and should work around the issue. We're working on publishing a new version of netlify-cli with this fix as well.

I can also confirm that this fixes the issue for our site on Netlify. It's odd because we've been using the same Yarn version for months without hitting this issue.

erezrokah · 2021-12-13T11:31:43Z

It's odd because we've been using the same Yarn version for months without hitting this issue.

The underlying problem is with yarn not handling circular dependencies well, see issue.
This doesn't seem to be a new issue with yarn, however a change in oclif triggered the yarn bug.
Since Node.js packages usually use a range to reference other dependencies, e.g. ^1.0.0 this means that yarn tries to install the latest minor/patch version automatically, hence the issue was triggered without a change on users' side.

It doesn't happen with the npm package manager since:

It might not have the same issue with circular dependencies
Netlify CLI uses a shrinkwrap file , which is a mechanism provided by npm to allow package authors pin dependencies of the published package.

ndr47 · 2022-01-09T10:06:34Z

Even with

"resolutions": {
    "@oclif/plugin-help": "3.2.14"
  },

in my package.json I still get the error.
Here is the output:

OS:

NAME="elementary OS"
VERSION="5.1.7 Hera"
ID=elementary
ID_LIKE=ubuntu
PRETTY_NAME="elementary OS 5.1.7 Hera"
LOGO=distributor-logo
VERSION_ID="5.1.7"

Yarn 1.22.17
Node v16.13.1

nbl7 · 2022-01-09T11:44:04Z

yarn cache clean

then add to package.json

"resolutions": {
    "@oclif/plugin-help": "3.2.14"
  },

then

yarn add --dev netlify-cli

AidanDaniel97 · 2022-01-10T09:52:25Z

Even with

"resolutions": {
    "@oclif/plugin-help": "3.2.14"
  },

in my package.json I still get the error. Here is the output:

OS:

NAME="elementary OS"
VERSION="5.1.7 Hera"
ID=elementary
ID_LIKE=ubuntu
PRETTY_NAME="elementary OS 5.1.7 Hera"
LOGO=distributor-logo
VERSION_ID="5.1.7"

Yarn 1.22.17
Node v16.13.1

Looks like this is related
#3981 (comment)

coelmay added the type: bug code to address defects in shipped code label Dec 10, 2021

zmwangx mentioned this issue Dec 10, 2021

yarn add netlify-cli fails with "JavaScript heap out of memory" #3789

Closed

erezrokah mentioned this issue Dec 10, 2021

yarn command fail，prompt Reached heap limit Allocation failed - JavaScript heap out of memory。 yarnpkg/yarn#8734

Open

erezrokah changed the title ~~FATAL ERROR: Ineffective mark-compacts near heap limit Allocation failed - JavaScript heap out of memory~~ yarn add netlify-cli fails with "JavaScript heap out of memory" Dec 10, 2021

erezrokah mentioned this issue Dec 10, 2021

fix: tweak imports to break cycle oclif/plugin-help#348

Merged

lukasholzer mentioned this issue Dec 10, 2021

fix: remove oclif framework and fixes an issue with sites:create #3717

Merged

53 tasks

peternhale mentioned this issue Dec 10, 2021

fix: recursive installation failure due to dep cycle oclif/command#301

Merged

ascorbic mentioned this issue Dec 10, 2021

[Bug]: getPageResolver generating incorrect paths on Windows netlify/next-runtime#941

Closed

2 tasks

erezrokah closed this as completed Dec 10, 2021

henrikq mentioned this issue Jan 10, 2022

Fail to install netlify cli action netlify/actions#54

Closed

ashkanahrabi mentioned this issue May 11, 2022

js heap dump on Netlify while using mapbox-gl #4602

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

yarn add netlify-cli fails with "JavaScript heap out of memory" #3788

yarn add netlify-cli fails with "JavaScript heap out of memory" #3788

coelmay commented Dec 10, 2021

overlordofmu commented Dec 10, 2021

overlordofmu commented Dec 10, 2021

overlordofmu commented Dec 10, 2021 •

edited

knupska commented Dec 10, 2021 •

edited

kirillgroshkov commented Dec 10, 2021

erezrokah commented Dec 10, 2021

erezrokah commented Dec 10, 2021 •

edited

erezrokah commented Dec 10, 2021 •

edited

knupska commented Dec 10, 2021

erezrokah commented Dec 10, 2021

erezrokah commented Dec 10, 2021

RodEsp commented Dec 10, 2021

RodEsp commented Dec 10, 2021

erezrokah commented Dec 10, 2021

stevepepple commented Dec 10, 2021

erezrokah commented Dec 13, 2021

ndr47 commented Jan 9, 2022 •

edited

nbl7 commented Jan 9, 2022

AidanDaniel97 commented Jan 10, 2022

yarn add netlify-cli fails with "JavaScript heap out of memory" #3788

yarn add netlify-cli fails with "JavaScript heap out of memory" #3788

Comments

coelmay commented Dec 10, 2021

overlordofmu commented Dec 10, 2021

overlordofmu commented Dec 10, 2021

overlordofmu commented Dec 10, 2021 • edited

knupska commented Dec 10, 2021 • edited

kirillgroshkov commented Dec 10, 2021

erezrokah commented Dec 10, 2021

erezrokah commented Dec 10, 2021 • edited

erezrokah commented Dec 10, 2021 • edited

knupska commented Dec 10, 2021

erezrokah commented Dec 10, 2021

erezrokah commented Dec 10, 2021

RodEsp commented Dec 10, 2021

RodEsp commented Dec 10, 2021

erezrokah commented Dec 10, 2021

stevepepple commented Dec 10, 2021

erezrokah commented Dec 13, 2021

ndr47 commented Jan 9, 2022 • edited

nbl7 commented Jan 9, 2022

AidanDaniel97 commented Jan 10, 2022

overlordofmu commented Dec 10, 2021 •

edited

knupska commented Dec 10, 2021 •

edited

erezrokah commented Dec 10, 2021 •

edited

erezrokah commented Dec 10, 2021 •

edited

ndr47 commented Jan 9, 2022 •

edited