Repo: After many website rebuilds, "FATAL ERROR: Reached heap limit Allocation failed - JavaScript heap out of memory"

[JoshuaKGoldberg]

Suggestion

Moving facebook/docusaurus#8142 over to this repo as it seems to be specific to our tooling.

Consistently when developing on the website (yarn start), after a dozen or two rebuilds, the JavaScript heap runs out of memory:

✔ Client: Compiled successfully in 110.32ms
client (webpack 5.74.0) compiled successfully
ℹ Compiling Client
✔ Client: Compiled successfully in 166.16ms
client (webpack 5.74.0) compiled successfully
ℹ Compiling Client

<--- Last few GCs --->

[95379:0x130048000]  1517409 ms: Mark-sweep (reduce) 4074.8 (4103.0) -> 4012.5 (4039.3) MB, 11.8 / 0.0 ms  (+ 32.0 ms in 1 steps since start of marking, biggest step 32.0 ms, walltime since start of marking 120 ms) (average mu = 0.946, current mu = 0.965)[95379:0x130048000]  1517589 ms: Mark-sweep (reduce) 4158.7 (4184.5) -> 4158.6 (4184.7) MB, 104.5 / 0.0 ms  (average mu = 0.856, current mu = 0.419) allocation failure; scavenge might not succeed


<--- JS stacktrace --->

FATAL ERROR: Reached heap limit Allocation failed - JavaScript heap out of memory
 1: 0x1005b7c60 node::Abort() [/Users/josh/.nvm/versions/node/v18.7.0/bin/node]
 2: 0x1005b7de8 node::ModifyCodeGenerationFromStrings(v8::Local<v8::Context>, v8::Local<v8::Value>, bool) [/Users/josh/.nvm/versions/node/v18.7.0/bin/node]
...
82: 0x1053b108c 
error Command failed with signal "SIGABRT".
info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.

Full stack trace here

 1: 0x1005b7c60 node::Abort() [/Users/josh/.nvm/versions/node/v18.7.0/bin/node]
 2: 0x1005b7de8 node::ModifyCodeGenerationFromStrings(v8::Local<v8::Context>, v8::Local<v8::Value>, bool) [/Users/josh/.nvm/versions/node/v18.7.0/bin/node]
 3: 0x1006fa350 v8::internal::V8::FatalProcessOutOfMemory(v8::internal::Isolate*, char const*, bool) [/Users/josh/.nvm/versions/node/v18.7.0/bin/node]
 4: 0x1008a5004 v8::internal::EmbedderStackStateScope::EmbedderStackStateScope(v8::internal::Heap*, v8::internal::EmbedderStackStateScope::Origin, cppgc::EmbedderStackState) [/Users/josh/.nvm/versions/node/v18.7.0/bin/node]
 5: 0x1008a3ae8 v8::internal::Heap::CollectGarbage(v8::internal::AllocationSpace, v8::internal::GarbageCollectionReason, v8::GCCallbackFlags) [/Users/josh/.nvm/versions/node/v18.7.0/bin/node]
 6: 0x100897d90 v8::internal::HeapAllocator::AllocateRawWithLightRetrySlowPath(int, v8::internal::AllocationType, v8::internal::AllocationOrigin, v8::internal::AllocationAlignment) [/Users/josh/.nvm/versions/node/v18.7.0/bin/node]
 7: 0x1008985c0 v8::internal::HeapAllocator::AllocateRawWithRetryOrFailSlowPath(int, v8::internal::AllocationType, v8::internal::AllocationOrigin, v8::internal::AllocationAlignment) [/Users/josh/.nvm/versions/node/v18.7.0/bin/node]
 8: 0x10087dcd8 v8::internal::Factory::AllocateRaw(int, v8::internal::AllocationType, v8::internal::AllocationAlignment) [/Users/josh/.nvm/versions/node/v18.7.0/bin/node]
 9: 0x100877860 v8::internal::MaybeHandle<v8::internal::SeqOneByteString> v8::internal::FactoryBase<v8::internal::Factory>::NewRawStringWithMap<v8::internal::SeqOneByteString>(int, v8::internal::Map, v8::internal::AllocationType) [/Users/josh/.nvm/versions/node/v18.7.0/bin/node]
10: 0x100c55e30 v8::internal::IncrementalStringBuilder::Extend() [/Users/josh/.nvm/versions/node/v18.7.0/bin/node]
11: 0x1009a9168 v8::internal::JsonStringifier::SerializeString(v8::internal::Handle<v8::internal::String>) [/Users/josh/.nvm/versions/node/v18.7.0/bin/node]
12: 0x1009ab294 v8::internal::JsonStringifier::Result v8::internal::JsonStringifier::Serialize_<false>(v8::internal::Handle<v8::internal::Object>, bool, v8::internal::Handle<v8::internal::Object>) [/Users/josh/.nvm/versions/node/v18.7.0/bin/node]
13: 0x1009ae5ec v8::internal::JsonStringifier::Result v8::internal::JsonStringifier::Serialize_<true>(v8::internal::Handle<v8::internal::Object>, bool, v8::internal::Handle<v8::internal::Object>) [/Users/josh/.nvm/versions/node/v18.7.0/bin/node]
14: 0x1009ac2d4 v8::internal::JsonStringifier::Result v8::internal::JsonStringifier::Serialize_<false>(v8::internal::Handle<v8::internal::Object>, bool, v8::internal::Handle<v8::internal::Object>) [/Users/josh/.nvm/versions/node/v18.7.0/bin/node]
15: 0x1009a5dec v8::internal::JsonStringify(v8::internal::Isolate*, v8::internal::Handle<v8::internal::Object>, v8::internal::Handle<v8::internal::Object>, v8::internal::Handle<v8::internal::Object>) [/Users/josh/.nvm/versions/node/v18.7.0/bin/node]
16: 0x100788890 v8::internal::Builtin_JsonStringify(int, unsigned long*, v8::internal::Isolate*) [/Users/josh/.nvm/versions/node/v18.7.0/bin/node]
17: 0x100f5d18c Builtins_CEntry_Return1_DontSaveFPRegs_ArgvOnStack_BuiltinExit [/Users/josh/.nvm/versions/node/v18.7.0/bin/node]
18: 0x1075ac744 
...
56: 0x1075b2b84 
57: 0x100f7beec Builtins_ArrayMap [/Users/josh/.nvm/versions/node/v18.7.0/bin/node]
58: 0x1063a87ac 
59: 0x100f19ef4 Builtins_AsyncFunctionAwaitResolveClosure [/Users/josh/.nvm/versions/node/v18.7.0/bin/node]
60: 0x100fa86f8 Builtins_PromiseFulfillReactionJob [/Users/josh/.nvm/versions/node/v18.7.0/bin/node]
61: 0x100f0bc4c Builtins_RunMicrotasks [/Users/josh/.nvm/versions/node/v18.7.0/bin/node]
62: 0x100ee63a4 Builtins_JSRunMicrotasksEntry [/Users/josh/.nvm/versions/node/v18.7.0/bin/node]
63: 0x1008268a4 v8::internal::(anonymous namespace)::Invoke(v8::internal::Isolate*, v8::internal::(anonymous namespace)::InvokeParams const&) [/Users/josh/.nvm/versions/node/v18.7.0/bin/node]
64: 0x100826d94 v8::internal::(anonymous namespace)::InvokeWithTryCatch(v8::internal::Isolate*, v8::internal::(anonymous namespace)::InvokeParams const&) [/Users/josh/.nvm/versions/node/v18.7.0/bin/node]
65: 0x100826f70 v8::internal::Execution::TryRunMicrotasks(v8::internal::Isolate*, v8::internal::MicrotaskQueue*, v8::internal::MaybeHandle<v8::internal::Object>*) [/Users/josh/.nvm/versions/node/v18.7.0/bin/node]
66: 0x10084d5c8 v8::internal::MicrotaskQueue::RunMicrotasks(v8::internal::Isolate*) [/Users/josh/.nvm/versions/node/v18.7.0/bin/node]
67: 0x10084de60 v8::internal::MicrotaskQueue::PerformCheckpoint(v8::Isolate*) [/Users/josh/.nvm/versions/node/v18.7.0/bin/node]
68: 0x100508c4c node::InternalCallbackScope::Close() [/Users/josh/.nvm/versions/node/v18.7.0/bin/node]
69: 0x100508fc8 node::InternalMakeCallback(node::Environment*, v8::Local<v8::Object>, v8::Local<v8::Object>, v8::Local<v8::Function>, int, v8::Local<v8::Value>*, node::async_context) [/Users/josh/.nvm/versions/node/v18.7.0/bin/node]
70: 0x10051d840 node::AsyncWrap::MakeCallback(v8::Local<v8::Function>, int, v8::Local<v8::Value>*) [/Users/josh/.nvm/versions/node/v18.7.0/bin/node]
71: 0x1005bbfe8 node::fs::FSReqCallback::Reject(v8::Local<v8::Value>) [/Users/josh/.nvm/versions/node/v18.7.0/bin/node]
72: 0x1005bc7cc node::fs::FSReqAfterScope::Reject(uv_fs_s*) [/Users/josh/.nvm/versions/node/v18.7.0/bin/node]
73: 0x1005bc8d4 node::fs::AfterNoArgs(uv_fs_s*) [/Users/josh/.nvm/versions/node/v18.7.0/bin/node]
74: 0x100ec27f8 uv__work_done [/Users/josh/.nvm/versions/node/v18.7.0/bin/node]
75: 0x100ec5f94 uv__async_io [/Users/josh/.nvm/versions/node/v18.7.0/bin/node]
76: 0x100ed7ce0 uv__io_poll [/Users/josh/.nvm/versions/node/v18.7.0/bin/node]
77: 0x100ec6424 uv_run [/Users/josh/.nvm/versions/node/v18.7.0/bin/node]
78: 0x1005096d4 node::SpinEventLoop(node::Environment*) [/Users/josh/.nvm/versions/node/v18.7.0/bin/node]
79: 0x1005f0c30 node::NodeMainInstance::Run(int*, node::Environment*) [/Users/josh/.nvm/versions/node/v18.7.0/bin/node]
80: 0x1005f0908 node::NodeMainInstance::Run() [/Users/josh/.nvm/versions/node/v18.7.0/bin/node]
81: 0x10058bf1c node::Start(int, char**) [/Users/josh/.nvm/versions/node/v18.7.0/bin/node]
82: 0x1053b108c 

This can generally be triggered by:

1. Running yarn start
2. Making small changes to .md files maybe 10-20 times, allowing a rebuild after each

Note: I don't think this is the same issue as facebook/docusaurus#4765. This is about rebuilds crashing, not performance.

[liuxingbaoyu]

After debugging, I found out that the problem comes from the swc of ts-node, which doesn't appear OOM after I disable it locally. (As I'm not sure what to do, I'm not opening a PR, I'll leave it to you to decide)

[vikdet]

I have same error but another reproducing.

Environment
Node version: 20.10.0
npm version: 10.2.4
Local ESLint version: 9.3.0
Global ESLint version: -
Operating System: Mac Os

Configuration
const tseslint = require('typescript-eslint');
const commonConfig = require('@wrike/eslint-config/node');
const common = require('@wrike/eslint-config/common');

module.exports = tseslint.config(
{
ignores: ['REPOS/'],
},
...commonConfig,
{
files: common.files,
rules: {
'@typescript-eslint/no-use-before-define': ['error', { functions: false }],
'@typescript-eslint/no-empty-function': 'off',
'@typescript-eslint/no-misused-promises': [
'error',
{
checksVoidReturn: false,
},
],
},
languageOptions: {
parserOptions: {
project: true,
tsconfigRootDir: __dirname,
},
},
},
);

When I run eslint REPOS/ it works:

Oops! Something went wrong! :(

ESLint: 9.3.0

You are linting "REPOS/", but all of the files matching the glob pattern "REPOS/" are ignored.

If you don't want to lint these files, remove the pattern "REPOS/" from the list of arguments passed to ESLint.

If you do want to lint these files, try the following solutions:

Check your .eslintignore file, or the eslintIgnore property in package.json, to ensure that the files are not configured to be ignored.
Explicitly list the files from this glob that you'd like to lint on the command-line, rather than providing a glob as an argument.

I run eslint . --fix and it fails

What did you expect to happen?
I expect eslint will ignore all content inside REPOS/ folder.

What actually happened?
It looks like it tries to analyse REPOS/ folder and fails with

<--- Last few GCs --->

[72317:0x160008000] 143421 ms: Mark-Compact (reduce) 4070.4 (4142.9) -> 4070.3 (4143.6) MB, 1605.96 / 0.00 ms (+ 56.1 ms in 15 steps since start of marking, biggest step 5.8 ms, walltime since start of marking 1684 ms) (average mu = 0.648, current mu =[72317:0x160008000] 144843 ms: Mark-Compact (reduce) 4071.6 (4143.9) -> 4071.2 (4144.6) MB, 1416.00 / 0.00 ms (average mu = 0.450, current mu = 0.004) allocation failure; scavenge might not succeed

<--- JS stacktrace --->

FATAL ERROR: Reached heap limit Allocation failed - JavaScript heap out of memory
----- Native stack trace -----

1: 0x10278553c node::Abort() [/Users/vitalii.bevzik/.nvm/versions/node/v20.11.0/bin/node]
2: 0x10278573c node::ModifyCodeGenerationFromStrings(v8::Localv8::Context, v8::Localv8::Value, bool) [/Users/vitalii.bevzik/.nvm/versions/node/v20.11.0/bin/node]
3: 0x10290a6c4 v8::internal::V8::FatalProcessOutOfMemory(v8::internal::Isolate*, char const*, v8::OOMDetails const&) [/Users/vitalii.bevzik/.nvm/versions/node/v20.11.0/bin/node]
4: 0x102adedb8 v8::internal::Heap::GarbageCollectionReasonToString(v8::internal::GarbageCollectionReason) [/Users/vitalii.bevzik/.nvm/versions/node/v20.11.0/bin/node]
5: 0x102add894 v8::internal::Heap::CollectGarbage(v8::internal::AllocationSpace, v8::internal::GarbageCollectionReason, v8::GCCallbackFlags) [/Users/vitalii.bevzik/.nvm/versions/node/v20.11.0/bin/node]
6: 0x102ad40ac v8::internal::HeapAllocator::AllocateRawWithLightRetrySlowPath(int, v8::internal::AllocationType, v8::internal::AllocationOrigin, v8::internal::AllocationAlignment) [/Users/vitalii.bevzik/.nvm/versions/node/v20.11.0/bin/node]
7: 0x102ad490c v8::internal::HeapAllocator::AllocateRawWithRetryOrFailSlowPath(int, v8::internal::AllocationType, v8::internal::AllocationOrigin, v8::internal::AllocationAlignment) [/Users/vitalii.bevzik/.nvm/versions/node/v20.11.0/bin/node]
8: 0x102ab997c v8::internal::Factory::NewFillerObject(int, v8::internal::AllocationAlignment, v8::internal::AllocationType, v8::internal::AllocationOrigin) [/Users/vitalii.bevzik/.nvm/versions/node/v20.11.0/bin/node]
9: 0x102ea1444 v8::internal::Runtime_AllocateInYoungGeneration(int, unsigned long*, v8::internal::Isolate*) [/Users/vitalii.bevzik/.nvm/versions/node/v20.11.0/bin/node]
10: 0x103200c44 Builtins_CEntry_Return1_ArgvOnStack_NoBuiltinExit [/Users/vitalii.bevzik/.nvm/versions/node/v20.11.0/bin/node]

Link to Minimal Reproducible Example
I don't have public link

Participation
I am willing to submit a pull request for this issue.
Additional comments
When I run eslint REPOS/ it works

Oops! Something went wrong! :(

ESLint: 9.3.0

You are linting "REPOS/", but all of the files matching the glob pattern "REPOS/" are ignored.

If you don't want to lint these files, remove the pattern "REPOS/" from the list of arguments passed to ESLint.

If you do want to lint these files, try the following solutions:

Check your .eslintignore file, or the eslintIgnore property in package.json, to ensure that the files are not configured to be ignored.
Explicitly list the files from this glob that you'd like to lint on the command-line, rather than providing a glob as an argument.

[bradzacher]

@vikdet as per the OP and the tags. This issue is not related to lint runs. It is an INTERNAL repo infrastructure issue.

[Josh-Cena]

@JoshuaKGoldberg Now that we no longer use ts-node in the website, does it still happen for you? Should we close it?

[JoshuaKGoldberg]

Yeah! I haven't seen this in a while. 🔪 thanks!