Skip to content

Experimental CLI OOMs on large cold-cache runs because file reads are not bounded by worker concurrency #114

Description

@cellison-figma

Environment/Setup

  • Prettier 3.9.3
  • Node 22.22.0
  • macOS ARM64
  • @prettier/plugin-oxc 0.2.0
  • @prettier/plugin-ruby 4.0.4
  • Command: prettier --cache --write --experimental-cli '**/*.{ts,tsx,js,jsx,cjs,mjs,css,mdx,rake,rb}'

Note: glob match in the command targets ~84k glob candidates (0.85GB) before ignore filtering

Description / Hypothesis

When running prettier with a cold cache against a large glob match of files, the experimental CLI crashes during worker pool scheduling. This appears to be due to the cache wrapper reading full file contents before submitting to the bounded pool.

// src/prettier_cached.ts
const fileContent =
  data?.content?.toString() ?? await readFile(filePath, "utf8")

const fileContentFormatted = await prettier.format(...)

With this behavior, prettier exceeds the default ~8GB V8 heap limit, as file reading and per-file operations appear to be unbounded. Expected behavior would be that memory usage is bounded roughly by worker parallelism.

Without the experimental CLI, this OOM crash does not occur.

Error log

> prettier --cache --write --experimental-cli '**/*.{ts,tsx,js,jsx,cjs,mjs,css,mdx,rake,rb}'


<--- Last few GCs --->

[84044:0xb2740c000]   131521 ms: Scavenge 8099.0 (8229.8) -> 8098.6 (8230.8) MB, pooled: 0 MB, 9.79 / 0.00 ms  (average mu = 0.403, current mu = 0.141) allocation failure; 
[84044:0xb2740c000]   131532 ms: Scavenge 8100.0 (8230.8) -> 8099.9 (8253.1) MB, pooled: 0 MB, 10.54 / 0.00 ms  (average mu = 0.403, current mu = 0.141) allocation failure; 


<--- JS stacktrace --->

FATAL ERROR: Reached heap limit Allocation failed - JavaScript heap out of memory
----- Native stack trace -----

 1: 0x104c2f040 node::OOMErrorHandler(char const*, v8::OOMDetails const&) [/usr/local/bin/node]
 2: 0x104e0b79c v8::internal::V8::FatalProcessOutOfMemory(v8::internal::Isolate*, char const*, v8::OOMDetails const&) [/usr/local/bin/node]
 3: 0x10501af30 v8::internal::Heap::stack() [/usr/local/bin/node]
 4: 0x1050192d0 v8::internal::Heap::CollectGarbage(v8::internal::AllocationSpace, v8::internal::GarbageCollectionReason, v8::GCCallbackFlags) [/usr/local/bin/node]
 5: 0x10500d8c4 v8::internal::HeapAllocator::AllocateRawWithLightRetrySlowPath(int, v8::internal::AllocationType, v8::internal::AllocationOrigin, v8::internal::AllocationAlignment) [/usr/local/bin/node]
 6: 0x10500e0fc v8::internal::HeapAllocator::AllocateRawWithRetryOrFailSlowPath(int, v8::internal::AllocationType, v8::internal::AllocationOrigin, v8::internal::AllocationAlignment) [/usr/local/bin/node]
 7: 0x104ff141c v8::internal::Factory::NewFillerObject(int, v8::internal::AllocationAlignment, v8::internal::AllocationType, v8::internal::AllocationOrigin) [/usr/local/bin/node]
 8: 0x10540cee0 v8::internal::Runtime_AllocateInYoungGeneration(int, unsigned long*, v8::internal::Isolate*) [/usr/local/bin/node]
 9: 0x1058cdaf4 Builtins_CEntry_Return1_ArgvOnStack_NoBuiltinExit [/usr/local/bin/node]
10: 0x105836510 Builtins_FastNewClosure [/usr/local/bin/node]
11: 0x1059b4670 Builtins_CreateClosureHandler [/usr/local/bin/node]
12: 0x105838ef0 Builtins_InterpreterEntryTrampoline [/usr/local/bin/node]
13: 0x1058fc65c Builtins_ArrayMap [/usr/local/bin/node]
14: 0x105838ef0 Builtins_InterpreterEntryTrampoline [/usr/local/bin/node]
15: 0x105875290 Builtins_AsyncFunctionAwaitResolveClosure [/usr/local/bin/node]
16: 0x1059404d8 Builtins_PromiseFulfillReactionJob [/usr/local/bin/node]
17: 0x105865594 Builtins_RunMicrotasks [/usr/local/bin/node]
18: 0x105836af4 Builtins_JSRunMicrotasksEntry [/usr/local/bin/node]
19: 0x104f72df8 v8::internal::(anonymous namespace)::Invoke(v8::internal::Isolate*, v8::internal::(anonymous namespace)::InvokeParams const&) [/usr/local/bin/node]
20: 0x104f736a0 v8::internal::(anonymous namespace)::InvokeWithTryCatch(v8::internal::Isolate*, v8::internal::(anonymous namespace)::InvokeParams const&) [/usr/local/bin/node]
21: 0x104f737d8 v8::internal::Execution::TryRunMicrotasks(v8::internal::Isolate*, v8::internal::MicrotaskQueue*) [/usr/local/bin/node]
22: 0x104fa09fc v8::internal::MicrotaskQueue::RunMicrotasks(v8::internal::Isolate*) [/usr/local/bin/node]
23: 0x104fa1180 v8::internal::MicrotaskQueue::PerformCheckpoint(v8::Isolate*) [/usr/local/bin/node]
24: 0x104b2d9dc node::InternalCallbackScope::Close() [/usr/local/bin/node]
25: 0x104b2d484 node::InternalCallbackScope::~InternalCallbackScope() [/usr/local/bin/node]
26: 0x104c332e4 node::fs::FileHandle::CloseReq::Resolve() [/usr/local/bin/node]
27: 0x104c4dc8c node::fs::FileHandle::ClosePromise()::$_0::__invoke(uv_fs_s*) [/usr/local/bin/node]
28: 0x104c26cfc node::MakeLibuvRequestCallback<uv_fs_s, void (*)(uv_fs_s*)>::Wrapper(uv_fs_s*) [/usr/local/bin/node]
29: 0x105812aec uv__work_done [/usr/local/bin/node]
30: 0x10581681c uv__async_io [/usr/local/bin/node]
31: 0x10582abb4 uv__io_poll [/usr/local/bin/node]
32: 0x105816d84 uv_run [/usr/local/bin/node]
33: 0x104b2e518 node::SpinEventLoopInternal(node::Environment*) [/usr/local/bin/node]
34: 0x104c7993c node::NodeMainInstance::Run() [/usr/local/bin/node]
35: 0x104be9d50 node::Start(int, char**) [/usr/local/bin/node]
36: 0x186e1be00 start [/usr/lib/dyld]
 ELIFECYCLE  Command failed.
 ELIFECYCLE  Command failed.
[1]    83966 abort      pnpm prettier

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions