JavaScript heap out of memory

[SciPlayD]

This is new in v6.2.3, I am pretty certain.

FATAL ERROR: Ineffective mark-compacts near heap limit Allocation failed - JavaScript heap out of memory
 1: 0x100063a65 node::Abort() [/Users/cache/node-v11.6.0-darwin-x64/bin/node]
 2: 0x100064104 node::errors::TryCatchScope::~TryCatchScope() [/Users/cache/node-v11.6.0-darwin-x64/bin/node]
 3: 0x10019d9a7 v8::Utils::ReportOOMFailure(v8::internal::Isolate*, char const*, bool) [/Users/cache/node-v11.6.0-darwin-x64/bin/node]
 4: 0x10019d944 v8::internal::V8::FatalProcessOutOfMemory(v8::internal::Isolate*, char const*, bool) [/Users/cache/node-v11.6.0-darwin-x64/bin/node]
 5: 0x1005a2122 v8::internal::Heap::FatalProcessOutOfMemory(char const*) [/Users/cache/node-v11.6.0-darwin-x64/bin/node]
 6: 0x1005a4653 v8::internal::Heap::CheckIneffectiveMarkCompact(unsigned long, double) [/Users/cache/node-v11.6.0-darwin-x64/bin/node]
 7: 0x1005a0b88 v8::internal::Heap::PerformGarbageCollection(v8::internal::GarbageCollector, v8::GCCallbackFlags) [/Users/cache/node-v11.6.0-darwin-x64/bin/node]
 8: 0x10059ed45 v8::internal::Heap::CollectGarbage(v8::internal::AllocationSpace, v8::internal::GarbageCollectionReason, v8::GCCallbackFlags) [/Users/cache/node-v11.6.0-darwin-x64/bin/node]
 9: 0x1005a7a7b v8::internal::Heap::FinalizeIncrementalMarkingIfComplete(v8::internal::GarbageCollectionReason) [/Users/cache/node-v11.6.0-darwin-x64/bin/node]
10: 0x1005b8e6f v8::internal::IncrementalMarkingJob::Task::RunInternal() [/Users/cache/node-v11.6.0-darwin-x64/bin/node]
11: 0x1000ba156 node::PerIsolatePlatformData::RunForegroundTask(std::__1::unique_ptr<v8::Task, std::__1::default_delete<v8::Task> >) [/Users/cache/node-v11.6.0-darwin-x64/bin/node]
12: 0x1000b987a node::PerIsolatePlatformData::FlushForegroundTasksInternal() [/Users/cache/node-v11.6.0-darwin-x64/bin/node]
13: 0x1009fbd2f uv__async_io [/Users/cache/node-v11.6.0-darwin-x64/bin/node]
14: 0x100a0b898 uv__io_poll [/Users/cache/node-v11.6.0-darwin-x64/bin/node]
15: 0x1009fc1b6 uv_run [/Users/cache/node-v11.6.0-darwin-x64/bin/node]
16: 0x100048371 node::Start(v8::Isolate*, node::IsolateData*, std::__1::vector<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, std::__1::allocator<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > > > const&, std::__1::vector<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, std::__1::allocator<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > > > const&) [/Users/cache/node-v11.6.0-darwin-x64/bin/node]
17: 0x10004784c node::Start(uv_loop_s*, std::__1::vector<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, std::__1::allocator<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > > > const&, std::__1::vector<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, std::__1::allocator<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > > > const&) [/Users/cache/node-v11.6.0-darwin-x64/bin/node]
18: 0x1000474b2 node::Start(int, char**) [/Users/cache/node-v11.6.0-darwin-x64/bin/node]
19: 0x100001034 start [/Users/cache/node-v11.6.0-darwin-x64/bin/node]

[stephen-palmer]

What stands out immediately to me is that you're running node v11.6 .. we're still on v8.12 LTS.

[SciPlayD]

I'll downgrade tonight. It wasn't clear whether v8.x was the required version, or the minimum required version.

[stephen-palmer]

Required. I wanted to move to v10 (latest LTS) but there are some incompatibilities that require investigation and fixing.

[mleenhardt]

We haven't yet seen an OOM error but have noticed a massive increase in RAM consumption with 6.2.3. We're using the fs module and are seeing each of the 8 node workers that we're running consuming up to 1.5GB of RAM. I'm under the impression that workers were barely using any RAM on 6.2.0.

We're running node 10.13.

[stephen-palmer]

@mleenhardt we don't support node v10 yet. Not saying that would make a difference in RAM usage but please use the version specified in the package.json file (or .nvmrc)

[stephen-palmer]

Published a fix for the worker memory leak in v6.2.4-beta2

[stephen-palmer]

Fixed in v6.2.4 (published to NPM)

[corngood]

We upgraded from 6.1.2 to 6.2.4 and we're seeing a huge increase in memory use. We have four workers, and after a day they are each using between 1.5 and 3.5GB of physical memory. This is on node 8.14 on linux.

[stephen-palmer]

I'll investigate further, thanks for reporting.