Empty Node Module Memory Usage

[masihyeganeh]

Unstoppable memory usage increase

Node is increasingly using RSS and Heap memory for doing nothing

My environment

• CPU: 2.3 GHz Intel Core i7
• OS: OSX Yosemite 10.10.1 (14B25)
• RAM: 16 GB 1600 MHz DDR3
• Node.JS verison: v0.11.14

How to reproduce problem

Just start an empty node module and log memory usage each seconds:

setInterval(function() {
    console.log(process.memoryUsage());
}, 1000);

For better readability, let's use this instead:

var unit = ['', 'K', 'M', 'G', 'T', 'P'];

function bytesToSize(input, precision)
{
    var index = Math.floor(Math.log(input) / Math.log(1024));
    if (unit >= unit.length) return input + ' B';
    return (input / Math.pow(1024, index)).toFixed(precision) + ' ' + unit[index] + 'B'
}

var usage;

console.log('==================================');
console.log('  Empty Node Module Memory Usage  ');
console.log('==================================');

setInterval(function() {
    usage = process.memoryUsage();
    console.log('RSS: ' + bytesToSize(usage.rss, 3), 'and Heap:', bytesToSize(usage.heapUsed, 3), 'of', bytesToSize(usage.heapTotal, 3), 'total');
}, 1000);

Expected result

Memory usage should be constant

Actual result

Heap and RSS memory are increasing

My results:

==================================
  Empty Node Module Memory Usage  
==================================
RSS: 13.746 MB and Heap: 3.583 MB of 6.191 MB total
RSS: 13.824 MB and Heap: 3.626 MB of 6.191 MB total
RSS: 13.828 MB and Heap: 3.634 MB of 6.191 MB total
RSS: 13.832 MB and Heap: 3.637 MB of 6.191 MB total
RSS: 13.836 MB and Heap: 3.641 MB of 6.191 MB total
RSS: 13.840 MB and Heap: 3.645 MB of 6.191 MB total
RSS: 13.848 MB and Heap: 3.649 MB of 6.191 MB total
RSS: 13.852 MB and Heap: 3.653 MB of 6.191 MB total
RSS: 13.855 MB and Heap: 3.657 MB of 6.191 MB total
RSS: 13.871 MB and Heap: 3.661 MB of 6.191 MB total
RSS: 13.875 MB and Heap: 3.665 MB of 6.191 MB total
RSS: 13.879 MB and Heap: 3.669 MB of 6.191 MB total
RSS: 13.883 MB and Heap: 3.673 MB of 6.191 MB total
...

[netpoetica]

I wonder if maybe this is some rampup time for some libs that are loading async in the background, or, if maybe this is libuv or other deps ramping up in the background?

I ran these on Mac OSX using node v0.10.33 (stable) on system Darwin12.5.0 Darwin Kernel Version 12.5.0: Sun Sep 29 13:33:47 PDT 2013; root:xnu-2050.48.12~1/RELEASE_X86_64 x86_64

Output from simple test:

// ~ 50 cycles
Begin:
rss: 12570624, heapTotal: 4083456, heapUsed: 2169696
End:
rss: 15073280, heapTotal: 6163968, heapUsed: 2658480
Results:
rss: +2502656, heapTotal: +2080512, heapUsed: +488784

In the second test:

// ~215 cycles
==================================
  Empty Node Module Memory Usage  
==================================
Begin:
RSS: 11.875 MB and Heap: 2.126 MB of 3.894 MB total
End:
RSS: 15.031 MB and Heap: 2.337 MB of 5.878 MB total
Difference:
RSS: +3.156 MB and Heap: +2.126 of +1.984 total

[netpoetica]

I left this running every 10 seconds for a good while to see if it was just a ramp up, but it still looks like it's just going and going and going: https://gist.github.com/netpoetica/4d16382c3cfbf6db0f58 - Mac OSX using node v0.10.33 (stable) on system Darwin12.5.0 Darwin Kernel Version 12.5.0: Sun Sep 29 13:33:47 PDT 2013; root:xnu-2050.48.12~1/RELEASE_X86_64 x86_64

Running the same test with Node v0.11.14: https://gist.github.com/netpoetica/1a0a6652fa6c22b76951

[abenhamdine]

@netpoetica in your test with node v0.10.33, we can see some decreases, eg lines 1397 or 1758.
But there's none in 0.11.14 ?

[netpoetica]

in 0.11 there was no initial dip and then went up leaving a small increase later:

==================================
  Empty Node Module Memory Usage  
==================================
BEGIN RSS: 16.273 MB and Heap: 3.466 MB of 6.191 MB total
END     RSS: 16.633 MB and Heap: 3.974 MB of 6.191 MB total

in 0.10 there was an initial dip for a little while (it decreased a bit), and then went up leaving a larger increase later but I do believe this test ran much longer:

==================================
  Empty Node Module Memory Usage  
==================================
BEGIN RSS: 11.828 MB and Heap: 2.116 MB of 3.894 MB total
END     RSS: 15.254 MB and Heap: 2.523 MB of 5.878 MB total

[chrisdickinson]

This is almost certainly the result of (unavoidable!) garbage being generated by the timer subsystem. V8 will happily grow the heap until it trips the compaction heuristic. Here's a link covering some bits about the GC. You may try running with --always-compact --expose-gc, and adding a gc() call to the setInterval callback and seeing if that changes the results at all. Closing this as "works as expected" for now. Please comment if I am in error, and I can reopen.

[renanbastos93]

Hello Guys,
There is function memoryUsage, accumulate in memory?

Att,