Memory leak

[richardscarrott]

I've recently ported ~5500 tests made up of ~700 test suites to take advantage of Jest's test isolation and parallel test runs (no auto mocking etc. as the tests are vanilla jasmine).

I'm seeing the first 600 or so test suites run incredibly fast (< 90s) however my machine then runs out of memory and resorts to using the swap disk which as you could imagine drastically slows the tests down adding another couple of minutes for the last 100 suites.

I'm certain my tests contain memory leaks however because the JSDom environment is disposed of after each test run I don't think this should cause an issue? i.e. it shouldn't be possible for tests themselves to leak once the JSDom global is destroyed?

On that note I did notice the JSDom window isn't being closed, the dispose method is just being passed as an arg to resolve so, unless I'm mis-reading it, it's never called? ...Saying that I did try calling it in the success and error callbacks myself but didn't see any improvement with memory consumption.

Short of throwing more RAM at it do you have any pointers / have you experienced similar problems?

[cpojer]

Oh man, did I mess up this thing in Promise.resolve? I thought Promise.resolve calls the passed in function. That is messed up. I'm sorry!

[richardscarrott]

It might be worth noting I'm running on an iMac with a quad core cpu which I think can handle multiple threads per core because os.cpus().length === 8 so there are 8 child processes spawned.

[cpojer]

Can you join me in #jest on discord so we can discuss there and resolve this? http://facebook.github.io/jest/support.html

[richardscarrott]

Hi @cpojer, I've created a reduced test case here https://github.com/richardscarrott/jest-mem-leak so hopefully that let's us rule out a few concerns.

It's probably worth noting I could reproduce the leak without deliberately using a lot of memory by duplicating the test 1000s of times (I just didn't want to have to check in that many files).

[richardscarrott]

@cpojer I spent some time looking at this today and (after fixing the dispose call) I've narrowed the leak down to FakeTimers. In particular it's related to the fake next tick as replacing this line with this._global.process.nextTick = function() {}; fixes the leak....but I can't see why.

[alitaheri]

@richardscarrott at this line:

  if (typeof global.process === 'object'
      && typeof global.process.nextTick === 'function') {
    this._originalTimerAPIs.nextTick = global.process.nextTick;
    this._fakeTimerAPIs.nextTick = mocks.getMockFn().mockImpl(
      this._fakeNextTick.bind(this)
    );
  }

Might the leak go further down to the the this._fakeNextTick function at this line? I mean, if the function gets called a lot we'll have so many captured arrays and variables. I'm not sure... depends on how often the function gets called.

[richardscarrott]

@subjectix ah yeah I've got it now -- I didn't realise this._global.process.nextTick = this._fakeTimerAPIs.nextTick was actually mutating Node's global process object.

This means each new instance of FakeTimers actually holds a reference to the previous instance because of this where the first instance does actually reference Node's process.nextTick but subsequent instances reference the fake version defined by the previous instance. The reason the entire FakeTimers instance (and therefore the JSDom window via this._global) is leaked is because the fakeTick function is bound here.

We could fix this by cloning global.process here so the global reference isn't mutated but is there really a need for the process object in a JSDom env?

[cpojer]

Thanks for all the debugging work. Using Object.assign for global.process does indeed fix memory usage a lot! I'm working on a fix for jest 0.7.0.

[cpojer]

Fixed in 4e12731 and I will tag 0.7.0 shortly. @richardscarrott would you mind trying to run your code like this (after upgrading):

node --expose-gc node_modules/jest-cli/bin/jest.js --runInBand --logHeapUsage

[alitaheri]

@cpojer Thanks for the fix :D

[richardscarrott]

Just tried out 0.7.0 and the memory pressure is really stable :)

node node_modules/jest-cli/bin/jest.js --runInBand
16 tests failed, 5600 tests passed (5616 total in 682 test suites, run time 330.041s)

node node_modules/jest-cli/bin/jest.js
16 tests failed, 5600 tests passed (5616 total in 682 test suites, run time 76.257s)

Thanks for sorting this @cpojer!

[github-actions]

This issue has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.
Please note this issue tracker is not a help forum. We recommend using StackOverflow or our discord channel for questions.