Memory leaking

[frozeman]

hi,

postet this already in the meteor-core group, but after testing more i guess this belongs here.

I figured, that when switching templates really often it stacks up memory. Which shouldn't be the case, because for large apps this can be fast at 100MB+ after a while.

Test case:

install the todos example of meteor $ meteor create --example todos

open the chrome dev tools and take a memory snapshot under "profiles".

then switch the todos lists like 20 times and takt one again and you will see an increase in the memory.

If you want to have a more obvious example got to the telescope demo app at http://demo.telesc.pe and do the same setps but switch the list with a lot of comments back and forth.

you will build up 50MB+ mb really fast, starting at 10MB!

This is obvious bad memeory management of the template engine and can lead to crashing apps in production environment with apps which are used for hours.

Thanks for looking into it.
Fabian

[frozeman]

btw, the same happens for the meteor.com site, when only switching between 'home' and 'examples'.
i tested the same on an ember.js app and the memory gets always properly cleared again.

[awatson1978]

YES!
I just posted something similar in the Google Talk group! Are you finding it crashes the Chrome Dev Tools themselves? After a bit of switching back and forth, I'm finding that the Dev Tools themselves start getting sluggish and eventually crash. Something has gotten leaky, unfortunately. :(

[frozeman]

no it doesnt crashes chromes dev tools for me, but stacking up al lot of memory, the more i use the website.
this is certainly critical for me, as i want to develop a quite large app with meteor and if this issue is not resolved i hav to choose another framework (which i would like to prevent :)

[frozeman]

A Blog post which could help. solving this problem:
http://coding.smashingmagazine.com/2012/11/05/writing-fast-memory-efficient-javascript/

[awatson1978]

Mmm. Yeah, I already adopted all those functional patterns into my coding style a few years ago. Never use 'new', no timers, avoid 'var' as much as possible, etc. Good practices though! :)

So, there's a related discussion happening in the google talk meteor list...
https://groups.google.com/forum/m/?fromgroups#!topic/meteor-talk/49q2JcAxgMA

First of all, check whether you're using Meteor.autosubscribe, whic is depreciated, instead of Meteor.autorun. It's looking like use of a depreciated auto publish may be a common factor of people having memory leaks.
http://www.meteor.com/blog/2013/02/13/meteor-055-devshop-code-and-community-contributions

If that doesn't fix things, lets start taking a close look at the changes made in 0.5.7. They did a lot of tinkering with how the Spark engine observes live-data updates, and if this is is a real memory leak (and not simply the result of using a depreciated function) it may have been introduced around then.
http://www.meteor.com/blog/2013/02/21/meteor-057-major-scaling-update-new-ddp-version-ejson

[frozeman]

Could help too:
http://code.google.com/p/leak-finder-for-javascript/

I use only collection and template., no subsciptions at all, as i only want to use the frontend part of meteor

[awatson1978]

Oh my. That there is some voodoo magic, aint it? But it might do the trick.

I guess maybe set goog.Disposable.MONITORING_MODE=goog.Disposable.MonitoringMode.INTERACTIVE in the Meteor.startup()?

[frozeman]

It add it says:
Uncaught ReferenceError: goog is not defined

[awatson1978]

Well, I almost got things working. I resolved the ReferenceError by copying the following files into my Meteor project:

// original files
// http://closure-library.googlecode.com/svn/trunk/closure/goog/disposable/disposable.js
// http://closure-library.googlecode.com/svn/trunk/closure/goog/disposable/idisposable.js
// http://closure-library.googlecode.com/svn/trunk/closure/goog/base.js

// file structure
client/compatibility/disposable.js
client/compatibility/core/idisposable.js
client/compatibility/core/base.js

Then hiding the following two files, so Meteor didn't parse them, like so:

/leak-finder/.doc
/pyautolib/.dom_mutation_observer.js

And adding the following to my application:

Meteor.startup(function(){
    goog.require('goog.Disposable');
    goog.Disposable.ENABLE_MONITORING = true;
    goog.Disposable.MONITORING_MODE = goog.Disposable.MonitoringMode.INTERACTIVE
});

At which point, after relaunching the Meteor server, and refreshing the page displaying the app, when I check the inspection on localhost:9222, the page I'm testing for a memory leak is sort of grayed out, and I'm getting the following message in the console:

INFO:root:Reading suppressions from "closure-disposable-suppressions.txt"
INFO:root:Taking heap snapshot

But then it just hangs there. Any ideas?

[frozeman]

Why you don't use the chrome dev tools timeline -> memory observer? Or snapshot?

You also can then observe objects etc.

And why you think the memory thing changed in 5.7?

I assume that the templates aren't disposed correctly and still hang around in the memory.

[frozeman]

Any progress here?
I tested it with ember.js, and filling up views and replacing them doesn't stack up memory like spark.

here the memory gets cleaned up properly so that my app never goes beyond 6mb, only for short periods when loading a heavy template.

So i guess, there is definitively something wrong with spark, or the way the template lifecycle works.

[frozeman]

Is there anyone working on this issue? As it seems rather important for app stability.

[avital]

@frozeman I'm looking into this.

[avital]

I loaded the todos app (on Meteor 0.6.4). I used Chrome Dev Console to take a heap snapshot right after pageload and found 4.4MB.

I then ran the following Javascript for a few minutes, which switches back and forth between different lists (which as I understand is the repro suggested by @frozeman)

setInterval(function () {
  Router.setList(Lists.find().fetch()[0]._id);
  setTimeout(function () {
    Router.setList(Lists.find().fetch()[1]._id)
  }, 150);
}, 300);

Notably, I do see the lists changing back and forth and all items in the list appearing each time. I then took a heap snapshot once every minute or so as this code was running. Heap size grew to around 5.5MB after which it stopped growing.

I believe this demonstrates that there is no memory leak (though it may be possible to consume less memory in general). Am I running a bad test?

[frozeman]

can you try this again using the "timeline" pane in chrome dev tools (swtch the memeory recoding buttons on)?

i tried my own app, i did a small test with the todo list app, but not long.
the most obvious it is in the telescope app.

What i did for testing, is to create a template with an {{#each}} over a cursor. Then i fill in that collection like 1000 objects, and switch the view back and forth to and from that list. Son it rises up to 50MB+ and never goes down, even if the template is not visible. and even if i press the "trashcan" button, to clean up the garbage collection in chrome dev > timeline.

I used the technic described below, but also used plain {{> templateWithLargeList}} helpers to test it with the same result.

the technic i used to dynamically create templates.
when i set template dynamically like this:

  // setting a session variable to trigger the right template and then i load it dynamically
  Session.set('myTemplate','someTemplate');

  Template.myContainerTemplate.showTemplate = function() {
   return Template[Session.get('myTemplate')]();
  }

Does this create always new instances of that template, which never gets cleaned up? And if so, how to properly clean up that template, as i find this solution for dynamically setting templates, very nice.

I also did the same setup in ember.js and it grows memeory but always after a few sec. clean it up and it basically never goes beyond 6mb, even if i had big lists...