GM_registerMenuCommand memory leak

[ghost]

A user of one of my scripts discovered a memory leak which occurs when Distill Web Monitor is running and I can confirm this with Fx 35 + GM 2.3 + DWM 1.1.2.

Distill Web Monitor is an extension to periodically check web pages for changes. It seems to be doing this in a way that trips up Greasemonkey. On each check, Greasemonkey injects its userscripts into an invisible window and if GM_registerMenuCommand gets called, memory consumption will increase over time.

Steps to reproduce (clean profile):

1. Install GM + DWM and restart
2. Create new userscript via GM, meta: @include *, @grant GM_registerMenuCommand, code: GM_registerMenuCommand('Test', function() { alert("Test!"); });
3. Go to http://www.buzzfeed.com and click "Watch Page" in DWM toolbar icon menu
4. Open menu again, go to "Watchlist" -> arrow to the left of BuzzFeed entry -> "Edit options" and set schedule to 20 seconds
5. Save changes and observe Firefox process growing

If you uncomment GM_registerMenuCommand, memory will stay roughly the same (140 to 180 MB on my system).

Is it possible to make Greasemonkey only run in "real" windows?

[tulip33]

GM+alertbox+个别脚本，导致firefox内存暴涨
GM＋MPIV＋alertbox曾导致火狐内存暴涨，MPIV作者改了行代码暂时解决了问题，但最新的enhanced word highlight 1.5.8＋alertbox＋GM2.3又出现内存暴涨现象，如果GM不解决此问题的话，其他脚本也可能导致内存泄漏，希望作者尽快修复此问题，谢谢!

[arantius]

I'm not denying anything, but the above steps are quite involved. Can you produce a reduced test case to help support your position?

Also, is this a "leak" i.e. the memory is permanently allocated even when no longer used? Or does it go away at an appropriate time (i.e. closing the tab)?

Also: // @noframes?

[tophf]

The described testcase seems to be as minimal and simple as possible while maintaining the connection to reality...

[ghost]

Step 4 is optional and you can replace BuzzFeed with any other site.

The test case boils down to start watching a page where a userscript calls GM_registerMenuCommand and wait. You just have to wait a little longer for the increase to be noticeable with default settings.

I didn't try @noframes and I don't recall seeing memory going down. I let Firefox just sit there minimized with an empty tab for 20 minutes or so.

I'm currently working around this by delaying GM_registerMenuCommand until the first document.onmouseover.

[simonzack]

(moved from other issue per @arantius's comment)

My own user script leaks memory significantly with GM_registerMenuCommand as well, until firefox crashes at ~1.5gb of memory used. Commenting it out and memory use becomes rather stable.

To reproduce:

• install https://gist.github.com/simonzack/d5ac206f0fe661cc0c27 with greasemonkey on firefox 38
• do some google searches in a new tab and check the script is active (it should be)
• close the google search tab
• you will see the permanent ghost compartments in about:memory

If you comment out the single call to GM_registerMenuCommand at line 216 and remove // @grant GM_registerMenuCommand, no ghost compartments are created and memory usage remains stable.

[zoocey]

I believe I am experiencing the same issue.

Since upgrading Greasemonkey past v2.3 after an hour+ of browsing Firefox becomes much slower and less responsive. With higher memory usage and randomly noticeable delay in scrolling, opening links in new tabs, closing tabs etc.

I narrowed it down to being caused by the following userscript http://userscripts-mirror.org/scripts/show/174606 (you will have to modify it to add grants for GM_getValue, GM_registerMenuCommand, GM_setValue)

Removing the GM_registerMenuCommand grant and commenting out the line calling the GM_registerMenuCommand function and so far Firefox seems to be running without decreased responsiveness.

Setup is Greasemonkey 3.1, Firefox 38 (beta) 64-bit.

[arantius]

shrug I did @simonzack 's steps. I visit about:memory, click measure, and it says " 0 ── ghost-windows". How many is "some" searches?

[simonzack]

@arantius I remember I did less than 5, let me check it again.

[zoocey]

I cant speak for @simonzack or his script but running the script I linked above (with the added grants) and I get a lot of ghost windows created. I cant figure out how to attach a file to the issue tracker so I uploaded a about:memory memory report to dropbox https://www.dropbox.com/s/apxw6sspgp0e28h/memory-report.json.gz?dl=0

[jerone]

I cant figure out how to attach a file to the issue tracker

Just use GitHub Gists...

[arantius]

Firefox: Mozilla/5.0 (X11; Linux x86_64; rv:40.0) Gecko/20100101 Firefox/40.0 Built from https://hg.mozilla.org/mozilla-central/rev/caf25344f73e
Greasemonkey: Source as of commit 8b2ccf0 .

1. Install the script whose source is below.
2. Restart firefox
3. Open about:memory, press measure, see 207.51 MB resident main process, 71.47 MB resident child process, 0 ghost windows, close this tab.
4. Load a page (and execute the script) ten times.
5. Open about:memory, press measure, see 246.24 MB resident main process, 79.58 MB resident child process, 0 ghost windows, close this tab.
6. Close the tab that had the ten page loads above.
7. Open about:memory, press measure, see 336.29 MB resident main process, 74.56 MB resident chid process, 0 ghost windows, close this tab.

// ==UserScript==
// @name        GM_registerMenuCommand Tester
// @namespace   https://github.com/arantius
// @include     *
// @version     1
// @grant       GM_registerMenuCommand
// ==/UserScript==

for (var i = 0; i < 99; i++) {
  GM_registerMenuCommand(
      'Test command #' + i,
      function() { alert('Clicked menu command!') });
}

If I disable the script, the numbers are similar. If I repeat the same steps several times, the specific numbers vary, but all stay in the same window regardless of whether the script is running or not. Ok, how about this script:

// ==UserScript==
// @name        GM_registerMenuCommand Tester
// @namespace   https://github.com/arantius
// @include     *
// @version     1
// @grant       GM_registerMenuCommand
// ==/UserScript==

var memHog = [];
for (var i = 0; i < 9999; i++) {
  // Just over 1k bytes, 10k times = ~10MB
  var val = ''
      + 'xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx'
      + 'xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx'
      + 'xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx'
      + 'xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx'
      + 'xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx'
      + 'xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx'
      + 'xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx'
      + 'xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx'
      + 'xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx'
      + 'xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx'
      + 'xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx'
      + 'xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx'
      + 'xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx'
      + 'xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx'
      + 'xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx'
      + 'xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx'
      + Math.random();
  memHog.push(val);
  // Plus who knows how much for the DOM?
  var p = document.createElement('p');
  p.textContent = val;
  document.body.appendChild(p);
}


GM_registerMenuCommand(
    'Test command #' + i,
    function() { alert('Clicked menu command!') });

This should consume tons of memory! If I comment out the GM_registerMenuCommand() call, then I see fresh launch->measure memory go from 160/60 (parent/child resident) then load a page once it goes to 260/200. Yes, I'm consuming memory. Close the tab and it's 300/120. Load/close: 320/120. Load/close: 320/120. Stable point?

Ok, restore the menu command in the script, fresh re-launch Firefox. Fresh, then after each load/close memory: 160/60, 260/220, 280/120, 300/120, 340/120, 400/140, 320/120, 340/140, 300/120. More jitter, but around the same point.

I don't see any memory leak. If you want to convince me, please provide reams of data and explanations a five year old could understand.

[ghost]

The situation has gotten significantly worse under Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:40.0) Gecko/20100101 Firefox/40.0. Distill Web Monitor is not involved this time.

After just 5 minutes of normal browsing with multiple tabs, Firefox starts to lock up every 10 seconds for one second. Typing, scrolling, tab animations - everything stops.

Currently with only about:memory open, I'm seeing 203 ghost windows and an explicit memory allocation of 734.96 MB. Pressing "Minimize memory usage" does virtually nothing except freeze Firefox for 10 seconds.

GM 3.2 is the only extension installed.
Flash and other plugins are disabled.
This is the only userscript running: GM_registerMenuCommand("Test...", function() { alert("Test!"); });

[arantius]

Can you repeat a test like mine above ? And if so what sort of results do you see?

Saying just ".. minutes of normal web browsing" is hard to reproduce. I've tried and failed to reproduce this. Or can you say specifically which page(s) trigger this?

[ghost]

I opened around 15 tabs with popular sites like reddit, amazon, imdb and wikipedia, clicked a link in each tab, scrolled around for 5 seconds and closed all tabs. I'm gonna try your tester script.

[Ventero]

Here are my steps with which I'm consistently able to reproduce a ghost window in a fresh profile with GM HEAD (6f5ba1c) installed in FF 39:

1. Install this script.
2. In one tab, open http://reddit.com/ and wait for it to load.
3. In another tab, open about:memory.
4. In the reddit tab, click the "all" link in the top left corner and wait for it to load.
5. Close the reddit tab.
6. In the about:memory tab, click "Measure". Observe a detached/ghost reddit.com window.
7. Click GC, CC, minimize memory usage a few times, then measure again. Still observe the ghost window.

If step 4 is left out, no ghost window appears. I think the issue might be related to contentWin in ContentObserver#pagehide sometimes being a DeadObject. Adding a call to Components.utils.nukeSandbox right after the call to runScriptInSandbox in ScriptRunner#injectScripts gets rid of the ghost window. Delaying the call to nukeSandbox until a non-persisting pagehide event is received however is not enough - probably again due to the fact that in some cases it's not possible to figure out what window object was hidden.

[ghost]

Removed my minimalist test script, installed arantius' memHog test script and restarted Firefox.

Memory consumption (private working set in Windows task manager) directly after start with empty page: 244 MB
After 15 seconds idling: 166 MB

Opened http://arstechnica.com.
Tab loading animation freezes after 2 seconds, tab bar flickers 7 times (unskinned window titlebar is visible), Firefox becomes unresponsive for 55 seconds.

Memory after loading finished: 1355 MB
After 20 seconds: 841 MB

Opened new empty tab. Closed first tab.

Memory: 754 MB.
After 20 seconds: 640 MB.

Opened about:memory and clicked "Minimize memory usage"

Memory 635 MB. Clicked a second time. 637 MB.

Clicked "Measure".

575.79 MB explicit allocations
21 ghost windows (15 http://arstechnica.com/, 6 http://arstechnica.com/wp-content/themes/arstechnica/assets/html/fif.html?v=2&appId=f7e3be68-c3d8-4c0a-bd01-80af938dba72&...)

[ghost]

Choosing arstechnica was a bad idea. Takes forever.
I repeated the test with http://example.com 5 times (open/close tab): 10 ghost windows, 420 MB.
After 10 times: 20 ghost windows, 720 MB.

[arantius]

1. Brand new blank Firefox profile.
2. Install GM 3.2 (restart).
3. Install memhog script.
4. Close all tabs/windows.
5. Launch new browser, to "new tab" page.
6. Five times: open new tab, navigate to example.com, close tab. (Each one takes a few seconds to run ten thousand iterations of that awful loop.)
7. New tab, navigate to about:memory, press measure. At the top it says 240.98 MB (100.0%) -- explicit, at the end it says: 0 ── ghost-windows.
8. Close and restart the browser, step 7 again shows 263.17 MB (100.0%) -- explicit. It definitely did not go up with several open tab/navigate/close tab actions.

I believe you, but we're seeing very different results. We need to know why. There's something you're doing differently. Perhaps something in your Firefox profile (can you try with a new blank profile)? Other actions you're taking? Clicking menus? Extensions installed?

[ghost]

I followed your steps exactly and got 376.34 MB and 0 ghost windows.

However, ghost windows do appear out of nowhere when I press "Minimize memory usage" and measure again. They do not appear when the GM_registerMenuCommand call is commented out.

[arantius]

Can confirm one ghost per open/navigate/close tab after clicking minimize.

[arantius]

Also, with the smaller (99 loop GM_rmc) script:

1. Open the scratchpad, set it to "Browser" environment.
2. Evaluate Object.keys( GM_MenuCommander.menuCommands ).length, get 1.
3. Open/navigate/close two tabs.
4. Length is now 3.
5. about:memory, minimize, measure: two ghosts show. Length is still 3.

That's pretty definitive leaking. The menu commands keep a reference to the window for too long.

[ghost]

Side note: I never press the minimize button while I'm in my everyday profile so I guess "heap-minimize notifications" get sent automatically from time to time.

[arantius]

Definitely, that button is just a nice way to deterministically reproduce when they happen.

[arantius]

If:

1. You're running Greasemonkey 3.2 and
2. You think you see this problem, please
3. Install Greasemonkey 3.3beta1

At let me know if things get better, worse, or no change. Somehow my steps to reproduce ghost windows from this morning don't work anymore, but I've done two things that may help the situation. Confirmation either way would be real nice.

[ghost]

Greasemonkey 3.3beta1 is a huge improvement but I still see a small number of ghost windows popping up after pressing "Minimize memory usage" - currently 18 after 3 hours in my main profile (13 addons, 41 userscripts of which 4 add menu commands, process size 496 MB after closing all tabs).

Unfortunately I haven't found a way to replicate this reliably with a clean profile. At first I thought it had to do with restored tabs from a previous session. Worked only twice though...

[gabrielfin]

I have noticed bursts of high CPU usage every 30 to 60 seconds with adsbypasser, during which Firefox's UI freezes. I concluded that the problem lies in GM_registerMenuCommand, since commenting the line that calls that function solves the problem.

I think it might be related to this issue, since on every burst, Firefox's RAM usage seems to drop a few MB. Perhaps to some periodic garbage collecting?

I seem to recall that the issue started occuring at the end of may, which coincides with the release of version 3.2, but it might be a coincidence. I'm going to try with 3.1

I have tried installing Greasemonkey 3.3beta1, but the issue persists.
I'm using Firefox 38.

[arantius]

I've prepared a much bigger, hopefully much better, change to target this issue. Please review (@Ventero perhaps?):

master...arantius:async-menu-command

[Ventero]

Will do, probably won't be able to finish it until tomorrow though.

[Ventero]

I'm not really sure I'm 100% comfortable with dispatching the greasemonkey-menu-command-run / greasemonkey-menu-command-list events on the content window. For one, this leads to the menu-command-run listener being called in every sandbox when any one menu command for that content scope is clicked. And secondly, as you correctly note, these events are visible to content, allowing them to detect (#1787, and possibly even intercept?) menu command usage.

I can't think of an easy way to avoid this though. Sure, only running the menu command that was actually clicked is easy enough, but I can't think of a way to dispatch this event privately somehow which does not involve storing a sandbox reference somewhere (at least implicitly in some kind of closure).

I'll follow this up with some additional review comments tomorrow.

[arantius]

Thanks for the early comments. This was the best idea I had to avoid storing references that might leak. But your comments have prompted me to think back. What I've done above is to implicitly rely on the implicit reference that the sandbox has to the content window, and traverse that implicit link by dispatching an event. Trusting that the sandbox will be GCed when the window object is.

Can we perhaps add an expando to the content window/browser to explicitly make this link, but outside of the content scope? If so does it retain the property of being GCed at the right time by default?

[Ventero]

arantius/greasemonkey@async-menu-command...ventero:async-menu-command is a small experiment with using a function reference chain to go from frame script to sandbox and back. It doesn't work very well though, as it seems that the message listener keeping a strong reference to the menu commander function returned from the sandbox means the sandbox is kept alive indefinitely.

Maybe your idea will work, since by storing the function returned by MenuCommandSandbox on the window/browser instead, we don't need to keep a reference to the sandbox alive in the message listener (though I guess the returned function itself already keeps a reference to the sandbox as its principal?). I have to admit though that the details of what is collected when goes a little over my head - especially once frame scripts and message managers are involved.

Maybe we could reach out to some Mozilla folks - after all, if anyone knows how to cross from chrome to a content sandbox through a frame script without leaking objects everywhere, it should be them ...

[arantius]

Especially now that the -run event is filtering for script ID properly, I'm not concerned with every script (within the scope of the frame, which is not the whole browser right?) seeing and immediately throwing away the event. This only happens as a response to manual action.

Now, on content detecting/intercepting menu commands. I put:

addEventListener('greasemonkey-menu-command-list', function(e) {
  document.body.innerHTML +=
      'Content scope detected greasemonkey-menu-command-list event!  Detail: '
      +JSON.stringify(e.detail)+'<br>';
}, true);
addEventListener('greasemonkey-menu-command-run', function(e) {
  document.body.innerHTML +=
      'Content scope detected greasemonkey-menu-command-run event!  Detail: '
      +JSON.stringify(e.detail)+'<br>';
}, true);

Into a content page, and it logged:

Content scope detected greasemonkey-menu-command-list event! Detail: 2
Content scope detected greasemonkey-menu-command-run event! Detail: "{"cookie":2,"scriptUuid":"04940564-08aa-4225-93d5-1509737c373d"}"

So, yeah. As-is, content can see A) when you open the monkey menu, and B) when you select a command. But it doesn't know which script (the UUIDs are random per user) or anything about which command besides its index in the list. And it's pretty trivial for content to register the listener before the sandbox runs (especially for document-end scripts). Content could probably .stopPropagation() (or .stopImmediatePropagation()?) the event.

Now I'm thinking of inserting an installation-random value into the event name. At that point it's infeasible for content to attach the right event listener, how do you feel about that?

[arantius]

Version 3.3beta2 ( https://addons.mozilla.org/firefox/downloads/file/329720/greasemonkey-3.3beta2-fx.xpi?src=devhub ) contains the above-discussed enhancements. All testing is highly appreciated!

[zoocey]

On Firefox 40 (64 bit) beta under Windows 7, trying GM 3.3beta2. Testing my usual scripts that caused ghost windows and lag (in minutes with prior GM versions) and after ~ 4 hours I have 0 ghost windows. Will comment again if that changes.

[yarrmateys]

try on firefox 39. ff40 had some changes that apparently fixed or reduced the problem, that ff39 lacked, according to #2200 (comment)

[Ventero]

No ghost windows whatsoever after using current git HEAD in FF39 over the weekend, both with general usage and also when following the steps with which I could 100% reproduce a ghost window before.

(Ab)Using the content window to communicate with the sandbox still feels a bit iffy to me, but I can't argue with the results. With random event names, I also don't see a way how untrusted content could interfere with the events anymore, so +1 from me.

[Alhaitham]

Positive reports here

Had the same issue with memory leaks with scripts that use "GM_registerMenuCommand" which is finally fixed in the latest dev build.

And

Wow, this was the fix for me as well. The last two months have been hell with lagging and freezing issues. I initially thought it was the Firefox 39 beta as it released at about the same time as Greasemonkey 3.2. My memory usage is much smaller over time now. Thanks!

[yarrmateys]

i'll join in on the positive reports as well. the beta seems to work much better than 3.2, at similar level as 2.3.1 did. i no longer have extreme browser lag and/or 3+ gb ram usage with it.

i've been running firefox nonstop for about a day, ram is still hovering around 700 mb-1 gb, cpu use is at less than 5%, and that's with nearly 50 tabs open, 2 of them being youtube with yt center.

[arantius]

Awesome, thanks for the data! I'll get right on a 3.3 release.