Live Update Memory Growth

[briend]

Splitting this issue from mypaint/libmypaint#99

Using Live Update can cause enormous memory growth and potential memory exhaustion.

Tested on a fresh config of plain master (libmypaint and mypaint) on Debian Stretch, and also on Windows 10 with build 1.3.0-alpha+git.f03e5ec5. Was able to reproduce this memory issue with just a plain deevad 2b pencil and jitter assigned to random input.
Steps to reproduce:

0. run mypaint -c /tmp/temporary_profile
1. edit devaad 2b brush and assign jitter setting a random input all the way to +25 (default slope)
2. scribble around and observe memory usage. It should be pretty low and static
3. enable live update and begin dragging the random input left and right and observer the live changes on the canvas. You may see memory usage creep up a bit, but dragging the slider skips a lot of values.
4. using the arrow keys, move the slider continuously through all the values -25 thru +25 on the random input
5. Observer memory usage continuously grow beyond 1GB

The memory does not seem to be released until you quit mypaint. Also, after the memory growth it seems like mypaint peformance lags considerably with complex brushes for a few minutes and then speeds up again, although the memory consumption remains high. Changing the brush can also make the performance go back to normal.

Even if you attach the same setting to "Stroke" you won't see the memory issue. I would think a stroke input w/ a long stroke length would be very similar to moving the sliders through all the values, but no, Live Update behaves totally differently than an actual brush. Also it's probably not related to the actual slope of mappings since I originally observed the issue with just changing the base_value on a setting (the beta gribmap setting).

[achadwick]

That sounds weird. It should be bouncing on the undo stack for that, and that ought never to grow that big. It shouldn't ever be pushing more than one thing total too.

Does forcing a gc fix it, or reveal any non-collectable objects?

cPython memory allocation is a bit odd, in that it's basically a big hash table whose allocation never diminishes even if objects get collected. The space will get reused, however.

[briend]

Actually on windows the memory does go back down pretty quickly after I stop dragging the sliders. It wasn't doing that on linux. I'll try garbage collection on linux. . .

[briend]

Ok I ran the "Print Memory Leak info to console" before and after doing the tasks to create 2GB of memory usage. The one that pops out to me is: INFO: lib.helpers: MEM: DELTA +1542 Processor <class 'lib.idletask.Processor'>

INFO: lib.helpers: MEM: garbage collector run, collected 1110 objects
INFO: lib.helpers: MEM: gc.garbage contains 0 items of uncollectible garbage
INFO: lib.helpers: MEM: collecting info (can take some time)...
INFO: lib.helpers: MEM: ...done collecting.
INFO: lib.helpers: MEM: Stored stats to compare with the next info collection.
INFO: lib.helpers: MEM: garbage collector run, collected 0 objects
INFO: lib.helpers: MEM: gc.garbage contains 0 items of uncollectible garbage
INFO: lib.helpers: MEM: collecting info (can take some time)...
INFO: lib.helpers: MEM: ...done collecting.
INFO: lib.helpers: MEM: DELTA +1 Actionable <class 'gi.repository.Gtk.Actionable'>
INFO: lib.helpers: MEM: DELTA +1 Activatable <class 'gi.repository.Gtk.Activatable'>
INFO: lib.helpers: MEM: DELTA +42 Adjustment <class 'gi.overrides.Gtk.Adjustment'>
INFO: lib.helpers: MEM: DELTA +1 Application <class 'gui.application.Application'>
INFO: lib.helpers: MEM: DELTA -2 AutosaveInfo <class 'lib.document.AutosaveInfo'>
INFO: lib.helpers: MEM: DELTA +1 Bin <class 'gi.repository.Gtk.Bin'>
INFO: lib.helpers: MEM: DELTA +1 BoundObserverMethod <class 'lib.observable.BoundObserverMethod'>
INFO: lib.helpers: MEM: DELTA +99 BrushEditorWindow <class 'gui.brusheditor.BrushEditorWindow'>
INFO: lib.helpers: MEM: DELTA +2 Brushwork <class 'lib.command.Brushwork'>
INFO: lib.helpers: MEM: DELTA +1 Buildable <class 'gi.repository.Gtk.Buildable'>
INFO: lib.helpers: MEM: DELTA +1 Builder <class 'gi.overrides.Gtk.Builder'>
INFO: lib.helpers: MEM: DELTA +17 Button <class 'gi.overrides.Gtk.Button'>
INFO: lib.helpers: MEM: DELTA +1 Button <class 'gi.repository.Gtk.Button'>
INFO: lib.helpers: MEM: DELTA +1 Container <class 'gi.overrides.Gtk.Container'>
INFO: lib.helpers: MEM: DELTA +1 Container <class 'gi.repository.Gtk.Container'>
INFO: lib.helpers: MEM: DELTA +75 CurveWidget <class 'gui.curve.CurveWidget'>
INFO: lib.helpers: MEM: DELTA +3 GObjectMeta <class 'gi.types.GObjectMeta'>
INFO: lib.helpers: MEM: DELTA +1 HSVColor <class 'lib.color.HSVColor'>
INFO: lib.helpers: MEM: DELTA +1 ImplementorIface <class 'gi.repository.Atk.ImplementorIface'>
INFO: lib.helpers: MEM: DELTA +1 InitiallyUnowned <class 'gi.repository.GObject.InitiallyUnowned'>
INFO: lib.helpers: MEM: DELTA +7 Keymap <class 'gi.repository.Gdk.Keymap'>
INFO: lib.helpers: MEM: DELTA +56 Label <class 'gi.overrides.Gtk.Label'>
INFO: lib.helpers: MEM: DELTA +1 MethodWithObservers <class 'lib.observable.MethodWithObservers'>
INFO: lib.helpers: MEM: DELTA +5 Object <class 'gi.overrides.GObject.Object'>
INFO: lib.helpers: MEM: DELTA +3 Object <class 'gi.repository.GObject.Object'>
INFO: lib.helpers: MEM: DELTA +2 Orientable <class 'gi.repository.Gtk.Orientable'>
INFO: lib.helpers: MEM: DELTA -2 ParseResult <class 'urlparse.ParseResult'>
INFO: lib.helpers: MEM: DELTA +3 Pixbuf <class 'gi.repository.GdkPixbuf.Pixbuf'>
INFO: lib.helpers: MEM: DELTA +1542 Processor <class 'lib.idletask.Processor'>
INFO: lib.helpers: MEM: DELTA +1 PyGIDeprecationWarning <class 'gi.PyGIDeprecationWarning'>
INFO: lib.helpers: MEM: DELTA +3 PyGTKDeprecationWarning <class 'gi.overrides.Gtk.PyGTKDeprecationWarning'>
INFO: lib.helpers: MEM: DELTA -1 RGBColor <class 'lib.color.RGBColor'>
INFO: lib.helpers: MEM: DELTA +4 ScaleButton <class 'gi.repository.Gtk.ScaleButton'>
INFO: lib.helpers: MEM: DELTA -2 SplitResult <class 'urlparse.SplitResult'>
INFO: lib.helpers: MEM: DELTA -1 StateDirs <class 'gui.application.StateDirs'>
INFO: lib.helpers: MEM: DELTA +13 StateType <class 'gi.repository.Gtk.StateType'>
INFO: lib.helpers: MEM: DELTA +2 Stroke <class 'lib.stroke.Stroke'>
INFO: lib.helpers: MEM: DELTA +2 StrokeShape <class 'lib.strokemap.StrokeShape'>
INFO: lib.helpers: MEM: DELTA +4 StrokemappedPaintingLayerSnapshot <class 'lib.layer.data.StrokemappedPaintingLayerSnapshot'>
INFO: lib.helpers: MEM: DELTA +63 TreePath <class 'gi.overrides.Gtk.TreePath'>
INFO: lib.helpers: MEM: DELTA +1 TreeStore <class 'gi.overrides.Gtk.TreeStore'>
INFO: lib.helpers: MEM: DELTA +18 Weight <class 'gi.repository.Pango.Weight'>
INFO: lib.helpers: MEM: DELTA +1 Widget <class 'gi.overrides.Gtk.Widget'>
INFO: lib.helpers: MEM: DELTA +1 Widget <class 'gi.repository.Gtk.Widget'>
INFO: lib.helpers: MEM: DELTA +20 WindowTypeHint <class 'gi.repository.Gdk.WindowTypeHint'>
INFO: lib.helpers: MEM: DELTA +4 X11Keymap <class 'gi.repository.GdkX11.X11Keymap'>

Here is a sample of me just loading mypaint, running the "Print Memory Leak.." and then just doodling for a bit with the same pencil (and running Print memory leak again). This is NOT going into the brush editor at all:

INFO: lib.helpers: MEM: garbage collector run, collected 1389 objects
INFO: lib.helpers: MEM: gc.garbage contains 0 items of uncollectible garbage
INFO: lib.helpers: MEM: collecting info (can take some time)...
INFO: lib.helpers: MEM: ...done collecting.
INFO: lib.helpers: MEM: Stored stats to compare with the next info collection.
INFO: lib.helpers: MEM: garbage collector run, collected 2332 objects
INFO: lib.helpers: MEM: gc.garbage contains 0 items of uncollectible garbage
INFO: lib.helpers: MEM: collecting info (can take some time)...
INFO: lib.helpers: MEM: ...done collecting.
INFO: lib.helpers: MEM: DELTA -2 AutosaveInfo <class 'lib.document.AutosaveInfo'>
INFO: lib.helpers: MEM: DELTA +17 Brushwork <class 'lib.command.Brushwork'>
INFO: lib.helpers: MEM: DELTA +3 GObjectMeta <class 'gi.types.GObjectMeta'>
INFO: lib.helpers: MEM: DELTA +12 GrabStatus <class 'gi.repository.Gdk.GrabStatus'>
INFO: lib.helpers: MEM: DELTA +4 IconInfo <class 'gi.repository.Gtk.IconInfo'>
INFO: lib.helpers: MEM: DELTA +7 Keymap <class 'gi.repository.Gdk.Keymap'>
INFO: lib.helpers: MEM: DELTA +7 Object <class 'gi.overrides.GObject.Object'>
INFO: lib.helpers: MEM: DELTA +3 Object <class 'gi.repository.GObject.Object'>
INFO: lib.helpers: MEM: DELTA -2 ParseResult <class 'urlparse.ParseResult'>
INFO: lib.helpers: MEM: DELTA +1 Pixbuf <class 'gi.repository.GdkPixbuf.Pixbuf'>
INFO: lib.helpers: MEM: DELTA +17 Processor <class 'lib.idletask.Processor'>
INFO: lib.helpers: MEM: DELTA +1 PyGIDeprecationWarning <class 'gi.PyGIDeprecationWarning'>
INFO: lib.helpers: MEM: DELTA -2 SplitResult <class 'urlparse.SplitResult'>
INFO: lib.helpers: MEM: DELTA -1 StateDirs <class 'gui.application.StateDirs'>
INFO: lib.helpers: MEM: DELTA +13 StateType <class 'gi.repository.Gtk.StateType'>
INFO: lib.helpers: MEM: DELTA +17 Stroke <class 'lib.stroke.Stroke'>
INFO: lib.helpers: MEM: DELTA +17 StrokeShape <class 'lib.strokemap.StrokeShape'>
INFO: lib.helpers: MEM: DELTA +34 StrokemappedPaintingLayerSnapshot <class 'lib.layer.data.StrokemappedPaintingLayerSnapshot'>
INFO: lib.helpers: MEM: DELTA +1 X11Cursor <class 'gi.repository.GdkX11.X11Cursor'>
INFO: lib.helpers: MEM: DELTA +4 X11Keymap <class 'gi.repository.GdkX11.X11Keymap'>