webp decoder is leaking memory

[totaam]

You must also free the pixel buffer when dealing with RGB data, just after calling PyBytes_FromStringAndSize.

I have written a simple standalone test case you can run to see the damage for yourself:
http://xpra.org/trac/browser/xpra/trunk/src/tests/xpra/codecs/test_Pillow.py

And you can see our fix here (for Cython code, but you get the idea):
http://xpra.org/trac/changeset/5945/xpra

[wiredfool]

Thanks, I'll take a look at it.

[wiredfool]

Notes from pycon -- look at pep 454, in py3.4, using tracemalloc. In session on memory leaks.
can get the traceback where an object was allocated, and compute the difference between two snapshots. Can compute statistics per line number.

Ubunbu package backport for 2.7 and 3.3 and have to recompile extensions.

import tracemalloc
tracemalloc.start(25)
sn = tracemalloc.take_snapshot()
tb = tracmalloc.get_objject_traceback(obj)

[wiredfool]

More exploration:

• Doesn't appear to be in the python layer. Does appear to be in the webp.decode method isolated from just calling the c object, not the full load/open stack.
• Doesn't appear in garbage collection gc.get_count()
• Webp.output appears to be freed, pointers are coming up invalid after the freebuf call, however the rgba and yuba memory pointers are still pointing to the memory where they were, but the private memory is not. The contents are different each time, which seems to be incorrect for running the same image through on each iteration. Something happens that munges the buffers on the free call. I'm guessing that it's a gaurd pattern or somthing.
• both the rgb and the yuv pointers point to the private memory block, and that memory is freed.
• PyPy's mark and sweep garbage collector doesn't leak memory in the same test where c-python's refcounting one does. Or, if it does, it leaks an order of magnitude less. (12M vs 200M/1000 decodes)

So. New thought, we're not decrefing something. Not sure why it's not showing up in the garbage collection counter though.

[aclark4life]

Would be nice to get a fix for this in to 2.5.0

[aclark4life]

Targeting future

[wiredfool]

#1114 fixes this OMM.

[gumblex]

@wiredfool Thanks for the fix. I have tested about a GB's pictures, and there is no longer obvious leaking.