Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Memory leak when using Typed Memory View and np array of objects (since 2013) #1638

Closed
ckald opened this issue Mar 20, 2017 · 5 comments
Closed

Comments

@ckald
Copy link

ckald commented Mar 20, 2017

Hello,

I encountered a problem with my compiled code: the memory is leaking with each call to the cpdef function. Memory profiling shows lots of memoryview objects remaining.

Googling quickly found a couple of similar bug reports, some of them pretty old. The main one, I think, is this one: https://www.mail-archive.com/cython-devel@python.org/msg03817.html

I checked the attached test there — the latest Cython 0.25.2 is leaking as well.

Please advise how to avoid this — my Cython functions are executed thousands to millions of times and 200 MB program grows to >10 GB because of the leak.

leaktest.tar.gz

@ckald
Copy link
Author

ckald commented Apr 24, 2017

Is anybody looking at these issues at all?

@stefanmh
Copy link

Bump. I'm experiencing this issue, as well.

Repro:

# Compile: cythonize -bi y.pyx

# Run: python -c 'import y; y.main()'

from cpython cimport array
import array
import time
import psutil


cdef f():
	cdef array.array a = array.array('d', [0.0] * 10000)
	cdef double[:] ca = a # Commenting this out prevents the leak

def mem():
	print str(round(psutil.Process().memory_info().rss/1024./1024., 2)) + ' MB'

def main():
	mem()
	for i in xrange(10000):
		f()
	mem()
	time.sleep(1.0)
	mem()

Output:

$ python -c 'import y; y.main()'
10.9 MB
774.6 MB
774.6 MB

Does anyone know of a workaround?

@stefanmh
Copy link

After applying the patch:

$ python -c 'import y; y.main()'
12.96 MB
13.23 MB
13.23 MB

@scoder
Copy link
Contributor

scoder commented Aug 18, 2017

Is this an issue only in Python 2.x or did anyone encounter it also in Python 3?

@stefanmh
Copy link

Only in Python2, by the looks of it:

$ python3 -c 'import y; y.main()'
11.64 MB
12.13 MB
12.13 MB

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

3 participants