New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

memory leak with typed ndarray and memory views #2828

Open
jhauschild opened this Issue Feb 8, 2019 · 1 comment

Comments

Projects
None yet
2 participants
@jhauschild
Copy link

jhauschild commented Feb 8, 2019

Hi,
I found a memory leak when using the np.ndarray[np.intp_t, ndim=2] syntax or memory views of dimension at least 2.
Strangely, the memory leak only appears if the second dimension is 0, irregardless of the other dimensions. I checked that it also appears with ndim=3 (also if the second dimension is zero).

Minimal example:
# a.pyx
# compile: cythonize -bi a.pyx
# run: python -c 'import a; a.main()'

cimport numpy as cnp
import numpy as np
import psutil

cnp.import_array()

def check_memleak(var):
    cdef cnp.ndarray[cnp.intp_t, ndim=2] c_var = var  # commenting this out prevents the leak
    #cdef cnp.intp_t[:, :] c_var = var  # memory views have the same problem
    return True

def mem():
    print(str(round(psutil.Process().memory_info().rss/1024./1024., 2)) + ' MB')

def main():
    for shape in [(2, 2), (2, 0), (0, 2), (0, 0)]:
        print("shape = ", shape)
        var = np.zeros(shape, dtype=np.intp)
        mem()
        for i in range(100000):
            check_memleak(var)
        mem()

Sample output:

$python -c 'import a; a.main()'
('shape = ', (2, 2))
27.95 MB
27.95 MB
('shape = ', (2, 0))
27.95 MB
43.81 MB
('shape = ', (0, 2))
40.21 MB
40.21 MB
('shape = ', (0, 0))
40.21 MB
43.82 MB

Any ideas what might cause this strange behaviour?

@jhauschild jhauschild changed the title memory leak with typed ndarray memory leak with typed ndarray and memory views Feb 12, 2019

@scoder

This comment has been minimized.

Copy link
Contributor

scoder commented Feb 15, 2019

I looked into this a bit and could reproduce it. Not sure where the problem is, though. It would help to see if moving either NumPy or Cython out of the way would show the same issue, so that we can see if it's due to Cython or NumPy (or a mix of both).

Also, arrays with a zero dimension, while certainly not useless, do not seem like a particularly critical use case to fix. That might also be the reason why it's not been found and/or reported so far.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment