New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
apparent memory leak using ctypes #69698
Comments
Attached file demonstrates a memory leak arising in ctypes (I think). Leak occurs in both 2.7 and 3.5, though at different rates. Humble apologies if I've screwed up somewhere. Very nervous about submitting this. |
I am running your script with the Python 3.5.0 from Arch Linux on an x86-64 computer. The reported numbers slowly increase and then plateau at 221 million: $ python ctype_stressor.py
1 0.0000000e+00 3.8010880e+07
100000 2.8538793e+00 1.4648934e+08
200000 2.9961207e+00 1.5189606e+08
300000 3.1383621e+00 1.5730278e+08
400000 3.2806034e+00 1.6270950e+08
500000 3.4512931e+00 1.6919757e+08
600000 3.5935345e+00 1.7460429e+08
700000 3.7357759e+00 1.8001101e+08
800000 3.8780172e+00 1.8541773e+08
900000 4.0202586e+00 1.9082445e+08
1000000 4.1909483e+00 1.9731251e+08
1100000 4.3331897e+00 2.0271923e+08
1200000 4.4754310e+00 2.0812595e+08
1300000 4.6176724e+00 2.1353267e+08
1400000 4.7599138e+00 2.1893939e+08
1500000 4.7883621e+00 2.2002074e+08
1600000 4.7883621e+00 2.2002074e+08
. . .
2700000 4.7883621e+00 2.2002074e+08
2800000 4.8043103e+00 2.2062694e+08
. . .
4100000 4.8043103e+00 2.2062694e+08
4200000 4.8133621e+00 2.2097101e+08
. . .
13000000 4.8133621e+00 2.2097101e+08 Monitoring the process with “htop”, I see the total virtual memory (VIRT) ramping up to about 79 MiB, and then jumping back to about 62 MiB, and the resident physical memory (RES) ramps within 36–53 MiB in a similar pattern. Similarly, Python 2.7.10 ramps within RES = 32–43 MiB, and your script is showing 177 million: 11400000 5.5962126e+00 1.7691443e+08 Is this the same as what you are seeing? How long do you have to you run your script for? The memory figures in your script seem to be out by a factor of four, assuming the third column is meant to be a number of bytes. But I don’t think there is a memory leak. At worst there is an unexpected dependency cycle between objects, meaning the memory builds up a bit until the garbage collector runs. |
cast() always creates a ref cycle, which is probably what you are seeing. >>> from ctypes import c_double, cast, POINTER
>>> samples = (c_double*23)()
>>> cast(samples, POINTER(c_double))
<__main__.LP_c_double object at 0x7f33bc920560>
>>> samples._objects
{139860183544448: <__main__.c_double_Array_23 object at 0x7f33bc8ef280>}
>>> samples
<__main__.c_double_Array_23 object at 0x7f33bc8ef280> |
I'm persuaded that this is not a sustained memory leak. Thanks to Martin Panter and Benjamin Peterson for the tests and the careful explanations. Apologies for distracting busy people. |
Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.
Show more details
GitHub fields:
bugs.python.org fields:
The text was updated successfully, but these errors were encountered: