Memory Leak Fix(es)

[WillAyd]

closes #155

this fixes the largest issue

valgrind still reports the following errors which are suspicious. The line 583 line is particularly suspect because we aren't introducing any new memory management structures in pantab at that point in time. Perhaps a leak in pandas?

==13638== 24 bytes in 1 blocks are definitely lost in loss record 406 of 1,789
==13638==    at 0x4845899: malloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==13638==    by 0x1815E3: raw_malloc (_tracemalloc.c:254)
==13638==    by 0x1815E3: traceback_new (_tracemalloc.c:444)
==13638==    by 0x18E33E: _PyTraceMalloc_NewReference (_tracemalloc.c:1798)
==13638==    by 0x192B15: _Py_NewReference (object.c:1825)
==13638==    by 0x192B15: PyList_New.cold.1512 (listobject.c:144)
==13638==    by 0x2D40FD: _PyEval_EvalFrameDefault (ceval.c:2753)
==13638==    by 0x28B752: _PyEval_EvalFrame (pycore_ceval.h:40)
==13638==    by 0x28B752: function_code_fastcall (call.c:330)
==13638==    by 0x28B752: _PyFunction_Vectorcall (call.c:367)
==13638==    by 0x26C3B6: _PyObject_VectorcallTstate (abstract.h:118)
==13638==    by 0x26C3B6: PyObject_CallOneArg (abstract.h:188)
==13638==    by 0x26C3B6: property_descr_get (descrobject.c:1573)
==13638==    by 0x23B733: _PyObject_GenericGetAttrWithDict (object.c:1201)
==13638==    by 0x23B733: PyObject_GenericGetAttr (object.c:1280)
==13638==    by 0x23B733: PyObject_GetAttr (object.c:890)
==13638==    by 0x2A90DA: PyObject_GetAttrString (object.c:795)
==13638==    by 0x4868025: write_to_hyper (writer.c:575)
==13638==    by 0x27C737: cfunction_call (methodobject.c:552)
==13638==    by 0x24C9EE: _PyObject_MakeTpCall (call.c:191)

==13638== 24 bytes in 1 blocks are definitely lost in loss record 407 of 1,789
==13638==    at 0x4845899: malloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==13638==    by 0x1815E3: raw_malloc (_tracemalloc.c:254)
==13638==    by 0x1815E3: traceback_new (_tracemalloc.c:444)
==13638==    by 0x181654: tracemalloc_add_trace (_tracemalloc.c:526)
==13638==    by 0x198D5D: tracemalloc_alloc (_tracemalloc.c:596)
==13638==    by 0x198DDC: tracemalloc_alloc_gil (_tracemalloc.c:695)
==13638==    by 0x198DDC: tracemalloc_malloc_gil (_tracemalloc.c:705)
==13638==    by 0x29CB53: PyObject_Malloc (obmalloc.c:685)
==13638==    by 0x29CB53: _PyObject_New (object.c:165)
==13638==    by 0x29CB53: make_range_object (rangeobject.c:58)
==13638==    by 0x29CB53: range_from_array (rangeobject.c:123)
==13638==    by 0x29CB53: range_vectorcall (rangeobject.c:153)
==13638==    by 0x202EBA: _PyObject_VectorcallTstate (abstract.h:118)
==13638==    by 0x202EBA: PyObject_Vectorcall (abstract.h:127)
==13638==    by 0x202EBA: call_function (ceval.c:5075)
==13638==    by 0x202EBA: _PyEval_EvalFrameDefault.cold.2975 (ceval.c:3518)
==13638==    by 0x28B752: _PyEval_EvalFrame (pycore_ceval.h:40)
==13638==    by 0x28B752: function_code_fastcall (call.c:330)
==13638==    by 0x28B752: _PyFunction_Vectorcall (call.c:367)
==13638==    by 0x26C3B6: _PyObject_VectorcallTstate (abstract.h:118)
==13638==    by 0x26C3B6: PyObject_CallOneArg (abstract.h:188)
==13638==    by 0x26C3B6: property_descr_get (descrobject.c:1573)
==13638==    by 0x23B733: _PyObject_GenericGetAttrWithDict (object.c:1201)
==13638==    by 0x23B733: PyObject_GenericGetAttr (object.c:1280)
==13638==    by 0x23B733: PyObject_GetAttr (object.c:890)
==13638==    by 0x2A90DA: PyObject_GetAttrString (object.c:795)
==13638==    by 0x4868025: write_to_hyper (writer.c:575)

==13638== 24 bytes in 1 blocks are definitely lost in loss record 409 of 1,789
==13638==    at 0x4845899: malloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==13638==    by 0x1815E3: raw_malloc (_tracemalloc.c:254)
==13638==    by 0x1815E3: traceback_new (_tracemalloc.c:444)
==13638==    by 0x181654: tracemalloc_add_trace (_tracemalloc.c:526)
==13638==    by 0x198D5D: tracemalloc_alloc (_tracemalloc.c:596)
==13638==    by 0x198DDC: tracemalloc_alloc_gil (_tracemalloc.c:695)
==13638==    by 0x198DDC: tracemalloc_malloc_gil (_tracemalloc.c:705)
==13638==    by 0x252F39: PyObject_Malloc (obmalloc.c:685)
==13638==    by 0x252F39: _PyLong_New (longobject.c:259)
==13638==    by 0x289092: PyLong_FromLongLong (longobject.c:1176)
==13638==    by 0x289092: builtin_len (bltinmodule.c:1549)
==13638==    by 0x25EF1A: cfunction_vectorcall_O (methodobject.c:516)
==13638==    by 0x202EBA: _PyObject_VectorcallTstate (abstract.h:118)
==13638==    by 0x202EBA: PyObject_Vectorcall (abstract.h:127)
==13638==    by 0x202EBA: call_function (ceval.c:5075)
==13638==    by 0x202EBA: _PyEval_EvalFrameDefault.cold.2975 (ceval.c:3518)
==13638==    by 0x28B752: _PyEval_EvalFrame (pycore_ceval.h:40)
==13638==    by 0x28B752: function_code_fastcall (call.c:330)
==13638==    by 0x28B752: _PyFunction_Vectorcall (call.c:367)
==13638==    by 0x28562D: _PyObject_VectorcallTstate (abstract.h:118)
==13638==    by 0x28562D: vectorcall_unbound (typeobject.c:1520)
==13638==    by 0x28562D: vectorcall_method (typeobject.c:1552)
==13638==    by 0x28562D: slot_sq_length (typeobject.c:6380)
==13638==    by 0x4868079: write_to_hyper (writer.c:583)

==13638== 48 bytes in 2 blocks are definitely lost in loss record 853 of 1,789
==13638==    at 0x4845899: malloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==13638==    by 0x1815E3: raw_malloc (_tracemalloc.c:254)
==13638==    by 0x1815E3: traceback_new (_tracemalloc.c:444)
==13638==    by 0x18E33E: _PyTraceMalloc_NewReference (_tracemalloc.c:1798)
==13638==    by 0x28B858: _Py_NewReference (object.c:1825)
==13638==    by 0x28B858: frame_alloc (frameobject.c:778)
==13638==    by 0x28B858: _PyFrame_New_NoTrack (frameobject.c:885)
==13638==    by 0x28B858: function_code_fastcall (call.c:319)
==13638==    by 0x28B858: _PyFunction_Vectorcall (call.c:367)
==13638==    by 0x26C3B6: _PyObject_VectorcallTstate (abstract.h:118)
==13638==    by 0x26C3B6: PyObject_CallOneArg (abstract.h:188)
==13638==    by 0x26C3B6: property_descr_get (descrobject.c:1573)
==13638==    by 0x23B733: _PyObject_GenericGetAttrWithDict (object.c:1201)
==13638==    by 0x23B733: PyObject_GenericGetAttr (object.c:1280)
==13638==    by 0x23B733: PyObject_GetAttr (object.c:890)
==13638==    by 0x2D2D30: _PyEval_EvalFrameDefault (ceval.c:2994)
==13638==    by 0x28B752: _PyEval_EvalFrame (pycore_ceval.h:40)
==13638==    by 0x28B752: function_code_fastcall (call.c:330)
==13638==    by 0x28B752: _PyFunction_Vectorcall (call.c:367)
==13638==    by 0x26C3B6: _PyObject_VectorcallTstate (abstract.h:118)
==13638==    by 0x26C3B6: PyObject_CallOneArg (abstract.h:188)
==13638==    by 0x26C3B6: property_descr_get (descrobject.c:1573)
==13638==    by 0x23B733: _PyObject_GenericGetAttrWithDict (object.c:1201)
==13638==    by 0x23B733: PyObject_GenericGetAttr (object.c:1280)
==13638==    by 0x23B733: PyObject_GetAttr (object.c:890)
==13638==    by 0x2A90DA: PyObject_GetAttrString (object.c:795)
==13638==    by 0x4868025: write_to_hyper (writer.c:575)

[WillAyd]

@maffka123

[WillAyd]

Ran this program under valgrind a few more times and it still leaks, but I can't find any of the leaks pointing back to the C code in pantab. You can see this by even commenting out the C calls from in _writer - leaks are still present.

The remaining leaks possibly go back to Python itself so going with this for now as I think its all the library can do. Thanks @maffka123 and if you find anything else let me know