Handles memory is ever growing and causes a memory leak

[sb2nov]

https://github.com/go-python/gopy/blob/master/gopyh/handle.go#L35 this map seems to be ever growing. This causes my python process to use more memory indefinitely.

Can you explain how are handles garbage collected?

[rcoreilly]

Yep that is a known problem with the design. In theory it might be possible to tap into the python garbage collector to delete unused items on the map, but that is probably a pretty tricky thing -- I haven't looked into it though (maybe it would not be too hard?)

In the mean time, trying to organize things so that the python-accessed part remains at a fairly broad outer-loop level would help mitigate this issue.

It might also be useful at some point to revisit some of the slice and map access code to make sure it is doing things as efficiently as possible.

[sb2nov]

Thanks for confirming. I was trying to use some go code in a Pyspark map so running it in an outer loop is not possible.

[rcoreilly]

this is now fixed and can be closed due to Cosmos Nicolau's fixes, right?

[sbinet]

yes. feel free to reopen if that's not the case.