Join GitHub today
GitHub is home to over 28 million developers working together to host and review code, manage projects, and build software together.Sign up
gevent.local.local() memory leak #981
There's a memory leak when making
What I've run:
You can reproduce the memory leak by the below simple code:
import gc import gevent.local def main(): while True: gevent.local.local() gc.collect() gevent.spawn(main).join()
The growth is much faster on PyPy.
It is by design that the local dictionaries aren't cleared until the greenlet exits, so I don't believe that's a leak. If you allocate and never destroy the greenlet they will not be cleared. I believe the prior implementation had the same behaviour (in fact it may have been even less timely).
Most locals are allocated at a static scope (e.g., module or class level) and assigned to a variable from which they are reachable, or occasionally they are allocated as instance attributes, and again, assigned to a variable. They have to be named to be useful, so simply allocating them in a loop is not a good representation of a real scenario.
As you said, we can make a local for an instance attribute. At this case, if we have some infinite greenlets that have touched the local at least once, the callbacks from the local will never be released even though the instance is deleted.
I just wanted to show you the memory leak from deleted locals with the shortest code. So I put the allocation without name in the loop. The below example would be more practical:
import gc import gevent.local class O(object): def __init__(self): self.local = gevent.local.local() def main_loop(): x = 0 objs =  while True: o = O() objs.append(o) x += 1 if x % 1000 == 0: del objs[:] gc.collect() gevent.spawn(main_loop).join()