New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
itertools.groupby() leaks memory with circular reference #46499
Comments
Quoting from my email to Raymond: In the Trac/Genshi community we've been tracking a bit obscure memory Please see http://trac.edgewall.org/ticket/6614 and then We reduced the case to the following Python only code and believe it is "Looks like genshi is not to blame. itertools.groupby has a grouper This can be demonstrated with the following code (testcase attachment import gc
from itertools import groupby
def run():
keyfunc = lambda x: x
for i, j in groupby(range(100), key=keyfunc):
keyfunc.x = j
for x in xrange(20):
gc.collect()
run()
print len(gc.get_objects()) On executing this in will show numerical output of the garbage "a frame, a grouper, a keyfunc and a groupby object" We have been unable to come up with a decent patch and thus I am |
With the following patch: =================================================================== --- Lib/test/test_itertools.py (revision 61284)
+++ Lib/test/test_itertools.py (working copy)
@@ -707,6 +707,12 @@
a = []
self.makecycle(takewhile(bool, [1, 0, a, a]), a)
+ def test_issue2246(self):
+ n = 10
+ keyfunc = lambda x: x
+ for i, j in groupby(xrange(n), key=keyfunc):
+ keyfunc.__dict__.setdefault('x',[]).append(j)
+
def R(seqn):
'Regular generator'
for i in seqn: $ ./python Lib/test/regrtest.py -R :: test_itertools reports n*3 + 13 reference leaks. This should give a clue ... |
It looks like the problem is that the internal grouper object becomes a |
No need. I'm already working on adding GC to the grouper. |
Oops. Here is my patch anyways. |
Damn, I wrote a patch too ;) |
r61286. Applied a patch substantially similar to Alexanders. Thanks |
Backport candidate |
Already backported in r61287. |
Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.
Show more details
GitHub fields:
bugs.python.org fields:
The text was updated successfully, but these errors were encountered: