-
-
Notifications
You must be signed in to change notification settings - Fork 30k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
pickling of large recursive structures crashes cPickle #46954
Comments
The documentation1 says: Trying to pickle a highly recursive data structure may exceed the The lightweight pickle module handles this problem correctly (in that it This was mentioned in bpo-2480, but that has now been changed to a |
What operating system and compiler are you using? |
Python 2.5 (r25:51908, Sep 19 2006, 09:52:17) [MSC v.1310 32 bit (Windows XP Professional 32 bits) |
Could you please tell me if this problem arises with this test? |
I have just quickly pasted it into an interpreter. Python 2.5 (r25:51908, Sep 19 2006, 09:52:17) [MSC v.1310 32 bit
(Intel)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> class Node(object):
... pass
...
>>> nodes = [Node() for i in range(500)]
>>> for n in nodes:
... n.connections = list(nodes)
... n.connections.remove(n)
...
>>> import cPickle
>>> s = cPickle.dumps( n ) After this line, the interpreter terminated without any further output Is it sufficient, or would you prefer me to run the test properly? |
Daniel, it'd be great, because it does not crash in linux, so I can not |
It works for me as a test case too: test_deep_recursive (main.cPickleDeepRecursive) ... ERROR ====================================================================== Traceback (most recent call last):
File "c:\devel\Python25\Lib\test\test_cpickle.py", line 107, in
test_deep_recursive
self.assertRaises(RuntimeError, cPickle.dumps, n)
File "C:\devel\Python25\lib\unittest.py", line 320, in failUnlessRaises
callableObj(*args, **kwargs)
KeyError: 13352688 |
Commited this patch to the test cases, and the patch from bpo-3165 to fix |
I reverted the patch, because of bpo-3179, so I'm reopening this. Note that I left the test in the suite, but commented out (if you find a |
It seems that each self->nesting++ should be balanced with |
Another version of the patch, that removes the self->nesting member and |
The tests pass on my machine (64 bit debian testing) with cpickle2.patch. |
Great, Amaury, I applied your second patch... it make all tests pass ok, :) Commited in 64595. Thank you all!! |
Please check issue bpo-3338 for a dup of this issue with a testcase that |
Thanks Darryl. We'll continue in that issue, as the patched commited in this one did |
Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.
Show more details
GitHub fields:
bugs.python.org fields:
The text was updated successfully, but these errors were encountered: