cPickle is seriously broken

[schmir]

BPO	3179
Nosy	@facundobatista, @jcea
Files	test-3179.diff: test case

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = 'https://github.com/facundobatista'
closed_at = <Date 2008-06-25.19:28:08.896>
created_at = <Date 2008-06-23.14:42:12.649>
labels = ['type-bug']
title = 'cPickle is seriously broken'
updated_at = <Date 2008-06-26.07:21:01.320>
user = 'https://bugs.python.org/schmir'

bugs.python.org fields:

activity = <Date 2008-06-26.07:21:01.320>
actor = 'jcea'
assignee = 'facundobatista'
closed = True
closed_date = <Date 2008-06-25.19:28:08.896>
closer = 'facundobatista'
components = []
creation = <Date 2008-06-23.14:42:12.649>
creator = 'schmir'
dependencies = []
files = ['10708']
hgrepos = []
issue_num = 3179
keywords = ['patch']
message_count = 8.0
messages = ['68629', '68631', '68632', '68633', '68635', '68669', '68670', '68746']
nosy_count = 4.0
nosy_names = ['facundobatista', 'jcea', 'schmir', 'cuerty']
pr_nums = []
priority = 'normal'
resolution = 'fixed'
stage = None
status = 'closed'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue3179'
versions = ['Python 2.6']

[schmir]

import cPickle

res=[]
for x in range(1,2000):
    res.append(dict(doc=x, similar=[]))

cPickle.dumps(res)

Traceback (most recent call last):
  File "pi.py", line 10, in <module>
cPickle.dumps(res)
RuntimeError: maximum recursion depth exceeded

svn r64471 seems to cause the problematic behaviour.

facundo, you committed that one.

[facundobatista]

This example works before my patch, exactly!

The reason of the patch I applied is that in some cases a Recursion
error should be raised, but it didn't happen, causing some serious
issues later.

I'm putting in copy to cuerty, for him to analyze if this case should
have caused Recursion error in the first place, or not.

In any case, it's a good example. Ralf, would you mind to generate a
test case from this example?

Thank you all!

[schmir]

Apparently there are some self->nesting-- calls missing in batch_list
and batch_dict.

[schmir]

Of course it should not raise an RecursionError.

for reference:
http://bugs.python.org/issue2702 is the original bugreport.

[schmir]

here is a test case. I cannot run it however:
~/pydev/trunk/ ./python Lib/test/test_cpickle.py                   
ralf@red ok
Traceback (most recent call last):
  File "Lib/test/test_cpickle.py", line 3, in <module>
from test.pickletester import AbstractPickleTests, AbstractPickleModuleTests
ImportError: No module named pickletester

???

[schmir]

the test works as expected (i.e. it fails). The problem I had was that
some Bittorrent bencode also installed a test package for me.

======================================================================
ERROR: test_flat_list (main.cPickleFlatList)
----------------------------------------------------------------------

Traceback (most recent call last):
  File "Lib/test/test_cpickle.py", line 113, in test_flat_list
    cPickle.dumps(lst)
RuntimeError: maximum recursion depth exceeded

---

Ran 159 tests in 0.457s

FAILED (errors=1)
Traceback (most recent call last):
  File "Lib/test/test_cpickle.py", line 128, in <module>
test_main()
  File "Lib/test/test_cpickle.py", line 124, in test_main
cPickleFlatList,
  File "/home/ralf/pydev/trunk/Lib/test/test_support.py", line 714, in
run_unittest
_run_suite(suite)
  File "/home/ralf/pydev/trunk/Lib/test/test_support.py", line 697, in
_run_suite
raise TestFailed(err)
test.test_support.TestFailed: Traceback (most recent call last):
  File "Lib/test/test_cpickle.py", line 113, in test_flat_list
    cPickle.dumps(lst)
RuntimeError: maximum recursion depth exceeded

[schmir]

btw. this should be a release blocker.

[facundobatista]

I reverted the patch, commented out the previous test, and included this
one in the test suite, to never break it again, ;)

I hope we now find a solution to the issue bpo-2702.

Thank you!!