cPickle.dumps change after for

[marcoberi]

BPO	640645
Nosy	@mwhudson, @tim-one, @arigo

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = <Date 2004-01-12.12:30:57.000>
created_at = <Date 2002-11-19.11:42:57.000>
labels = ['invalid', 'library']
title = 'cPickle.dumps change after for'
updated_at = <Date 2004-01-12.12:30:57.000>
user = 'https://bugs.python.org/marcoberi'

bugs.python.org fields:

activity = <Date 2004-01-12.12:30:57.000>
actor = 'arigo'
assignee = 'none'
closed = True
closed_date = None
closer = None
components = ['Library (Lib)']
creation = <Date 2002-11-19.11:42:57.000>
creator = 'marcoberi'
dependencies = []
files = []
hgrepos = []
issue_num = 640645
keywords = []
message_count = 9.0
messages = ['13394', '13395', '13396', '13397', '13398', '13399', '13400', '13401', '13402']
nosy_count = 4.0
nosy_names = ['mwh', 'tim.peters', 'arigo', 'marcoberi']
pr_nums = []
priority = 'normal'
resolution = 'not a bug'
stage = None
status = 'closed'
superseder = None
type = None
url = 'https://bugs.python.org/issue640645'
versions = []

[marcoberi]

try this program:

import cPickle
a=([{}])
print cPickle.dumps(a)
for x in a:
  pass
cPickle.dumps(a)

It prints:
(lp1
(da.

Why on earth it does dumps return different string?
And after just a for with a pass in it...

[mwhudson]

Logged In: YES
user_id=6656

Umm:

>>> a=([{}])
>>> cPickle.dumps(a)
'(lp1\n(da.'

you're only printing the first pickle...

[marcoberi]

Logged In: YES
user_id=588604

I forgot to print the second dumps but the bug remains:
Before the for:
(lp1
(da.
After the for:
(lp1
(dp2
a.

[mwhudson]

Logged In: YES
user_id=6656

You're right, sorry.

[mwhudson]

Logged In: YES
user_id=6656

OK, I think I know what's happening.

Look at this:

>>> a = {}       
>>> cPickle.dumps(a)
'(dp1\n.'
>>> cPickle.dumps({})
'(d.'

the only difference is in the number of references to the
object getting pickled. I don't know why that matters, but
it seems to.

Why does this matter to you?

[marcoberi]

Logged In: YES
user_id=588604

I would like to use hash(cPickle.dumps(a)) to have a unique
hash also for dict that aren't hashable.

[mwhudson]

Logged In: YES
user_id=6656

Well, that's asking for something you're not promised. It
takes some imagination to categorize this as a bug, and I'm
not going to put any of my time into fixing it.

It seems using pickle in place of cPickle *may* work, fwiw.

[tim-one]

Logged In: YES
user_id=31435

Closed as Not-A-Bug. The internals of pickle strings aren't
guaranteed, just that "they work" when unpickled again, and
these do. If you want a hash code for a dict, don't dare use
pickle for this either, even if it appears "to work": it doesn't.
The order in which dict keys are enumerated isn't defined
either, and can and does vary across releases, and even
across program runs.

So a reliable hash code for a dict needs to be independent of
key iteration order, and no version of pickle spends time
trying to force that issue (waste of time -- it isn't needed for
pickling or unpickling).

[arigo]

Logged In: YES
user_id=4771

Someone else ran into the problem of unexpectedly varying
cPickle strings:

http://sourceforge.net/tracker/?func=detail&atid=429622&aid=875331&group_id=41036

Maybe this behavior should be documented.