StringIO.StringIO pickled in 2.7 is not unpickleable on 3.x

[serhiy-storchaka]

BPO	22890
Nosy	@pitrou, @avassalotti, @serhiy-storchaka, @JelleZijlstra

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = <Date 2016-06-26.21:59:59.540>
created_at = <Date 2014-11-17.07:41:25.551>
labels = ['type-bug', 'library']
title = 'StringIO.StringIO pickled in 2.7 is not unpickleable on 3.x'
updated_at = <Date 2016-06-26.21:59:59.539>
user = 'https://github.com/serhiy-storchaka'

bugs.python.org fields:

activity = <Date 2016-06-26.21:59:59.539>
actor = 'serhiy.storchaka'
assignee = 'none'
closed = True
closed_date = <Date 2016-06-26.21:59:59.540>
closer = 'serhiy.storchaka'
components = ['Library (Lib)']
creation = <Date 2014-11-17.07:41:25.551>
creator = 'serhiy.storchaka'
dependencies = []
files = []
hgrepos = []
issue_num = 22890
keywords = []
message_count = 5.0
messages = ['231268', '236784', '236799', '267016', '269320']
nosy_count = 4.0
nosy_names = ['pitrou', 'alexandre.vassalotti', 'serhiy.storchaka', 'JelleZijlstra']
pr_nums = []
priority = 'normal'
resolution = 'wont fix'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue22890'
versions = ['Python 3.4', 'Python 3.5']

[serhiy-storchaka]

StringIO.StringIO is pickleable and unpickleable on 2.7 but is not unpickleable on 3.x.

Python 2.7:
>>> import pickle, StringIO
>>> pickle.dumps(StringIO.StringIO('abc'), 2)
'\x80\x02(cStringIO\nStringIO\nq\x00oq\x01}q\x02(U\tsoftspaceq\x03K\x00U\x07buflistq\x04]q\x05U\x03posq\x06K\x00U\x03lenq\x07K\x03U\x06closedq\x08\x89U\x03bufq\tU\x03abcq\nub.'
>>> pickle.loads(b'\x80\x02(cStringIO\nStringIO\nq\x00oq\x01}q\x02(U\tsoftspaceq\x03K\x00U\x07buflistq\x04]q\x05U\x03posq\x06K\x00U\x03lenq\x07K\x03U\x06closedq\x08\x89U\x03bufq\tU\x03abcq\nub.')
<StringIO.StringIO instance at 0xb70c122c>

On 3.x I got an error:
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: _io.StringIO.__setstate__ argument should be 4-tuple, got dict

[BreamoreBoy]

Seems fine on Windows 8.1

c:\Users\Mark\Documents\MyPython>c:\cpython\PCbuild\amd64\python.exe
Python 3.5.0a1+ (default:344d57c521b9+, Feb 27 2015, 13:39:56) [MSC v.1800 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import pickle, io
>>> pickle.dumps(io.StringIO('abc'), 2)
b'\x80\x02c_io\nStringIO\nq\x00)\x81q\x01(X\x03\x00\x00\x00abcq\x02X\x01\x00\x00\x00\nq\x03K\x00Ntq\x04b.'
>>> pickle.loads(b'\x80\x02c_io\nStringIO\nq\x00)\x81q\x01(X\x03\x00\x00\x00abcq\x02X\x01\x00\x00\x00\nq\x03K\x00Ntq\x04b.')
<_io.StringIO object at 0x0000004C6E604288>
>>>

[serhiy-storchaka]

The issue is about StringIO.StringIO, not io.StringIO.

[JelleZijlstra]

Unless I'm missing something, StringIO doesn't exist as a module in Python 3, as mentioned in https://docs.python.org/3.0/whatsnew/3.0.html#text-vs-data-instead-of-unicode-vs-8-bit.

If you're trying to unpickle in 3.x an object pickled by 2.7.x, I'm not sure there is a guarantee that unpickling works for objects in modules that were renamed between 2.x and 3.x. This also doesn't work if you pickle a ConfigParser in 2.7 and unpickle it in 3.x.

[serhiy-storchaka]

There is the _compat_pickle module that purposed to support backward and forward compatibility for renamed or moved classes and modules.

StringIO.StringIO in 2.x corresponds to io.StringIO or io.BytesIO. The problem is that we can't determine what of classes is a match in every concrete case, especially if StringIO.StringIO is empty.

But since cStringIO.StringIO is not pickleable either, I now think that it is not worth to support to support compatibility with pickled StringIO.StringIO instances.