File objects should not pickleable

[pitrou]

BPO	10180
Nosy	@birkenfeld, @amauryfa, @pitrou, @avassalotti, @benjaminp, @alex
Dependencies	bpo-10173: Don't pickle TestCase instances in test_multiprocessing
Files	pickleio.patch

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = <Date 2010-11-05.21:16:14.377>
created_at = <Date 2010-10-23.17:19:26.446>
labels = ['type-bug', 'library', 'expert-IO']
title = 'File objects should not pickleable'
updated_at = <Date 2010-11-05.21:16:14.376>
user = 'https://github.com/pitrou'

bugs.python.org fields:

activity = <Date 2010-11-05.21:16:14.376>
actor = 'pitrou'
assignee = 'none'
closed = True
closed_date = <Date 2010-11-05.21:16:14.377>
closer = 'pitrou'
components = ['Library (Lib)', 'IO']
creation = <Date 2010-10-23.17:19:26.446>
creator = 'pitrou'
dependencies = ['10173']
files = ['19344']
hgrepos = []
issue_num = 10180
keywords = ['patch']
message_count = 9.0
messages = ['119450', '119457', '119459', '119470', '119471', '119473', '119474', '119475', '120536']
nosy_count = 6.0
nosy_names = ['georg.brandl', 'amaury.forgeotdarc', 'pitrou', 'alexandre.vassalotti', 'benjamin.peterson', 'alex']
pr_nums = []
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue10180'
versions = ['Python 3.2']

[pitrou]

In Python 3, pickle accepts to serialize a file object, but the result is nonsensical when unpickled. I think we should explicitly forbid pickling of FileIO, Buffered{Reader,Writer} and TextIOWrapper objects.

[birkenfeld]

Sounds like a good idea to me.

[pitrou]

Here is a patch, but bpo-10173 must probably be fixed first.

[amauryfa]

The patch modifies _io.TextIOWrapper, but not _pyio.TextIOWrapper. Is there a reason?

[pitrou]

The patch modifies _io.TextIOWrapper, but not _pyio.TextIOWrapper. Is
there a reason?

Yes, two of them:

• modifying _pyio.Buffered* is enough to trigger the TypeError
• _pyio.StringIO inherits from TextIOWrapper, and it must be pickleable

[alex]

I don't see why Buffered or TextIO's shouldn't be pickleable, ISTM their pickleability should be based on what the underlying file obj is.

[birkenfeld]

What would be the use case for that?

[pitrou]

I don't see why Buffered or TextIO's shouldn't be pickleable, ISTM
their pickleability should be based on what the underlying file obj
is.

That could be. Right now, though, pickling them gives nonsensical
results and I think it would be better to give an explicit error
message.

(a problem with making them pickleable is that, by exposing details of
the internal structure in the pickle, you're limiting what
implementation changes you can do in the future)

[pitrou]

Committed in r86220. I won't backport it since it would risk breaking existing code, although relying on this is really a bug in itself.