Deprecate the 'U' open mode

[serhiy-storchaka]

BPO	15204
Nosy	@birkenfeld, @rhettinger, @pitrou, @bitdancer, @asvetlov, @cjerdonek, @vadmium, @serhiy-storchaka, @vajrasky
Files	deprecate-U-mode.patch deprecate-U-mode_2.patch deprecate-U-mode_3.patch

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = <Date 2013-11-24.21:15:30.677>
created_at = <Date 2012-06-27.10:29:50.642>
labels = ['type-bug', 'library', 'expert-IO', 'docs']
title = "Deprecate the 'U' open mode"
updated_at = <Date 2014-10-19.20:20:15.229>
user = 'https://github.com/serhiy-storchaka'

bugs.python.org fields:

activity = <Date 2014-10-19.20:20:15.229>
actor = 'serhiy.storchaka'
assignee = 'docs@python'
closed = True
closed_date = <Date 2013-11-24.21:15:30.677>
closer = 'serhiy.storchaka'
components = ['Documentation', 'Library (Lib)', 'IO']
creation = <Date 2012-06-27.10:29:50.642>
creator = 'serhiy.storchaka'
dependencies = []
files = ['31416', '32708', '32809']
hgrepos = []
issue_num = 15204
keywords = ['patch']
message_count = 22.0
messages = ['164142', '164144', '164145', '164153', '164232', '164296', '168247', '171546', '171547', '171549', '179190', '195865', '195875', '203421', '203432', '204081', '204092', '204094', '204269', '213002', '229527', '229693']
nosy_count = 15.0
nosy_names = ['georg.brandl', 'rhettinger', 'pitrou', 'nadeem.vawda', 'stutzbach', 'Arfrever', 'r.david.murray', 'asvetlov', 'chris.jerdonek', 'docs@python', 'python-dev', 'martin.panter', 'serhiy.storchaka', 'vajrasky', '\xd0\x92\xd0\xb0\xd1\x81\xd0\xb8\xd0\xbb\xd0\xb8\xd0\xb9.\xd0\x9c\xd0\xb0\xd0\xba\xd0\xb0\xd1\x80\xd0\xbe\xd0\xb2']
pr_nums = []
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue15204'
versions = ['Python 3.4']

[serhiy-storchaka]

Since Python 2.3 many open functions supports "Universal line mode" (PEP-278). Since 3.0 (and 2.6) PEP-3116 suggests better alternative -- io.TextWrapper.

Now support for the 'U' mode in the different open functions is heterogeneous. Some functions simply ignore the 'U' mode (but accept it), others perceive it as a synonym for text-mode, others just pass it on lower lever, other attempt to implement it, but the implementation is obtained imperfect and contradictory (as in ZipExtFile). The documentation for built-in open does not advise the use of the 'U' mode. The 'U' mode support cumbersome. I propose to deprecate the 'U' mode. If someone wanted to work with the universal line support, he'll surely need to work with encodings too, and io.TextWrapper provides is better choise.

The deprecation plan for the 'U' mode of open functions might be as follow:

3.3. Deprecating the 'U' mode in docs for all opens (building open, io.open, codecs.open, gzip.open, ZipFile.open, etc). Add suggestion about io.TextWrapper.
3.4. Raise a warning on use of the 'U' mode.
3.5. Raise an exception on use of the 'U' mode.
3.6 (or 4.0?). Remove the 'U' mode processing code.

As the first stage involves only the changes to the documentation, I hope deprecation can starts in 3.3.

[serhiy-storchaka]

Related issues: bpo-2091, bpo-5148, bpo-6759, bpo-12900.

[pitrou]

Starting to deprecate "U" in the 3.3 docs sounds reasonable to me.

[bitdancer]

Unless there are places where it is actually broken, I don't think there is a good reason to have step 3.5, though. Just add the deprecation warning and remove it in 4.0.

[serhiy-storchaka]

Unless there are places where it is actually broken, I don't think there is a good reason to have step 3.5, though. Just add the deprecation warning and remove it in 4.0.

Well. In any case, the 'U' mode in most cases has no effect, and the
code, where it has an effect (in zipfile), is very rarely used (and it
is actually complicated and broken).

Here are the patches for the first (documentation) and the second stage
(warnings).

[nadeemvawda]

+1 for the general idea of deprecating and eventually removing the "U"
modes.

But I agree with David, that it doesn't make sense to have separate steps
for 3.5 and 3.6/4.0. If you make the code raise an exception when "U" is
used, how is that different from what will happen when you remove the
code for processing it? Surely we want it to eventually be treated just
like any other invalid mode string?

[rhettinger]

The only people affected by deprecating "U" are the people who are currently using it for some reason -- presumably they are expecting that it does something useful for their code.

Ideally, this proposed deprecation should be mentioned on python-dev so that those folks will have an opportunity to respond.

[serhiy-storchaka]

Ideally, this proposed deprecation should be mentioned on python-dev so that those folks will have an opportunity to respond.

http://comments.gmane.org/gmane.comp.python.devel/134996

[serhiy-storchaka]

Chris, you did a great job on improving the documentation (including about universal newlines). Can you help with this issue?

For the first stage should be clearly shown that "U" mode is not only should not be used in new code, but must be removed from the old code.

Stage 1 patch updated to resolve conflicts with Chris changes.

[cjerdonek]

Thanks, Serhiy. :) Sure, I should be able to help with the documentation portion of this issue if the community is in agreement. Would the deprecation need to be moved up to 3.4 though now?

[pitrou]

Would the deprecation need to be moved up to 3.4 though now?

Yes, I think so.

[serhiy-storchaka]

Here is an updated patch. I think we can combine 1 and 2 stages (change the documentation and add warnings).

[vajrasky]

http://bugs.python.org/review/15204/diff/9032/Tools/iobench/iobench.py

Line 10 in New:
import io

But io module is never used.

[serhiy-storchaka]

Could anyone please review the patch?

[serhiy-storchaka]

Updated patch addresses Victor's comments.

[serhiy-storchaka]

Updated patch addresses Ezio's comments.

[python-dev]

New changeset 70bd6f7e013b by Serhiy Storchaka in branch 'default':
Issue bpo-15204: Deprecated the 'U' mode in file-like objects.
http://hg.python.org/cpython/rev/70bd6f7e013b

[serhiy-storchaka]

Thanks Victor and Ezio for the reviews.

[python-dev]

New changeset 694e2708b4a8 by Serhiy Storchaka in branch 'default':
Issue bpo-15204: Silence and check the 'U' mode deprecation warnings in tests.
http://hg.python.org/cpython/rev/694e2708b4a8

[python-dev]

New changeset 2d5544afb510 by R David Murray in branch 'default':
whatsnew: 'U' mode deprecation (bpo-15204).
http://hg.python.org/cpython/rev/2d5544afb510

[ghost]

open() documentation is probably broken a little now. Here is what one can see at the end of open() description:
"...
Deprecated since version 3.4, will be removed in version 4.0.

The 'U' mode."

Reader may assume open() function is what will be removed. Which is frightening :)

[serhiy-storchaka]

Please open new issue for this.