Allow backslashreplace error handler to be used on input

[ncoghlan]

BPO	22286
Nosy	@ncoghlan, @vstinner, @bitdancer, @vadmium, @serhiy-storchaka, @zooba
Dependencies	bpo-22470: Possible integer overflow in error handlers
Files	backslashreplace_decode.patch backslashreplace_decode_2.patch

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = 'https://github.com/serhiy-storchaka'
closed_at = <Date 2015-01-27.08:54:56.094>
created_at = <Date 2014-08-27.10:50:20.603>
labels = ['extension-modules', 'type-feature']
title = 'Allow backslashreplace error handler to be used on input'
updated_at = <Date 2015-01-27.08:54:56.094>
user = 'https://github.com/ncoghlan'

bugs.python.org fields:

activity = <Date 2015-01-27.08:54:56.094>
actor = 'serhiy.storchaka'
assignee = 'serhiy.storchaka'
closed = True
closed_date = <Date 2015-01-27.08:54:56.094>
closer = 'serhiy.storchaka'
components = ['Extension Modules']
creation = <Date 2014-08-27.10:50:20.603>
creator = 'ncoghlan'
dependencies = ['22470']
files = ['36697', '37274']
hgrepos = []
issue_num = 22286
keywords = ['patch', 'needs review']
message_count = 9.0
messages = ['225966', '227352', '231655', '234144', '234607', '234691', '234694', '234706', '234784']
nosy_count = 8.0
nosy_names = ['ncoghlan', 'vstinner', 'Arfrever', 'r.david.murray', 'python-dev', 'martin.panter', 'serhiy.storchaka', 'steve.dower']
pr_nums = []
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'enhancement'
url = 'https://bugs.python.org/issue22286'
versions = ['Python 3.5']

[ncoghlan]

In the discussion on bpo-18814, Antoine pointed out that in a Python 3 world, using backslashescape during decoding actually makes sense - it lets you accurately report arbitrary bytes in the sequence, without needing surrogateescape or surrogatepass to be used when encoding later.

[serhiy-storchaka]

Here is a patch.

[serhiy-storchaka]

Synchronized with the tip (resolved conflicts with bpo-22470 and bpo-19676).

[serhiy-storchaka]

Could you make a review Nick to get this feature in the first alpha.

[ncoghlan]

+1 from me for merging, although I suspect you'll need to adjust the codecs.rst changes first.

[python-dev]

New changeset dd8a03e98158 by Serhiy Storchaka in branch 'default':
Issue bpo-22286: The "backslashreplace" error handlers now works with
https://hg.python.org/cpython/rev/dd8a03e98158

[bitdancer]

Looks like the buildbots aren't happy with this change.

[serhiy-storchaka]

Yes, I see. The patch exposed existing bug in decoding error handing. See bpo-23321 for this.

[python-dev]

New changeset 1e8937861ee3 by Victor Stinner in branch 'default':
Issue bpo-22286, bpo-23321: Fix failing test on Windows code page 932
https://hg.python.org/cpython/rev/1e8937861ee3