Suspect test.test_codeccallbacks.test_mutatingdecodehandler

[amauryfa]

BPO	16577
Nosy	@doerwalter, @amauryfa, @vstinner, @ezio-melotti

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = None
created_at = <Date 2012-11-29.12:51:48.278>
labels = ['type-feature', 'expert-unicode']
title = 'Suspect test.test_codeccallbacks.test_mutatingdecodehandler'
updated_at = <Date 2016-04-21.03:15:44.877>
user = 'https://github.com/amauryfa'

bugs.python.org fields:

activity = <Date 2016-04-21.03:15:44.877>
actor = 'berker.peksag'
assignee = 'none'
closed = False
closed_date = None
closer = None
components = ['Unicode']
creation = <Date 2012-11-29.12:51:48.278>
creator = 'amaury.forgeotdarc'
dependencies = []
files = []
hgrepos = []
issue_num = 16577
keywords = []
message_count = 2.0
messages = ['176645', '176661']
nosy_count = 4.0
nosy_names = ['doerwalter', 'amaury.forgeotdarc', 'vstinner', 'ezio.melotti']
pr_nums = []
priority = 'normal'
resolution = None
stage = 'needs patch'
status = 'open'
superseder = None
type = 'enhancement'
url = 'https://bugs.python.org/issue16577'
versions = ['Python 3.6']

[amauryfa]

The test created by 504084c04ac0 passes for bad reasons, at least for the second part:

• "test.mutating" error handler is never used.
• If I replace the last line of the test, it still passes, but the TypeError is raised by exc.object[:] = b"" (bytes are not mutable!)

I suggest to revert this old change completely. The premises ("the decoder might use memory that's no longer in use") are no longer true.

[doerwalter]

True, the second test uses the wrong error handler.

And yes, you're correct, bytes are now immutable. And even if I try to decode a bytearray, what the callback gets to see is still an immutable bytes object::

   import codecs

   def mutating(exc):
      if isinstance(exc, UnicodeDecodeError):
         exc.object[:] = b""
         return ("\u4242", 0)
      else:
         raise TypeError("don't know how to handle %r" % exc)

   codecs.register_error('mutating', mutating)

   input = bytearray(b'bbb\xff')

   input.decode('ascii', 'mutating')

This still raises:

TypeError: 'bytes' object does not support item assignment

So the change should indeed be reverted.