Skip to content

gh-47482: Reject non-bytes-like input in the idna codec decoder#154699

Open
Cnashn wants to merge 2 commits into
python:mainfrom
Cnashn:fix-idna-decode-input-type
Open

gh-47482: Reject non-bytes-like input in the idna codec decoder#154699
Cnashn wants to merge 2 commits into
python:mainfrom
Cnashn:fix-idna-decode-input-type

Conversation

@Cnashn

@Cnashn Cnashn commented Jul 25, 2026

Copy link
Copy Markdown

Codec.decode in Lib/encodings/idna.py called bytes(input) on any input that
was not already bytes:

# IDNA allows decoding to operate on Unicode strings, too.
if not isinstance(input, bytes):
    # XXX obviously wrong, see #3232
    input = bytes(input)

bytes() accepts more than buffers, so objects it happens to understand were
silently coerced instead of rejected:

>>> codecs.decode(5, "idna")
'\x00\x00\x00\x00\x00'
>>> codecs.decode([65, 66], "idna")
'AB'
>>> codecs.decode(None, "idna")
''
>>> codecs.decode("python.org", "idna")
TypeError: string argument without an encoding

The first three return nonsense rather than raising, and the last leaks an
error message from bytes() that never mentions what was actually expected.
Every other bytes-to-text decoder raises TypeError: a bytes-like object is required, not 'str'.

This PR raises TypeError with that same message, and keeps accepting
bytes-like objects, so bytearray and memoryview continue to work:

>>> codecs.decode(bytearray(b"xn--pythn-mua.org"), "idna")
'pythön.org'
>>> codecs.decode(5, "idna")
TypeError: a bytes-like object is required, not 'int'

The type check now runs before the empty-input shortcut, so codecs.decode("", "idna")
raises instead of returning '', matching codecs.decode("", "utf-8").

This is the narrow half of the issue that already has agreement in the
discussion: rejecting non-bytes input "deliberately and explicitly, rather than
as a result of a bogus forward-port". I have deliberately not touched the
separate proposal in that thread to add a new decode_idna function for
un-IDNA-ing str input, since that is a feature and still undecided.

The issue is marked "test needed" and noted that the behaviour was not covered
by the test suite, so IDNACodecTest gains two tests: one that bytes-like
input still decodes, and one that non-bytes-like input raises TypeError with
the expected message.

test_codecs, test_socket, test_ssl, test_httplib, test_urllib,
test_urllib2, test_email and test_str all pass.

One thing I noticed but left alone: IncrementalDecoder._buffer_decode has an
isinstance(input, str) branch that is unreachable, because
codecs.BufferedIncrementalDecoder concatenates its bytes buffer to the input
before _buffer_decode is called. Happy to address that separately if wanted.

The decoder called bytes(input) on any non-bytes input, a bogus Python 3
forward-port. Objects that bytes() accepts were silently coerced instead
of rejected: codecs.decode(5, "idna") returned five NUL characters and
codecs.decode([65, 66], "idna") returned "AB".

Raise TypeError with the same message other codecs use, while continuing
to accept bytes-like objects such as bytearray and memoryview.
@python-cla-bot

python-cla-bot Bot commented Jul 25, 2026

Copy link
Copy Markdown

All commit authors signed the Contributor License Agreement.

CLA signed

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant