Improve Unicode Documentation with Known Caveats

[mitsuhiko]

BPO	11573
Nosy	@vstinner, @mitsuhiko

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = None
created_at = <Date 2011-03-16.17:15:40.852>
labels = ['expert-unicode', 'docs']
title = 'Improve Unicode Documentation with Known Caveats'
updated_at = <Date 2011-03-17.23:37:21.430>
user = 'https://github.com/mitsuhiko'

bugs.python.org fields:

activity = <Date 2011-03-17.23:37:21.430>
actor = 'eric.araujo'
assignee = 'docs@python'
closed = False
closed_date = None
closer = None
components = ['Documentation', 'Unicode']
creation = <Date 2011-03-16.17:15:40.852>
creator = 'aronacher'
dependencies = []
files = []
hgrepos = []
issue_num = 11573
keywords = []
message_count = 2.0
messages = ['131143', '131291']
nosy_count = 3.0
nosy_names = ['vstinner', 'aronacher', 'docs@python']
pr_nums = []
priority = 'normal'
resolution = None
stage = 'needs patch'
status = 'open'
superseder = None
type = None
url = 'https://bugs.python.org/issue11573'
versions = ['Python 3.1', 'Python 3.2', 'Python 3.3']

[mitsuhiko]

The documentation should explain some of the common problems with Unicode on Python 3.

• locale's affect the text default encoding
• SSH clients can set the locale on a remote server
• filesystem encoding is set by the SSH client as well

[vstinner]

Can you provide a patch?

[petsuter]

Surprisingly https://docs.python.org/3/howto/unicode.html does not mention the default encoding for open().

It should maybe also give an overview of:

• https://peps.python.org/pep-3120/ Using UTF-8 as the default source encoding (Python 3.0)

• https://peps.python.org/pep-0528/ Change Windows console encoding to UTF-8 (Python 3.6)

• https://peps.python.org/pep-0529/ Change Windows filesystem encoding to UTF-8 (Python 3.6)

• https://peps.python.org/pep-0538/ Coercing the legacy C locale to a UTF-8 based locale (Python 3.7)

• https://peps.python.org/pep-0540/ Add a new UTF-8 Mode (Python 3.7)

• https://peps.python.org/pep-0686/ Make UTF-8 mode default (Python 3.15)

• https://peps.python.org/pep-0672/ Unicode-related Security Considerations for Python