Deprecate getlocale() and normalize() functions

[vstinner]

BPO	43557
Nosy	@malemburg, @vstinner
PRs	bpo-46659: Deprecate locale.getdefaultlocale() #31206

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = None
created_at = <Date 2021-03-19.13:27:27.551>
labels = ['library', '3.10']
title = 'Deprecate getlocale() and normalize() functions'
updated_at = <Date 2022-04-03.03:54:41.038>
user = 'https://github.com/vstinner'

bugs.python.org fields:

activity = <Date 2022-04-03.03:54:41.038>
actor = 'vstinner'
assignee = 'none'
closed = False
closed_date = None
closer = None
components = ['Library (Lib)']
creation = <Date 2021-03-19.13:27:27.551>
creator = 'vstinner'
dependencies = []
files = []
hgrepos = []
issue_num = 43557
keywords = ['patch']
message_count = 4.0
messages = ['389086', '389096', '412648', '413911']
nosy_count = 2.0
nosy_names = ['lemburg', 'vstinner']
pr_nums = ['31206']
priority = 'normal'
resolution = None
stage = 'patch review'
status = 'open'
superseder = None
type = None
url = 'https://bugs.python.org/issue43557'
versions = ['Python 3.10']

[vstinner]

I propose to deprecate getdefaultlocale(), getlocale() and normalize() functions since they have multiple issues, and remove them in Python 3.12.

The normalize() function uses the locale.locale_alias dictionary which was copied from the X11 locale database in 2000. It's hard to keep this dictionary up to date and to support all locales of all platforms supported by Python. There are multiple issues on macOS for example.

getdefaultlocale() and getlocale() use heuristics to get an encoding from the locale name. These heuristics are not reliable.

getdefaultlocale() only rely on environment variables. When setlocale() is called, environment variables are not updated, and so the encoding returned by getdefaultlocale() is not the effective LC_CTYPE locale encoding. Example:
https://bugs.python.org/issue43552#msg389069

getlocale() open issues:

• bpo-20088: locale.getlocale() fails if locale name doesn't include encoding
• bpo-23425: Windows getlocale unix-like with french, german, portuguese, spanish
• bpo-33934: locale.getlocale() seems wrong when the locale is yet unset (python3 on linux)
• bpo-38805: locale.getlocale() returns a non RFC1766 language code
• bpo-43115: locale.getlocale fails if locale is set

getdefaultlocale() open issue:

• bpo-6981: locale.getdefaultlocale() envvars default code and documentation mismatch
• bpo-30755: locale.normalize() and getdefaultlocale() convert C.UTF-8 to en_US.UTF-8

Replacements:

• getdefaultlocale()[1] => getpreferredencoding(False) or get_current_locale_encoding(), see bpo-43552
• getlocale(loc) => setlocale(loc) or setlocale(loc, None)
• normalize => no replacement. There is no standard way to normalize a locale name.

[malemburg]

+1 on getdefaultlocale() as mentioned in https://bugs.python.org/issue43552

However, -1 on getlocale() and normalize().

Those two are needed to access and successfully set the locale on
Linux: the lib C setlocale() is very picky about locale names and
so normalization helps in finding the right form and getting
usable results across platforms.

The issues open for these should be addressed and fixed.

[vstinner]

Deprecating these functions is complex. I prefer to start with the least controversial part: bpo-46659.

[vstinner]

Deprecating these functions is complex. I prefer to start with the least controversial part: bpo-46659.

locale.getdefaultlocale() is now deprecated in Python 3.11.

[vstinner]

Sadly, I don't have the bandwidth to work on this issue, so I just close it. At least, locale.getdefaultlocale() is now deprecated in Python 3.11.