bpo-18378: Recognize "UTF-8" as a valid name in locale._parse_localename

[ronaldoussoren]

"UTF-8" is a valid value for LC_CTYPE on macOS, and the variable is set to that value during interpreter startup by the UTF-8 coercion code in pylifecycle.c.

https://bugs.python.org/issue18378

[ronaldoussoren]

The pull-request enables recognition of "UTF-8" as a valid value on all platforms. I've primarily done this because the value of LC_CTYPE will usually leak into the target system when you use SSH to log on those systems.

[ronaldoussoren]

Victor, I've asked for your review because you're involved with the code that sets LC_CTYPE during interpreter startup.

[vstinner]

locale.getpreferredencoding() is based on nl_langinfo(CODESET) which is reliable.

I never understood the purpose of the locale.getdefaultlocale() function, especially locale._parse_localename(). What is the use case for locale.getdefaultlocale()?

[ronaldoussoren]

What is the use case for locale.getdefaultlocale()?

I don't know what the use case is, but this is a public API that's currently broken on macOS. I've run into this issue in the past when using software that uses this API, but to be honest I don't recall which software that was. The BPO issue mentions (older versions of?) docutils that use this API.

[ned-deily]

LGTM

[miss-islington]

Thanks @ronaldoussoren for the PR, and @ned-deily for merging it 🌮🎉.. I'm working now to backport this PR to: 3.7, 3.8.
🐍🍒⛏🤖

[bedevere-bot]

GH-15569 is a backport of this pull request to the 3.8 branch.

[bedevere-bot]

GH-15570 is a backport of this pull request to the 3.7 branch.