Bug in C locale coercion with PYTHONCOERCECLOCALE=1

[vstinner]

BPO	35336
Nosy	@vstinner
PRs	bpo-35336: Fix PYTHONCOERCECLOCALE=1 #10806 [3.7] bpo-35336: Fix PYTHONCOERCECLOCALE=1 (GH-10806) #10813

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = <Date 2018-11-30.11:20:02.030>
created_at = <Date 2018-11-28.10:42:16.416>
labels = ['interpreter-core', '3.7', '3.8']
title = 'Bug in C locale coercion with PYTHONCOERCECLOCALE=1'
updated_at = <Date 2018-11-30.11:20:02.030>
user = 'https://github.com/vstinner'

bugs.python.org fields:

activity = <Date 2018-11-30.11:20:02.030>
actor = 'vstinner'
assignee = 'none'
closed = True
closed_date = <Date 2018-11-30.11:20:02.030>
closer = 'vstinner'
components = ['Interpreter Core']
creation = <Date 2018-11-28.10:42:16.416>
creator = 'vstinner'
dependencies = []
files = []
hgrepos = []
issue_num = 35336
keywords = ['patch']
message_count = 4.0
messages = ['330590', '330768', '330771', '330772']
nosy_count = 1.0
nosy_names = ['vstinner']
pr_nums = ['10806', '10813']
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = None
url = 'https://bugs.python.org/issue35336'
versions = ['Python 3.7', 'Python 3.8']

[vstinner]

PYTHONCOERCECLOCALE=1 should not force C locale coercion, Python should still check if the LC_CTYPE locale is "C".

Bug:

$ ./python -c 'import locale; print(locale.setlocale(locale.LC_CTYPE, None))'
fr_FR.UTF-8
$ PYTHONCOERCECLOCALE=1 ./python -c 'import locale; print(locale.setlocale(locale.LC_CTYPE, None))'
C.UTF-8

It should be fr_FR.UTF-8 as well in the second example :-( It seems to be a regression that I introduced in the middle of my refactoring on the Python initialization.

The bug is around:

static void
config_init_locale(_PyCoreConfig *config)
{
    if (config->coerce_c_locale < 0) {
        /* The C locale enables the C locale coercion (PEP 538) */
        if (_Py_LegacyLocaleDetected()) {
            config->coerce_c_locale = 1;
        }
    }
    ...
}

The 3.7 branch and the 3.7.0 release are affected :-(

$ ./python -V
Python 3.7.0
$ PYTHONCOERCECLOCALE=1 ./python -c 'import locale; print(locale.setlocale(locale.LC_CTYPE, None))'
C.UTF-8

[vstinner]

New changeset 55e4980 by Victor Stinner in branch 'master':
bpo-35336: Fix PYTHONCOERCECLOCALE=1 (GH-10806)
55e4980

[vstinner]

PYTHONCOERCECLOCALE=1 should not force C locale coercion, Python should still check if the LC_CTYPE locale is "C".

Reference:

https://docs.python.org/dev/using/cmdline.html#envvar-PYTHONCOERCECLOCALE

"If (...) the current locale reported for the LC_CTYPE category is either the default C locale, or else the explicitly ASCII-based POSIX locale, (...)"

[vstinner]

New changeset df738d5 by Victor Stinner in branch '3.7':
bpo-35336: Fix PYTHONCOERCECLOCALE=1 (GH-10806) (GH-10813)
df738d5