-
-
Notifications
You must be signed in to change notification settings - Fork 31.7k
localeconv() doesn't support LC_MONETARY encoding different than LC_CTYPE encoding #72790
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Comments
This issue was originally reported on Fedora's Bugzilla: https://bugzilla.redhat.com/show_bug.cgi?id=1391280 Description of problem: Version-Release number of selected component (if applicable): How reproducible: Steps to Reproduce:
Actual results:
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/usr/lib64/python3.5/locale.py", line 110, in localeconv
d = _localeconv()
UnicodeDecodeError: 'locale' codec can't decode byte 0xa3 in position 0: Invalid or incomplete multibyte or wide character Expected results: Note: |
I suspect this issue is similar to bpo-25812. en_GB has non-ut8 encoding (likely iso8859-1). Currency symbol £ is encoded with this encoding as b'\xa3'. But Python tries to decode b'\xa3' with an encoding determined by other locale setting (LC_CTYPE). |
This causes test_float.py to fail with glibc > 2.26. ERROR: test_float_with_comma (main.GeneralFloatCases) Traceback (most recent call last):
File "/home/abuild/rpmbuild/BUILD/Python-3.6.2/Lib/test/support/__init__.py", line 1590, in inner
return func(*args, **kwds)
File "Lib/test/test_float.py", line 150, in test_float_with_comma
if not locale.localeconv()['decimal_point'] == ',':
File "/home/abuild/rpmbuild/BUILD/Python-3.6.2/Lib/locale.py", line 110, in localeconv
d = _localeconv()
UnicodeDecodeError: 'locale' codec can't decode byte 0xa0 in position 0: Invalid or incomplete multibyte or wide character |
Example of the bug: import locale
# LC_CTYPE: latin1 encoding
locale.setlocale(locale.LC_ALL, "en_GB")
# LC_MONETARY: utf8 encoding
locale.setlocale(locale.LC_MONETARY, "ar_SA.UTF-8")
lc = locale.localeconv()
for attr in (
"mon_grouping",
"int_curr_symbol",
"currency_symbol",
"mon_decimal_point",
"mon_thousands_sep",
):
print(f"{attr}: {lc[attr]!a}") Python 3.7 output: mon_grouping: [] Expected output: mon_grouping: [] Tested on Fedora 29. |
See also bpo-33954: float.__format__('n') fails with _PyUnicode_CheckConsistency assertion error for locales with non-ascii thousands separator. It may be nice to fix these two bugs at the same times, since they are related :-) |
I tested manually PR 10606: LC_ALL= LC_CTYPE=xxx LC_MONETARY=xxx ./python -c 'import locale; locale.setlocale(locale.LC_ALL, ""); print(ascii(locale.localeconv()["currency_symbol"]))'
'\xa3' Result (bug = result/error without the fix):
Locale encodings:
|
It seems like my change introduced a regression: bpo-35290. |
See also bpo-31900: localeconv() should decode numeric fields from LC_NUMERIC encoding, not from LC_CTYPE encoding. |
Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.
Show more details
GitHub fields:
bugs.python.org fields:
The text was updated successfully, but these errors were encountered: