wcscoll is broken on Android and test_locale fails

[xdegaye]

BPO	28996
Nosy	@xdegaye, @yan12125
Files	test_locale_strcoll.patch

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = 'https://github.com/xdegaye'
closed_at = <Date 2016-12-19.10:16:19.749>
created_at = <Date 2016-12-17.08:35:18.014>
labels = ['3.7', 'type-bug', 'tests']
title = 'wcscoll is broken on Android and test_locale fails'
updated_at = <Date 2016-12-19.10:16:19.748>
user = 'https://github.com/xdegaye'

bugs.python.org fields:

activity = <Date 2016-12-19.10:16:19.748>
actor = 'xdegaye'
assignee = 'xdegaye'
closed = True
closed_date = <Date 2016-12-19.10:16:19.749>
closer = 'xdegaye'
components = ['Tests']
creation = <Date 2016-12-17.08:35:18.014>
creator = 'xdegaye'
dependencies = []
files = ['45942']
hgrepos = []
issue_num = 28996
keywords = ['patch']
message_count = 5.0
messages = ['283474', '283496', '283497', '283500', '283598']
nosy_count = 3.0
nosy_names = ['xdegaye', 'python-dev', 'yan12125']
pr_nums = []
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue28996'
versions = ['Python 3.6', 'Python 3.7']

[xdegaye]

These failures happen now that bpo-28596 has been fixed and that locale.getpreferredencoding(False) returns 'UTF-8'.

======================================================================
FAIL: test_strcoll_with_diacritic (test.test_locale.TestEnUSCollation)
----------------------------------------------------------------------

Traceback (most recent call last):
  File "/sdcard/org.bitbucket.pyona/lib/python3.7/test/test_locale.py", line 362, in test_strcoll_with_diacritic
    self.assertLess(locale.strcoll('à', 'b'), 0)
AssertionError: 1 not less than 0

======================================================================
FAIL: test_strxfrm_with_diacritic (test.test_locale.TestEnUSCollation)
----------------------------------------------------------------------

Traceback (most recent call last):
  File "/sdcard/org.bitbucket.pyona/lib/python3.7/test/test_locale.py", line 365, in test_strxfrm_with_diacritic
    self.assertLess(locale.strxfrm('à'), locale.strxfrm('b'))
AssertionError: 'à' not less than 'b'

---

[xdegaye]

Both strcoll() and strxfrm() are broken (character 'à' unicode code point is 'e0'):

>>> import locale
>>> locale.setlocale(locale.LC_ALL, 'en_US.UTF-8')
'C.UTF-8'
>>> locale.strcoll('\u00e0', 'b')
1
>>> locale.strxfrm('\u00e0') < locale.strxfrm('b')
False

The correct results are -1 and True.

[xdegaye]

Here is a patch that skips test_strcoll_with_diacritic and test_strxfrm_with_diacritic.

[yan12125]

I'm afraid that the sentence "wcscoll/wcsxfrm have known bugs" is misleading for people who are not quite familiar with Android. The actual cause is that BioniC's setlocale() behaves differently than other platforms. Most implementations returns NULL if en_US.UTF-8 is not available, but BioniC returns C.UTF-8. I guess it's better to add some comments for the real cause on Android.

[python-dev]

New changeset 781c56168484 by Xavier de Gaye in branch '3.6':
Issue bpo-28996: Skip two tests that fail on Android with the locale strcoll() and
https://hg.python.org/cpython/rev/781c56168484

New changeset 5c5cf7687dc1 by Xavier de Gaye in branch 'default':
Issue bpo-28996: Merge 3.6.
https://hg.python.org/cpython/rev/5c5cf7687dc1