Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

android: test_sys fails #71107

Closed
xdegaye mannequin opened this issue May 3, 2016 · 12 comments
Closed

android: test_sys fails #71107

xdegaye mannequin opened this issue May 3, 2016 · 12 comments
Assignees
Labels
3.7 interpreter-core Interpreter core (Objects, Python, Grammar, and Parser dirs) type-bug An unexpected behavior, bug, or error

Comments

@xdegaye
Copy link
Mannequin

xdegaye mannequin commented May 3, 2016

BPO 26920
Nosy @malemburg, @loewis, @vstinner, @xdegaye, @serhiy-storchaka, @moreati
PRs
  • [Do Not Merge] Convert Misc/NEWS so that it is managed by towncrier #552
  • Files
  • test_ioencoding_nonascii.patch
  • test_c_locale_surrogateescape.patch
  • Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

    Show more details

    GitHub fields:

    assignee = 'https://github.com/xdegaye'
    closed_at = <Date 2016-11-16.07:19:52.513>
    created_at = <Date 2016-05-03.09:26:22.543>
    labels = ['interpreter-core', 'type-bug', '3.7']
    title = 'android: test_sys fails'
    updated_at = <Date 2017-03-31.16:36:16.821>
    user = 'https://github.com/xdegaye'

    bugs.python.org fields:

    activity = <Date 2017-03-31.16:36:16.821>
    actor = 'dstufft'
    assignee = 'xdegaye'
    closed = True
    closed_date = <Date 2016-11-16.07:19:52.513>
    closer = 'xdegaye'
    components = ['Interpreter Core']
    creation = <Date 2016-05-03.09:26:22.543>
    creator = 'xdegaye'
    dependencies = []
    files = ['42844', '42845']
    hgrepos = []
    issue_num = 26920
    keywords = ['patch']
    message_count = 12.0
    messages = ['264705', '265463', '265465', '265480', '265509', '265511', '265512', '265518', '265522', '265548', '279822', '280917']
    nosy_count = 7.0
    nosy_names = ['lemburg', 'loewis', 'vstinner', 'xdegaye', 'python-dev', 'serhiy.storchaka', 'Alex.Willmer']
    pr_nums = ['552']
    priority = 'normal'
    resolution = 'fixed'
    stage = 'resolved'
    status = 'closed'
    superseder = None
    type = 'behavior'
    url = 'https://bugs.python.org/issue26920'
    versions = ['Python 3.6', 'Python 3.7']

    @xdegaye
    Copy link
    Mannequin Author

    xdegaye mannequin commented May 3, 2016

    test_sys fails on an android emulator running an x86 system image at API level 21.

    ...
    test_implementation (test.test_sys.SysModuleTest) ... ok
    test_intern (test.test_sys.SysModuleTest) ... ok
    test_ioencoding (test.test_sys.SysModuleTest) ... ok
    test_ioencoding_nonascii (test.test_sys.SysModuleTest) ... Traceback (most recent call last):
      File "<string>", line 1, in <module>
    UnicodeEncodeError: 'ascii' codec can't encode character '\xe6' in position 0: ordinal not in range(
    128)
    FAIL
    test_is_finalizing (test.test_sys.SysModuleTest) ... ok
    test_lost_displayhook (test.test_sys.SysModuleTest) ... ok
    ...

    ======================================================================
    FAIL: test_c_locale_surrogateescape (test.test_sys.SysModuleTest)
    ----------------------------------------------------------------------

    Traceback (most recent call last):
      File "/sdcard/org.bitbucket.pyona/lib/python3.6/test/test_sys.py", line 709, in test_c_locale_surr
    ogateescape
        'stdin: surrogateescape\n'
    AssertionError: 'stdin: strict\nstdout: strict\nstderr: backslashreplace\n' != 'stdin: surrogateesca
    pe\nstdout: surrogateescape\nstderr: backslashreplace\n'
    - stdin: strict
    - stdout: strict
    + stdin: surrogateescape
    + stdout: surrogateescape
      stderr: backslashreplace

    ======================================================================
    FAIL: test_ioencoding_nonascii (test.test_sys.SysModuleTest)
    ----------------------------------------------------------------------

    Traceback (most recent call last):
      File "/sdcard/org.bitbucket.pyona/lib/python3.6/test/test_sys.py", line 638, in test_ioencoding_no
    nascii
        self.assertEqual(out, os.fsencode(test.support.FS_NONASCII))
    AssertionError: b'' != b'\xc3\xa6'

    Ran 43 tests in 0.913s

    FAILED (failures=2, skipped=2)
    test test_sys failed
    1 test failed:
    test_sys
    Total duration: 0:00:01

    @xdegaye xdegaye mannequin added extension-modules C modules in the Modules dir build The build process and cross-build type-bug An unexpected behavior, bug, or error labels May 3, 2016
    @xdegaye
    Copy link
    Mannequin Author

    xdegaye mannequin commented May 13, 2016

    test_ioencoding_nonascii does not fail when LANG is set to en_GB.UTF-8 in the environment.

    @xdegaye
    Copy link
    Mannequin Author

    xdegaye mannequin commented May 13, 2016

    test_ioencoding_nonascii does not fail when LANG is set to en_GB.UTF-8 in the environment.

    When LANG is not set, we have on an android emulator:

    >>> from test.support import FS_NONASCII
    >>> print(FS_NONASCII)
    Traceback (most recent call last):
      File "<stdin>", line 1, in <module>
    UnicodeEncodeError: 'ascii' codec can't encode character '\xe6' in position 0: ordinal not in range(128)

    But on linux when LANG is not set:

    $ LANG= ./python
    Python 3.6.0a0 (default:eee959fee5f5+, May 13 2016, 11:32:27) 
    [GCC 6.1.1 20160501] on linux
    Type "help", "copyright", "credits" or "license" for more information.
    >>> from test.support import FS_NONASCII
    >>> print(FS_NONASCII)
    None
    >>> 

    And test_ioencoding_nonascii is skipped accordingly on linux for the following printed reason: 'requires OS support of non-ASCII encodings'.

    On Android, os.fsdecode(os.fsencode('\xe6')) does not raise UnicodeError in the test.support module. And locale.getpreferredencoding() returns 'ascii'. So it seems that this criterion (FS_NONASCII is not None) is not sufficient to decide when this test should be run and not skipped.

    @xdegaye
    Copy link
    Mannequin Author

    xdegaye mannequin commented May 13, 2016

    On Android, os.fsdecode(os.fsencode('\xe6')) does not raise UnicodeError in the test.support module.

    Because of changeset ad6be34ce8c9.

    @xdegaye
    Copy link
    Mannequin Author

    xdegaye mannequin commented May 14, 2016

    This patch fixes test_ioencoding_nonascii.

    @xdegaye
    Copy link
    Mannequin Author

    xdegaye mannequin commented May 14, 2016

    About the failures in test_c_locale_surrogateescape, it seems that on Android the locale is not set correctly on startup when LC_ALL is set to C:

    root@generic_x86:/data/local/tmp # LC_ALL=C python
    Python 3.6.0a0 (default:eee959fee5f5+, May 14 2016, 10:19:09) 
    [GCC 4.9 20150123 (prerelease)] on linux
    Type "help", "copyright", "credits" or "license" for more information.
    >>> import locale; locale.getlocale()
    ('en_US', 'UTF-8')
    >>> locale.setlocale(locale.LC_ALL, '')
    'C'
    >>> locale.getlocale()
    (None, None)
    >>>

    @xdegaye
    Copy link
    Mannequin Author

    xdegaye mannequin commented May 14, 2016

    This patch fixes the locale setting on startup when the LC_ALL environment variable is set to C, and as a consequence test_c_locale_surrogateescape does not fail anymore. Note that on Android HAVE_LANGINFO_H is undefined, see issue bpo-22747.

    @serhiy-storchaka
    Copy link
    Member

    serhiy-storchaka commented May 14, 2016

    See also bpo-19058.

    @xdegaye
    Copy link
    Mannequin Author

    xdegaye mannequin commented May 14, 2016

    With Serhiy's latest patch named sys_test_ioencoding.patch in bpo-19058, test_sys fails only in test_c_locale_surrogateescape on the android-21-x86 emulator.

    @vstinner
    Copy link
    Member

    vstinner commented May 14, 2016

    test_c_locale_surrogateescape.patch LGTM.

    @xdegaye
    Copy link
    Mannequin Author

    xdegaye mannequin commented Oct 31, 2016

    For the record, on the Android emulator we have now (not sure where this change has been made):
    >>> sys.getfilesystemencoding()
    'utf-8'
    >>> locale.getpreferredencoding(False)
    'ascii'

    So test_ioencoding_nonascii succeeds now. Anyway the problem with this test is being fixed at bpo-19058.

    @xdegaye xdegaye mannequin added 3.7 interpreter-core Interpreter core (Objects, Python, Grammar, and Parser dirs) and removed extension-modules C modules in the Modules dir build The build process and cross-build labels Oct 31, 2016
    @xdegaye xdegaye mannequin self-assigned this Oct 31, 2016
    @python-dev
    Copy link
    Mannequin

    python-dev mannequin commented Nov 16, 2016

    New changeset 73bd6bbbb5e2 by Xavier de Gaye in branch '3.6':
    Issue bpo-26920: Fix not getting the locale's charset upon initializing the interpreter,
    https://hg.python.org/cpython/rev/73bd6bbbb5e2

    New changeset f358d849c14e by Xavier de Gaye in branch 'default':
    Issue bpo-26920: Merge 3.6
    https://hg.python.org/cpython/rev/f358d849c14e

    @xdegaye xdegaye mannequin closed this as completed Nov 16, 2016
    @ezio-melotti ezio-melotti transferred this issue from another repository Apr 10, 2022
    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
    Labels
    3.7 interpreter-core Interpreter core (Objects, Python, Grammar, and Parser dirs) type-bug An unexpected behavior, bug, or error
    Projects
    None yet
    Development

    No branches or pull requests

    2 participants