New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

PEP 538: update notes for linux-sig feedback #171

Closed
ncoghlan opened this Issue Jan 4, 2017 · 3 comments

Comments

Projects
None yet
1 participant
@ncoghlan
Contributor

ncoghlan commented Jan 4, 2017

Notes from the linux-sig thread at: https://mail.python.org/pipermail/linux-sig/2017-January/000014.html

  • change the sense of the environment variable to be an explicit off-switch for the coercion behaviour (PYTHONCOERCECLOCALE=0) rather than being an on-switch for more permissive behaviour
  • add a "--with[out]-c-locale-coercion" configure flag and a related PY_COERCE_C_LOCALE preprocessor definition (for the command line app coercion behaviour)
  • add a "--with[out]-c-locale-warning" configure flag and a related PY_WARN_ON_C_LOCALE preprocessor definition (for the runtime library warning behaviour)
  • in the command line app, check the result of the C.UTF-8 locale coercion, complain if it doesn't work, and then try the en_US.UTF-8 locale. If that still doesn't work, complain again, then continue with the C locale
  • simplify the library warning to:

Python runtime initialized with LC_CTYPE=C (a locale with default ASCII encoding), which may cause Unicode compatibility problems. Using C.UTF-8 as an alternative Unicode-compatible locale is recommended.

@ncoghlan

This comment has been minimized.

Contributor

ncoghlan commented Jan 5, 2017

Additional notes based on further discussion:

  • PY_COERCE_C_LOCALE and PY_WARN_ON_C_LOCALE should both be unset when building on Mac OS X (since this problem doesn't occur there)
  • PY_COERCE_C_LOCALE and PY_WARN_ON_C_LOCALE should both be unset when building on Windows (while this will happen by default, and the entire code blocks where these occur are skipped in the Windows builds anyway, there's no harm in being explicit about it in the PEP)
@ncoghlan

This comment has been minimized.

Contributor

ncoghlan commented Jan 6, 2017

Additional notes based on https://www.python.org/dev/peps/pep-0540/:

  • if not already set, PYTHONIOENCODING should be set to UTF-8:surrogateescape if the locale is successfully coerced to one based on UTF-8 (NOT done: added a section explaining the use of "strict" instead)

  • emphasise that the proposed approach matches what people already do to get Python 3 working reliably in Docker containers and other environments where the system locale information isn't readily available (and/or isn't relevant)

  • emphasise that the proposed solution can be implemented as a wrapper script for earlier Python 3.x versions:

    #!/bin/sh
    LC_ALL=C.UTF-8 LANG=C.UTF-8 python3 $@

@ncoghlan

This comment has been minimized.

Contributor

ncoghlan commented Jan 7, 2017

Changes incorporated via 221099d

@ncoghlan ncoghlan closed this Jan 7, 2017

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment