Skip to content
Branch: master
Commits on May 23, 2019
  1. bpo-36763: Fix _PyPreConfig_InitCompatConfig() utf8_mode (GH-13518)

    vstinner committed May 23, 2019
    * _PyPreConfig_InitCompatConfig() sets utf8_mode to 0.
    * Change Py_UTF8Mode default value to 0.
    * Fix _PyPreConfig_Copy(): copy also _config_init attrbibute.
    * _PyPreConfig_AsDict() exports _config_init
    * Fix _PyPreConfig_GetGlobalConfig(): use Py_UTF8Mode if it's greater
      than 0, even if utf8_mode >= 0.
    * Add unit tests on environment variables using Python API.
  2. bpo-18748: _pyio.IOBase emits unraisable exception (GH-13512)

    vstinner committed May 23, 2019
    In development (-X dev) mode and in a debug build, IOBase finalizer
    of the _pyio module now logs the exception if the close() method
    fails. The exception is ignored silently by default in release build.
    test_io: test_error_through_destructor() now uses
    support.catch_unraisable_exception() rather than capturing stderr.
  3. bpo-36721: Add --embed option to python-config (GH-13500)

    vstinner committed May 23, 2019
    To embed Python into an application, a new --embed option must be
    passed to "python3-config --libs --embed" to get "-lpython3.8" (link
    the application to libpython). To support both 3.8 and older, try
    "python3-config --libs --embed" first and fallback to "python3-config
    --libs" (without --embed) if the previous command fails.
    Add a pkg-config "python-3.8-embed" module to embed Python into an
    application: "pkg-config python-3.8-embed --libs" includes
    "-lpython3.8".  To support both 3.8 and older, try "pkg-config
    python-X.Y-embed --libs" first and fallback to "pkg-config python-X.Y
    --libs" (without --embed) if the previous command fails (replace
    "X.Y" with the Python version).
    On the other hand, "pkg-config python3.8 --libs" no longer contains
    "-lpython3.8". C extensions must not be linked to libpython (except
    on Android, case handled by the script); this change is backward
    incompatible on purpose.
    "make install" now also installs "python-3.8-embed.pc".
Commits on May 22, 2019
  1. bpo-36829: PyErr_WriteUnraisable() normalizes exception (GH-13507)

    vstinner committed May 22, 2019
    PyErr_WriteUnraisable() now creates a traceback object if there is no
    current traceback. Moreover, call PyErr_NormalizeException() and
    PyException_SetTraceback() to normalize the exception value. Ignore
    silently any error.
  2. bpo-36763: Rename private Python initialization functions (GH-13511)

    vstinner committed May 22, 2019
    * Rename private C functions:
      * _Py_Initialize_ReconfigureCore => pyinit_core_reconfigure
      * _Py_InitializeCore_impl => pyinit_core_config
      * _Py_InitializeCore = > pyinit_core
      * _Py_InitializeMainInterpreter => pyinit_main
      * init_python => pyinit_python
    * Rename _testembed.c commands: add "test_" prefix.
  3. bpo-36763: Add _PyPreConfig._config_init (GH-13481)

    vstinner committed May 22, 2019
    * _PyPreConfig_GetGlobalConfig() and  _PyCoreConfig_GetGlobalConfig()
      now do nothing if the configuration was not initialized with
      _PyPreConfig_InitCompatConfig() and _PyCoreConfig_InitCompatConfig()
    * Remove utf8_mode=-2 special case: use utf8_mode=-1 instead.
    * Fix _PyPreConfig_InitPythonConfig():
      * isolated = 0 instead of -1
      * use_environment = 1 instead of -1
    * Rename _PyConfig_INIT to  _PyConfig_INIT_COMPAT
    * Rename _PyPreConfig_Init() to _PyPreConfig_InitCompatConfig()
    * Rename _PyCoreConfig_Init() to _PyCoreConfig_InitCompatConfig()
    * PyInterpreterState_New() now uses _PyCoreConfig_InitPythonConfig()
      as default configuration, but it's very quickly overriden anyway.
    * _freeze_importlib.c uses _PyCoreConfig_SetString() to set
    * Cleanup preconfig_init_utf8_mode(): cmdline is always non-NULL.
  4. bpo-36829: Add (GH-13490)

    vstinner committed May 22, 2019
    * Copy test_exceptions.test_unraisable() to
    * Use catch_unraisable_exception() in test_coroutines,
      test_exceptions, test_generators.
  5. bpo-35907, CVE-2019-9948: urllib rejects local_file:// scheme (GH-13474)

    vstinner and push0ebp committed May 22, 2019
    CVE-2019-9948: Avoid file reading as disallowing the unnecessary URL
    scheme in URLopener().open() and URLopener().retrieve()
    of urllib.request.
    Co-Authored-By: SH <>
  6. bpo-36829: Add sys.unraisablehook() (GH-13187)

    vstinner committed May 22, 2019
    Add new sys.unraisablehook() function which can be overridden to
    control how "unraisable exceptions" are handled. It is called when an
    exception has occurred but there is no way for Python to handle it.
    For example, when a destructor raises an exception or during garbage
    collection (gc.collect()).
    * Add an internal UnraisableHookArgs type used to pass arguments to
    * Add _PyErr_WriteUnraisableDefaultHook().
    * The default hook now ignores exception on writing the traceback.
    * test_sys now uses unittest.main() to automatically discover tests:
      remove test_main().
    * Add _PyErr_Init().
    * Fix PyErr_WriteUnraisable(): hold a strong reference to sys.stderr
      while using it
Commits on May 21, 2019
  1. Revert "bpo-36084: Add native thread ID to threading.Thread objects (G…

    vstinner committed May 21, 2019
    …H-11993)" (GH-13458)
    This reverts commit 4959c33.
Commits on May 20, 2019
  1. bpo-36763: Fix _PyRuntime.preconfig.coerce_c_locale (GH-13444)

    vstinner committed May 20, 2019
    _PyRuntime.preconfig.coerce_c_locale can now be used to
    check if the C locale has been coerced.
    * Fix _Py_LegacyLocaleDetected(): don't attempt to coerce the
      C locale if LC_ALL environment variable is set. Add 'warn'
      parameter: emit_stderr_warning_for_legacy_locale() must not
      the LC_ALL env var.
    * _PyPreConfig_Write() sets coerce_c_locale to 0 if
      _Py_CoerceLegacyLocale() fails.
  2. bpo-36763: Fix encoding/locale tests in test_embed (GH-13443)

    vstinner committed May 20, 2019
    * Fix encoding and locale tests in test_embed.InitConfigTests.
    * InitConfigTests now only computes EXPECTED_CONFIG once.
    * Add tests for PYTHONWARNINGS and PYTHONPATH env vars
  3. bpo-36763: Fix Python preinitialization (GH-13432)

    vstinner committed May 20, 2019
    * Add _PyPreConfig.parse_argv
    * Add _PyCoreConfig._config_init field and _PyCoreConfigInitEnum enum
    * Initialization functions: reject preconfig=NULL and config=NULL
    * Add config parameter to _PyCoreConfig_DecodeLocaleErr(): pass
      config->argv to _Py_PreInitializeFromPyArgv(), to parse config
      command line arguments in preinitialization.
    * Add config parameter to _PyCoreConfig_SetString(). It now
      preinitializes Python.
    * _PyCoreConfig_SetPyArgv() now also preinitializes Python for wide
    * Fix _Py_PreInitializeFromCoreConfig(): don't pass args to
      _Py_PreInitializeFromPyArgv() if config.parse_argv=0.
    * Use "char * const *" and "wchar_t * const *" types for 'argv'
      parameters and _PyArgv.argv.
    * Add unit test on preinitialization from argv.
    * _PyPreConfig.allocator type becomes int
    * Add _PyPreConfig_InitFromPreConfig() and
      _PyPreConfig_InitFromCoreConfig() helper functions
  4. bpo-35134: Register new traceback.h header files (GH-13431)

    vstinner committed May 20, 2019
    Add new cpython/traceback.h and pycore_traceback.h header files to and PCbuild/ project.
Commits on May 19, 2019
  1. bpo-35134: Split traceback.h header (GH-13430)

    vstinner committed May 19, 2019
    Add new Include/cpython/traceback.h and Include/internal/traceback.h
    header files.
Commits on May 18, 2019
  1. bpo-36763: Remove _PyCoreConfig.dll_path (GH-13402)

    vstinner committed May 18, 2019
  2. bpo-36763: Use _PyCoreConfig_InitPythonConfig() (GH-13398)

    vstinner committed May 18, 2019
    _PyPreConfig_InitPythonConfig() and _PyCoreConfig_InitPythonConfig()
    no longer inherit their values from global configuration variables.
    * _PyPreCmdline_Read() now ignores -X dev and PYTHONDEVMODE
      if dev_mode is already set.
    * Inline _PyPreConfig_INIT macro into _PyPreConfig_Init() function.
    * Inline _PyCoreConfig_INIT macro into _PyCoreConfig_Init() function.
    * Replace _PyCoreConfig_Init() with _PyCoreConfig_InitPythonConfig()
      in most tests of _testembed.c.
    * Replace _PyCoreConfig_Init() with _PyCoreConfig_InitIsolatedConfig()
      in _freeze_importlib.c.
    * Move some initialization functions from the internal
      to the private API.
Commits on May 17, 2019
  1. bpo-36763: _Py_InitializeFromArgs() argc becomes Py_ssize_t (GH-13396)

    vstinner committed May 17, 2019
    * The type of initlization function 'argc' parameters becomes
      Py_ssize_t, instead of int.
    * Change _PyPreConfig_Copy() return type to void, instead of int.
      The function cannot fail anymore.
    * Fix compilation warnings on Windows.
  2. bpo-36763: Add _PyInitError functions (GH-13395)

    vstinner committed May 17, 2019
    * Add _PyInitError functions:
      * _PyInitError_Ok()
      * _PyInitError_Error()
      * _PyInitError_NoMemory()
      * _PyInitError_Exit()
      * _PyInitError_IsError()
      * _PyInitError_IsExit()
      * _PyInitError_Failed()
    * frozenmain.c and _testembed.c now use functions rather than macros.
    * Move _Py_INIT_xxx() macros to the internal API.
    * Move _PyWstrList_INIT macro to the internal API.
  3. bpo-36763: _Py_RunMain() doesn't call Py_Exit() anymore (GH-13390)

    vstinner committed May 17, 2019
    Py_Main() and _Py_RunMain() now return the exitcode rather than
    calling Py_Exit(exitcode) when calling PyErr_Print() if the current
    exception type is SystemExit.
    * Add _Py_HandleSystemExit().
    * Add pymain_exit_err_print().
    * Add pymain_exit_print().
  4. bpo-36945: Add _PyPreConfig.configure_locale (GH-13368)

    vstinner committed May 17, 2019
    _PyPreConfig_InitIsolatedConfig() sets configure_locale to 0 to
    prevent Python to modify the LC_CTYPE locale. In that case,
    coerce_c_locale an coerce_c_locale_warn are set to 0 as well.
  5. bpo-36763: Add _PyCoreConfig_InitPythonConfig() (GH-13388)

    vstinner committed May 17, 2019
    Add new functions to get the Python interpreter behavior:
    * _PyPreConfig_InitPythonConfig()
    * _PyCoreConfig_InitPythonConfig()
    Add new functions to get an isolated configuration:
    * _PyPreConfig_InitIsolatedConfig()
    * _PyCoreConfig_InitIsolatedConfig()
    Replace _PyPreConfig_INIT and _PyCoreConfig_INIT with new functions
    _PyPreConfig_Init() and _PyCoreConfig_Init().
    _PyCoreConfig: set configure_c_stdio and parse_argv to 0 by default
    to behave as Python 3.6 in the default configuration.
    _PyCoreConfig_Read() no longer sets coerce_c_locale_warn to 1 if it's
    equal to 0. coerce_c_locale_warn must now be set to -1 (ex: using
    _PyCoreConfig_InitPythonConfig()) to enable C locale coercion
    Add unit tests for _PyCoreConfig_InitPythonConfig()
    and _PyCoreConfig_InitIsolatedConfig().
    * Rename _PyCoreConfig_GetCoreConfig() to _PyPreConfig_GetCoreConfig()
    * Fix core_read_precmdline(): handle parse_argv=0
    * Fix _Py_PreInitializeFromCoreConfig(): pass coreconfig.argv
      to _Py_PreInitializeFromPyArgv(), except if parse_argv=0
  6. bpo-36763: Add PyMemAllocatorName (GH-13387)

    vstinner committed May 17, 2019
    * Add PyMemAllocatorName enum
    * _PyPreConfig.allocator type becomes PyMemAllocatorName, instead of
    * Remove _PyPreConfig_Clear()
    * Add _PyMem_GetAllocatorName()
    * Rename _PyMem_GetAllocatorsName() to
    * Remove _PyPreConfig_SetAllocator(): just call
      _PyMem_SetupAllocators() directly, we don't have do reallocate the
      configuration with the new allocator anymore!
    * _PyPreConfig_Write() parameter becomes const, as it should be in
      the first place!
  7. bpo-36763: Remove _PyCoreConfig.program (GH-13373)

    vstinner committed May 17, 2019
    Use _PyCoreConfig.program_name instead.
  8. bpo-36763: Cleanup precmdline in _PyCoreConfig_Read() (GH-13371)

    vstinner committed May 17, 2019
    precmdline is only needed in _PyCoreConfig_Read(), not in
    config_read_cmdline(). Reorganize _PyCoreConfig_Read().
Commits on May 16, 2019
  1. bpo-36763: Add _PyCoreConfig.configure_c_stdio (GH-13363)

    vstinner committed May 16, 2019
    Add tests for configure_c_stdio and pathconfig_warnings parameters.
  2. bpo-36763: Add _Py_InitializeMain() (GH-13362)

    vstinner committed May 16, 2019
    * Add a private _Py_InitializeMain() function.
    * Add again _PyCoreConfig._init_main.
    * _Py_InitializeFromConfig() now uses _init_main to decide
      if _Py_InitializeMainInterpreter() should be called.
    * _PyCoreConfig: rename _frozen to pathconfig_warnings, its value is
      now the opposite of Py_FrozenFlag.
    * Add an unit test for _init_main=0 and _Py_InitializeMain().
  3. bpo-36763: Add _PyCoreConfig.parse_argv (GH-13361)

    vstinner committed May 16, 2019
    * _PyCoreConfig_Read() doesn't parse nor update argv
      if parse_argv is 0.
    * Move path configuration fields in _PyCoreConfig.
    * Add an unit test for parse_argv=0.
    * Remove unused "done": label in _Py_RunMain().
  4. bpo-36763: _PyInitError always use int for exitcode (GH-13360)

    vstinner committed May 16, 2019
    We cannot use "unsigned int" for exitcode on Windows, since
    Py_Main() and _Py_RunMain() always return an "int".
    * _PyPathConfig_ComputeSysPath0() now returns -1 if an exception is
    * pymain_run_python() no longer uses _PyInitError but display the
      exception and set exitcode to 1 in case of error.
    * Fix _Py_RunMain(): return an exitcode rather than calling
      exit() on pymain_run_python() failure.
    * _Py_ExitInitError() no longer uses ExitProcess() on Windows, use
      exit() on all platforms.
    * _Py_ExitInitError() now fails with a fatal error if 'err' is not an
      error not an exit.
Commits on May 15, 2019
  1. bpo-36763: InitConfigTests tests all core config (GH-13331)

    vstinner committed May 15, 2019
    Remove UNTESTED_CORE_CONFIG from test_embed.InitConfigTests: all core
    config fields are now tested!
    * Test also dll_path on Windows
    * Add run_main_config unit test: test config using _Py_RunMain().
Commits on May 14, 2019
  1. bpo-36763: Add test for _PyCoreConfig_SetString() (GH-13275)

    vstinner committed May 14, 2019
    test_embed: add test_init_read_set() to test newly added APIs: test
    module_search_paths and executable.
  2. bpo-36618: Don't add -fmax-type-align=8 flag for clang (GH-13320)

    vstinner committed May 14, 2019
    Python 3.8 now respects the x86-64 ABI: memory allocations are
    aligned on 16 bytes. The clang flag was only used as a temporary
  3. bpo-36900: Replace global conf vars with config (GH-13299)

    vstinner committed May 14, 2019
    Replace global configuration variables with core_config read from the
    current interpreter.
    Cleanup dynload_hpux.c.
  4. bpo-36915: regrtest always remove tempdir of worker processes (GH-13312)

    vstinner committed May 14, 2019
    When using multiprocessing (-jN option), worker processes now create
    their temporary directory inside the temporary directory of the
    main process. So the main process is able to remove temporary
    directories of worker processes even if they crash or when they are
    killed by regrtest on KeyboardInterrupt (CTRL+c).
    Rework also how multiprocessing arguments are parsed in
  5. bpo-36719: Fix regrtest MultiprocessThread (GH-13301)

    vstinner committed May 14, 2019
    MultiprocessThread.kill() now closes stdout and stderr to prevent
    popen.communicate() to hang.
You can’t perform that action at this time.