python/cpython

_PyCoreConfig_SetPyArgv() and _PyCoreConfig_SetWideString() now
pre-initialize Python if needed to ensure that the locale encoding is
properly configured.

* Add _Py_PreInitializeFromPyArgv() internal function.
* Add 'args' parameter to _Py_PreInitializeFromCoreConfig()

_PyCoreConfig: Change filesystem_encoding, filesystem_errors,
stdio_encoding and stdio_errors fields type from char* to wchar_t*.

Changes:

* PyInterpreterState: replace fscodec_initialized (int) with fs_codec
  structure.
* Add get_error_handler_wide() and unicode_encode_utf8() helper
  functions.
* Add error_handler parameter to unicode_encode_locale()
  and unicode_decode_locale().
* Remove _PyCoreConfig_SetString().
* Rename _PyCoreConfig_SetWideString() to _PyCoreConfig_SetString().
* Rename _PyCoreConfig_SetWideStringFromString()
  to _PyCoreConfig_DecodeLocale().

Add private _config_version field to _PyPreConfig and _PyCoreConfig
to prepare future ABI compatibility.

Move get_codec_name() and initfsencoding() from pylifecycle.c to
unicodeobject.c.

Rename also "init" functions in pylifecycle.c.

Add _Py_FORCE_UTF8_LOCALE and _Py_FORCE_UTF8_FS_ENCODING macros to
avoid factorize "#if defined(__ANDROID__) || defined(__VXWORKS__)"
and "#if defined(__APPLE__)".

Cleanup also config_init_fs_encoding().

_PyCoreConfig: Rename _check_hash_pycs_mode field to
check_hash_pycs_mode (make it public) and change its type from "const
char*" to "wchar_t*".

Add 3 new config methods:

* _PyCoreConfig_SetString()
* _PyCoreConfig_SetWideString()
* _PyCoreConfig_SetWideStringFromString()

Changes:

* _PyCoreConfig_Copy() returns _PyInitError.
* Add CONFIG_GET_ENV_DUP().

* Remove _PyInitError.user_err field and _Py_INIT_USER_ERR() macro:
  use _Py_INIT_ERR() instead. _Py_ExitInitError() now longer calls
  abort() on error: exit with exit code 1 instead.
* Add _PyInitError._type private field.
* exitcode field type is now unsigned int on Windows.
* Rename prefix field to _func.
* Rename msg field to err_msg.

* Add 2 new config methods:

  * _PyCoreConfig_SetArgv()
  * _PyCoreConfig_SetWideArgv()

* Add also an internal _PyCoreConfig_SetPyArgv() method.
* Remove 'args' parameter from _PyCoreConfig_Read().

Fix memory leak in Py_SetStandardStreamEncoding(): release memory
if the function is called twice.

Initialize "stack_t current_stack" to zero using memset().

This reverts commit b36e5d6.

* Add _PyEval_FiniThreads2(). _PyEval_FiniThreads() now only clears
  the pending lock, whereas _PyEval_FiniThreads2() destroys the GIL.
* pymain_free() now calls _PyEval_FiniThreads2().
* Py_FinalizeEx() now calls _PyEval_FiniThreads().

Py_TRACE_REFS ABI is incompatible with release and debug (Py_DEBUG)
ABI.

Misc/python-config.sh lives in the build directory, not in the source
directory.

Add PYTHONPATH=$(srcdir) to run $(PYTHON_FOR_REGEN) -m Parser.pgen,
so it's possible to build Python from a different directory.

…2968)

Add _Py_INIT_HAS_EXITCODE() macro.

Regrtest.finalize() now closes explicitly the WindowsLoadTracker
instance.

Properly handle a test which fail but then pass.

Add test_rerun_success() unit test.

regrtest now always detects uncollectable objects. Previously, the
check was only enabled by --findleaks. The check now also works with
-jN/--multiprocess N.

--findleaks becomes a deprecated alias to --fail-env-changed.

Rewrite run_tests_multiprocess() function as a new MultiprocessRunner
class with multiple methods to better report errors and stop
immediately when needed.

Changes:

* Worker processes are now killed immediately if tests are
  interrupted or if a test does crash (CHILD_ERROR): worker
  processes are killed.
* Rewrite how errors in a worker thread are reported to
  the main thread. No longer ignore BaseException or parsing errors
  silently.
* Remove 'finished' variable: use worker.is_alive() instead
* Always compute omitted tests. Add Regrtest.get_executed() method.

Py_FinalizeEx() now clears _PyRuntime.warnings variables and
_PyRuntime.exitfuncs.

Changes:

* Add _PyWarnings_Fini(): called by Py_FinalizeEx()
* call_ll_exitfuncs() now clears _PyRuntime.exitfuncs while iterating
  on it (on backward order).

_PyThreadState_Delete() has no return value.

* Add TestResult and MultiprocessResult types to ensure that results
  always have the same fields.
* runtest() now handles KeyboardInterrupt
* accumulate_result() and format_test_result() now takes a TestResult
* cleanup_test_droppings() is now called by runtest() and mark the
  test as ENV_CHANGED if the test leaks support.TESTFN file.
* runtest() now includes code "around" the test in the test timing
* Add print_warning() in test.libregrtest.utils to standardize how
  libregrtest logs warnings to ease parsing the test output.
* support.unload() is now called with abstest rather than test_name
* Rename 'test' variable/parameter to 'test_name'
* dash_R(): remove unused the_module parameter
* Remove unused imports

Add 'state' or 'runtime' parameter to functions in gcmodule.c to
avoid to rely directly on the global variable _PyRuntime.

Add 'gilstate', 'runtime' or 'xidregistry' parameter to many
functions on pystate.c to avoid lying on _PyRuntime global.

In debug build, import now also looks for C extensions compiled in
release mode and for C extensions compiled in the stable ABI.

On Unix, C extensions are no longer linked to libpython.

It is now possible to load a C extension built using a shared library
Python with a statically linked Python.

When Python is embedded, libpython must not be loaded with
RTLD_LOCAL, but RTLD_GLOBAL instead. Previously, using RTLD_LOCAL, it
was already not possible to load C extensions which were not linked
to libpython, like C extensions of the standard library built by the
"*shared*" section of Modules/Setup.

distutils, python-config and python-config.py have been modified.

Document that subprocess.Popen no longer raise an exception on error
like missing program on very specific platforms when using
os.posix_spawn() is used.

bpo-28552, bpo-7774: Fix distutils.sysconfig if sys.executable is
None or an empty string: use os.getcwd() to initialize project_base.

Fix also the distutils build command: don't use sys.executable if
it's evaluated as false (None or empty string).

Fix Py_DEBUG constant: check for sys.gettotalrefcount attribute
rather than sys.getobjects.

Update also SpecialBuilds.txt documentation.

Release build and debug build are now ABI compatible: the Py_DEBUG
define no longer implies Py_TRACE_REFS define which introduces the
only ABI incompatibility.

A new "./configure --with-trace-refs" build option is now required to
get Py_TRACE_REFS define which adds sys.getobjects() function and
PYTHONDUMPREFS environment variable.

Changes:

* Add ./configure --with-trace-refs
* Py_DEBUG no longer implies Py_TRACE_REFS