Merge remote-tracking branch 'origin/master' into bpo_41073_add_types…

…lots1

* origin/master: (63 commits)
  bpo-41627: Distinguish 32 and 64-bit user site packages on Windows (pythonGH-22098)
  bpo-38585: Remove references to defusedexpat (pythonGH-22095)
  bpo-41721: Add xlc options (pythonGH-22096)
  bpo-40486: Specify what happens if directory content change diring iteration (pythonGH-22025)
  bpo-41638: Improve ProgrammingError message for absent parameter. (pythonGH-21999)
  bpo-41713: _signal doesn't use multi-phase init (pythonGH-22087)
  bpo-41700: Skip test if the locale is not supported (pythonGH-22081)
  [doc] Update documentation on logging optimization. (pythonGH-22075)
  Fix 'gather' rules in the python parser generator (pythonGH-22021)
  bpo-41697: Correctly handle KeywordOrStarred when parsing arguments in the parser (pythonGH-22077)
  [doc] Fix a typo in the graphlib docs (python#22030)
  bpo-1635741: Port _signal module to multi-phase init (PEP 489) (pythonGH-22049)
  bpo-39883: Use BSD0 license for code in docs (pythonGH-17635)
  bpo-39010: Improve test shutdown (python#22066)
  bpo-41696: Fix handling of debug mode in asyncio.run (python#22069)
  bpo-41690: Use a loop to collect args in the parser instead of recursion (pythonGH-22053)
  closes bpo-41689: Preserve text signature from tp_doc in C heap type creation. (pythonGH-22058)
  Fix invalid escape sequences in the peg_highlight Sphinx extension (pythonGH-22047)
  bpo-41675: Modernize siginterrupt calls (pythonGH-22028)
  bpo-41685: Don't pin setuptools version anymore in Doc/Makefile (pythonGH-22062)
  ...

Doc/Makefile

@@ -143,7 +143,7 @@ clean:
 venv:
 	$(PYTHON) -m venv $(VENVDIR)
 	$(VENVDIR)/bin/python3 -m pip install -U pip setuptools
-	$(VENVDIR)/bin/python3 -m pip install -U Sphinx==2.3.1 blurb python-docs-theme
+	$(VENVDIR)/bin/python3 -m pip install -U Sphinx==3.2.1 blurb python-docs-theme
 	@echo "The venv has been created in the $(VENVDIR) directory"
 
 dist:

Doc/c-api/type.rst

@@ -118,6 +118,13 @@ Type Objects
    If no module is associated with the given type, sets :py:class:`TypeError`
    and returns ``NULL``.
 
+   This function is usually used to get the module in which a method is defined.
+   Note that in such a method, ``PyType_GetModule(Py_TYPE(self))``
+   may not return the intended result.
+   ``Py_TYPE(self)`` may be a *subclass* of the intended class, and subclasses
+   are not necessarily defined in the same module as their superclass.
+   See :c:type:`PyCMethod` to get the class that defines the method.
+
    .. versionadded:: 3.9
 
 .. c:function:: void* PyType_GetModuleState(PyTypeObject *type)
@@ -152,9 +159,12 @@ The following functions and structs are used to create
    If *bases* is ``NULL``, the *Py_tp_base* slot is used instead.
    If that also is ``NULL``, the new type derives from :class:`object`.
 
-   The *module* must be a module object or ``NULL``.
+   The *module* argument can be used to record the module in which the new
+   class is defined. It must be a module object or ``NULL``.
    If not ``NULL``, the module is associated with the new type and can later be
    retreived with :c:func:`PyType_GetModule`.
+   The associated module is not inherited by subclasses; it must be specified
+   for each class individually.
 
    This function calls :c:func:`PyType_Ready` on the new type.
 
@@ -216,7 +226,8 @@ The following functions and structs are used to create
       * ``Py_nb_add`` to set :c:member:`PyNumberMethods.nb_add`
       * ``Py_sq_length`` to set :c:member:`PySequenceMethods.sq_length`
 
-      The following fields cannot be set using :c:type:`PyType_Spec` and :c:type:`PyType_Slot`:
+      The following fields cannot be set at all using :c:type:`PyType_Spec` and
+      :c:type:`PyType_Slot`:
 
       * :c:member:`~PyTypeObject.tp_dict`
       * :c:member:`~PyTypeObject.tp_mro`
@@ -230,13 +241,21 @@ The following functions and structs are used to create
         (see :ref:`PyMemberDef <pymemberdef-offsets>`)
       * :c:member:`~PyTypeObject.tp_vectorcall_offset`
         (see :ref:`PyMemberDef <pymemberdef-offsets>`)
+
+      The following fields cannot be set using :c:type:`PyType_Spec` and
+      :c:type:`PyType_Slot` under the limited API:
+
       * :c:member:`~PyBufferProcs.bf_getbuffer`
       * :c:member:`~PyBufferProcs.bf_releasebuffer`
 
       Setting :c:data:`Py_tp_bases` may be problematic on some platforms.
       To avoid issues, use the *bases* argument of
       :py:func:`PyType_FromSpecWithBases` instead.
 
+     .. versionchanged:: 3.9
+
+        Slots in :c:type:`PyBufferProcs` in may be set in the unlimited API.
+
    .. c:member:: void *PyType_Slot.pfunc
 
       The desired value of the slot. In most cases, this is a pointer

Doc/faq/design.rst

@@ -573,8 +573,7 @@ whether an instance or a class implements a particular ABC.  The
 :class:`~collections.abc.MutableMapping`.
 
 For Python, many of the advantages of interface specifications can be obtained
-by an appropriate test discipline for components.  There is also a tool,
-PyChecker, which can be used to find problems due to subclassing.
+by an appropriate test discipline for components.
 
 A good test suite for a module can both provide a regression test and serve as a
 module interface specification and a set of examples.  Many Python modules can

Doc/faq/general.rst

@@ -142,9 +142,9 @@ to fix critical bugs.
 Alpha, beta and release candidate versions have an additional suffix.  The
 suffix for an alpha version is "aN" for some small number N, the suffix for a
 beta version is "bN" for some small number N, and the suffix for a release
-candidate version is "cN" for some small number N.  In other words, all versions
+candidate version is "rcN" for some small number N.  In other words, all versions
 labeled 2.0aN precede the versions labeled 2.0bN, which precede versions labeled
-2.0cN, and *those* precede 2.0.
+2.0rcN, and *those* precede 2.0.
 
 You may also find version numbers with a "+" suffix, e.g. "2.2+".  These are
 unreleased versions, built directly from the CPython development repository.  In
@@ -309,8 +309,8 @@ releases.
 The latest stable releases can always be found on the `Python download page
 <https://www.python.org/downloads/>`_.  There are two production-ready versions
 of Python: 2.x and 3.x. The recommended version is 3.x, which is supported by
-most widely used libraries.  Although 2.x is still widely used, `it will not
-be maintained after January 1, 2020 <https://www.python.org/dev/peps/pep-0373/>`_.
+most widely used libraries.  Although 2.x is still widely used, `it is not
+maintained anymore <https://www.python.org/dev/peps/pep-0373/>`_.
 
 How many people are using Python?
 ---------------------------------

Doc/faq/programming.rst

@@ -35,12 +35,6 @@ for Windows Extensions <https://sourceforge.net/projects/pywin32/>`__ project an
 as a part of the ActivePython distribution (see
 https://www.activestate.com/activepython\ ).
 
-`Boa Constructor <http://boa-constructor.sourceforge.net/>`_ is an IDE and GUI
-builder that uses wxWidgets.  It offers visual frame creation and manipulation,
-an object inspector, many views on the source like object browsers, inheritance
-hierarchies, doc string generated html documentation, an advanced debugger,
-integrated help, and Zope support.
-
 `Eric <http://eric-ide.python-projects.org/>`_ is an IDE built on PyQt
 and the Scintilla editing component.
 
@@ -57,22 +51,14 @@ They include:
 * PyCharm (https://www.jetbrains.com/pycharm/)
 
 
-Is there a tool to help find bugs or perform static analysis?
+Are there tools to help find bugs or perform static analysis?
 -------------------------------------------------------------
 
 Yes.
 
-PyChecker is a static analysis tool that finds bugs in Python source code and
-warns about code complexity and style.  You can get PyChecker from
-http://pychecker.sourceforge.net/.
-
-`Pylint <https://www.pylint.org/>`_ is another tool that checks
-if a module satisfies a coding standard, and also makes it possible to write
-plug-ins to add a custom feature.  In addition to the bug checking that
-PyChecker performs, Pylint offers some additional features such as checking line
-length, whether variable names are well-formed according to your coding
-standard, whether declared interfaces are fully implemented, and more.
-https://docs.pylint.org/ provides a full list of Pylint's features.
+`Pylint <https://www.pylint.org/>`_ and
+`Pyflakes <https://github.com/PyCQA/pyflakes>`_ do basic checking that will
+help you catch bugs sooner.
 
 Static type checkers such as `Mypy <http://mypy-lang.org/>`_,
 `Pyre <https://pyre-check.org/>`_, and

Doc/howto/logging.rst

@@ -1078,20 +1078,22 @@ need more precise control over what logging information is collected. Here's a
 list of things you can do to avoid processing during logging which you don't
 need:
 
-+-----------------------------------------------+----------------------------------------+
-| What you don't want to collect                | How to avoid collecting it             |
-+===============================================+========================================+
-| Information about where calls were made from. | Set ``logging._srcfile`` to ``None``.  |
-|                                               | This avoids calling                    |
-|                                               | :func:`sys._getframe`, which may help  |
-|                                               | to speed up your code in environments  |
-|                                               | like PyPy (which can't speed up code   |
-|                                               | that uses :func:`sys._getframe`).      |
-+-----------------------------------------------+----------------------------------------+
-| Threading information.                        | Set ``logging.logThreads`` to ``0``.   |
-+-----------------------------------------------+----------------------------------------+
-| Process information.                          | Set ``logging.logProcesses`` to ``0``. |
-+-----------------------------------------------+----------------------------------------+
++-----------------------------------------------------+---------------------------------------------------+
+| What you don't want to collect                      | How to avoid collecting it                        |
++=====================================================+===================================================+
+| Information about where calls were made from.       | Set ``logging._srcfile`` to ``None``.             |
+|                                                     | This avoids calling :func:`sys._getframe`, which  |
+|                                                     | may help to speed up your code in environments    |
+|                                                     | like PyPy (which can't speed up code that uses    |
+|                                                     | :func:`sys._getframe`).                           |
++-----------------------------------------------------+---------------------------------------------------+
+| Threading information.                              | Set ``logging.logThreads`` to ``False``.          |
++-----------------------------------------------------+---------------------------------------------------+
+| Current process ID (:func:`os.getpid`)              | Set ``logging.logProcesses`` to ``False``.        |
++-----------------------------------------------------+---------------------------------------------------+
+| Current process name when using ``multiprocessing`` | Set ``logging.logMultiprocessing`` to ``False``.  |
+| to manage multiple processes.                       |                                                   |
++-----------------------------------------------------+---------------------------------------------------+
 
 Also note that the core logging module only includes the basic handlers. If
 you don't import :mod:`logging.handlers` and :mod:`logging.config`, they won't

Doc/library/argparse.rst

@@ -1162,8 +1162,8 @@ keyword argument to :meth:`~ArgumentParser.add_argument`::
    >>> parser.parse_args(['--foo', 'BAR'])
    Namespace(foo='BAR')
    >>> parser.parse_args([])
-   usage: argparse.py [-h] [--foo FOO]
-   argparse.py: error: option --foo is required
+   usage: [-h] --foo FOO
+   : error: the following arguments are required: --foo
 
 As the example shows, if an option is marked as ``required``,
 :meth:`~ArgumentParser.parse_args` will report an error if that option is not

Doc/library/asyncio-dev.rst

@@ -107,6 +107,16 @@ The :meth:`loop.run_in_executor` method can be used with a
 blocking code in a different OS thread without blocking the OS thread
 that the event loop runs in.
 
+There is currently no way to schedule coroutines or callbacks directly
+from a different process (such as one started with
+:mod:`multiprocessing`). The :ref:`Event Loop Methods <asyncio-event-loop>`
+section lists APIs that can read from pipes and watch file descriptors
+without blocking the event loop. In addition, asyncio's
+:ref:`Subprocess <asyncio-subprocess>` APIs provide a way to start a
+process and communicate with it from the event loop. Lastly, the
+aforementioned :meth:`loop.run_in_executor` method can also be used
+with a :class:`concurrent.futures.ProcessPoolExecutor` to execute
+code in a different process.
 
 .. _asyncio-handle-blocking:
 

Doc/library/curses.rst

@@ -717,7 +717,7 @@ the following methods and attributes:
             window.addch(y, x, ch[, attr])
 
    Paint character *ch* at ``(y, x)`` with attributes *attr*, overwriting any
-   character previously painter at that location.  By default, the character
+   character previously painted at that location.  By default, the character
    position and attributes are the current settings for the window object.
 
    .. note::

Doc/library/functions.rst

@@ -1734,6 +1734,9 @@ are always available.  They are listed here in alphabetical order.
    locals dictionary is only useful for reads since updates to the locals
    dictionary are ignored.
 
+   A :exc:`TypeError` exception is raised if an object is specified but
+   it doesn't have a :attr:`~object.__dict__` attribute (for example, if
+   its class defines the :attr:`~object.__slots__` attribute).
 
 .. function:: zip(*iterables, strict=False)
 

Doc/library/glob.rst

@@ -43,7 +43,9 @@ For example, ``'[?]'`` matches the character ``'?'``.
    (like :file:`/usr/src/Python-1.5/Makefile`) or relative (like
    :file:`../../Tools/\*/\*.gif`), and can contain shell-style wildcards. Broken
    symlinks are included in the results (as in the shell). Whether or not the
-   results are sorted depends on the file system.
+   results are sorted depends on the file system.  If a file that satisfies
+   conditions is removed or added during the call of this function, whether
+   a path name for that file be included is unspecified.
 
    If *root_dir* is not ``None``, it should be a :term:`path-like object`
    specifying the root directory for searching.  It has the same effect on

Doc/library/graphlib.rst

@@ -121,7 +121,7 @@
           if ts.is_active():
               ...
 
-      if possible to simply do::
+      it is possible to simply do::
 
           if ts:
               ...

Doc/library/json.rst

@@ -745,7 +745,7 @@ Command line options
 
 .. cmdoption:: --indent, --tab, --no-indent, --compact
 
-   Mutually exclusive options for whitespace control
+   Mutually exclusive options for whitespace control.
 
    .. versionadded:: 3.9
 

Doc/library/logging.rst

@@ -1166,9 +1166,9 @@ functions.
    +--------------+---------------------------------------------+
    | Format       | Description                                 |
    +==============+=============================================+
-   | *filename*   | Specifies that a FileHandler be created,    |
-   |              | using the specified filename, rather than a |
-   |              | StreamHandler.                              |
+   | *filename*   | Specifies that a :class:`FileHandler` be    |
+   |              | created, using the specified filename,      |
+   |              | rather than a :class:`StreamHandler`.       |
    +--------------+---------------------------------------------+
    | *filemode*   | If *filename* is specified, open the file   |
    |              | in this :ref:`mode <filemodes>`. Defaults   |
@@ -1192,9 +1192,10 @@ functions.
    |              | :ref:`level <levels>`.                      |
    +--------------+---------------------------------------------+
    | *stream*     | Use the specified stream to initialize the  |
-   |              | StreamHandler. Note that this argument is   |
-   |              | incompatible with *filename* - if both      |
-   |              | are present, a ``ValueError`` is raised.    |
+   |              | :class:`StreamHandler`. Note that this      |
+   |              | argument is incompatible with *filename* -  |
+   |              | if both are present, a ``ValueError`` is    |
+   |              | raised.                                     |
    +--------------+---------------------------------------------+
    | *handlers*   | If specified, this should be an iterable of |
    |              | already created handlers to add to the root |
@@ -1213,18 +1214,18 @@ functions.
    +--------------+---------------------------------------------+
    | *encoding*   | If this keyword argument is specified along |
    |              | with *filename*, its value is used when the |
-   |              | FileHandler is created, and thus used when  |
-   |              | opening the output file.                    |
+   |              | :class:`FileHandler` is created, and thus   |
+   |              | used when opening the output file.          |
    +--------------+---------------------------------------------+
    | *errors*     | If this keyword argument is specified along |
    |              | with *filename*, its value is used when the |
-   |              | FileHandler is created, and thus used when  |
-   |              | opening the output file. If not specified,  |
-   |              | the value 'backslashreplace' is used. Note  |
-   |              | that if ``None`` is specified, it will be   |
-   |              | passed as such to func:`open`, which means  |
-   |              | that it will be treated the same as passing |
-   |              | 'errors'.                                   |
+   |              | :class:`FileHandler` is created, and thus   |
+   |              | used when opening the output file. If not   |
+   |              | specified, the value 'backslashreplace' is  |
+   |              | used. Note that if ``None`` is specified,   |
+   |              | it will be passed as such to :func:`open`,  |
+   |              | which means that it will be treated the     |
+   |              | same as passing 'errors'.                   |
    +--------------+---------------------------------------------+
 
    .. versionchanged:: 3.2

Doc/library/multiprocessing.rst

@@ -861,7 +861,7 @@ For an example of the usage of queues for interprocess communication see
 
       A better name for this method might be
       ``allow_exit_without_flush()``.  It is likely to cause enqueued
-      data to lost, and you almost certainly will not need to use it.
+      data to be lost, and you almost certainly will not need to use it.
       It is really only there if you need the current process to exit
       immediately without waiting to flush enqueued data to the
       underlying pipe, and you don't care about lost data.

Doc/library/os.rst

@@ -1852,6 +1852,8 @@ features:
    Return a list containing the names of the entries in the directory given by
    *path*.  The list is in arbitrary order, and does not include the special
    entries ``'.'`` and ``'..'`` even if they are present in the directory.
+   If a file is removed from or added to the directory during the call of
+   this function, whether a name for that file be included is unspecified.
 
    *path* may be a :term:`path-like object`.  If *path* is of type ``bytes``
    (directly or indirectly through the :class:`PathLike` interface),
@@ -2257,7 +2259,9 @@ features:
    Return an iterator of :class:`os.DirEntry` objects corresponding to the
    entries in the directory given by *path*. The entries are yielded in
    arbitrary order, and the special entries ``'.'`` and ``'..'`` are not
-   included.
+   included.  If a file is removed from or added to the directory after
+   creating the iterator, whether an entry for that file be included is
+   unspecified.
 
    Using :func:`scandir` instead of :func:`listdir` can significantly
    increase the performance of code that also needs file type or file
@@ -3007,7 +3011,10 @@ features:
    *filenames* is a list of the names of the non-directory files in *dirpath*.
    Note that the names in the lists contain no path components.  To get a full path
    (which begins with *top*) to a file or directory in *dirpath*, do
-   ``os.path.join(dirpath, name)``.
+   ``os.path.join(dirpath, name)``.  Whether or not the lists are sorted
+   depends on the file system.  If a file is removed from or added to the
+   *dirpath* directory during generating the lists, whether a name for that
+   file be included is unspecified.
 
    If optional argument *topdown* is ``True`` or not specified, the triple for a
    directory is generated before the triples for any of its subdirectories

Doc/library/pathlib.rst

@@ -890,6 +890,11 @@ call fails (for example because the path doesn't exist).
       PosixPath('docs/_static')
       PosixPath('docs/Makefile')
 
+   The children are yielded in arbitrary order, and the special entries
+   ``'.'`` and ``'..'`` are not included.  If a file is removed from or added
+   to the directory after creating the iterator, whether an path object for
+   that file be included is unspecified.
+
 .. method:: Path.lchmod(mode)
 
    Like :meth:`Path.chmod` but, if the path points to a symbolic link, the