Merge branch 'main' into pythongh-110109-simplify-test-setup

.pre-commit-config.yaml

@@ -1,6 +1,6 @@
 repos:
   - repo: https://github.com/astral-sh/ruff-pre-commit
-    rev: v0.0.288
+    rev: v0.0.292
     hooks:
       - id: ruff
         name: Run Ruff on Lib/test/

Doc/c-api/init.rst

@@ -870,6 +870,19 @@ code, or when embedding the Python interpreter:
    When the current thread state is ``NULL``, this issues a fatal error (so that
    the caller needn't check for ``NULL``).
 
+   See also :c:func:`PyThreadState_GetUnchecked`.
+
+
+.. c:function:: PyThreadState* PyThreadState_GetUnchecked()
+
+   Similar to :c:func:`PyThreadState_Get`, but don't kill the process with a
+   fatal error if it is NULL. The caller is responsible to check if the result
+   is NULL.
+
+   .. versionadded:: 3.13
+      In Python 3.5 to 3.12, the function was private and known as
+      ``_PyThreadState_UncheckedGet()``.
+
 
 .. c:function:: PyThreadState* PyThreadState_Swap(PyThreadState *tstate)
 

Doc/c-api/object.rst

@@ -489,3 +489,21 @@ Object Protocol
    :c:macro:`Py_TPFLAGS_ITEMS_AT_END` set.
 
    .. versionadded:: 3.12
+
+.. c:function:: int PyObject_VisitManagedDict(PyObject *obj, visitproc visit, void *arg)
+
+   Visit the managed dictionary of *obj*.
+
+   This function must only be called in a traverse function of the type which
+   has the :c:macro:`Py_TPFLAGS_MANAGED_DICT` flag set.
+
+   .. versionadded:: 3.13
+
+.. c:function:: void PyObject_ClearManagedDict(PyObject *obj)
+
+   Clear the managed dictionary of *obj*.
+
+   This function must only be called in a traverse function of the type which
+   has the :c:macro:`Py_TPFLAGS_MANAGED_DICT` flag set.
+
+   .. versionadded:: 3.13

Doc/c-api/typeobj.rst

@@ -1131,6 +1131,9 @@ and :c:data:`PyType_Type` effectively act as defaults.)
 
       If this flag is set, :c:macro:`Py_TPFLAGS_HAVE_GC` should also be set.
 
+      The type traverse function must call :c:func:`PyObject_VisitManagedDict`
+      and its clear function must call :c:func:`PyObject_ClearManagedDict`.
+
       .. versionadded:: 3.12
 
       **Inheritance:**
@@ -1368,6 +1371,23 @@ and :c:data:`PyType_Type` effectively act as defaults.)
    debugging aid you may want to visit it anyway just so the :mod:`gc` module's
    :func:`~gc.get_referents` function will include it.
 
+   Heap types (:c:macro:`Py_TPFLAGS_HEAPTYPE`) must visit their type with::
+
+       Py_VISIT(Py_TYPE(self));
+
+   It is only needed since Python 3.9. To support Python 3.8 and older, this
+   line must be conditionnal::
+
+       #if PY_VERSION_HEX >= 0x03090000
+           Py_VISIT(Py_TYPE(self));
+       #endif
+
+   If the :c:macro:`Py_TPFLAGS_MANAGED_DICT` bit is set in the
+   :c:member:`~PyTypeObject.tp_flags` field, the traverse function must call
+   :c:func:`PyObject_VisitManagedDict` like this::
+
+       PyObject_VisitManagedDict((PyObject*)self, visit, arg);
+
    .. warning::
        When implementing :c:member:`~PyTypeObject.tp_traverse`, only the
        members that the instance *owns* (by having :term:`strong references
@@ -1451,6 +1471,12 @@ and :c:data:`PyType_Type` effectively act as defaults.)
    so that *self* knows the contained object can no longer be used.  The
    :c:func:`Py_CLEAR` macro performs the operations in a safe order.
 
+   If the :c:macro:`Py_TPFLAGS_MANAGED_DICT` bit is set in the
+   :c:member:`~PyTypeObject.tp_flags` field, the traverse function must call
+   :c:func:`PyObject_ClearManagedDict` like this::
+
+       PyObject_ClearManagedDict((PyObject*)self);
+
    Note that :c:member:`~PyTypeObject.tp_clear` is not *always* called
    before an instance is deallocated. For example, when reference counting
    is enough to determine that an object is no longer used, the cyclic garbage
@@ -1801,7 +1827,7 @@ and :c:data:`PyType_Type` effectively act as defaults.)
    field is ``NULL`` then no :attr:`~object.__dict__` gets created for instances.
 
    If the :c:macro:`Py_TPFLAGS_MANAGED_DICT` bit is set in the
-   :c:member:`~PyTypeObject.tp_dict` field, then
+   :c:member:`~PyTypeObject.tp_flags` field, then
    :c:member:`~PyTypeObject.tp_dictoffset` will be set to ``-1``, to indicate
    that it is unsafe to use this field.
 

Doc/constraints.txt

@@ -10,8 +10,7 @@ colorama<0.5
 imagesize<1.5
 Jinja2<3.2
 packaging<24
-# Pygments==2.15.0 breaks CI
-Pygments<2.16,!=2.15.0
+Pygments>=2.16.1,<3
 requests<3
 snowballstemmer<3
 sphinxcontrib-applehelp<1.1

Doc/library/__main__.rst

@@ -238,9 +238,9 @@ package.  For more details, see :ref:`intra-package-references` in the
 Idiomatic Usage
 ^^^^^^^^^^^^^^^
 
-The contents of ``__main__.py`` typically isn't fenced with
-``if __name__ == '__main__'`` blocks.  Instead, those files are kept short,
-functions to execute from other modules.  Those other modules can then be
+The content of ``__main__.py`` typically isn't fenced with an
+``if __name__ == '__main__'`` block.  Instead, those files are kept
+short and import functions to execute from other modules.  Those other modules can then be
 easily unit-tested and are properly reusable.
 
 If used, an ``if __name__ == '__main__'`` block will still work as expected

Doc/library/ast.rst

@@ -2483,26 +2483,26 @@ The following options are accepted:
 
 .. program:: ast
 
-.. cmdoption:: -h, --help
+.. option:: -h, --help
 
    Show the help message and exit.
 
-.. cmdoption:: -m <mode>
-               --mode <mode>
+.. option:: -m <mode>
+            --mode <mode>
 
    Specify what kind of code must be compiled, like the *mode* argument
    in :func:`parse`.
 
-.. cmdoption:: --no-type-comments
+.. option:: --no-type-comments
 
    Don't parse type comments.
 
-.. cmdoption:: -a, --include-attributes
+.. option:: -a, --include-attributes
 
    Include attributes such as line numbers and column offsets.
 
-.. cmdoption:: -i <indent>
-               --indent <indent>
+.. option:: -i <indent>
+            --indent <indent>
 
    Indentation of nodes in AST (number of spaces).
 

Doc/library/compileall.rst

@@ -26,74 +26,74 @@ compile Python sources.
 
 .. program:: compileall
 
-.. cmdoption:: directory ...
-               file ...
+.. option:: directory ...
+            file ...
 
    Positional arguments are files to compile or directories that contain
    source files, traversed recursively.  If no argument is given, behave as if
    the command line was :samp:`-l {<directories from sys.path>}`.
 
-.. cmdoption:: -l
+.. option:: -l
 
    Do not recurse into subdirectories, only compile source code files directly
    contained in the named or implied directories.
 
-.. cmdoption:: -f
+.. option:: -f
 
    Force rebuild even if timestamps are up-to-date.
 
-.. cmdoption:: -q
+.. option:: -q
 
    Do not print the list of files compiled. If passed once, error messages will
    still be printed. If passed twice (``-qq``), all output is suppressed.
 
-.. cmdoption:: -d destdir
+.. option:: -d destdir
 
    Directory prepended to the path to each file being compiled.  This will
    appear in compilation time tracebacks, and is also compiled in to the
    byte-code file, where it will be used in tracebacks and other messages in
    cases where the source file does not exist at the time the byte-code file is
    executed.
 
-.. cmdoption:: -s strip_prefix
-.. cmdoption:: -p prepend_prefix
+.. option:: -s strip_prefix
+.. option:: -p prepend_prefix
 
    Remove (``-s``) or append (``-p``) the given prefix of paths
    recorded in the ``.pyc`` files.
    Cannot be combined with ``-d``.
 
-.. cmdoption:: -x regex
+.. option:: -x regex
 
    regex is used to search the full path to each file considered for
    compilation, and if the regex produces a match, the file is skipped.
 
-.. cmdoption:: -i list
+.. option:: -i list
 
    Read the file ``list`` and add each line that it contains to the list of
    files and directories to compile.  If ``list`` is ``-``, read lines from
    ``stdin``.
 
-.. cmdoption:: -b
+.. option:: -b
 
    Write the byte-code files to their legacy locations and names, which may
    overwrite byte-code files created by another version of Python.  The default
    is to write files to their :pep:`3147` locations and names, which allows
    byte-code files from multiple versions of Python to coexist.
 
-.. cmdoption:: -r
+.. option:: -r
 
    Control the maximum recursion level for subdirectories.
    If this is given, then ``-l`` option will not be taken into account.
    :program:`python -m compileall <directory> -r 0` is equivalent to
    :program:`python -m compileall <directory> -l`.
 
-.. cmdoption:: -j N
+.. option:: -j N
 
    Use *N* workers to compile the files within the given directory.
-   If ``0`` is used, then the result of :func:`os.cpu_count()`
+   If ``0`` is used, then the result of :func:`os.process_cpu_count()`
    will be used.
 
-.. cmdoption:: --invalidation-mode [timestamp|checked-hash|unchecked-hash]
+.. option:: --invalidation-mode [timestamp|checked-hash|unchecked-hash]
 
    Control how the generated byte-code files are invalidated at runtime.
    The ``timestamp`` value, means that ``.pyc`` files with the source timestamp
@@ -106,17 +106,17 @@ compile Python sources.
    variable is not set, and ``checked-hash`` if the ``SOURCE_DATE_EPOCH``
    environment variable is set.
 
-.. cmdoption:: -o level
+.. option:: -o level
 
    Compile with the given optimization level. May be used multiple times
    to compile for multiple levels at a time (for example,
    ``compileall -o 1 -o 2``).
 
-.. cmdoption:: -e dir
+.. option:: -e dir
 
    Ignore symlinks pointing outside the given directory.
 
-.. cmdoption:: --hardlink-dupes
+.. option:: --hardlink-dupes
 
    If two ``.pyc`` files with different optimization level have
    the same content, use hard links to consolidate duplicate files.

Doc/library/concurrent.futures.rst

@@ -188,6 +188,10 @@ And::
       ThreadPoolExecutor now reuses idle worker threads before starting
       *max_workers* worker threads too.
 
+   .. versionchanged:: 3.13
+      Default value of *max_workers* is changed to
+      ``min(32, (os.process_cpu_count() or 1) + 4)``.
+
 
 .. _threadpoolexecutor-example:
 
@@ -243,7 +247,7 @@ to a :class:`ProcessPoolExecutor` will result in deadlock.
 
    An :class:`Executor` subclass that executes calls asynchronously using a pool
    of at most *max_workers* processes.  If *max_workers* is ``None`` or not
-   given, it will default to the number of processors on the machine.
+   given, it will default to :func:`os.process_cpu_count`.
    If *max_workers* is less than or equal to ``0``, then a :exc:`ValueError`
    will be raised.
    On Windows, *max_workers* must be less than or equal to ``61``. If it is not
@@ -301,6 +305,10 @@ to a :class:`ProcessPoolExecutor` will result in deadlock.
       different start method. See the :func:`os.fork` documentation for
       further explanation.
 
+   .. versionchanged:: 3.13
+      *max_workers* uses :func:`os.process_cpu_count` by default, instead of
+      :func:`os.cpu_count`.
+
 .. _processpoolexecutor-example:
 
 ProcessPoolExecutor Example

Doc/library/exceptions.rst

@@ -220,10 +220,16 @@ The following exceptions are the exceptions that are usually raised.
    load a module.  Also raised when the "from list" in ``from ... import``
    has a name that cannot be found.
 
-   The :attr:`name` and :attr:`path` attributes can be set using keyword-only
-   arguments to the constructor. When set they represent the name of the module
-   that was attempted to be imported and the path to any file which triggered
-   the exception, respectively.
+   The optional *name* and *path* keyword-only arguments
+   set the corresponding attributes:
+
+   .. attribute:: name
+
+      The name of the module that was attempted to be imported.
+
+   .. attribute:: path
+
+      The path to any file which triggered the exception.
 
    .. versionchanged:: 3.3
       Added the :attr:`name` and :attr:`path` attributes.

Doc/library/gzip.rst

@@ -268,23 +268,23 @@ Once executed the :mod:`gzip` module keeps the input file(s).
 Command line options
 ^^^^^^^^^^^^^^^^^^^^
 
-.. cmdoption:: file
+.. option:: file
 
    If *file* is not specified, read from :data:`sys.stdin`.
 
-.. cmdoption:: --fast
+.. option:: --fast
 
    Indicates the fastest compression method (less compression).
 
-.. cmdoption:: --best
+.. option:: --best
 
    Indicates the slowest compression method (best compression).
 
-.. cmdoption:: -d, --decompress
+.. option:: -d, --decompress
 
    Decompress the given file.
 
-.. cmdoption:: -h, --help
+.. option:: -h, --help
 
    Show the help message.
 

Doc/library/inspect.rst

@@ -1655,6 +1655,6 @@ By default, accepts the name of a module and prints the source of that
 module. A class or function within the module can be printed instead by
 appended a colon and the qualified name of the target object.
 
-.. cmdoption:: --details
+.. option:: --details
 
    Print information about the specified object rather than the source code