Merge branch 'superopt' into superopt_spec

* superopt: (82 commits)
  pythongh-101517: fix line number propagation in code generated for except* (python#103550)
  pythongh-103780: Use patch instead of mock in asyncio unix events test (python#103782)
  pythongh-102498 Clean up unused variables and imports in the email module  (python#102482)
  pythongh-99184: Bypass instance attribute access in `repr` of `weakref.ref` (python#99244)
  pythongh-99032: datetime docs: Encoding is no longer relevant (python#93365)
  pythongh-94300: Update datetime.strptime documentation (python#95318)
  pythongh-103776: Remove explicit uses of $(SHELL) from Makefile (pythonGH-103778)
  pythongh-87092: fix a few cases of incorrect error handling in compiler (python#103456)
  pythonGH-103727: Avoid advancing tokenizer too far in f-string mode (pythonGH-103775)
  Revert "Add tests for empty range equality (python#103751)" (python#103770)
  pythongh-94518: Port 23-argument `_posixsubprocess.fork_exec` to Argument Clinic (python#94519)
  pythonGH-65022: Fix description of copyreg.pickle function (python#102656)
  pythongh-103323: Get the "Current" Thread State from a Thread-Local Variable (pythongh-103324)
  pythongh-91687: modernize dataclass example typing (python#103773)
  pythongh-103746: Test `types.UnionType` and `Literal` types together (python#103747)
  pythongh-103765: Fix 'Warning: py:class reference target not found: ModuleSpec' (pythonGH-103769)
  pythongh-87452: Improve the Popen.returncode docs
  Removed unnecessary escaping of asterisks (python#103714)
  pythonGH-102973: Slim down Fedora packages in the dev container (python#103283)
  pythongh-103091: Add PyUnstable_Type_AssignVersionTag (python#103095)
  ...

.devcontainer/Dockerfile

@@ -9,8 +9,8 @@ ENV WASMTIME_HOME=/opt/wasmtime
 ENV WASMTIME_VERSION=7.0.0
 ENV WASMTIME_CPU_ARCH=x86_64
 
-RUN dnf -y --nodocs install git clang xz python3-blurb dnf-plugins-core && \
-    dnf -y --nodocs builddep python3 && \
+RUN dnf -y --nodocs --setopt=install_weak_deps=False install /usr/bin/{blurb,clang,curl,git,ln,tar,xz} 'dnf-command(builddep)' && \
+    dnf -y --nodocs --setopt=install_weak_deps=False builddep python3 && \
     dnf -y clean all
 
 RUN mkdir ${WASI_SDK_PATH} && \

.gitattributes

@@ -34,6 +34,7 @@ Lib/test/xmltestdata/*                     noeol
 
 # Shell scripts should have LF even on Windows because of Cygwin
 Lib/venv/scripts/common/activate text eol=lf
+Lib/venv/scripts/posix/* text eol=lf
 
 # CRLF files
 [attr]dos text eol=crlf

.github/workflows/verify-ensurepip-wheels.yml

@@ -1,4 +1,4 @@
-name: Verify bundled pip and setuptools
+name: Verify bundled wheels
 
 on:
   workflow_dispatch:
@@ -29,5 +29,5 @@ jobs:
       - uses: actions/setup-python@v4
         with:
           python-version: '3'
-      - name: Compare checksums of bundled pip and setuptools to ones published on PyPI
+      - name: Compare checksum of bundled wheels to the ones published on PyPI
         run: ./Tools/build/verify_ensurepip_wheels.py

Doc/c-api/type.rst

@@ -232,6 +232,15 @@ Type Objects
 
    .. versionadded:: 3.11
 
+.. c:function:: int PyUnstable_Type_AssignVersionTag(PyTypeObject *type)
+
+   Attempt to assign a version tag to the given type.
+
+   Returns 1 if the type already had a valid version tag or a new one was
+   assigned, or 0 if a new tag could not be assigned.
+
+   .. versionadded:: 3.12
+
 
 Creating Heap-Allocated Types
 .............................

Doc/conf.py

@@ -254,8 +254,31 @@
 # Options for the link checker
 # ----------------------------
 
-# Ignore certain URLs.
-linkcheck_ignore = [r'https://bugs.python.org/(issue)?\d+']
+linkcheck_allowed_redirects = {
+    # bpo-NNNN -> BPO -> GH Issues
+    r'https://bugs.python.org/issue\?@action=redirect&bpo=\d+': 'https://github.com/python/cpython/issues/\d+',
+    # GH-NNNN used to refer to pull requests
+    r'https://github.com/python/cpython/issues/\d+': 'https://github.com/python/cpython/pull/\d+',
+    # :source:`something` linking files in the repository
+    r'https://github.com/python/cpython/tree/.*': 'https://github.com/python/cpython/blob/.*'
+}
+
+linkcheck_anchors_ignore = [
+    # ignore anchors that start with a '/', e.g. Wikipedia media files:
+    # https://en.wikipedia.org/wiki/Walrus#/media/File:Pacific_Walrus_-_Bull_(8247646168).jpg
+    r'\/.*',
+]
+
+linkcheck_ignore = [
+    # The crawler gets "Anchor not found"
+    r'https://developer.apple.com/documentation/.+?#.*',
+    r'https://devguide.python.org.+?/#.*',
+    r'https://github.com.+?#.*',
+    # Robot crawlers not allowed: "403 Client Error: Forbidden"
+    r'https://support.enthought.com/hc/.*',
+    # SSLError CertificateError, even though it is valid
+    r'https://unix.org/version2/whatsnew/lp64_wp.html',
+]
 
 
 # Options for extensions

Doc/distributing/index.rst

@@ -129,14 +129,10 @@ involved in creating and publishing a project:
 * `Uploading the project to the Python Package Index`_
 * `The .pypirc file`_
 
-.. _Project structure: \
-    https://packaging.python.org/tutorials/packaging-projects/#packaging-python-projects
-.. _Building and packaging the project: \
-   https://packaging.python.org/tutorials/packaging-projects/#creating-the-package-files
-.. _Uploading the project to the Python Package Index: \
-   https://packaging.python.org/tutorials/packaging-projects/#uploading-the-distribution-archives
-.. _The .pypirc file: \
-   https://packaging.python.org/specifications/pypirc/
+.. _Project structure: https://packaging.python.org/tutorials/packaging-projects/#packaging-python-projects
+.. _Building and packaging the project: https://packaging.python.org/tutorials/packaging-projects/#creating-the-package-files
+.. _Uploading the project to the Python Package Index: https://packaging.python.org/tutorials/packaging-projects/#uploading-the-distribution-archives
+.. _The .pypirc file: https://packaging.python.org/specifications/pypirc/
 
 
 How do I...?

Doc/extending/newtypes.rst

@@ -337,7 +337,7 @@ Here is an example::
        }
 
        PyErr_Format(PyExc_AttributeError,
-                    "'%.50s' object has no attribute '%.400s'",
+                    "'%.100s' object has no attribute '%.400s'",
                     tp->tp_name, name);
        return NULL;
    }

Doc/faq/library.rst

@@ -780,7 +780,7 @@ socket to :meth:`select.select` to check if it's writable.
    The :mod:`asyncio` module provides a general purpose single-threaded and
    concurrent asynchronous library, which can be used for writing non-blocking
    network code.
-   The third-party `Twisted <https://twistedmatrix.com/trac/>`_ library is
+   The third-party `Twisted <https://twisted.org/>`_ library is
    a popular and feature-rich alternative.
 
 

Doc/howto/descriptor.rst

@@ -1273,37 +1273,86 @@ Using the non-data descriptor protocol, a pure Python version of
 
 .. testcode::
 
+    import functools
+
     class StaticMethod:
         "Emulate PyStaticMethod_Type() in Objects/funcobject.c"
 
         def __init__(self, f):
             self.f = f
+            functools.update_wrapper(self, f)
 
         def __get__(self, obj, objtype=None):
             return self.f
 
         def __call__(self, *args, **kwds):
             return self.f(*args, **kwds)
 
+The :func:`functools.update_wrapper` call adds a ``__wrapped__`` attribute
+that refers to the underlying function.  Also it carries forward
+the attributes necessary to make the wrapper look like the wrapped
+function: ``__name__``, ``__qualname__``, ``__doc__``, and ``__annotations__``.
+
 .. testcode::
     :hide:
 
     class E_sim:
         @StaticMethod
-        def f(x):
-            return x * 10
+        def f(x: int) -> str:
+            "Simple function example"
+            return "!" * x
 
     wrapped_ord = StaticMethod(ord)
 
 .. doctest::
     :hide:
 
     >>> E_sim.f(3)
-    30
+    '!!!'
     >>> E_sim().f(3)
-    30
+    '!!!'
+
+    >>> sm = vars(E_sim)['f']
+    >>> type(sm).__name__
+    'StaticMethod'
+    >>> f = E_sim.f
+    >>> type(f).__name__
+    'function'
+    >>> sm.__name__
+    'f'
+    >>> f.__name__
+    'f'
+    >>> sm.__qualname__
+    'E_sim.f'
+    >>> f.__qualname__
+    'E_sim.f'
+    >>> sm.__doc__
+    'Simple function example'
+    >>> f.__doc__
+    'Simple function example'
+    >>> sm.__annotations__
+    {'x': <class 'int'>, 'return': <class 'str'>}
+    >>> f.__annotations__
+    {'x': <class 'int'>, 'return': <class 'str'>}
+    >>> sm.__module__ == f.__module__
+    True
+    >>> sm(3)
+    '!!!'
+    >>> f(3)
+    '!!!'
+
     >>> wrapped_ord('A')
     65
+    >>> wrapped_ord.__module__ == ord.__module__
+    True
+    >>> wrapped_ord.__wrapped__ == ord
+    True
+    >>> wrapped_ord.__name__ == ord.__name__
+    True
+    >>> wrapped_ord.__qualname__ == ord.__qualname__
+    True
+    >>> wrapped_ord.__doc__ == ord.__doc__
+    True
 
 
 Class methods
@@ -1359,11 +1408,14 @@ Using the non-data descriptor protocol, a pure Python version of
 
 .. testcode::
 
+    import functools
+
     class ClassMethod:
         "Emulate PyClassMethod_Type() in Objects/funcobject.c"
 
         def __init__(self, f):
             self.f = f
+            functools.update_wrapper(self, f)
 
         def __get__(self, obj, cls=None):
             if cls is None:
@@ -1380,8 +1432,9 @@ Using the non-data descriptor protocol, a pure Python version of
     # Verify the emulation works
     class T:
         @ClassMethod
-        def cm(cls, x, y):
-            return (cls, x, y)
+        def cm(cls, x: int, y: str) -> tuple[str, int, str]:
+            "Class method that returns a tuple"
+            return (cls.__name__, x, y)
 
         @ClassMethod
         @property
@@ -1393,17 +1446,40 @@ Using the non-data descriptor protocol, a pure Python version of
     :hide:
 
     >>> T.cm(11, 22)
-    (<class 'T'>, 11, 22)
+    ('T', 11, 22)
 
     # Also call it from an instance
     >>> t = T()
     >>> t.cm(11, 22)
-    (<class 'T'>, 11, 22)
+    ('T', 11, 22)
 
     # Check the alternate path for chained descriptors
     >>> T.__doc__
     "A doc for 'T'"
 
+    # Verify that T uses our emulation
+    >>> type(vars(T)['cm']).__name__
+    'ClassMethod'
+
+    # Verify that update_wrapper() correctly copied attributes
+    >>> T.cm.__name__
+    'cm'
+    >>> T.cm.__qualname__
+    'T.cm'
+    >>> T.cm.__doc__
+    'Class method that returns a tuple'
+    >>> T.cm.__annotations__
+    {'x': <class 'int'>, 'y': <class 'str'>, 'return': tuple[str, int, str]}
+
+    # Verify that __wrapped__ was added and works correctly
+    >>> f = vars(T)['cm'].__wrapped__
+    >>> type(f).__name__
+    'function'
+    >>> f.__name__
+    'cm'
+    >>> f(T, 11, 22)
+    ('T', 11, 22)
+
 
 The code path for ``hasattr(type(self.f), '__get__')`` was added in
 Python 3.9 and makes it possible for :func:`classmethod` to support
@@ -1423,6 +1499,12 @@ chained together.  In Python 3.11, this functionality was deprecated.
     >>> G.__doc__
     "A doc for 'G'"
 
+The :func:`functools.update_wrapper` call in ``ClassMethod`` adds a
+``__wrapped__`` attribute that refers to the underlying function.  Also
+it carries forward the attributes necessary to make the wrapper look
+like the wrapped function: ``__name__``, ``__qualname__``, ``__doc__``,
+and ``__annotations__``.
+
 
 Member objects and __slots__
 ----------------------------

Doc/howto/enum.rst

@@ -36,8 +36,10 @@ inherits from :class:`Enum` itself.
 
 .. note:: Case of Enum Members
 
-    Because Enums are used to represent constants we recommend using
-    UPPER_CASE names for members, and will be using that style in our examples.
+    Because Enums are used to represent constants, and to help avoid issues
+    with name clashes between mixin-class methods/attributes and enum names,
+    we strongly recommend using UPPER_CASE names for members, and will be using
+    that style in our examples.
 
 Depending on the nature of the enum a member's value may or may not be
 important, but either way that value can be used to get the corresponding
@@ -490,6 +492,10 @@ the :meth:`~Enum.__repr__` omits the inherited class' name.  For example::
 Use the :func:`!dataclass` argument ``repr=False``
 to use the standard :func:`repr`.
 
+.. versionchanged:: 3.12
+   Only the dataclass fields are shown in the value area, not the dataclass'
+   name.
+
 
 Pickling
 --------
@@ -992,7 +998,9 @@ but remain normal attributes.
 Enum members are instances of their enum class, and are normally accessed as
 ``EnumClass.member``.  In certain situations, such as writing custom enum
 behavior, being able to access one member directly from another is useful,
-and is supported.
+and is supported; however, in order to avoid name clashes between member names
+and attributes/methods from mixed-in classes, upper-case names are strongly
+recommended.
 
 .. versionchanged:: 3.5
 

Doc/howto/functional.rst

@@ -1208,8 +1208,8 @@ General
 -------
 
 **Structure and Interpretation of Computer Programs**, by Harold Abelson and
-Gerald Jay Sussman with Julie Sussman.  Full text at
-https://mitpress.mit.edu/sicp/.  In this classic textbook of computer science,
+Gerald Jay Sussman with Julie Sussman.  The book can be found at
+https://mitpress.mit.edu/sicp.  In this classic textbook of computer science,
 chapters 2 and 3 discuss the use of sequences and streams to organize the data
 flow inside a program.  The book uses Scheme for its examples, but many of the
 design approaches described in these chapters are applicable to functional-style

Doc/howto/urllib2.rst

@@ -86,7 +86,7 @@ response::
 
     import urllib.request
 
-    req = urllib.request.Request('http://www.voidspace.org.uk')
+    req = urllib.request.Request('http://python.org/')
     with urllib.request.urlopen(req) as response:
        the_page = response.read()
 
@@ -458,7 +458,7 @@ To illustrate creating and installing a handler we will use the
 ``HTTPBasicAuthHandler``. For a more detailed discussion of this subject --
 including an explanation of how Basic Authentication works - see the `Basic
 Authentication Tutorial
-<http://www.voidspace.org.uk/python/articles/authentication.shtml>`_.
+<https://web.archive.org/web/20201215133350/http://www.voidspace.org.uk/python/articles/authentication.shtml>`__.
 
 When authentication is required, the server sends a header (as well as the 401
 error code) requesting authentication.  This specifies the authentication scheme

Doc/library/argparse.rst

@@ -67,7 +67,7 @@ default_               Default value used when an argument is not provided
 dest_                  Specify the attribute name used in the result namespace
 help_                  Help message for an argument
 metavar_               Alternate display name for the argument as shown in help
-nargs_                 Number of times the argument can be used                    :class:`int`, ``'?'``, ``'*'``, ``'+'``, or ``argparse.REMAINDER``
+nargs_                 Number of times the argument can be used                    :class:`int`, ``'?'``, ``'*'``, or ``'+'``
 required_              Indicate whether an argument is required or optional        ``True`` or ``False``
 type_                  Automatically convert an argument to the given type         :class:`int`, :class:`float`, ``argparse.FileType('w')``, or callable function
 ====================== =========================================================== ==========================================================================================================================
@@ -2218,7 +2218,7 @@ support this parsing style.
 
 These parsers do not support all the argparse features, and will raise
 exceptions if unsupported features are used.  In particular, subparsers,
-``argparse.REMAINDER``, and mutually exclusive groups that include both
+and mutually exclusive groups that include both
 optionals and positionals are not supported.
 
 The following example shows the difference between