Merge branch 'main' into remove-import-trampoline

Doc/c-api/list.rst

@@ -86,6 +86,10 @@ List Objects
    Macro form of :c:func:`PyList_SetItem` without error checking. This is
    normally only used to fill in new lists where there is no previous content.
 
+   Bounds checking is performed as an assertion if Python is built in
+   :ref:`debug mode <debug-build>` or :option:`with assertions
+   <--with-assertions>`.
+
    .. note::
 
       This macro "steals" a reference to *item*, and, unlike

Doc/c-api/tuple.rst

@@ -89,6 +89,9 @@ Tuple Objects
    Like :c:func:`PyTuple_SetItem`, but does no error checking, and should *only* be
    used to fill in brand new tuples.
 
+   Bounds checking is performed as an assertion if Python is built in
+   :ref:`debug mode <debug-build>` or :option:`with assertions <--with-assertions>`.
+
    .. note::
 
       This function "steals" a reference to *o*, and, unlike
@@ -194,12 +197,17 @@ type.
 .. c:function:: PyObject* PyStructSequence_GetItem(PyObject *p, Py_ssize_t pos)
 
    Return the object at position *pos* in the struct sequence pointed to by *p*.
-   No bounds checking is performed.
+
+   Bounds checking is performed as an assertion if Python is built in
+   :ref:`debug mode <debug-build>` or :option:`with assertions <--with-assertions>`.
 
 
 .. c:function:: PyObject* PyStructSequence_GET_ITEM(PyObject *p, Py_ssize_t pos)
 
-   Macro equivalent of :c:func:`PyStructSequence_GetItem`.
+   Alias to :c:func:`PyStructSequence_GetItem`.
+
+   .. versionchanged:: 3.13
+      Now implemented as an alias to :c:func:`PyStructSequence_GetItem`.
 
 
 .. c:function:: void PyStructSequence_SetItem(PyObject *p, Py_ssize_t pos, PyObject *o)
@@ -208,16 +216,17 @@ type.
    :c:func:`PyTuple_SET_ITEM`, this should only be used to fill in brand new
    instances.
 
+   Bounds checking is performed as an assertion if Python is built in
+   :ref:`debug mode <debug-build>` or :option:`with assertions <--with-assertions>`.
+
    .. note::
 
       This function "steals" a reference to *o*.
 
 
 .. c:function:: void PyStructSequence_SET_ITEM(PyObject *p, Py_ssize_t *pos, PyObject *o)
 
-   Similar to :c:func:`PyStructSequence_SetItem`, but implemented as a static
-   inlined function.
+   Alias to :c:func:`PyStructSequence_SetItem`.
 
-   .. note::
-
-      This function "steals" a reference to *o*.
+   .. versionchanged:: 3.13
+      Now implemented as an alias to :c:func:`PyStructSequence_SetItem`.

Doc/glossary.rst

@@ -83,8 +83,8 @@ Glossary
 
    asynchronous context manager
       An object which controls the environment seen in an
-      :keyword:`async with` statement by defining :meth:`__aenter__` and
-      :meth:`__aexit__` methods.  Introduced by :pep:`492`.
+      :keyword:`async with` statement by defining :meth:`~object.__aenter__` and
+      :meth:`~object.__aexit__` methods.  Introduced by :pep:`492`.
 
    asynchronous generator
       A function which returns an :term:`asynchronous generator iterator`.  It
@@ -104,26 +104,26 @@ Glossary
       An object created by a :term:`asynchronous generator` function.
 
       This is an :term:`asynchronous iterator` which when called using the
-      :meth:`__anext__` method returns an awaitable object which will execute
+      :meth:`~object.__anext__` method returns an awaitable object which will execute
       the body of the asynchronous generator function until the next
       :keyword:`yield` expression.
 
       Each :keyword:`yield` temporarily suspends processing, remembering the
       location execution state (including local variables and pending
       try-statements).  When the *asynchronous generator iterator* effectively
-      resumes with another awaitable returned by :meth:`__anext__`, it
+      resumes with another awaitable returned by :meth:`~object.__anext__`, it
       picks up where it left off.  See :pep:`492` and :pep:`525`.
 
    asynchronous iterable
       An object, that can be used in an :keyword:`async for` statement.
       Must return an :term:`asynchronous iterator` from its
-      :meth:`__aiter__` method.  Introduced by :pep:`492`.
+      :meth:`~object.__aiter__` method.  Introduced by :pep:`492`.
 
    asynchronous iterator
-      An object that implements the :meth:`__aiter__` and :meth:`__anext__`
-      methods.  ``__anext__`` must return an :term:`awaitable` object.
+      An object that implements the :meth:`~object.__aiter__` and :meth:`~object.__anext__`
+      methods.  :meth:`~object.__anext__` must return an :term:`awaitable` object.
       :keyword:`async for` resolves the awaitables returned by an asynchronous
-      iterator's :meth:`__anext__` method until it raises a
+      iterator's :meth:`~object.__anext__` method until it raises a
       :exc:`StopAsyncIteration` exception.  Introduced by :pep:`492`.
 
    attribute
@@ -140,7 +140,7 @@ Glossary
 
    awaitable
       An object that can be used in an :keyword:`await` expression.  Can be
-      a :term:`coroutine` or an object with an :meth:`__await__` method.
+      a :term:`coroutine` or an object with an :meth:`~object.__await__` method.
       See also :pep:`492`.
 
    BDFL

Doc/library/asyncio-eventloop.rst

@@ -895,6 +895,9 @@ TLS Upgrade
    object only because the coder caches *protocol*-side data and sporadically
    exchanges extra TLS session packets with *transport*.
 
+   In some situations (e.g. when the passed transport is already closing) this
+   may return ``None``.
+
    Parameters:
 
    * *transport* and *protocol* instances that methods like

Doc/library/asyncio-task.rst

@@ -654,16 +654,16 @@ Timeouts
     If ``long_running_task`` takes more than 10 seconds to complete,
     the context manager will cancel the current task and handle
     the resulting :exc:`asyncio.CancelledError` internally, transforming it
-    into an :exc:`asyncio.TimeoutError` which can be caught and handled.
+    into a :exc:`TimeoutError` which can be caught and handled.
 
     .. note::
 
       The :func:`asyncio.timeout` context manager is what transforms
-      the :exc:`asyncio.CancelledError` into an :exc:`asyncio.TimeoutError`,
-      which means the :exc:`asyncio.TimeoutError` can only be caught
+      the :exc:`asyncio.CancelledError` into a :exc:`TimeoutError`,
+      which means the :exc:`TimeoutError` can only be caught
       *outside* of the context manager.
 
-    Example of catching :exc:`asyncio.TimeoutError`::
+    Example of catching :exc:`TimeoutError`::
 
         async def main():
             try:

Doc/library/exceptions.rst

@@ -450,7 +450,7 @@ The following exceptions are the exceptions that are usually raised.
 
 .. exception:: StopAsyncIteration
 
-   Must be raised by :meth:`__anext__` method of an
+   Must be raised by :meth:`~object.__anext__` method of an
    :term:`asynchronous iterator` object to stop the iteration.
 
    .. versionadded:: 3.5

Doc/library/optparse.rst

@@ -42,8 +42,8 @@ on the command-line, for example::
    <yourscript> --file=outfile -q
 
 As it parses the command line, :mod:`optparse` sets attributes of the
-``options`` object returned by :meth:`parse_args` based on user-supplied
-command-line values.  When :meth:`parse_args` returns from parsing this command
+``options`` object returned by :meth:`~OptionParser.parse_args` based on user-supplied
+command-line values.  When :meth:`~OptionParser.parse_args` returns from parsing this command
 line, ``options.filename`` will be ``"outfile"`` and ``options.verbose`` will be
 ``False``.  :mod:`optparse` supports both long and short options, allows short
 options to be merged together, and allows options to be associated with their
@@ -285,10 +285,10 @@ program's command line::
 
    (options, args) = parser.parse_args()
 
-(If you like, you can pass a custom argument list to :meth:`parse_args`, but
+(If you like, you can pass a custom argument list to :meth:`~OptionParser.parse_args`, but
 that's rarely necessary: by default it uses ``sys.argv[1:]``.)
 
-:meth:`parse_args` returns two values:
+:meth:`~OptionParser.parse_args` returns two values:
 
 * ``options``, an object containing values for all of your options---e.g. if
   ``--file`` takes a single string argument, then ``options.file`` will be the
@@ -339,7 +339,7 @@ Now let's make up a fake command line and ask :mod:`optparse` to parse it::
 
 When :mod:`optparse` sees the option string ``-f``, it consumes the next
 argument, ``foo.txt``, and stores it in ``options.filename``.  So, after this
-call to :meth:`parse_args`, ``options.filename`` is ``"foo.txt"``.
+call to :meth:`~OptionParser.parse_args`, ``options.filename`` is ``"foo.txt"``.
 
 Some other option types supported by :mod:`optparse` are ``int`` and ``float``.
 Here's an option that expects an integer argument::
@@ -453,7 +453,8 @@ Again, the default value for ``verbose`` will be ``True``: the last default
 value supplied for any particular destination is the one that counts.
 
 A clearer way to specify default values is the :meth:`set_defaults` method of
-OptionParser, which you can call at any time before calling :meth:`parse_args`::
+OptionParser, which you can call at any time before calling
+:meth:`~OptionParser.parse_args`::
 
    parser.set_defaults(verbose=True)
    parser.add_option(...)
@@ -1338,35 +1339,37 @@ Parsing arguments
 ^^^^^^^^^^^^^^^^^
 
 The whole point of creating and populating an OptionParser is to call its
-:meth:`parse_args` method::
+:meth:`~OptionParser.parse_args` method.
 
-   (options, args) = parser.parse_args(args=None, values=None)
+.. method:: OptionParser.parse_args(args=None, values=None)
 
-where the input parameters are
+   Parse the command-line options found in *args*.
 
-``args``
-   the list of arguments to process (default: ``sys.argv[1:]``)
+   The input parameters are
 
-``values``
-   an :class:`optparse.Values` object to store option arguments in (default: a
-   new instance of :class:`Values`) -- if you give an existing object, the
-   option defaults will not be initialized on it
+   ``args``
+      the list of arguments to process (default: ``sys.argv[1:]``)
 
-and the return values are
+   ``values``
+      an :class:`Values` object to store option arguments in (default: a
+      new instance of :class:`Values`) -- if you give an existing object, the
+      option defaults will not be initialized on it
 
-``options``
-   the same object that was passed in as ``values``, or the optparse.Values
-   instance created by :mod:`optparse`
+   and the return value is a pair ``(options, args)`` where
 
-``args``
-   the leftover positional arguments after all options have been processed
+   ``options``
+      the same object that was passed in as *values*, or the ``optparse.Values``
+      instance created by :mod:`optparse`
+
+   ``args``
+      the leftover positional arguments after all options have been processed
 
 The most common usage is to supply neither keyword argument.  If you supply
 ``values``, it will be modified with repeated :func:`setattr` calls (roughly one
 for every option argument stored to an option destination) and returned by
-:meth:`parse_args`.
+:meth:`~OptionParser.parse_args`.
 
-If :meth:`parse_args` encounters any errors in the argument list, it calls the
+If :meth:`~OptionParser.parse_args` encounters any errors in the argument list, it calls the
 OptionParser's :meth:`error` method with an appropriate end-user error message.
 This ultimately terminates your process with an exit status of 2 (the
 traditional Unix exit status for command-line errors).
@@ -1661,7 +1664,7 @@ where
       the current list of leftover arguments, ie. arguments that have been
       consumed but are neither options nor option arguments. Feel free to modify
       ``parser.largs``, e.g. by adding more arguments to it.  (This list will
-      become ``args``, the second return value of :meth:`parse_args`.)
+      become ``args``, the second return value of :meth:`~OptionParser.parse_args`.)
 
    ``parser.rargs``
       the current list of remaining arguments, ie. with ``opt_str`` and

Doc/whatsnew/3.13.rst

@@ -441,6 +441,12 @@ New Features
   ``NULL`` if the referent is no longer live.
   (Contributed by Victor Stinner in :gh:`105927`.)
 
+* If Python is built in :ref:`debug mode <debug-build>` or :option:`with
+  assertions <--with-assertions>`, :c:func:`PyTuple_SET_ITEM` and
+  :c:func:`PyList_SET_ITEM` now check the index argument with an assertion.
+  If the assertion fails, make sure that the size is set before.
+  (Contributed by Victor Stinner in :gh:`106168`.)
+
 Porting to Python 3.13
 ----------------------
 

Include/cpython/listobject.h

@@ -41,6 +41,8 @@ static inline Py_ssize_t PyList_GET_SIZE(PyObject *op) {
 static inline void
 PyList_SET_ITEM(PyObject *op, Py_ssize_t index, PyObject *value) {
     PyListObject *list = _PyList_CAST(op);
+    assert(0 <= index);
+    assert(index < Py_SIZE(list));
     list->ob_item[index] = value;
 }
 #define PyList_SET_ITEM(op, index, value) \

Include/cpython/tupleobject.h

@@ -31,6 +31,8 @@ static inline Py_ssize_t PyTuple_GET_SIZE(PyObject *op) {
 static inline void
 PyTuple_SET_ITEM(PyObject *op, Py_ssize_t index, PyObject *value) {
     PyTupleObject *tuple = _PyTuple_CAST(op);
+    assert(0 <= index);
+    assert(index < Py_SIZE(tuple));
     tuple->ob_item[index] = value;
 }
 #define PyTuple_SET_ITEM(op, index, value) \

Include/internal/pycore_list.h

@@ -49,8 +49,8 @@ _PyList_AppendTakeRef(PyListObject *self, PyObject *newitem)
     Py_ssize_t allocated = self->allocated;
     assert((size_t)len + 1 < PY_SSIZE_T_MAX);
     if (allocated > len) {
-        PyList_SET_ITEM(self, len, newitem);
         Py_SET_SIZE(self, len + 1);
+        PyList_SET_ITEM(self, len, newitem);
         return 0;
     }
     return _PyList_AppendTakeRefListResize(self, newitem);

Include/structseq.h

@@ -31,18 +31,15 @@ PyAPI_FUNC(PyTypeObject*) PyStructSequence_NewType(PyStructSequence_Desc *desc);
 
 PyAPI_FUNC(PyObject *) PyStructSequence_New(PyTypeObject* type);
 
+PyAPI_FUNC(void) PyStructSequence_SetItem(PyObject*, Py_ssize_t, PyObject*);
+PyAPI_FUNC(PyObject*) PyStructSequence_GetItem(PyObject*, Py_ssize_t);
+
 #ifndef Py_LIMITED_API
 typedef PyTupleObject PyStructSequence;
-
-/* Macro, *only* to be used to fill in brand new objects */
-#define PyStructSequence_SET_ITEM(op, i, v) PyTuple_SET_ITEM((op), (i), (v))
-
-#define PyStructSequence_GET_ITEM(op, i) PyTuple_GET_ITEM((op), (i))
+#define PyStructSequence_SET_ITEM PyStructSequence_SetItem
+#define PyStructSequence_GET_ITEM PyStructSequence_GetItem
 #endif
 
-PyAPI_FUNC(void) PyStructSequence_SetItem(PyObject*, Py_ssize_t, PyObject*);
-PyAPI_FUNC(PyObject*) PyStructSequence_GetItem(PyObject*, Py_ssize_t);
-
 #ifdef __cplusplus
 }
 #endif

Lib/poplib.py

@@ -451,7 +451,6 @@ def stls(self, context=None):
     __all__.append("POP3_SSL")
 
 if __name__ == "__main__":
-    import sys
     a = POP3(sys.argv[1])
     print(a.getwelcome())
     a.user(sys.argv[2])

Lib/test/libregrtest/runtest_mp.py

@@ -277,6 +277,7 @@ def _runtest(self, test_name: str) -> MultiprocessResult:
             encoding = locale.getencoding()
         else:
             encoding = sys.stdout.encoding
+
         # gh-94026: Write stdout+stderr to a tempfile as workaround for
         # non-blocking pipes on Emscripten with NodeJS.
         with tempfile.TemporaryFile('w+', encoding=encoding) as stdout_fh:
@@ -298,7 +299,14 @@ def _runtest(self, test_name: str) -> MultiprocessResult:
                 retcode = self._run_process(test_name, None, stdout_fh)
                 tmp_files = ()
             stdout_fh.seek(0)
-            stdout = stdout_fh.read().strip()
+
+            try:
+                stdout = stdout_fh.read().strip()
+            except Exception as exc:
+                # gh-101634: Catch UnicodeDecodeError if stdout cannot be
+                # decoded from encoding
+                err_msg = f"Cannot read process stdout: {exc}"
+                return self.mp_result_error(ChildError(test_name), '', err_msg)
 
         if retcode is None:
             return self.mp_result_error(Timeout(test_name), stdout)
@@ -481,6 +489,8 @@ def _process_result(self, item: QueueOutput) -> bool:
             # Thread got an exception
             format_exc = item[1]
             print_warning(f"regrtest worker thread failed: {format_exc}")
+            result = ChildError("<regrtest worker>")
+            self.regrtest.accumulate_result(result)
             return True
 
         self.test_index += 1

Lib/test/test_array.py

@@ -100,6 +100,7 @@ def test_empty(self):
 class ArrayReconstructorTest(unittest.TestCase):
 
     def setUp(self):
+        self.enterContext(warnings.catch_warnings())
         warnings.filterwarnings(
             "ignore",
             message="The 'u' type code is deprecated and "

Lib/test/test_buffer.py

@@ -4722,7 +4722,7 @@ def __buffer__(self, flags):
         with self.assertRaises(ValueError):
             c.buffer.tobytes()
 
-    def test_multiple_inheritance_buffer_last(self):
+    def test_multiple_inheritance_buffer_last_raising(self):
         class A:
             def __buffer__(self, flags):
                 raise RuntimeError("should not be called")