[pull] main from python:main

Doc/library/argparse.rst

@@ -637,25 +637,22 @@ are set.
 
 .. versionadded:: 3.14
 
-To highlight inline code in your description or epilog text, you can use
-backticks::
+To highlight inline code in your description, epilog, or argument ``help``
+text, you can use single or double backticks::
 
    >>> parser = argparse.ArgumentParser(
    ...     formatter_class=argparse.RawDescriptionHelpFormatter,
+   ...     description='Run ``python -m myapp`` to start.',
    ...     epilog='''Examples:
    ...   `python -m myapp --verbose`
-   ...   `python -m myapp --config settings.json`
+   ...   ``python -m myapp --config settings.json``
    ... ''')
+   >>> parser.add_argument('--foo', help='set the `foo` value')
 
 When colors are enabled, the text inside backticks will be displayed in a
 distinct color to help examples stand out. When colors are disabled, backticks
 are preserved as-is, which is readable in plain text.
 
-.. note::
-
-   Backtick markup only applies to description and epilog text. It does not
-   apply to individual argument ``help`` strings.
-
 .. versionadded:: 3.15
 
 

Doc/library/urllib.robotparser.rst

@@ -18,7 +18,7 @@
 This module provides a single class, :class:`RobotFileParser`, which answers
 questions about whether or not a particular user agent can fetch a URL on the
 website that published the :file:`robots.txt` file.  For more details on the
-structure of :file:`robots.txt` files, see http://www.robotstxt.org/orig.html.
+structure of :file:`robots.txt` files, see :rfc:`9309`.
 
 
 .. class:: RobotFileParser(url='')

Doc/whatsnew/3.15.rst

@@ -492,6 +492,47 @@ Improved error messages
                ^^^^^^^^^^^^^^
      AttributeError: 'Container' object has no attribute 'area'. Did you mean '.inner.area' instead of '.area'?
 
+* When an :exc:`AttributeError` on a builtin type has no close match via
+  Levenshtein distance, the error message now checks a static table of common
+  method names from other languages (JavaScript, Java, Ruby, C#) and suggests
+  the Python equivalent:
+
+  .. doctest::
+
+     >>> [1, 2, 3].push(4)  # doctest: +ELLIPSIS
+     Traceback (most recent call last):
+     ...
+     AttributeError: 'list' object has no attribute 'push'. Did you mean '.append'?
+
+     >>> 'hello'.toUpperCase()  # doctest: +ELLIPSIS
+     Traceback (most recent call last):
+     ...
+     AttributeError: 'str' object has no attribute 'toUpperCase'. Did you mean '.upper'?
+
+  When the Python equivalent is a language construct rather than a method,
+  the hint describes the construct directly:
+
+  .. doctest::
+
+     >>> {}.put("a", 1)  # doctest: +ELLIPSIS
+     Traceback (most recent call last):
+     ...
+     AttributeError: 'dict' object has no attribute 'put'. Use d[k] = v.
+
+  When a mutable method is called on an immutable type, the hint suggests
+  the mutable counterpart:
+
+  .. doctest::
+
+     >>> (1, 2, 3).append(4)  # doctest: +ELLIPSIS
+     Traceback (most recent call last):
+     ...
+     AttributeError: 'tuple' object has no attribute 'append'. Did you mean to use a 'list' object?
+
+  These hints also work for subclasses of builtin types.
+
+  (Contributed by Matt Van Horn in :gh:`146406`.)
+
 * The interpreter now tries to provide a suggestion when
   :func:`delattr` fails due to a missing attribute.
   When an attribute name that closely resembles an existing attribute is used,
@@ -715,10 +756,14 @@ argparse
   default to ``True``. This enables suggestions for mistyped arguments by default.
   (Contributed by Jakob Schluse in :gh:`140450`.)
 
-* Added backtick markup support in description and epilog text to highlight
-  inline code when color output is enabled.
+* Added backtick markup support in :class:`~argparse.ArgumentParser` description
+  and epilog text to highlight inline code when color output is enabled.
   (Contributed by Savannah Ostrowski in :gh:`142390`.)
 
+* Extended backtick markup to argument ``help`` text and added support for
+  double backticks (RST inline-literal style).
+  (Contributed by Hugo van Kemenade in :gh:`149375`.)
+
 
 array
 -----
@@ -1045,7 +1090,7 @@ mimetypes
   (Contributed by Benedikt Johannes, Charlie Lin, Foolbar, Gil Forcada and
   John Franey
   in :gh:`144217`, :gh:`145720`, :gh:`140937`, :gh:`139959`, :gh:`145698`,
-  :gh:`145718` and :gh:`144213`.)
+  :gh:`145718`, :gh:`145918`, and :gh:`144213`.)
 * Rename ``application/x-texinfo`` to ``application/texinfo``.
   (Contributed by Charlie Lin in :gh:`140165`.)
 * Changed the MIME type for ``.ai`` files to ``application/pdf``.

Include/internal/pycore_debug_offsets.h

@@ -158,8 +158,16 @@ typedef struct _Py_DebugOffsets {
         uint64_t tp_name;
         uint64_t tp_repr;
         uint64_t tp_flags;
+        uint64_t tp_basicsize;
+        uint64_t tp_dictoffset;
     } type_object;
 
+    // PyHeapTypeObject offset;
+    struct _heap_type_object {
+        uint64_t size;
+        uint64_t ht_cached_keys;
+    } heap_type_object;
+
     // PyTuple object offset;
     struct _tuple_object {
         uint64_t size;
@@ -330,6 +338,12 @@ typedef struct _Py_DebugOffsets {
         .tp_name = offsetof(PyTypeObject, tp_name), \
         .tp_repr = offsetof(PyTypeObject, tp_repr), \
         .tp_flags = offsetof(PyTypeObject, tp_flags), \
+        .tp_basicsize = offsetof(PyTypeObject, tp_basicsize), \
+        .tp_dictoffset = offsetof(PyTypeObject, tp_dictoffset), \
+    }, \
+    .heap_type_object = { \
+        .size = sizeof(PyHeapTypeObject), \
+        .ht_cached_keys = offsetof(PyHeapTypeObject, ht_cached_keys), \
     }, \
     .tuple_object = { \
         .size = sizeof(PyTupleObject), \

Include/internal/pycore_gc.h

@@ -223,12 +223,14 @@ static inline void _PyObject_GC_TRACK(
                           "object is in generation which is garbage collected",
                           filename, lineno, __func__);
 
-    PyGC_Head *generation0 = _PyInterpreterState_GET()->gc.generation0;
+    struct _gc_runtime_state *gcstate = &_PyInterpreterState_GET()->gc;
+    PyGC_Head *generation0 = gcstate->generation0;
     PyGC_Head *last = (PyGC_Head*)(generation0->_gc_prev);
     _PyGCHead_SET_NEXT(last, gc);
     _PyGCHead_SET_PREV(gc, last);
     _PyGCHead_SET_NEXT(gc, generation0);
     generation0->_gc_prev = (uintptr_t)gc;
+    gcstate->heap_size++;
 #endif
 }
 
@@ -263,6 +265,8 @@ static inline void _PyObject_GC_UNTRACK(
     _PyGCHead_SET_PREV(next, prev);
     gc->_gc_next = 0;
     gc->_gc_prev &= _PyGC_PREV_MASK_FINALIZED;
+    struct _gc_runtime_state *gcstate = &_PyInterpreterState_GET()->gc;
+    gcstate->heap_size--;
 #endif
 }
 

Include/internal/pycore_interp_structs.h

@@ -191,6 +191,8 @@ struct gc_generation_stats {
     Py_ssize_t candidates;
     // Total duration of the collection in seconds:
     double duration;
+    /* heap_size on the start of the collection */
+    Py_ssize_t heap_size;
 };
 
 #ifdef Py_GIL_DISABLED
@@ -226,7 +228,6 @@ struct _gc_runtime_state {
     /* linked lists of container objects */
 #ifndef Py_GIL_DISABLED
     struct gc_generation generations[NUM_GENERATIONS];
-    PyGC_Head *generation0;
 #else
     struct gc_generation young;
     struct gc_generation old[2];
@@ -244,6 +245,9 @@ struct _gc_runtime_state {
     /* a list of callbacks to be invoked when collection is performed */
     PyObject *callbacks;
 
+    /* The number of live objects. */
+    Py_ssize_t heap_size;
+
     /* This is the number of objects that survived the last full
        collection. It approximates the number of long lived objects
        tracked by the GC.
@@ -269,6 +273,8 @@ struct _gc_runtime_state {
 
     /* Mutex held for gc_should_collect_mem_usage(). */
     PyMutex mutex;
+#else
+    PyGC_Head *generation0;
 #endif
 };
 
@@ -278,7 +284,8 @@ struct _gc_runtime_state {
         { .threshold = 2000, }, \
         { .threshold = 10, }, \
         { .threshold = 10, }, \
-    },
+    }, \
+    .heap_size = 0,
 #else
 #define GC_GENERATION_INIT \
     .young = { .threshold = 2000, }, \

Lib/argparse.py

@@ -529,16 +529,16 @@ def _apply_text_markup(self, text):
         """Apply color markup to text.
 
         Supported markup:
-          `...` - inline code (rendered with prog_extra color)
+          `...` or ``...`` - inline code (rendered with prog_extra color)
 
         When colors are disabled, backticks are preserved as-is.
         """
         t = self._theme
         if not t.reset:
             return text
         text = _re.sub(
-            r'`([^`]+)`',
-            rf'{t.prog_extra}\1{t.reset}',
+            r'(`{1,2})([^`]+)\1',
+            rf'{t.prog_extra}\2{t.reset}',
             text,
         )
         return text
@@ -682,7 +682,7 @@ def _format_args(self, action, default_metavar):
     def _expand_help(self, action):
         help_string = self._get_help_string(action)
         if '%' not in help_string:
-            return help_string
+            return self._apply_text_markup(help_string)
         params = dict(vars(action), prog=self._prog)
         for name in list(params):
             value = params[name]
@@ -726,7 +726,9 @@ def colorize(match):
             # bare %s etc. - format with full params dict, no colorization
             return spec % params
 
-        return _re.sub(fmt_spec, colorize, help_string, flags=_re.VERBOSE)
+        return self._apply_text_markup(
+            _re.sub(fmt_spec, colorize, help_string, flags=_re.VERBOSE)
+        )
 
     def _iter_indented_subactions(self, action):
         try:
@@ -2758,7 +2760,7 @@ def _check_value(self, action, value):
 
         if value not in choices:
             args = {'value': str(value),
-                    'choices': ', '.join(map(str, action.choices))}
+                    'choices': ', '.join(repr(str(choice)) for choice in action.choices)}
             msg = _('invalid choice: %(value)r (choose from %(choices)s)')
 
             if self.suggest_on_error and isinstance(value, str):

Lib/ast.py

@@ -21,7 +21,6 @@
 :license: Python License.
 """
 from _ast import *
-lazy from _colorize import can_colorize, get_theme
 
 
 def parse(source, filename='<unknown>', mode='exec', *,
@@ -142,6 +141,8 @@ def dump(
     If show_empty is False, then empty lists and fields that are None
     will be omitted from the output for better readability.
     """
+    from _colorize import get_theme
+
     t = get_theme(force_color=color, force_no_color=not color).ast
 
     def _format(node, level=0):
@@ -665,7 +666,7 @@ def main(args=None):
 
     parser = argparse.ArgumentParser(formatter_class=argparse.ArgumentDefaultsHelpFormatter)
     parser.add_argument('infile', nargs='?', default='-',
-                        help='the file to parse; defaults to stdin')
+                        help='the file to parse; defaults to `stdin`')
     parser.add_argument('-m', '--mode', default='exec',
                         choices=('exec', 'single', 'eval', 'func_type'),
                         help='specify what kind of code must be parsed')
@@ -678,8 +679,8 @@ def main(args=None):
                         help='indentation of nodes (number of spaces)')
     parser.add_argument('--feature-version',
                         type=str, default=None, metavar='VERSION',
-                        help='Python version in the format 3.x '
-                             '(for example, 3.10)')
+                        help='Python version in the format `3.x` '
+                             '(for example, `3.10`)')
     parser.add_argument('-O', '--optimize',
                         type=int, default=-1, metavar='LEVEL',
                         help='optimization level for parser')
@@ -708,6 +709,7 @@ def main(args=None):
 
     tree = parse(source, name, args.mode, type_comments=args.no_type_comments,
                  feature_version=feature_version, optimize=args.optimize)
+    from _colorize import can_colorize
     print(dump(tree, include_attributes=args.include_attributes,
                color=can_colorize(file=sys.stdout),
                indent=args.indent, show_empty=args.show_empty))

Lib/asyncio/__main__.py

@@ -159,7 +159,6 @@ def interrupt(self) -> None:
     parser = argparse.ArgumentParser(
         prog="python3 -m asyncio",
         description="Interactive asyncio shell and CLI tools",
-        color=True,
     )
     subparsers = parser.add_subparsers(help="sub-commands", dest="command")
     ps = subparsers.add_parser(

Lib/calendar.py

@@ -922,7 +922,7 @@ def main(args=None):
         "-t", "--type",
         default="text",
         choices=("text", "html"),
-        help="output type (text or html)"
+        help="output type (`text` or `html`)"
     )
     parser.add_argument(
         "-f", "--first-weekday",

Lib/code.py

@@ -385,7 +385,7 @@ def interact(banner=None, readfunc=None, local=None, exitmsg=None, local_exit=Fa
 if __name__ == "__main__":
     import argparse
 
-    parser = argparse.ArgumentParser(color=True)
+    parser = argparse.ArgumentParser()
     parser.add_argument('-q', action='store_true',
                        help="don't print version and copyright messages")
     args = parser.parse_args()

Lib/compileall.py

@@ -326,7 +326,6 @@ def main():
 
     parser = argparse.ArgumentParser(
         description='Utilities to support installing Python libraries.',
-        color=True,
     )
     parser.add_argument('-l', action='store_const', const=0,
                         default=None, dest='maxlevels',
@@ -338,10 +337,10 @@ def main():
     parser.add_argument('-f', action='store_true', dest='force',
                         help='force rebuild even if timestamps are up to date')
     parser.add_argument('-q', action='count', dest='quiet', default=0,
-                        help='output only error messages; -qq will suppress '
+                        help='output only error messages; `-qq` will suppress '
                              'the error messages as well.')
     parser.add_argument('-b', action='store_true', dest='legacy',
-                        help='use legacy (pre-PEP3147) compiled file locations')
+                        help='use legacy (pre-PEP 3147) compiled file locations')
     parser.add_argument('-d', metavar='DESTDIR',  dest='ddir', default=None,
                         help=('directory to prepend to file paths for use in '
                               'compile-time tracebacks and in runtime '
@@ -367,28 +366,28 @@ def main():
                               'of each file considered for compilation'))
     parser.add_argument('-i', metavar='FILE', dest='flist',
                         help=('add all the files and directories listed in '
-                              'FILE to the list considered for compilation; '
-                              'if "-", names are read from stdin'))
+                              '`FILE` to the list considered for compilation; '
+                              'if `"-"`, names are read from `stdin`'))
     parser.add_argument('compile_dest', metavar='FILE|DIR', nargs='*',
                         help=('zero or more file and directory names '
                               'to compile; if no arguments given, defaults '
-                              'to the equivalent of -l sys.path'))
+                              'to the equivalent of `-l` `sys.path`'))
     parser.add_argument('-j', '--workers', default=1,
                         type=int, help='Run compileall concurrently')
     invalidation_modes = [mode.name.lower().replace('_', '-')
                           for mode in py_compile.PycInvalidationMode]
     parser.add_argument('--invalidation-mode',
                         choices=sorted(invalidation_modes),
-                        help=('set .pyc invalidation mode; defaults to '
-                              '"checked-hash" if the SOURCE_DATE_EPOCH '
+                        help=('set `.pyc` invalidation mode; defaults to '
+                              '`"checked-hash"` if the `SOURCE_DATE_EPOCH` '
                               'environment variable is set, and '
-                              '"timestamp" otherwise.'))
+                              '`"timestamp"` otherwise.'))
     parser.add_argument('-o', action='append', type=int, dest='opt_levels',
                         help=('Optimization levels to run compilation with. '
-                              'Default is -1 which uses the optimization level '
-                              'of the Python interpreter itself (see -O).'))
+                              'Default is `-1` which uses the optimization level '
+                              'of the Python interpreter itself (see `-O`).'))
     parser.add_argument('-e', metavar='DIR', dest='limit_sl_dest',
-                        help='Ignore symlinks pointing outsite of the DIR')
+                        help='Ignore symlinks pointing outsite of the `DIR`')
     parser.add_argument('--hardlink-dupes', action='store_true',
                         dest='hardlink_dupes',
                         help='Hardlink duplicated pyc files')