diff --git a/.gitattributes b/.gitattributes index c66e765266382f..fd303806dac21a 100644 --- a/.gitattributes +++ b/.gitattributes @@ -57,6 +57,7 @@ Doc/library/token-list.inc linguist-generated=true Include/token.h linguist-generated=true Lib/token.py linguist-generated=true Parser/token.c linguist-generated=true +Programs/test_frozenmain.h linguist-generated=true # Language aware diff headers # https://tekin.co.uk/2020/10/better-git-diff-output-for-ruby-python-elixir-and-more diff --git a/.github/CODEOWNERS b/.github/CODEOWNERS index 5b86d39dc9c259..8dc0a31c1ba271 100644 --- a/.github/CODEOWNERS +++ b/.github/CODEOWNERS @@ -53,11 +53,11 @@ Python/ast_opt.c @isidentical # SSL -**/*ssl* @python/crypto-team @tiran -**/*.pem @python/crypto-team @tiran +**/*ssl* @python/crypto-team +**/*.pem @python/crypto-team # CSPRNG -Python/bootstrap_hash.c @python/crypto-team @tiran +Python/bootstrap_hash.c @python/crypto-team # Dates and times **/*datetime* @pganssle @abalkin diff --git a/.github/SECURITY.md b/.github/SECURITY.md index 28aea946623cc5..82ae4ca8c30977 100644 --- a/.github/SECURITY.md +++ b/.github/SECURITY.md @@ -10,9 +10,8 @@ https://devguide.python.org/#status-of-python-branches ## Reporting a Vulnerability Please read the guidelines on reporting security issues [on the -official website]( -https://www.python.org/news/security/#reporting-security-issues-in-python -) for instructions on how to report a security-related problem to +official website](https://www.python.org/dev/security/) for +instructions on how to report a security-related problem to the Python team responsibly. To reach the response team, email `security at python dot org`. diff --git a/.github/workflows/build.yml b/.github/workflows/build.yml index d50696d4ff2882..aaa95dbec86c81 100644 --- a/.github/workflows/build.yml +++ b/.github/workflows/build.yml @@ -6,16 +6,18 @@ name: Tests on: push: branches: - - main - - 3.9 - - 3.8 - - 3.7 + - 'main' + - '3.10' + - '3.9' + - '3.8' + - '3.7' pull_request: branches: - - main - - 3.9 - - 3.8 - - 3.7 + - 'main' + - '3.10' + - '3.9' + - '3.8' + - '3.7' jobs: check_source: @@ -148,7 +150,7 @@ jobs: echo "LD_LIBRARY_PATH=${GITHUB_WORKSPACE}/multissl/openssl/${OPENSSL_VER}/lib" >> $GITHUB_ENV - name: 'Restore OpenSSL build' id: cache-openssl - uses: actions/cache@v2.1.5 + uses: actions/cache@v2.1.6 with: path: ./multissl/openssl/${{ env.OPENSSL_VER }} key: ${{ runner.os }}-multissl-openssl-${{ env.OPENSSL_VER }} @@ -177,7 +179,7 @@ jobs: strategy: fail-fast: false matrix: - openssl_ver: [1.1.1k, 3.0.0-alpha16] + openssl_ver: [1.1.1k, 3.0.0-alpha17] env: OPENSSL_VER: ${{ matrix.openssl_ver }} MULTISSL_DIR: ${{ github.workspace }}/multissl @@ -196,7 +198,7 @@ jobs: echo "LD_LIBRARY_PATH=${GITHUB_WORKSPACE}/multissl/openssl/${OPENSSL_VER}/lib" >> $GITHUB_ENV - name: 'Restore OpenSSL build' id: cache-openssl - uses: actions/cache@v2.1.5 + uses: actions/cache@v2.1.6 with: path: ./multissl/openssl/${{ env.OPENSSL_VER }} key: ${{ runner.os }}-multissl-openssl-${{ env.OPENSSL_VER }} diff --git a/.github/workflows/build_msi.yml b/.github/workflows/build_msi.yml index 074f70df4e7a6b..db5eaa17573f27 100644 --- a/.github/workflows/build_msi.yml +++ b/.github/workflows/build_msi.yml @@ -3,18 +3,20 @@ name: TestsMSI on: push: branches: - - main - - 3.9 - - 3.8 - - 3.7 + - 'main' + - '3.10' + - '3.9' + - '3.8' + - '3.7' paths: - 'Tools/msi/**' pull_request: branches: - - main - - 3.9 - - 3.8 - - 3.7 + - 'main' + - '3.10' + - '3.9' + - '3.8' + - '3.7' paths: - 'Tools/msi/**' diff --git a/.github/workflows/doc.yml b/.github/workflows/doc.yml index 3fcdcbc97f2534..1bc14a22ac6976 100644 --- a/.github/workflows/doc.yml +++ b/.github/workflows/doc.yml @@ -3,18 +3,20 @@ name: Docs on: #push: # branches: - # - main - # - 3.9 - # - 3.8 - # - 3.7 + # - 'main' + # - '3.10' + # - '3.9' + # - '3.8' + # - '3.7' # paths: # - 'Doc/**' pull_request: branches: - - main - - 3.9 - - 3.8 - - 3.7 + - 'main' + - '3.10' + - '3.9' + - '3.8' + - '3.7' paths: - 'Doc/**' - 'Misc/**' diff --git a/Doc/about.rst b/Doc/about.rst index 3ea311fa629dd2..f0b908487b2d09 100644 --- a/Doc/about.rst +++ b/Doc/about.rst @@ -23,9 +23,8 @@ Many thanks go to: and writer of much of the content; * the `Docutils `_ project for creating reStructuredText and the Docutils suite; -* Fredrik Lundh for his `Alternative Python Reference - `_ project from which Sphinx got many good - ideas. +* Fredrik Lundh for his Alternative Python Reference project from which Sphinx + got many good ideas. Contributors to the Python Documentation diff --git a/Doc/c-api/gcsupport.rst b/Doc/c-api/gcsupport.rst index 55ed9d4f7fad48..4bd2fb3837a1a0 100644 --- a/Doc/c-api/gcsupport.rst +++ b/Doc/c-api/gcsupport.rst @@ -33,6 +33,18 @@ Constructors for container types must conform to two rules: #. Once all the fields which may contain references to other containers are initialized, it must call :c:func:`PyObject_GC_Track`. + .. warning:: + If a type adds the Py_TPFLAGS_HAVE_GC, then it *must* implement at least + a :c:member:`~PyTypeObject.tp_traverse` handler or explicitly use one + from its subclass or subclasses. + + When calling :c:func:`PyType_Ready` or some of the APIs that indirectly + call it like :c:func:`PyType_FromSpecWithBases` or + :c:func:`PyType_FromSpec` the interpreter will automatically populate the + :c:member:`~PyTypeObject.tp_flags`, :c:member:`~PyTypeObject.tp_traverse` + and :c:member:`~PyTypeObject.tp_clear` fields if the type inherits from a + class that implements the garbage collector protocol and the child class + does *not* include the :const:`Py_TPFLAGS_HAVE_GC` flag. .. c:function:: TYPE* PyObject_GC_New(TYPE, PyTypeObject *type) diff --git a/Doc/c-api/structures.rst b/Doc/c-api/structures.rst index 20d5485d5544c2..1a6e14eae59239 100644 --- a/Doc/c-api/structures.rst +++ b/Doc/c-api/structures.rst @@ -99,7 +99,10 @@ the definition of all other Python objects. Return a :term:`borrowed reference`. - The :c:func:`Py_SET_TYPE` function must be used to set an object type. + Use the :c:func:`Py_SET_TYPE` function to set an object type. + + .. versionchanged:: 3.11 + :c:func:`Py_TYPE()` is changed to an inline static function. .. c:function:: int Py_IS_TYPE(PyObject *o, PyTypeObject *type) @@ -121,9 +124,10 @@ the definition of all other Python objects. Get the reference count of the Python object *o*. + Use the :c:func:`Py_SET_REFCNT()` function to set an object reference count. + .. versionchanged:: 3.10 :c:func:`Py_REFCNT()` is changed to the inline static function. - Use :c:func:`Py_SET_REFCNT()` to set an object reference count. .. c:function:: void Py_SET_REFCNT(PyObject *o, Py_ssize_t refcnt) @@ -137,7 +141,10 @@ the definition of all other Python objects. Get the size of the Python object *o*. - The :c:func:`Py_SET_SIZE` function must be used to set an object size. + Use the :c:func:`Py_SET_SIZE` function to set an object size. + + .. versionchanged:: 3.11 + :c:func:`Py_SIZE()` is changed to an inline static function. .. c:function:: void Py_SET_SIZE(PyVarObject *o, Py_ssize_t size) diff --git a/Doc/c-api/type.rst b/Doc/c-api/type.rst index bdb636dff326f2..7a677593d073d5 100644 --- a/Doc/c-api/type.rst +++ b/Doc/c-api/type.rst @@ -97,6 +97,15 @@ Type Objects from a type's base class. Return ``0`` on success, or return ``-1`` and sets an exception on error. + .. note:: + If some of the base classes implements the GC protocol and the provided + type does not include the :const:`Py_TPFLAGS_HAVE_GC` in its flags, then + the GC protocol will be automatically implemented from its parents. On + the contrary, if the type being created does include + :const:`Py_TPFLAGS_HAVE_GC` in its flags then it **must** implement the + GC protocol itself by at least implementing the + :c:member:`~PyTypeObject.tp_traverse` handle. + .. c:function:: void* PyType_GetSlot(PyTypeObject *type, int slot) Return the function pointer stored in the given slot. If the diff --git a/Doc/c-api/typeobj.rst b/Doc/c-api/typeobj.rst index 82f2ab53451166..ea811158fa8739 100644 --- a/Doc/c-api/typeobj.rst +++ b/Doc/c-api/typeobj.rst @@ -2453,6 +2453,8 @@ Async Object Structures See :c:func:`PyIter_Send` for details. This slot may be set to ``NULL``. + .. versionadded:: 3.10 + .. _slot-typedefs: diff --git a/Doc/c-api/unicode.rst b/Doc/c-api/unicode.rst index 8322d3da76dca4..ddc2346e92dc08 100644 --- a/Doc/c-api/unicode.rst +++ b/Doc/c-api/unicode.rst @@ -421,6 +421,12 @@ APIs: :c:func:`PyUnicode_KIND`). The *buffer* must point to an array of *size* units of 1, 2 or 4 bytes per character, as given by the kind. + If necessary, the input *buffer* is copied and transformed into the + canonical representation. For example, if the *buffer* is a UCS4 string + (:c:macro:`PyUnicode_4BYTE_KIND`) and it consists only of codepoints in + the UCS1 range, it will be transformed into UCS1 + (:c:macro:`PyUnicode_1BYTE_KIND`). + .. versionadded:: 3.3 diff --git a/Doc/data/stable_abi.dat b/Doc/data/stable_abi.dat index d582204f5626b9..50207ac91ec8a8 100644 --- a/Doc/data/stable_abi.dat +++ b/Doc/data/stable_abi.dat @@ -783,7 +783,6 @@ var,Py_FileSystemDefaultEncodeErrors,3.10, var,Py_FileSystemDefaultEncoding,3.2, function,Py_Finalize,3.2, function,Py_FinalizeEx,3.6, -function,Py_FrozenMain,3.10, function,Py_GenericAlias,3.9, function,Py_GenericAliasType,3.9, function,Py_GetBuildInfo,3.2, diff --git a/Doc/faq/gui.rst b/Doc/faq/gui.rst index 781da467d18013..a322fa231669bc 100644 --- a/Doc/faq/gui.rst +++ b/Doc/faq/gui.rst @@ -92,7 +92,7 @@ FLTK Python bindings for `the FLTK toolkit `_, a simple yet powerful and mature cross-platform windowing system, are available from `the -PyFLTK project `_. +PyFLTK project `_. OpenGL ------ diff --git a/Doc/faq/library.rst b/Doc/faq/library.rst index 4d27abd4351ec0..d03656272ebed5 100644 --- a/Doc/faq/library.rst +++ b/Doc/faq/library.rst @@ -106,9 +106,6 @@ support, pads, and mouse support. This means the module isn't compatible with operating systems that only have BSD curses, but there don't seem to be any currently maintained OSes that fall into this category. -For Windows: use `the consolelib module -`_. - Is there an equivalent to C's onexit() in Python? ------------------------------------------------- diff --git a/Doc/faq/programming.rst b/Doc/faq/programming.rst index ed03c494388805..921357d8fcdd2c 100644 --- a/Doc/faq/programming.rst +++ b/Doc/faq/programming.rst @@ -66,6 +66,8 @@ Static type checkers such as `Mypy `_, source code. +.. _faq-create-standalone-binary: + How can I create a stand-alone binary from a Python script? ----------------------------------------------------------- @@ -89,14 +91,15 @@ only contains those built-in modules which are actually used in the program. It then compiles the generated C code and links it with the rest of the Python interpreter to form a self-contained binary which acts exactly like your script. -Obviously, freeze requires a C compiler. There are several other utilities -which don't: - -* `py2exe `_ for Windows binaries -* `py2app `_ for Mac OS X binaries -* `cx_Freeze `_ for cross-platform - binaries +The following packages can help with the creation of console and GUI +executables: +* `Nuitka `_ (Cross-platform) +* `PyInstaller `_ (Cross-platform) +* `PyOxidizer `_ (Cross-platform) +* `cx_Freeze `_ (Cross-platform) +* `py2app `_ (macOS only) +* `py2exe `_ (Windows only) Are there coding standards or a style guide for Python programs? ---------------------------------------------------------------- diff --git a/Doc/faq/windows.rst b/Doc/faq/windows.rst index 186dac2e255b37..0153a4f316ee82 100644 --- a/Doc/faq/windows.rst +++ b/Doc/faq/windows.rst @@ -140,9 +140,8 @@ offender. How do I make an executable from a Python script? ------------------------------------------------- -See `cx_Freeze `_ and -`py2exe `_, both are distutils extensions -that allow you to create console and GUI executables from Python code. +See :ref:`faq-create-standalone-binary` for a list of tools that can be used to +make executables. Is a ``*.pyd`` file the same as a DLL? diff --git a/Doc/glossary.rst b/Doc/glossary.rst index 29c68ed72c6d70..afb5da48d256ef 100644 --- a/Doc/glossary.rst +++ b/Doc/glossary.rst @@ -712,7 +712,7 @@ Glossary On Unix, it is the encoding of the LC_CTYPE locale. It can be set with ``locale.setlocale(locale.LC_CTYPE, new_locale)``. - On Windows, it is is the ANSI code page (ex: ``cp1252``). + On Windows, it is the ANSI code page (ex: ``cp1252``). ``locale.getpreferredencoding(False)`` can be used to get the locale encoding. diff --git a/Doc/howto/curses.rst b/Doc/howto/curses.rst index cc4b4785b12290..c0149ffff37716 100644 --- a/Doc/howto/curses.rst +++ b/Doc/howto/curses.rst @@ -55,11 +55,7 @@ everything, though. The Windows version of Python doesn't include the :mod:`curses` module. A ported version called `UniCurses -`_ is available. You could -also try `the Console module `_ -written by Fredrik Lundh, which doesn't -use the same API as curses but provides cursor-addressable text output -and full support for mouse and keyboard input. +`_ is available. The Python curses module diff --git a/Doc/includes/sqlite3/countcursors.py b/Doc/includes/sqlite3/countcursors.py deleted file mode 100644 index 112f47703a2ff4..00000000000000 --- a/Doc/includes/sqlite3/countcursors.py +++ /dev/null @@ -1,17 +0,0 @@ -import sqlite3 - -class CountCursorsConnection(sqlite3.Connection): - def __init__(self, *args, **kwargs): - sqlite3.Connection.__init__(self, *args, **kwargs) - self.numcursors = 0 - - def cursor(self, *args, **kwargs): - self.numcursors += 1 - return sqlite3.Connection.cursor(self, *args, **kwargs) - -con = sqlite3.connect(":memory:", factory=CountCursorsConnection) -cur1 = con.cursor() -cur2 = con.cursor() -print(con.numcursors) - -con.close() diff --git a/Doc/includes/sqlite3/createdb.py b/Doc/includes/sqlite3/createdb.py deleted file mode 100644 index ee2950bdf81646..00000000000000 --- a/Doc/includes/sqlite3/createdb.py +++ /dev/null @@ -1,28 +0,0 @@ -# Not referenced from the documentation, but builds the database file the other -# code snippets expect. - -import sqlite3 -import os - -DB_FILE = "mydb" - -if os.path.exists(DB_FILE): - os.remove(DB_FILE) - -con = sqlite3.connect(DB_FILE) -cur = con.cursor() -cur.execute(""" - create table people - ( - name_last varchar(20), - age integer - ) - """) - -cur.execute("insert into people (name_last, age) values ('Yeltsin', 72)") -cur.execute("insert into people (name_last, age) values ('Putin', 51)") - -con.commit() - -cur.close() -con.close() diff --git a/Doc/includes/sqlite3/ctx_manager.py b/Doc/includes/sqlite3/ctx_manager.py index 6db77d45046e1f..2e1175ef44c641 100644 --- a/Doc/includes/sqlite3/ctx_manager.py +++ b/Doc/includes/sqlite3/ctx_manager.py @@ -1,19 +1,19 @@ import sqlite3 con = sqlite3.connect(":memory:") -con.execute("create table person (id integer primary key, firstname varchar unique)") +con.execute("create table lang (id integer primary key, name varchar unique)") # Successful, con.commit() is called automatically afterwards with con: - con.execute("insert into person(firstname) values (?)", ("Joe",)) + con.execute("insert into lang(name) values (?)", ("Python",)) # con.rollback() is called after the with block finishes with an exception, the # exception is still raised and must be caught try: with con: - con.execute("insert into person(firstname) values (?)", ("Joe",)) + con.execute("insert into lang(name) values (?)", ("Python",)) except sqlite3.IntegrityError: - print("couldn't add Joe twice") + print("couldn't add Python twice") # Connection object used as context manager only commits or rollbacks transactions, # so the connection object should be closed manually diff --git a/Doc/includes/sqlite3/execsql_fetchonerow.py b/Doc/includes/sqlite3/execsql_fetchonerow.py deleted file mode 100644 index 115bcb50c7c754..00000000000000 --- a/Doc/includes/sqlite3/execsql_fetchonerow.py +++ /dev/null @@ -1,19 +0,0 @@ -import sqlite3 - -con = sqlite3.connect("mydb") - -cur = con.cursor() -SELECT = "select name_last, age from people order by age, name_last" - -# 1. Iterate over the rows available from the cursor, unpacking the -# resulting sequences to yield their elements (name_last, age): -cur.execute(SELECT) -for (name_last, age) in cur: - print('%s is %d years old.' % (name_last, age)) - -# 2. Equivalently: -cur.execute(SELECT) -for row in cur: - print('%s is %d years old.' % (row[0], row[1])) - -con.close() diff --git a/Doc/includes/sqlite3/execsql_printall_1.py b/Doc/includes/sqlite3/execsql_printall_1.py deleted file mode 100644 index 19306e6e3ca7d1..00000000000000 --- a/Doc/includes/sqlite3/execsql_printall_1.py +++ /dev/null @@ -1,15 +0,0 @@ -import sqlite3 - -# Create a connection to the database file "mydb": -con = sqlite3.connect("mydb") - -# Get a Cursor object that operates in the context of Connection con: -cur = con.cursor() - -# Execute the SELECT statement: -cur.execute("select * from people order by age") - -# Retrieve all rows as a sequence and print that sequence: -print(cur.fetchall()) - -con.close() diff --git a/Doc/includes/sqlite3/execute_1.py b/Doc/includes/sqlite3/execute_1.py index 42aad4d5839f06..ee0000e2b94a32 100644 --- a/Doc/includes/sqlite3/execute_1.py +++ b/Doc/includes/sqlite3/execute_1.py @@ -2,22 +2,21 @@ con = sqlite3.connect(":memory:") cur = con.cursor() -cur.execute("create table lang (lang_name, lang_age)") +cur.execute("create table lang (name, first_appeared)") # This is the qmark style: -cur.execute("insert into lang values (?, ?)", ("C", 49)) +cur.execute("insert into lang values (?, ?)", ("C", 1972)) # The qmark style used with executemany(): lang_list = [ - ("Fortran", 64), - ("Python", 30), - ("Go", 11), + ("Fortran", 1957), + ("Python", 1991), + ("Go", 2009), ] cur.executemany("insert into lang values (?, ?)", lang_list) # And this is the named style: -cur.execute("select * from lang where lang_name=:name and lang_age=:age", - {"name": "C", "age": 49}) +cur.execute("select * from lang where first_appeared=:year", {"year": 1972}) print(cur.fetchall()) con.close() diff --git a/Doc/includes/sqlite3/insert_more_people.py b/Doc/includes/sqlite3/insert_more_people.py deleted file mode 100644 index 10cf937243f6da..00000000000000 --- a/Doc/includes/sqlite3/insert_more_people.py +++ /dev/null @@ -1,18 +0,0 @@ -import sqlite3 - -con = sqlite3.connect("mydb") - -cur = con.cursor() - -newPeople = ( - ('Lebed' , 53), - ('Zhirinovsky' , 57), - ) - -for person in newPeople: - cur.execute("insert into people (name_last, age) values (?, ?)", person) - -# The changes will not be saved unless the transaction is committed explicitly: -con.commit() - -con.close() diff --git a/Doc/includes/sqlite3/parse_colnames.py b/Doc/includes/sqlite3/parse_colnames.py deleted file mode 100644 index 5f01dbfe1cb524..00000000000000 --- a/Doc/includes/sqlite3/parse_colnames.py +++ /dev/null @@ -1,10 +0,0 @@ -import sqlite3 -import datetime - -con = sqlite3.connect(":memory:", detect_types=sqlite3.PARSE_COLNAMES) -cur = con.cursor() -cur.execute('select ? as "x [timestamp]"', (datetime.datetime.now(),)) -dt = cur.fetchone()[0] -print(dt, type(dt)) - -con.close() diff --git a/Doc/includes/sqlite3/shared_cache.py b/Doc/includes/sqlite3/shared_cache.py deleted file mode 100644 index 30e71c935ff62e..00000000000000 --- a/Doc/includes/sqlite3/shared_cache.py +++ /dev/null @@ -1,6 +0,0 @@ -import sqlite3 - -# The shared cache is only available in SQLite versions 3.3.3 or later -# See the SQLite documentation for details. - -sqlite3.enable_shared_cache(True) diff --git a/Doc/includes/sqlite3/shortcut_methods.py b/Doc/includes/sqlite3/shortcut_methods.py index 98a39411495cba..48ea6fad15a898 100644 --- a/Doc/includes/sqlite3/shortcut_methods.py +++ b/Doc/includes/sqlite3/shortcut_methods.py @@ -1,23 +1,23 @@ import sqlite3 -persons = [ - ("Hugo", "Boss"), - ("Calvin", "Klein") - ] +langs = [ + ("C++", 1985), + ("Objective-C", 1984), +] con = sqlite3.connect(":memory:") # Create the table -con.execute("create table person(firstname, lastname)") +con.execute("create table lang(name, first_appeared)") # Fill the table -con.executemany("insert into person(firstname, lastname) values (?, ?)", persons) +con.executemany("insert into lang(name, first_appeared) values (?, ?)", langs) # Print the table contents -for row in con.execute("select firstname, lastname from person"): +for row in con.execute("select name, first_appeared from lang"): print(row) -print("I just deleted", con.execute("delete from person").rowcount, "rows") +print("I just deleted", con.execute("delete from lang").rowcount, "rows") # close is not a shortcut method and it's not called automatically, # so the connection object should be closed manually diff --git a/Doc/includes/sqlite3/simple_tableprinter.py b/Doc/includes/sqlite3/simple_tableprinter.py deleted file mode 100644 index 148a1707f948bc..00000000000000 --- a/Doc/includes/sqlite3/simple_tableprinter.py +++ /dev/null @@ -1,28 +0,0 @@ -import sqlite3 - -FIELD_MAX_WIDTH = 20 -TABLE_NAME = 'people' -SELECT = 'select * from %s order by age, name_last' % TABLE_NAME - -con = sqlite3.connect("mydb") - -cur = con.cursor() -cur.execute(SELECT) - -# Print a header. -for fieldDesc in cur.description: - print(fieldDesc[0].ljust(FIELD_MAX_WIDTH), end=' ') -print() # Finish the header with a newline. -print('-' * 78) - -# For each row, print the value of each field left-justified within -# the maximum possible width of that field. -fieldIndices = range(len(cur.description)) -for row in cur: - for fieldIndex in fieldIndices: - fieldValue = str(row[fieldIndex]) - print(fieldValue.ljust(FIELD_MAX_WIDTH), end=' ') - - print() # Finish the row with a newline. - -con.close() diff --git a/Doc/library/asyncio-eventloop.rst b/Doc/library/asyncio-eventloop.rst index ca91efec260db2..83ab7360a9b6e2 100644 --- a/Doc/library/asyncio-eventloop.rst +++ b/Doc/library/asyncio-eventloop.rst @@ -445,7 +445,7 @@ Opening network connections and *local_addr* should be specified. * *local_addr*, if given, is a ``(local_host, local_port)`` tuple used - to bind the socket to locally. The *local_host* and *local_port* + to bind the socket locally. The *local_host* and *local_port* are looked up using ``getaddrinfo()``, similarly to *host* and *port*. * *ssl_handshake_timeout* is (for a TLS connection) the time in seconds @@ -523,7 +523,7 @@ Opening network connections Other arguments: * *local_addr*, if given, is a ``(local_host, local_port)`` tuple used - to bind the socket to locally. The *local_host* and *local_port* + to bind the socket locally. The *local_host* and *local_port* are looked up using :meth:`getaddrinfo`. * *remote_addr*, if given, is a ``(remote_host, remote_port)`` tuple used @@ -1440,7 +1440,7 @@ Do not instantiate the class directly. Start accepting connections. This method is idempotent, so it can be called when - the server is already being serving. + the server is already serving. The *start_serving* keyword-only parameter to :meth:`loop.create_server` and diff --git a/Doc/library/asyncio-queue.rst b/Doc/library/asyncio-queue.rst index 289ad1b014c356..e6f26bb959869f 100644 --- a/Doc/library/asyncio-queue.rst +++ b/Doc/library/asyncio-queue.rst @@ -105,6 +105,13 @@ Queue Raises :exc:`ValueError` if called more times than there were items placed in the queue. + .. deprecated-removed:: 3.8 3.10 + + The ``loop`` parameter. This function has been implicitly getting the + current running loop since 3.7. See + :ref:`What's New in 3.10's Removed section ` + for more information. + Priority Queue ============== diff --git a/Doc/library/asyncio-stream.rst b/Doc/library/asyncio-stream.rst index ad3c7442ad56cd..b3e229c24f07d1 100644 --- a/Doc/library/asyncio-stream.rst +++ b/Doc/library/asyncio-stream.rst @@ -70,6 +70,14 @@ and work with streams: The *ssl_handshake_timeout* parameter. + .. deprecated-removed:: 3.8 3.10 + + The ``loop`` parameter. This function has been implicitly getting the + current running loop since 3.7. See + :ref:`What's New in 3.10's Removed section ` + for more information. + + .. coroutinefunction:: start_server(client_connected_cb, host=None, \ port=None, *, limit=None, \ family=socket.AF_UNSPEC, \ @@ -100,6 +108,13 @@ and work with streams: The *ssl_handshake_timeout* and *start_serving* parameters. + .. deprecated-removed:: 3.8 3.10 + + The ``loop`` parameter. This function has been implicitly getting the + current running loop since 3.7. See + :ref:`What's New in 3.10's Removed section ` + for more information. + .. rubric:: Unix Sockets @@ -124,6 +139,13 @@ and work with streams: The *path* parameter can now be a :term:`path-like object` + .. deprecated-removed:: 3.8 3.10 + + The ``loop`` parameter. This function has been implicitly getting the + current running loop since 3.7. See + :ref:`What's New in 3.10's Removed section ` + for more information. + .. coroutinefunction:: start_unix_server(client_connected_cb, path=None, \ *, limit=None, sock=None, backlog=100, ssl=None, \ @@ -145,6 +167,13 @@ and work with streams: The *path* parameter can now be a :term:`path-like object`. + .. deprecated-removed:: 3.8 3.10 + + The ``loop`` parameter. This function has been implicitly getting the + current running loop since 3.7. See + :ref:`What's New in 3.10's Removed section ` + for more information. + StreamReader ============ diff --git a/Doc/library/asyncio-subprocess.rst b/Doc/library/asyncio-subprocess.rst index ef4d9bcc434c9f..fd1f9c99578794 100644 --- a/Doc/library/asyncio-subprocess.rst +++ b/Doc/library/asyncio-subprocess.rst @@ -75,6 +75,13 @@ Creating Subprocesses See the documentation of :meth:`loop.subprocess_exec` for other parameters. + .. deprecated-removed:: 3.8 3.10 + + The ``loop`` parameter. This function has been implicitly getting the + current running loop since 3.7. See + :ref:`What's New in 3.10's Removed section ` + for more information. + .. coroutinefunction:: create_subprocess_shell(cmd, stdin=None, \ stdout=None, stderr=None, limit=None, **kwds) @@ -99,6 +106,13 @@ Creating Subprocesses escape whitespace and special shell characters in strings that are going to be used to construct shell commands. + .. deprecated-removed:: 3.8 3.10 + + The ``loop`` parameter. This function has been implicitly getting the + current running loop since 3.7. See + :ref:`What's New in 3.10's Removed section ` + for more information. + .. note:: Subprocesses are available for Windows if a :class:`ProactorEventLoop` is diff --git a/Doc/library/asyncio-sync.rst b/Doc/library/asyncio-sync.rst index d12630afc6a326..88e523af0b8088 100644 --- a/Doc/library/asyncio-sync.rst +++ b/Doc/library/asyncio-sync.rst @@ -63,6 +63,12 @@ Lock finally: lock.release() + .. deprecated-removed:: 3.8 3.10 + The ``loop`` parameter. This class has been implicitly getting the + current running loop since 3.7. See + :ref:`What's New in 3.10's Removed section ` + for more information. + .. coroutinemethod:: acquire() Acquire the lock. @@ -105,6 +111,12 @@ Event :meth:`clear` method. The :meth:`~Event.wait` method blocks until the flag is set to *true*. The flag is set to *false* initially. + .. deprecated-removed:: 3.8 3.10 + The ``loop`` parameter. This class has been implicitly getting the + current running loop since 3.7. See + :ref:`What's New in 3.10's Removed section ` + for more information. + .. _asyncio_example_sync_event: Example:: @@ -177,6 +189,12 @@ Condition ``None``. In the latter case a new Lock object is created automatically. + .. deprecated-removed:: 3.8 3.10 + The ``loop`` parameter. This class has been implicitly getting the + current running loop since 3.7. See + :ref:`What's New in 3.10's Removed section ` + for more information. + The preferred way to use a Condition is an :keyword:`async with` statement:: @@ -273,6 +291,12 @@ Semaphore internal counter (``1`` by default). If the given value is less than ``0`` a :exc:`ValueError` is raised. + .. deprecated-removed:: 3.8 3.10 + The ``loop`` parameter. This class has been implicitly getting the + current running loop since 3.7. See + :ref:`What's New in 3.10's Removed section ` + for more information. + The preferred way to use a Semaphore is an :keyword:`async with` statement:: @@ -325,6 +349,13 @@ BoundedSemaphore a :exc:`ValueError` in :meth:`~Semaphore.release` if it increases the internal counter above the initial *value*. + .. deprecated-removed:: 3.8 3.10 + + The ``loop`` parameter. This class has been implicitly getting the + current running loop since 3.7. See + :ref:`What's New in 3.10's Removed section ` + for more information. + --------- diff --git a/Doc/library/asyncio-task.rst b/Doc/library/asyncio-task.rst index 69e965cfc1d2d3..bbdef3345a4d42 100644 --- a/Doc/library/asyncio-task.rst +++ b/Doc/library/asyncio-task.rst @@ -297,6 +297,12 @@ Sleeping tasks to run. This can be used by long-running functions to avoid blocking the event loop for the full duration of the function call. + .. deprecated-removed:: 3.8 3.10 + The ``loop`` parameter. This function has been implicitly getting the + current running loop since 3.7. See + :ref:`What's New in 3.10's Removed section ` + for more information. + .. _asyncio_example_sleep: Example of coroutine displaying the current date every second @@ -317,6 +323,14 @@ Sleeping asyncio.run(display_date()) + .. deprecated-removed:: 3.8 3.10 + + The ``loop`` parameter. This function has been implicitly getting the + current running loop since 3.7. See + :ref:`What's New in 3.10's Removed section ` + for more information. + + Running Tasks Concurrently ========================== @@ -349,6 +363,12 @@ Running Tasks Concurrently cancellation of one submitted Task/Future to cause other Tasks/Futures to be cancelled. + .. deprecated-removed:: 3.8 3.10 + The ``loop`` parameter. This function has been implicitly getting the + current running loop since 3.7. See + :ref:`What's New in 3.10's Removed section ` + for more information. + .. _asyncio_example_gather: Example:: @@ -400,6 +420,12 @@ Running Tasks Concurrently If the *gather* itself is cancelled, the cancellation is propagated regardless of *return_exceptions*. + .. deprecated-removed:: 3.8 3.10 + The ``loop`` parameter. This function has been implicitly getting the + current running loop since 3.7. See + :ref:`What's New in 3.10's Removed section ` + for more information. + .. deprecated:: 3.10 Deprecation warning is emitted if no positional arguments are provided or not all positional arguments are Future-like objects @@ -442,6 +468,12 @@ Shielding From Cancellation except CancelledError: res = None + .. deprecated-removed:: 3.8 3.10 + The ``loop`` parameter. This function has been implicitly getting the + current running loop since 3.7. See + :ref:`What's New in 3.10's Removed section ` + for more information. + .. deprecated:: 3.10 Deprecation warning is emitted if *aw* is not Future-like object and there is no running event loop. @@ -473,6 +505,12 @@ Timeouts If the wait is cancelled, the future *aw* is also cancelled. + .. deprecated-removed:: 3.8 3.10 + The ``loop`` parameter. This function has been implicitly getting the + current running loop since 3.7. See + :ref:`What's New in 3.10's Removed section ` + for more information. + .. _asyncio_example_waitfor: Example:: @@ -500,6 +538,12 @@ Timeouts for *aw* to be cancelled. Previously, it raised :exc:`asyncio.TimeoutError` immediately. + .. deprecated-removed:: 3.8 3.10 + The ``loop`` parameter. This function has been implicitly getting the + current running loop since 3.7. See + :ref:`What's New in 3.10's Removed section ` + for more information. + Waiting Primitives ================== @@ -556,6 +600,12 @@ Waiting Primitives ``wait()`` directly is deprecated as it leads to :ref:`confusing behavior `. + .. deprecated-removed:: 3.8 3.10 + The ``loop`` parameter. This function has been implicitly getting the + current running loop since 3.7. See + :ref:`What's New in 3.10's Removed section ` + for more information. + .. _asyncio_example_wait_coroutine: .. note:: @@ -583,6 +633,13 @@ Waiting Primitives if task in done: # Everything will work as expected now. + .. deprecated-removed:: 3.8 3.10 + + The ``loop`` parameter. This function has been implicitly getting the + current running loop since 3.7. See + :ref:`What's New in 3.10's Removed section ` + for more information. + .. deprecated-removed:: 3.8 3.11 Passing coroutine objects to ``wait()`` directly is @@ -599,12 +656,24 @@ Waiting Primitives Raises :exc:`asyncio.TimeoutError` if the timeout occurs before all Futures are done. + .. deprecated-removed:: 3.8 3.10 + The ``loop`` parameter. This function has been implicitly getting the + current running loop since 3.7. See + :ref:`What's New in 3.10's Removed section ` + for more information. + Example:: for coro in as_completed(aws): earliest_result = await coro # ... + .. deprecated-removed:: 3.8 3.10 + The ``loop`` parameter. This function has been implicitly getting the + current running loop since 3.7. See + :ref:`What's New in 3.10's Removed section ` + for more information. + .. deprecated:: 3.10 Deprecation warning is emitted if not all awaitable objects in the *aws* iterable are Future-like objects and there is no running event loop. diff --git a/Doc/library/builtins.rst b/Doc/library/builtins.rst index 8fb1fef6da2954..7e4f8fe0531567 100644 --- a/Doc/library/builtins.rst +++ b/Doc/library/builtins.rst @@ -25,7 +25,7 @@ that wants to implement an :func:`open` function that wraps the built-in return UpperCaser(f) class UpperCaser: - '''Wrapper around a file that converts output to upper-case.''' + '''Wrapper around a file that converts output to uppercase.''' def __init__(self, f): self._f = f diff --git a/Doc/library/configparser.rst b/Doc/library/configparser.rst index 646e8a317f52c3..b0c2a2cfb06a47 100644 --- a/Doc/library/configparser.rst +++ b/Doc/library/configparser.rst @@ -1153,6 +1153,13 @@ ConfigParser Objects *space_around_delimiters* is true, delimiters between keys and values are surrounded by spaces. + .. note:: + + Comments in the original configuration file are not preserved when + writing the configuration back. + What is considered a comment, depends on the given values for + *comment_prefix* and *inline_comment_prefix*. + .. method:: remove_option(section, option) diff --git a/Doc/library/contextlib.rst b/Doc/library/contextlib.rst index b92f703509fcac..c9065be32e6386 100644 --- a/Doc/library/contextlib.rst +++ b/Doc/library/contextlib.rst @@ -267,8 +267,9 @@ Functions and classes provided: .. function:: suppress(*exceptions) Return a context manager that suppresses any of the specified exceptions - if they occur in the body of a with statement and then resumes execution - with the first statement following the end of the with statement. + if they occur in the body of a :keyword:`!with` statement and then + resumes execution with the first statement following the end of the + :keyword:`!with` statement. As with any other mechanism that completely suppresses exceptions, this context manager should be used only to cover very specific errors where @@ -312,10 +313,11 @@ Functions and classes provided: For example, the output of :func:`help` normally is sent to *sys.stdout*. You can capture that output in a string by redirecting the output to an - :class:`io.StringIO` object:: + :class:`io.StringIO` object. The replacement stream is returned from the + ``__enter__`` method and so is available as the target of the + :keyword:`with` statement:: - f = io.StringIO() - with redirect_stdout(f): + with redirect_stdout(io.StringIO()) as f: help(pow) s = f.getvalue() @@ -578,7 +580,7 @@ Functions and classes provided: The :meth:`close` method is not implemented, :meth:`aclose` must be used instead. - .. method:: enter_async_context(cm) + .. coroutinemethod:: enter_async_context(cm) Similar to :meth:`enter_context` but expects an asynchronous context manager. @@ -592,7 +594,7 @@ Functions and classes provided: Similar to :meth:`callback` but expects a coroutine function. - .. method:: aclose() + .. coroutinemethod:: aclose() Similar to :meth:`close` but properly handles awaitables. diff --git a/Doc/library/dis.rst b/Doc/library/dis.rst index 193e40014d1ce4..a4746bcabc76bc 100644 --- a/Doc/library/dis.rst +++ b/Doc/library/dis.rst @@ -1176,6 +1176,18 @@ All of the following opcodes use their arguments. .. versionadded:: 3.7 +.. opcode:: CALL_METHOD_KW (argc) + + Calls a method in a similar fashion as :opcode:`CALL_METHOD`, but also supports keyword arguments. + *argc* is the number of positional and keyword arguments. + This opcode is designed to be used with :opcode:`LOAD_METHOD`. TOS is a + tuple of keyword argument names. Argument values are below that. + Below them, the two items described in :opcode:`LOAD_METHOD` are on the + stack (either ``self`` and an unbound method object or ``NULL`` and an + arbitrary callable). All of them are popped from the stack and the return value is pushed. + + .. versionadded:: 3.11 + .. opcode:: MAKE_FUNCTION (flags) Pushes a new function object on the stack. From bottom to top, the consumed diff --git a/Doc/library/enum.rst b/Doc/library/enum.rst index b755622afc3680..d9b06fb9efda24 100644 --- a/Doc/library/enum.rst +++ b/Doc/library/enum.rst @@ -597,8 +597,8 @@ Utilites and Decorators A :keyword:`class` decorator specifically for enumerations. It replaces the :meth:`__repr__` method with one that shows *module_name*.*member_name*. It - also injects the members, and their aliases, into the the global namespace - they were defined in. + also injects the members, and their aliases, into the global namespace they + were defined in. .. versionadded:: 3.10 diff --git a/Doc/library/fileinput.rst b/Doc/library/fileinput.rst index 819640045cadf7..3880ed3d2bfc94 100644 --- a/Doc/library/fileinput.rst +++ b/Doc/library/fileinput.rst @@ -205,7 +205,7 @@ The two following opening hooks are provided by this module: modules. If the filename extension is not ``'.gz'`` or ``'.bz2'``, the file is opened normally (ie, using :func:`open` without any decompression). - The *encoding* and *errors* values are passed to to :class:`io.TextIOWrapper` + The *encoding* and *errors* values are passed to :class:`io.TextIOWrapper` for compressed files and open for normal files. Usage example: ``fi = fileinput.FileInput(openhook=fileinput.hook_compressed, encoding="utf-8")`` diff --git a/Doc/library/functions.rst b/Doc/library/functions.rst index 7d8a669f9e16d4..bc55d42fa86051 100644 --- a/Doc/library/functions.rst +++ b/Doc/library/functions.rst @@ -1022,7 +1022,7 @@ are always available. They are listed here in alphabetical order. .. _func-memoryview: -.. class:: memoryview(obj) +.. class:: memoryview(object) :noindex: Return a "memory view" object created from the given argument. See diff --git a/Doc/library/gettext.rst b/Doc/library/gettext.rst index ec2c12806b4160..624501952421cc 100644 --- a/Doc/library/gettext.rst +++ b/Doc/library/gettext.rst @@ -46,16 +46,6 @@ class-based API instead. returned. [#]_ -.. function:: bind_textdomain_codeset(domain, codeset=None) - - Bind the *domain* to *codeset*, changing the encoding of byte strings - returned by the :func:`lgettext`, :func:`ldgettext`, :func:`lngettext` - and :func:`ldngettext` functions. - If *codeset* is omitted, then the current binding is returned. - - .. deprecated-removed:: 3.8 3.10 - - .. function:: textdomain(domain=None) Change or query the current global domain. If *domain* is ``None``, then the @@ -108,29 +98,6 @@ class-based API instead. .. versionadded:: 3.8 -.. function:: lgettext(message) -.. function:: ldgettext(domain, message) -.. function:: lngettext(singular, plural, n) -.. function:: ldngettext(domain, singular, plural, n) - - Equivalent to the corresponding functions without the ``l`` prefix - (:func:`.gettext`, :func:`dgettext`, :func:`ngettext` and :func:`dngettext`), - but the translation is returned as a byte string encoded in the preferred - system encoding if no other encoding was explicitly set with - :func:`bind_textdomain_codeset`. - - .. warning:: - - These functions should be avoided in Python 3, because they return - encoded bytes. It's much better to use alternatives which return - Unicode strings instead, since most Python applications will want to - manipulate human readable text as strings instead of bytes. Further, - it's possible that you may get unexpected Unicode-related exceptions - if there are encoding problems with the translated strings. - - .. deprecated-removed:: 3.8 3.10 - - Note that GNU :program:`gettext` also defines a :func:`dcgettext` method, but this was deemed not useful and so it is currently unimplemented. @@ -181,7 +148,7 @@ install themselves in the built-in namespace as the function :func:`_`. the environment variables. -.. function:: translation(domain, localedir=None, languages=None, class_=None, fallback=False, codeset=None) +.. function:: translation(domain, localedir=None, languages=None, class_=None, fallback=False) Return a :class:`*Translations` instance based on the *domain*, *localedir*, and *languages*, which are first passed to :func:`find` to get a list of the @@ -205,15 +172,13 @@ install themselves in the built-in namespace as the function :func:`_`. .. versionchanged:: 3.3 :exc:`IOError` used to be raised instead of :exc:`OSError`. - .. deprecated-removed:: 3.8 3.10 - The *codeset* parameter. - + .. versionchanged:: 3.11 + *codeset* parameter is removed. -.. function:: install(domain, localedir=None, codeset=None, names=None) +.. function:: install(domain, localedir=None, *, names=None) This installs the function :func:`_` in Python's builtins namespace, based on - *domain*, *localedir*, and *codeset* which are passed to the function - :func:`translation`. + *domain* and *localedir* which are passed to the function :func:`translation`. For the *names* parameter, please see the description of the translation object's :meth:`~NullTranslations.install` method. @@ -228,9 +193,8 @@ install themselves in the built-in namespace as the function :func:`_`. builtins namespace, so it is easily accessible in all modules of your application. - .. deprecated-removed:: 3.8 3.10 - The *codeset* parameter. - + .. versionchanged:: 3.11 + *names* is now a keyword-only parameter. The :class:`NullTranslations` class ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ @@ -294,22 +258,6 @@ are the methods of :class:`!NullTranslations`: .. versionadded:: 3.8 - .. method:: lgettext(message) - .. method:: lngettext(singular, plural, n) - - Equivalent to :meth:`.gettext` and :meth:`.ngettext`, but the translation - is returned as a byte string encoded in the preferred system encoding - if no encoding was explicitly set with :meth:`set_output_charset`. - Overridden in derived classes. - - .. warning:: - - These methods should be avoided in Python 3. See the warning for the - :func:`lgettext` function. - - .. deprecated-removed:: 3.8 3.10 - - .. method:: info() Return the "protected" :attr:`_info` variable, a dictionary containing @@ -321,21 +269,6 @@ are the methods of :class:`!NullTranslations`: Return the encoding of the message catalog file. - .. method:: output_charset() - - Return the encoding used to return translated messages in :meth:`.lgettext` - and :meth:`.lngettext`. - - .. deprecated-removed:: 3.8 3.10 - - - .. method:: set_output_charset(charset) - - Change the encoding used to return translated messages. - - .. deprecated-removed:: 3.8 3.10 - - .. method:: install(names=None) This method installs :meth:`.gettext` into the built-in namespace, @@ -450,22 +383,6 @@ unexpected, or if other problems occur while reading the file, instantiating a .. versionadded:: 3.8 - .. method:: lgettext(message) - .. method:: lngettext(singular, plural, n) - - Equivalent to :meth:`.gettext` and :meth:`.ngettext`, but the translation - is returned as a byte string encoded in the preferred system encoding - if no encoding was explicitly set with - :meth:`~NullTranslations.set_output_charset`. - - .. warning:: - - These methods should be avoided in Python 3. See the warning for the - :func:`lgettext` function. - - .. deprecated-removed:: 3.8 3.10 - - Solaris message catalog support ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ diff --git a/Doc/library/idle.rst b/Doc/library/idle.rst index 3c302115b5f408..faa34e69ff15d7 100644 --- a/Doc/library/idle.rst +++ b/Doc/library/idle.rst @@ -613,6 +613,12 @@ keywords, builtin class and function names, names following ``class`` and ``def``, strings, and comments. For any text window, these are the cursor (when present), found text (when possible), and selected text. +IDLE also highlights the :ref:`soft keywords ` :keyword:`match`, +:keyword:`case `, and :keyword:`_ ` in +pattern-matching statements. However, this highlighting is not perfect and +will be incorrect in some rare cases, including some ``_``-s in ``case`` +patterns. + Text coloring is done in the background, so uncolorized text is occasionally visible. To change the color scheme, use the Configure IDLE dialog Highlighting tab. The marking of debugger breakpoint lines in the editor and diff --git a/Doc/library/importlib.rst b/Doc/library/importlib.rst index 50297e0d73e7c8..b5ee7a6b9659af 100644 --- a/Doc/library/importlib.rst +++ b/Doc/library/importlib.rst @@ -491,7 +491,7 @@ ABC hierarchy:: .. class:: ResourceReader - *Superseded by TraversableReader* + *Superseded by TraversableResources* An :term:`abstract base class` to provide the ability to read *resources*. @@ -622,7 +622,7 @@ ABC hierarchy:: .. method:: is_package(fullname) - An abstract method to return a true value if the module is a package, a + An optional method to return a true value if the module is a package, a false value otherwise. :exc:`ImportError` is raised if the :term:`loader` cannot find the module. @@ -816,7 +816,7 @@ ABC hierarchy:: .. versionadded:: 3.9 -.. class:: TraversableReader +.. class:: TraversableResources An abstract base class for resource readers capable of serving the ``files`` interface. Subclasses ResourceReader and provides diff --git a/Doc/library/inspect.rst b/Doc/library/inspect.rst index b9e8be1234e259..ed33cbb8a61ab2 100644 --- a/Doc/library/inspect.rst +++ b/Doc/library/inspect.rst @@ -834,10 +834,10 @@ function. >>> str(param.replace(default=Parameter.empty, annotation='spam')) "foo:'spam'" - .. versionchanged:: 3.4 - In Python 3.3 Parameter objects were allowed to have ``name`` set - to ``None`` if their ``kind`` was set to ``POSITIONAL_ONLY``. - This is no longer permitted. + .. versionchanged:: 3.4 + In Python 3.3 Parameter objects were allowed to have ``name`` set + to ``None`` if their ``kind`` was set to ``POSITIONAL_ONLY``. + This is no longer permitted. .. class:: BoundArguments diff --git a/Doc/library/logging.rst b/Doc/library/logging.rst index 70a703dde18a03..313cff4ff9658f 100644 --- a/Doc/library/logging.rst +++ b/Doc/library/logging.rst @@ -1111,6 +1111,14 @@ functions. .. note:: If you are thinking of defining your own levels, please see the section on :ref:`custom-levels`. +.. function:: getLevelNamesMapping() + + Returns a mapping from level names to their corresponding logging levels. For example, the + string "CRITICAL" maps to :const:`CRITICAL`. The returned mapping is copied from an internal + mapping on each call to this function. + + .. versionadded:: 3.11 + .. function:: getLevelName(level) Returns the textual or numeric representation of logging level *level*. diff --git a/Doc/library/msilib.rst b/Doc/library/msilib.rst index 83b3d4973bf0d9..21a2a205c39391 100644 --- a/Doc/library/msilib.rst +++ b/Doc/library/msilib.rst @@ -119,7 +119,7 @@ structures. .. function:: gen_uuid() Return a new UUID, in the format that MSI typically requires (i.e. in curly - braces, and with all hexdigits in upper-case). + braces, and with all hexdigits in uppercase). .. seealso:: diff --git a/Doc/library/poplib.rst b/Doc/library/poplib.rst index 9bf9212d917a55..27202a168e7c17 100644 --- a/Doc/library/poplib.rst +++ b/Doc/library/poplib.rst @@ -118,7 +118,7 @@ One exception is defined as an attribute of the :mod:`poplib` module: POP3 Objects ------------ -All POP3 commands are represented by methods of the same name, in lower-case; +All POP3 commands are represented by methods of the same name, in lowercase; most return the response text sent by the server. An :class:`POP3` instance has the following methods: diff --git a/Doc/library/socket.rst b/Doc/library/socket.rst index 8a1fc96e0362e2..da3d0d8f1e1b0b 100755 --- a/Doc/library/socket.rst +++ b/Doc/library/socket.rst @@ -188,7 +188,7 @@ created. Socket addresses are represented as follows: - ``PACKET_HOST`` (the default) - Packet addressed to the local host. - ``PACKET_BROADCAST`` - Physical-layer broadcast packet. - - ``PACKET_MULTIHOST`` - Packet sent to a physical-layer multicast address. + - ``PACKET_MULTICAST`` - Packet sent to a physical-layer multicast address. - ``PACKET_OTHERHOST`` - Packet to some other host that has been caught by a device driver in promiscuous mode. - ``PACKET_OUTGOING`` - Packet originating from the local host that is diff --git a/Doc/library/sqlite3.rst b/Doc/library/sqlite3.rst index d0f28db12fda16..4010e1a4daff27 100644 --- a/Doc/library/sqlite3.rst +++ b/Doc/library/sqlite3.rst @@ -213,7 +213,7 @@ Module functions and constants The :mod:`sqlite3` module internally uses a statement cache to avoid SQL parsing overhead. If you want to explicitly set the number of statements that are cached for the connection, you can set the *cached_statements* parameter. The currently - implemented default is to cache 100 statements. + implemented default is to cache 128 statements. If *uri* is true, *database* is interpreted as a URI. This allows you to specify options. For example, to open a database in read-only mode @@ -648,7 +648,8 @@ Cursor Objects This is a nonstandard convenience method for executing multiple SQL statements at once. It issues a ``COMMIT`` statement first, then executes the SQL script it - gets as a parameter. + gets as a parameter. This method disregards :attr:`isolation_level`; any + transation control must be added to *sql_script*. *sql_script* can be an instance of :class:`str`. @@ -1048,6 +1049,9 @@ setting :attr:`isolation_level` to ``None``. This will leave the underlying control the transaction state by explicitly issuing ``BEGIN``, ``ROLLBACK``, ``SAVEPOINT``, and ``RELEASE`` statements in your code. +Note that :meth:`~Cursor.executescript` disregards +:attr:`isolation_level`; any transaction control must be added explicitly. + .. versionchanged:: 3.6 :mod:`sqlite3` used to implicitly commit an open transaction before DDL statements. This is no longer the case. diff --git a/Doc/library/statistics.rst b/Doc/library/statistics.rst index a65c9840b8113a..bb03a2ce6ee971 100644 --- a/Doc/library/statistics.rst +++ b/Doc/library/statistics.rst @@ -43,7 +43,7 @@ or sample. ======================= =============================================================== :func:`mean` Arithmetic mean ("average") of data. -:func:`fmean` Fast, floating point arithmetic mean. +:func:`fmean` Fast, floating point arithmetic mean, with optional weighting. :func:`geometric_mean` Geometric mean of data. :func:`harmonic_mean` Harmonic mean of data. :func:`median` Median (middle value) of data. @@ -76,7 +76,7 @@ These functions calculate statistics regarding relations between two inputs. ========================= ===================================================== :func:`covariance` Sample covariance for two variables. :func:`correlation` Pearson's correlation coefficient for two variables. -:func:`linear_regression` Intercept and slope for simple linear regression. +:func:`linear_regression` Slope and intercept for simple linear regression. ========================= ===================================================== @@ -128,7 +128,7 @@ However, for reading convenience, most of the examples show sorted sequences. ``mean(data)`` is equivalent to calculating the true population mean μ. -.. function:: fmean(data) +.. function:: fmean(data, weights=None) Convert *data* to floats and compute the arithmetic mean. @@ -141,8 +141,25 @@ However, for reading convenience, most of the examples show sorted sequences. >>> fmean([3.5, 4.0, 5.25]) 4.25 + Optional weighting is supported. For example, a professor assigns a + grade for a course by weighting quizzes at 20%, homework at 20%, a + midterm exam at 30%, and a final exam at 30%: + + .. doctest:: + + >>> grades = [85, 92, 83, 91] + >>> weights = [0.20, 0.20, 0.30, 0.30] + >>> fmean(grades, weights) + 87.6 + + If *weights* is supplied, it must be the same length as the *data* or + a :exc:`ValueError` will be raised. + .. versionadded:: 3.8 + .. versionchanged:: 3.11 + Added support for *weights*. + .. function:: geometric_mean(data) @@ -626,37 +643,38 @@ However, for reading convenience, most of the examples show sorted sequences. .. versionadded:: 3.10 -.. function:: linear_regression(regressor, dependent_variable) +.. function:: linear_regression(x, y, /) - Return the intercept and slope of `simple linear regression + Return the slope and intercept of `simple linear regression `_ parameters estimated using ordinary least squares. Simple linear - regression describes the relationship between *regressor* and - *dependent variable* in terms of this linear function: + regression describes the relationship between an independent variable *x* and + a dependent variable *y* in terms of this linear function: - *dependent_variable = intercept + slope \* regressor + noise* + *y = slope \* x + intercept + noise* - where ``intercept`` and ``slope`` are the regression parameters that are - estimated, and noise represents the + where ``slope`` and ``intercept`` are the regression parameters that are + estimated, and ``noise`` represents the variability of the data that was not explained by the linear regression (it is equal to the difference between predicted and actual values - of dependent variable). + of the dependent variable). - Both inputs must be of the same length (no less than two), and regressor - needs not to be constant; otherwise :exc:`StatisticsError` is raised. + Both inputs must be of the same length (no less than two), and + the independent variable *x* cannot be constant; + otherwise a :exc:`StatisticsError` is raised. For example, we can use the `release dates of the Monty - Python films `_, and used - it to predict the cumulative number of Monty Python films + Python films `_ + to predict the cumulative number of Monty Python films that would have been produced by 2019 - assuming that they kept the pace. + assuming that they had kept the pace. .. doctest:: >>> year = [1971, 1975, 1979, 1982, 1983] >>> films_total = [1, 2, 3, 4, 5] - >>> intercept, slope = linear_regression(year, films_total) - >>> round(intercept + slope * 2019) + >>> slope, intercept = linear_regression(year, films_total) + >>> round(slope * 2019 + intercept) 16 .. versionadded:: 3.10 diff --git a/Doc/library/stdtypes.rst b/Doc/library/stdtypes.rst index c4e6b4d1186d04..e34a888639c189 100644 --- a/Doc/library/stdtypes.rst +++ b/Doc/library/stdtypes.rst @@ -3632,17 +3632,16 @@ Memory Views of an object that supports the :ref:`buffer protocol ` without copying. -.. class:: memoryview(obj) +.. class:: memoryview(object) - Create a :class:`memoryview` that references *obj*. *obj* must support the - buffer protocol. Built-in objects that support the buffer protocol include - :class:`bytes` and :class:`bytearray`. + Create a :class:`memoryview` that references *object*. *object* must + support the buffer protocol. Built-in objects that support the buffer + protocol include :class:`bytes` and :class:`bytearray`. A :class:`memoryview` has the notion of an *element*, which is the - atomic memory unit handled by the originating object *obj*. For many - simple types such as :class:`bytes` and :class:`bytearray`, an element - is a single byte, but other types such as :class:`array.array` may have - bigger elements. + atomic memory unit handled by the originating *object*. For many simple + types such as :class:`bytes` and :class:`bytearray`, an element is a single + byte, but other types such as :class:`array.array` may have bigger elements. ``len(view)`` is equal to the length of :class:`~memoryview.tolist`. If ``view.ndim = 0``, the length is 1. If ``view.ndim = 1``, the length diff --git a/Doc/library/tempfile.rst b/Doc/library/tempfile.rst index 2970252036780d..49396b5f4bdc4d 100644 --- a/Doc/library/tempfile.rst +++ b/Doc/library/tempfile.rst @@ -87,6 +87,9 @@ The module defines the following user-callable items: attribute is the underlying true file object. This file-like object can be used in a :keyword:`with` statement, just like a normal file. + On POSIX (only), a process that is terminated abruptly with SIGKILL + cannot automatically delete any NamedTemporaryFiles it created. + .. audit-event:: tempfile.mkstemp fullpath tempfile.NamedTemporaryFile .. versionchanged:: 3.8 diff --git a/Doc/library/test.rst b/Doc/library/test.rst index e4f779bd83eb87..eb4f04f96e4f18 100644 --- a/Doc/library/test.rst +++ b/Doc/library/test.rst @@ -928,8 +928,16 @@ The :mod:`test.support` module defines the following functions: .. versionadded:: 3.10 +.. function:: check_disallow_instantiation(test_case, tp, *args, **kwds) + + Assert that type *tp* cannot be instantiated using *args* and *kwds*. + + .. versionadded:: 3.11 + + The :mod:`test.support` module defines the following classes: + .. class:: SuppressCrashReport() A context manager used to try to prevent crash dialog popups on tests that diff --git a/Doc/library/timeit.rst b/Doc/library/timeit.rst index 668fcb860cea87..d4e8b749db4808 100644 --- a/Doc/library/timeit.rst +++ b/Doc/library/timeit.rst @@ -15,8 +15,8 @@ This module provides a simple way to time small bits of Python code. It has both a :ref:`timeit-command-line-interface` as well as a :ref:`callable ` one. It avoids a number of common traps for measuring execution times. -See also Tim Peters' introduction to the "Algorithms" chapter in the *Python -Cookbook*, published by O'Reilly. +See also Tim Peters' introduction to the "Algorithms" chapter in the second +edition of *Python Cookbook*, published by O'Reilly. Basic Examples diff --git a/Doc/library/tkinter.rst b/Doc/library/tkinter.rst index 7739f2f60a7980..beeffb8a548ee5 100644 --- a/Doc/library/tkinter.rst +++ b/Doc/library/tkinter.rst @@ -34,9 +34,6 @@ installed, so you can read the Tcl/Tk documentation specific to that version. `Tkinter 8.5 reference: a GUI for Python `_ On-line reference material. - `Tkinter docs from effbot `_ - Online reference for tkinter supported by effbot.org. - `Programming Python `_ Book by Mark Lutz, has excellent coverage of Tkinter. diff --git a/Doc/library/traceback.rst b/Doc/library/traceback.rst index e938dd58b05312..bd53bc066becc5 100644 --- a/Doc/library/traceback.rst +++ b/Doc/library/traceback.rst @@ -271,6 +271,13 @@ capture data for later printing in a lightweight fashion. Note that when locals are captured, they are also shown in the traceback. + .. method:: print(*, file=None, chain=True) + + Print to *file* (default ``sys.stderr``) the exception information returned by + :meth:`format`. + + .. versionadded:: 3.11 + .. method:: format(*, chain=True) Format the exception. diff --git a/Doc/library/turtle.rst b/Doc/library/turtle.rst index 6a9d61916ad1a5..9e8e85da2bbb96 100644 --- a/Doc/library/turtle.rst +++ b/Doc/library/turtle.rst @@ -1894,7 +1894,7 @@ Input methods Pop up a dialog window for input of a number. title is the title of the dialog window, prompt is a text mostly describing what numerical information to input. default: default value, minval: minimum value for input, - maxval: maximum value for input + maxval: maximum value for input. The number input must be in the range minval .. maxval if these are given. If not, a hint is issued and the dialog remains open for correction. @@ -1938,7 +1938,7 @@ Settings and special methods :param cmode: one of the values 1.0 or 255 Return the colormode or set it to 1.0 or 255. Subsequently *r*, *g*, *b* - values of color triples have to be in the range 0..\ *cmode*. + values of color triples have to be in the range 0..*cmode*. .. doctest:: :skipif: _tkinter is None @@ -2002,7 +2002,7 @@ Settings and special methods >>> screen.register_shape("triangle", ((5,-3), (0,5), (-5,-3))) - (3) *name* is an arbitrary string and shape is a (compound) :class:`Shape` + (3) *name* is an arbitrary string and *shape* is a (compound) :class:`Shape` object: Install the corresponding compound shape. Add a turtle shape to TurtleScreen's shapelist. Only thusly registered @@ -2349,12 +2349,12 @@ Short explanation of selected entries: auto``. - If you set e.g. ``language = italian`` the docstringdict :file:`turtle_docstringdict_italian.py` will be loaded at import time (if - present on the import path, e.g. in the same directory as :mod:`turtle`. + present on the import path, e.g. in the same directory as :mod:`turtle`). - The entries *exampleturtle* and *examplescreen* define the names of these objects as they occur in the docstrings. The transformation of method-docstrings to function-docstrings will delete these names from the docstrings. -- *using_IDLE*: Set this to ``True`` if you regularly work with IDLE and its -n +- *using_IDLE*: Set this to ``True`` if you regularly work with IDLE and its ``-n`` switch ("no subprocess"). This will prevent :func:`exitonclick` to enter the mainloop. diff --git a/Doc/library/types.rst b/Doc/library/types.rst index b3fac293ecc9e1..2dfc0f2fcadf37 100644 --- a/Doc/library/types.rst +++ b/Doc/library/types.rst @@ -278,8 +278,8 @@ Standard names are defined for the following types: .. attribute:: __spec__ - A record of the the module's import-system-related state. Expected to be - an instance of :class:`importlib.machinery.ModuleSpec`. + A record of the module's import-system-related state. Expected to be an + instance of :class:`importlib.machinery.ModuleSpec`. .. versionadded:: 3.4 diff --git a/Doc/library/unittest.mock.rst b/Doc/library/unittest.mock.rst index 1604731b99e3fe..f1c0757c510fa6 100644 --- a/Doc/library/unittest.mock.rst +++ b/Doc/library/unittest.mock.rst @@ -328,8 +328,8 @@ the *new_callable* argument to :func:`patch`. .. method:: assert_called_once_with(*args, **kwargs) - Assert that the mock was called exactly once and that that call was - with the specified arguments. + Assert that the mock was called exactly once and that call was with the + specified arguments. >>> mock = Mock(return_value=None) >>> mock('foo', bar='baz') diff --git a/Doc/library/xml.etree.elementtree.rst b/Doc/library/xml.etree.elementtree.rst index 87f4ee347d604d..e3932bc9e659fe 100644 --- a/Doc/library/xml.etree.elementtree.rst +++ b/Doc/library/xml.etree.elementtree.rst @@ -363,13 +363,6 @@ These two approaches both output:: |--> Commander Clement -Additional resources -^^^^^^^^^^^^^^^^^^^^ - -See http://effbot.org/zone/element-index.htm for tutorials and links to other -docs. - - .. _elementtree-xpath: XPath support diff --git a/Doc/library/xmlrpc.client.rst b/Doc/library/xmlrpc.client.rst index 51279b35fd71b7..8d9db53ef1f0da 100644 --- a/Doc/library/xmlrpc.client.rst +++ b/Doc/library/xmlrpc.client.rst @@ -169,12 +169,6 @@ between conformable Python objects and XML on the wire. `XML-RPC Specification `_ The official specification. - `Unofficial XML-RPC Errata `_ - Fredrik Lundh's "unofficial errata, intended to clarify certain - details in the XML-RPC specification, as well as hint at - 'best practices' to use when designing your own XML-RPC - implementations." - .. _serverproxy-objects: ServerProxy Objects diff --git a/Doc/tools/susp-ignored.csv b/Doc/tools/susp-ignored.csv index d56a2b9fd0bfb9..1fde253feac2fa 100644 --- a/Doc/tools/susp-ignored.csv +++ b/Doc/tools/susp-ignored.csv @@ -209,8 +209,7 @@ library/smtplib,,:port,method must support that as well as a regular host:port library/socket,,::,'5aef:2b::8' library/socket,,:can,"return (can_id, can_dlc, data[:can_dlc])" library/socket,,:len,fds.frombytes(cmsg_data[:len(cmsg_data) - (len(cmsg_data) % fds.itemsize)]) -library/sqlite3,,:name,"cur.execute(""select * from lang where lang_name=:name and lang_age=:age""," -library/sqlite3,,:age,"cur.execute(""select * from lang where lang_name=:name and lang_age=:age""," +library/sqlite3,,:year,"cur.execute(""select * from lang where first_appeared=:year"", {""year"": 1972})" library/sqlite3,,:memory, library/sqlite3,,:path,"db = sqlite3.connect('file:path/to/database?mode=ro', uri=True)" library/ssl,,:My,"Organizational Unit Name (eg, section) []:My Group" diff --git a/Doc/tutorial/errors.rst b/Doc/tutorial/errors.rst index cd5fc878f9c946..27e39dfe257716 100644 --- a/Doc/tutorial/errors.rst +++ b/Doc/tutorial/errors.rst @@ -305,7 +305,7 @@ disabled by using ``from None`` idiom: >>> try: ... open('database.sqlite') - ... except IOError: + ... except OSError: ... raise RuntimeError from None ... Traceback (most recent call last): diff --git a/Doc/tutorial/inputoutput.rst b/Doc/tutorial/inputoutput.rst index 4e27cff83ce59f..7f83c4d4612eb3 100644 --- a/Doc/tutorial/inputoutput.rst +++ b/Doc/tutorial/inputoutput.rst @@ -480,7 +480,8 @@ If you have an object ``x``, you can view its JSON string representation with a simple line of code:: >>> import json - >>> json.dumps([1, 'simple', 'list']) + >>> x = [1, 'simple', 'list'] + >>> json.dumps(x) '[1, "simple", "list"]' Another variant of the :func:`~json.dumps` function, called :func:`~json.dump`, diff --git a/Doc/using/configure.rst b/Doc/using/configure.rst index c37540c7e031d6..b1d1623a38cbee 100644 --- a/Doc/using/configure.rst +++ b/Doc/using/configure.rst @@ -175,8 +175,8 @@ recommended for best performance. Enable Link Time Optimization (LTO) in any build (disabled by default). - The C compiler Clang requires ``llvm-ar`` for LTO, as well as an LTO-aware - linker (``ld.gold`` or ``lld``). + The C compiler Clang requires ``llvm-ar`` for LTO (``ar`` on macOS), as well + as an LTO-aware linker (``ld.gold`` or ``lld``). .. versionadded:: 3.6 @@ -227,7 +227,7 @@ Effects of a debug build: * Install :ref:`debug hooks on memory allocators ` to detect buffer overflow and other memory errors. * Define ``Py_DEBUG`` and ``Py_REF_DEBUG`` macros. -* Add runtime checks: code surroundeded by ``#ifdef Py_DEBUG`` and ``#endif``. +* Add runtime checks: code surrounded by ``#ifdef Py_DEBUG`` and ``#endif``. Enable ``assert(...)`` and ``_PyObject_ASSERT(...)`` assertions: don't set the ``NDEBUG`` macro (see also the :option:`--with-assertions` configure option). Main runtime checks: @@ -550,7 +550,7 @@ Built-in modules have no ``__file__`` attribute:: File "", line 1, in AttributeError: module 'sys' has no attribute '__file__' -Other C extensins are built as dynamic libraires, like the ``_asyncio`` module. +Other C extensions are built as dynamic libraires, like the ``_asyncio`` module. They are built with the ``Py_BUILD_CORE_MODULE`` macro defined. Example on Linux x86-64:: diff --git a/Doc/whatsnew/3.10.rst b/Doc/whatsnew/3.10.rst index 926679e6f32dc5..bc0f938870376f 100644 --- a/Doc/whatsnew/3.10.rst +++ b/Doc/whatsnew/3.10.rst @@ -172,7 +172,7 @@ These improvements are inspired by previous work in the PyPy interpreter. :issue:`40176`.) :exc:`SyntaxError` exceptions raised by the interpreter will now highlight the -full error range of the expression that consistutes the syntax error itself, +full error range of the expression that constitutes the syntax error itself, instead of just where the problem is detected. In this way, instead of displaying (before Python 3.10): @@ -274,6 +274,20 @@ have been incorporated. Some of the most notable ones: (Contributed by Pablo Galindo in :issue:`43823`) +* ``try`` blocks without ``except`` or ``finally`` blocks: + + .. code-block:: python + + >>> try + ... x = 2 + ... something = 3 + File "", line 3 + something = 3 + ^^^^^^^^^ + SyntaxError: expected 'except' or 'finally' block + + (Contributed by Pablo Galindo in :issue:`44305`) + * Usage of ``=`` instead of ``==`` in comparisons: .. code-block:: python @@ -1030,14 +1044,23 @@ Terry Jan Reedy in :issue:`37892`.) We expect to backport these shell changes to a future 3.9 maintenance release. +Highlight the new :ref:`soft keywords ` :keyword:`match`, +:keyword:`case `, and :keyword:`_ ` in +pattern-matching statements. However, this highlighting is not perfect +and will be incorrect in some rare cases, including some ``_``-s in +``case`` patterns. (Contributed by Tal Einat in bpo-44010.) + importlib.metadata ------------------ -Feature parity with ``importlib_metadata`` 3.7. +Feature parity with ``importlib_metadata`` 4.4 +(`history `_). -:func:`importlib.metadata.entry_points` now provides a nicer experience +:ref:`importlib.metadata entry points ` +now provides a nicer experience for selecting entry points by group and name through a new -:class:`importlib.metadata.EntryPoints` class. +:class:`importlib.metadata.EntryPoints` class. See the Compatibility +Note in the docs for more info on the deprecation and usage. Added :func:`importlib.metadata.packages_distributions` for resolving top-level Python modules and packages to their @@ -1128,6 +1151,9 @@ identification from `freedesktop.org os-release pprint ------ +:func:`pprint.pprint` now accepts a new ``underscore_numbers`` keyword argument. +(Contributed by sblondon in :issue:`42914`.) + :mod:`pprint` can now pretty-print :class:`dataclasses.dataclass` instances. (Contributed by Lewis Gaul in :issue:`43080`.) @@ -1311,6 +1337,14 @@ Add new function :func:`typing.is_typeddict` to introspect if an annotation is a :class:`typing.TypedDict`. (Contributed by Patrick Reader in :issue:`41792`) +Subclasses of ``typing.Protocol`` which only have data variables declared +will now raise a ``TypeError`` when checked with ``isinstance`` unless they +are decorated with :func:`runtime_checkable`. Previously, these checks +passed silently. Users should decorate their +subclasses with the :func:`runtime_checkable` decorator +if they want runtime protocols. +(Contributed by Yurii Karabas in :issue:`38908`) + unittest -------- @@ -1561,9 +1595,10 @@ Deprecated Python 3.12. Use :meth:`pathlib.Path.hardlink_to` instead. (Contributed by Barney Gale in :issue:`39950`.) -* ``cgi.log()`` is deprecated and slated for for removal in Python 3.12. +* ``cgi.log()`` is deprecated and slated for removal in Python 3.12. (Contributed by Inada Naoki in :issue:`41139`.) +.. _whatsnew310-removed: Removed ======= @@ -1913,6 +1948,10 @@ Porting to Python 3.10 instead. (Contributed by Victor Stinner and Erlend E. Aasland in :issue:`43908`.) +* The undocumented function ``Py_FrozenMain`` has been removed from the + limited API. The function is mainly useful for custom builds of Python. + (Contributed by Petr Viktorin in :issue:`26241`) + Deprecated ---------- diff --git a/Doc/whatsnew/3.11.rst b/Doc/whatsnew/3.11.rst index 503688294072a3..1ea8cbaf9a82a3 100644 --- a/Doc/whatsnew/3.11.rst +++ b/Doc/whatsnew/3.11.rst @@ -90,7 +90,7 @@ Improved Modules Optimizations ============= -* Compiler optimizes now simple C-style formatting with literal format +* Compiler now optimizes simple C-style formatting with literal format containing only format codes ``%s``, ``%r`` and ``%a`` and makes it as fast as corresponding f-string expression. (Contributed by Serhiy Storchaka in :issue:`28307`.) @@ -99,6 +99,20 @@ Optimizations almost eliminated when no exception is raised. (Contributed by Mark Shannon in :issue:`40222`.) +* Method calls with keywords are now faster due to bytecode + changes which avoid creating bound method instances. Previously, this + optimization was applied only to method calls with purely positional + arguments. + (Contributed by Ken Jin and Mark Shannon in :issue:`26110`, based on ideas + implemented in PyPy.) + +CPython bytecode changes +======================== + +* Added a new :opcode:`CALL_METHOD_KW` opcode. Calls a method in a similar + fashion as :opcode:`CALL_METHOD`, but also supports keyword arguments. Works + in tandem with :opcode:`LOAD_METHOD`. + Build Changes ============= @@ -130,6 +144,39 @@ New Features Porting to Python 3.11 ---------------------- +* The :c:func:`PyType_Ready` function now raises an error if a type is defined + with the :const:`Py_TPFLAGS_HAVE_GC` flag set but has no traverse function + (:c:member:`PyTypeObject.tp_traverse`). + (Contributed by Victor Stinner in :issue:`44263`.) + +* Since :c:func:`Py_TYPE()` is changed to a inline static function, + ``Py_TYPE(obj) = new_type`` must be replaced with + ``Py_SET_TYPE(obj, new_type)``: see the :c:func:`Py_SET_TYPE()` function + (available since Python 3.9). For backward compatibility, this macro can be + used:: + + #if PY_VERSION_HEX < 0x030900A4 && !defined(Py_SET_TYPE) + static inline void _Py_SET_TYPE(PyObject *ob, PyTypeObject *type) + { ob->ob_type = type; } + #define Py_SET_TYPE(ob, type) _Py_SET_TYPE(_PyObject_CAST(ob), type) + #endif + + (Contributed by Victor Stinner in :issue:`39573`.) + +* Since :c:func:`Py_SIZE()` is changed to a inline static function, + ``Py_SIZE(obj) = new_size`` must be replaced with + ``Py_SET_SIZE(obj, new_size)``: see the :c:func:`Py_SET_SIZE()` function + (available since Python 3.9). For backward compatibility, this macro can be + used:: + + #if PY_VERSION_HEX < 0x030900A4 && !defined(Py_SET_SIZE) + static inline void _Py_SET_SIZE(PyVarObject *ob, Py_ssize_t size) + { ob->ob_size = size; } + #define Py_SET_SIZE(ob, size) _Py_SET_SIZE((PyVarObject*)(ob), size) + #endif + + (Contributed by Victor Stinner in :issue:`39573`.) + Deprecated ---------- @@ -155,3 +202,15 @@ Removed Use the new :c:type:`PyConfig` API of the :ref:`Python Initialization Configuration ` instead (:pep:`587`). (Contributed by Victor Stinner in :issue:`44113`.) + +* The following deprecated functions and methods are removed in the :mod:`gettext` + module: :func:`~gettext.lgettext`, :func:`~gettext.ldgettext`, + :func:`~gettext.lngettext` and :func:`~gettext.ldngettext`. + + Function :func:`~gettext.bind_textdomain_codeset`, methods + :meth:`~gettext.NullTranslations.output_charset` and + :meth:`~gettext.NullTranslations.set_output_charset`, and the *codeset* + parameter of functions :func:`~gettext.translation` and + :func:`~gettext.install` are also removed, since they are only used for + the ``l*gettext()`` functions. + (Contributed by Dong-hee Na and Serhiy Storchaka in :issue:`44235`.) diff --git a/Grammar/python.gram b/Grammar/python.gram index 6b815ab0c518d4..d0f9bb0bc4f277 100644 --- a/Grammar/python.gram +++ b/Grammar/python.gram @@ -509,13 +509,13 @@ star_named_expression[expr_ty]: | '*' a=bitwise_or { _PyAST_Starred(a, Load, EXTRA) } | named_expression -named_expression[expr_ty]: - | a=NAME ':=' ~ b=expression { _PyAST_NamedExpr(CHECK(expr_ty, _PyPegen_set_expr_context(p, a, Store)), b, EXTRA) } - | invalid_named_expression - | expression !':=' -direct_named_expression[expr_ty]: +assigment_expression[expr_ty]: | a=NAME ':=' ~ b=expression { _PyAST_NamedExpr(CHECK(expr_ty, _PyPegen_set_expr_context(p, a, Store)), b, EXTRA) } + +named_expression[expr_ty]: + | assigment_expression + | invalid_named_expression | expression !':=' annotated_rhs[expr_ty]: yield_expr | star_expressions @@ -706,7 +706,7 @@ group[expr_ty]: | '(' a=(yield_expr | named_expression) ')' { a } | invalid_group genexp[expr_ty]: - | '(' a=direct_named_expression b=for_if_clauses ')' { _PyAST_GeneratorExp(a, b, EXTRA) } + | '(' a=( assigment_expression | expression !':=') b=for_if_clauses ')' { _PyAST_GeneratorExp(a, b, EXTRA) } | invalid_comprehension set[expr_ty]: '{' a=star_named_expressions '}' { _PyAST_Set(a, EXTRA) } setcomp[expr_ty]: @@ -745,11 +745,13 @@ arguments[expr_ty] (memo): | a=args [','] &')' { a } | invalid_arguments args[expr_ty]: - | a[asdl_expr_seq*]=','.(starred_expression | direct_named_expression !'=')+ b=[',' k=kwargs {k}] { _PyPegen_collect_call_seqs(p, a, b, EXTRA) } + | a[asdl_expr_seq*]=','.(starred_expression | ( assigment_expression | expression !':=') !'=')+ b=[',' k=kwargs {k}] { + _PyPegen_collect_call_seqs(p, a, b, EXTRA) } | a=kwargs { _PyAST_Call(_PyPegen_dummy_name(p), CHECK_NULL_ALLOWED(asdl_expr_seq*, _PyPegen_seq_extract_starred_exprs(p, a)), CHECK_NULL_ALLOWED(asdl_keyword_seq*, _PyPegen_seq_delete_starred_exprs(p, a)), EXTRA) } + kwargs[asdl_seq*]: | a=','.kwarg_or_starred+ ',' b=','.kwarg_or_double_starred+ { _PyPegen_join_sequences(p, a, b) } | ','.kwarg_or_starred+ @@ -757,15 +759,15 @@ kwargs[asdl_seq*]: starred_expression[expr_ty]: | '*' a=expression { _PyAST_Starred(a, Load, EXTRA) } kwarg_or_starred[KeywordOrStarred*]: + | invalid_kwarg | a=NAME '=' b=expression { _PyPegen_keyword_or_starred(p, CHECK(keyword_ty, _PyAST_keyword(a->v.Name.id, b, EXTRA)), 1) } | a=starred_expression { _PyPegen_keyword_or_starred(p, a, 0) } - | invalid_kwarg kwarg_or_double_starred[KeywordOrStarred*]: + | invalid_kwarg | a=NAME '=' b=expression { _PyPegen_keyword_or_starred(p, CHECK(keyword_ty, _PyAST_keyword(a->v.Name.id, b, EXTRA)), 1) } | '**' a=expression { _PyPegen_keyword_or_starred(p, CHECK(keyword_ty, _PyAST_keyword(NULL, a, EXTRA)), 1) } - | invalid_kwarg # NOTE: star_targets may contain *bitwise_or, targets may not. star_targets[expr_ty]: @@ -838,29 +840,37 @@ invalid_arguments: | a=args ',' '*' { RAISE_SYNTAX_ERROR_KNOWN_LOCATION(a, "iterable argument unpacking follows keyword argument unpacking") } | a=expression b=for_if_clauses ',' [args | expression for_if_clauses] { RAISE_SYNTAX_ERROR_KNOWN_RANGE(a, PyPegen_last_item(b, comprehension_ty)->target, "Generator expression must be parenthesized") } + | a=NAME b='=' expression for_if_clauses { + RAISE_SYNTAX_ERROR_KNOWN_RANGE(a, b, "invalid syntax. Maybe you meant '==' or ':=' instead of '='?")} | a=args for_if_clauses { _PyPegen_nonparen_genexp_in_call(p, a) } | args ',' a=expression b=for_if_clauses { RAISE_SYNTAX_ERROR_KNOWN_RANGE(a, asdl_seq_GET(b, b->size-1)->target, "Generator expression must be parenthesized") } | a=args ',' args { _PyPegen_arguments_parsing_error(p, a) } invalid_kwarg: - | a=expression b='=' { + | a=NAME b='=' expression for_if_clauses { + RAISE_SYNTAX_ERROR_KNOWN_RANGE(a, b, "invalid syntax. Maybe you meant '==' or ':=' instead of '='?")} + | !(NAME '=') a=expression b='=' { RAISE_SYNTAX_ERROR_KNOWN_RANGE( a, b, "expression cannot contain assignment, perhaps you meant \"==\"?") } +expression_without_invalid[expr_ty]: + | a=disjunction 'if' b=disjunction 'else' c=expression { _PyAST_IfExp(b, a, c, EXTRA) } + | disjunction + | lambdef invalid_expression: # !(NAME STRING) is not matched so we don't show this error with some invalid string prefixes like: kf"dsfsdf" # Soft keywords need to also be ignored because they can be parsed as NAME NAME - | !(NAME STRING | SOFT_KEYWORD) a=disjunction b=expression { + | !(NAME STRING | SOFT_KEYWORD) a=disjunction b=expression_without_invalid { RAISE_SYNTAX_ERROR_KNOWN_RANGE(a, b, "invalid syntax. Perhaps you forgot a comma?") } invalid_named_expression: | a=expression ':=' expression { RAISE_SYNTAX_ERROR_KNOWN_LOCATION( a, "cannot use assignment expressions with %s", _PyPegen_get_expr_name(a)) } - | a=NAME '=' b=bitwise_or !('='|':='|',') { - RAISE_SYNTAX_ERROR_KNOWN_RANGE(a, b, "invalid syntax. Maybe you meant '==' or ':=' instead of '='?") } - | !(list|tuple|genexp|'True'|'None'|'False') a=bitwise_or b='=' bitwise_or !('='|':='|',') { - RAISE_SYNTAX_ERROR_KNOWN_LOCATION(a, "cannot assign to %s here. Maybe you meant '==' instead of '='?", + | a=NAME '=' b=bitwise_or !('='|':=') { + p->in_raw_rule ? NULL : RAISE_SYNTAX_ERROR_KNOWN_RANGE(a, b, "invalid syntax. Maybe you meant '==' or ':=' instead of '='?") } + | !(list|tuple|genexp|'True'|'None'|'False') a=bitwise_or b='=' bitwise_or !('='|':=') { + p->in_raw_rule ? NULL : RAISE_SYNTAX_ERROR_KNOWN_LOCATION(a, "cannot assign to %s here. Maybe you meant '==' instead of '='?", _PyPegen_get_expr_name(a)) } invalid_assignment: @@ -954,6 +964,7 @@ invalid_with_stmt_indent: invalid_try_stmt: | a='try' ':' NEWLINE !INDENT { RAISE_INDENTATION_ERROR("expected an indented block after 'try' statement on line %d", a->lineno) } + | 'try' ':' block !('except' | 'finally') { RAISE_SYNTAX_ERROR("expected 'except' or 'finally' block") } invalid_except_stmt: | 'except' a=expression ',' expressions ['as' NAME ] ':' { RAISE_SYNTAX_ERROR_STARTING_FROM(a, "multiple exception types must be parenthesized") } diff --git a/Include/cpython/code.h b/Include/cpython/code.h index 330f1f54d15203..5c0fae47e79f2c 100644 --- a/Include/cpython/code.h +++ b/Include/cpython/code.h @@ -17,31 +17,62 @@ typedef struct _PyOpcache _PyOpcache; /* Bytecode object */ struct PyCodeObject { PyObject_HEAD + + /* Note only the following fields are used in hash and/or comparisons + * + * - co_name + * - co_argcount + * - co_posonlyargcount + * - co_kwonlyargcount + * - co_nlocals + * - co_stacksize + * - co_flags + * - co_firstlineno + * - co_code + * - co_consts + * - co_names + * - co_varnames + * - co_freevars + * - co_cellvars + * + * This is done to preserve the name and line number for tracebacks + * and debuggers; otherwise, constant de-duplication would collapse + * identical functions/lambdas defined on different lines. + */ + + /* These fields are set with provided values on new code objects. */ + + // The hottest fields (in the eval loop) are grouped here at the top. + PyObject *co_code; /* instruction opcodes */ + PyObject *co_consts; /* list (constants used) */ + PyObject *co_names; /* list of strings (names used) */ + int co_flags; /* CO_..., see below */ + // The rest are not so impactful on performance. int co_argcount; /* #arguments, except *args */ int co_posonlyargcount; /* #positional only arguments */ int co_kwonlyargcount; /* #keyword only arguments */ - int co_nlocals; /* #local variables */ int co_stacksize; /* #entries needed for evaluation stack */ - int co_flags; /* CO_..., see below */ int co_firstlineno; /* first source line number */ - PyObject *co_code; /* instruction opcodes */ - PyObject *co_consts; /* list (constants used) */ - PyObject *co_names; /* list of strings (names used) */ PyObject *co_varnames; /* tuple of strings (local variable names) */ - PyObject *co_freevars; /* tuple of strings (free variable names) */ PyObject *co_cellvars; /* tuple of strings (cell variable names) */ - /* The rest aren't used in either hash or comparisons, except for co_name, - used in both. This is done to preserve the name and line number - for tracebacks and debuggers; otherwise, constant de-duplication - would collapse identical functions/lambdas defined on different lines. - */ - Py_ssize_t *co_cell2arg; /* Maps cell vars which are arguments. */ + PyObject *co_freevars; /* tuple of strings (free variable names) */ PyObject *co_filename; /* unicode (where it was loaded from) */ PyObject *co_name; /* unicode (name, for reference) */ PyObject *co_linetable; /* string (encoding addr<->lineno mapping) See Objects/lnotab_notes.txt for details. */ PyObject *co_exceptiontable; /* Byte string encoding exception handling table */ - void *co_zombieframe; /* for optimization only (see frameobject.c) */ + + /* These fields are set with computed values on new code objects. */ + + int *co_cell2arg; /* Maps cell vars which are arguments. */ + // These are redundant but offer some performance benefit. + int co_nlocalsplus; /* number of local + cell + free variables */ + int co_nlocals; /* number of local variables */ + int co_ncellvars; /* number of cell variables */ + int co_nfreevars; /* number of free variables */ + + /* The remaining fields are zeroed out on new code objects. */ + PyObject *co_weakreflist; /* to support weakrefs to code objects */ /* Scratch space for extra data relating to the code object. Type is a void* to keep the format private in codeobject.c to force diff --git a/Include/cpython/dictobject.h b/Include/cpython/dictobject.h index 6822a65cad95e8..de3c1160b489ce 100644 --- a/Include/cpython/dictobject.h +++ b/Include/cpython/dictobject.h @@ -82,3 +82,7 @@ typedef struct { PyAPI_FUNC(PyObject *) _PyDictView_New(PyObject *, PyTypeObject *); PyAPI_FUNC(PyObject *) _PyDictView_Intersect(PyObject* self, PyObject *other); + +/* Gets a version number unique to the current state of the keys of dict, if possible. + * Returns the version number, or zero if it was not possible to get a version number. */ +uint32_t _PyDictKeys_GetVersionForCurrentState(PyDictObject *dict); diff --git a/Include/cpython/frameobject.h b/Include/cpython/frameobject.h index 581664775cdedc..fc20bc2ff89b0c 100644 --- a/Include/cpython/frameobject.h +++ b/Include/cpython/frameobject.h @@ -20,12 +20,9 @@ enum _framestate { typedef signed char PyFrameState; struct _frame { - PyObject_VAR_HEAD + PyObject_HEAD struct _frame *f_back; /* previous frame, or NULL */ PyCodeObject *f_code; /* code segment */ - PyObject *f_builtins; /* builtin symbol table (PyDictObject) */ - PyObject *f_globals; /* global symbol table (PyDictObject) */ - PyObject *f_locals; /* local symbol table (any mapping) */ PyObject **f_valuestack; /* points after the last local */ PyObject *f_trace; /* Trace function */ /* Borrowed reference to a generator, or NULL */ @@ -36,7 +33,8 @@ struct _frame { PyFrameState f_state; /* What state the frame is in */ char f_trace_lines; /* Emit per-line trace events? */ char f_trace_opcodes; /* Emit per-opcode trace events? */ - PyObject *f_localsplus[1]; /* locals+stack, dynamically sized */ + char f_own_locals_memory; /* This frame owns the memory for the locals */ + PyObject **f_localsptr; /* Pointer to locals, cells, free */ }; static inline int _PyFrame_IsRunnable(struct _frame *f) { @@ -62,7 +60,7 @@ PyAPI_FUNC(PyFrameObject *) PyFrame_New(PyThreadState *, PyCodeObject *, /* only internal use */ PyFrameObject* -_PyFrame_New_NoTrack(PyThreadState *, PyFrameConstructor *, PyObject *); +_PyFrame_New_NoTrack(PyThreadState *, PyFrameConstructor *, PyObject *, PyObject **); /* The rest of the interface is specific for frame objects */ diff --git a/Include/cpython/pylifecycle.h b/Include/cpython/pylifecycle.h index 6fe46a544016f4..bb5b07ef5901c8 100644 --- a/Include/cpython/pylifecycle.h +++ b/Include/cpython/pylifecycle.h @@ -2,6 +2,10 @@ # error "this header file must not be included directly" #endif +/* Py_FrozenMain is kept out of the Limited API until documented and present + in all builds of Python */ +PyAPI_FUNC(int) Py_FrozenMain(int argc, char **argv); + /* Only used by applications that embed the interpreter and need to * override the standard encoding determination mechanism */ diff --git a/Include/cpython/pystate.h b/Include/cpython/pystate.h index e3ccc543560849..1049eda3e51609 100644 --- a/Include/cpython/pystate.h +++ b/Include/cpython/pystate.h @@ -57,6 +57,12 @@ typedef struct _err_stackitem { } _PyErr_StackItem; +typedef struct _stack_chunk { + struct _stack_chunk *previous; + size_t size; + size_t top; + PyObject * data[1]; /* Variable sized */ +} _PyStackChunk; // The PyThreadState typedef is in Include/pystate.h. struct _ts { @@ -107,6 +113,12 @@ struct _ts { PyObject *async_exc; /* Asynchronous exception to raise */ unsigned long thread_id; /* Thread id where this tstate was created */ + /* Native thread id where this tstate was created. This will be 0 except on + * those platforms that have the notion of native thread id, for which the + * macro PY_HAVE_THREAD_NATIVE_ID is then defined. + */ + unsigned long native_thread_id; + int trash_delete_nesting; PyObject *trash_delete_later; @@ -149,6 +161,9 @@ struct _ts { CFrame root_cframe; + _PyStackChunk *datastack_chunk; + PyObject **datastack_top; + PyObject **datastack_limit; /* XXX signal handlers should also be here */ }; diff --git a/Include/errcode.h b/Include/errcode.h index f2671d6c9b30b4..2e07fc2c963280 100644 --- a/Include/errcode.h +++ b/Include/errcode.h @@ -28,6 +28,7 @@ extern "C" { #define E_DECODE 22 /* Error in decoding into Unicode */ #define E_LINECONT 25 /* Unexpected characters after a line continuation */ #define E_BADSINGLE 27 /* Ill-formed single statement input */ +#define E_INTERACT_STOP 28 /* Interactive mode stopped tokenization */ #ifdef __cplusplus } diff --git a/Include/genobject.h b/Include/genobject.h index e965334a0140c8..094d4e14fbe7cf 100644 --- a/Include/genobject.h +++ b/Include/genobject.h @@ -18,7 +18,7 @@ extern "C" { /* Note: gi_frame can be NULL if the generator is "finished" */ \ PyFrameObject *prefix##_frame; \ /* The code object backing the generator */ \ - PyObject *prefix##_code; \ + PyCodeObject *prefix##_code; \ /* List of weak reference. */ \ PyObject *prefix##_weakreflist; \ /* Name of the generator. */ \ diff --git a/Include/internal/pycore_code.h b/Include/internal/pycore_code.h index f1e89d96b9ebb0..dab6c34dd1732d 100644 --- a/Include/internal/pycore_code.h +++ b/Include/internal/pycore_code.h @@ -3,7 +3,8 @@ #ifdef __cplusplus extern "C" { #endif - + + typedef struct { PyObject *ptr; /* Cached pointer (borrowed reference) */ uint64_t globals_ver; /* ma_version of global dict */ @@ -24,7 +25,54 @@ struct _PyOpcache { char optimized; }; + +struct _PyCodeConstructor { + /* metadata */ + PyObject *filename; + PyObject *name; + int flags; + + /* the code */ + PyObject *code; + int firstlineno; + PyObject *linetable; + + /* used by the code */ + PyObject *consts; + PyObject *names; + + /* mapping frame offsets to information */ + PyObject *varnames; + PyObject *cellvars; + PyObject *freevars; + + /* args (within varnames) */ + int argcount; + int posonlyargcount; + int kwonlyargcount; + + /* needed to create the frame */ + int stacksize; + + /* used by the eval loop */ + PyObject *exceptiontable; +}; + +// Using an "arguments struct" like this is helpful for maintainability +// in a case such as this with many parameters. It does bear a risk: +// if the struct changes and callers are not updated properly then the +// compiler will not catch problems (like a missing argument). This can +// cause hard-to-debug problems. The risk is mitigated by the use of +// check_code() in codeobject.c. However, we may decide to switch +// back to a regular function signature. Regardless, this approach +// wouldn't be appropriate if this weren't a strictly internal API. +// (See the comments in https://github.com/python/cpython/pull/26258.) +PyAPI_FUNC(int) _PyCode_Validate(struct _PyCodeConstructor *); +PyAPI_FUNC(PyCodeObject *) _PyCode_New(struct _PyCodeConstructor *); + + /* Private API */ + int _PyCode_InitOpcache(PyCodeObject *co); diff --git a/Include/internal/pycore_condvar.h b/Include/internal/pycore_condvar.h index 8b89d709510a33..edb7dc8193cb8a 100644 --- a/Include/internal/pycore_condvar.h +++ b/Include/internal/pycore_condvar.h @@ -60,7 +60,7 @@ typedef CRITICAL_SECTION PyMUTEX_T; with a Semaphore. Semaphores are available on Windows XP (2003 server) and later. We use a Semaphore rather than an auto-reset event, because although - an auto-resent event might appear to solve the lost-wakeup bug (race + an auto-reset event might appear to solve the lost-wakeup bug (race condition between releasing the outer lock and waiting) because it maintains state even though a wait hasn't happened, there is still a lost wakeup problem if more than one thread are interrupted in the diff --git a/Include/internal/pycore_frame.h b/Include/internal/pycore_frame.h new file mode 100644 index 00000000000000..44f58fb6948712 --- /dev/null +++ b/Include/internal/pycore_frame.h @@ -0,0 +1,38 @@ +#ifndef Py_INTERNAL_FRAME_H +#define Py_INTERNAL_FRAME_H +#ifdef __cplusplus +extern "C" { +#endif + +enum { + FRAME_SPECIALS_GLOBALS_OFFSET = 0, + FRAME_SPECIALS_BUILTINS_OFFSET = 1, + FRAME_SPECIALS_LOCALS_OFFSET = 2, + FRAME_SPECIALS_SIZE = 3 +}; + +static inline PyObject ** +_PyFrame_Specials(PyFrameObject *f) { + return &f->f_valuestack[-FRAME_SPECIALS_SIZE]; +} + +/* Returns a *borrowed* reference. */ +static inline PyObject * +_PyFrame_GetGlobals(PyFrameObject *f) +{ + return _PyFrame_Specials(f)[FRAME_SPECIALS_GLOBALS_OFFSET]; +} + +/* Returns a *borrowed* reference. */ +static inline PyObject * +_PyFrame_GetBuiltins(PyFrameObject *f) +{ + return _PyFrame_Specials(f)[FRAME_SPECIALS_BUILTINS_OFFSET]; +} + +int _PyFrame_TakeLocals(PyFrameObject *f); + +#ifdef __cplusplus +} +#endif +#endif /* !Py_INTERNAL_FRAME_H */ diff --git a/Include/internal/pycore_pymem.h b/Include/internal/pycore_pymem.h index e4e35c16ce8eda..30db3f2b6701ec 100644 --- a/Include/internal/pycore_pymem.h +++ b/Include/internal/pycore_pymem.h @@ -42,7 +42,7 @@ PyAPI_FUNC(int) _PyMem_SetDefaultAllocator( fills newly allocated memory with CLEANBYTE (0xCD) and newly freed memory with DEADBYTE (0xDD). Detect also "untouchable bytes" marked with FORBIDDENBYTE (0xFD). */ -static inline int _PyMem_IsPtrFreed(void *ptr) +static inline int _PyMem_IsPtrFreed(const void *ptr) { uintptr_t value = (uintptr_t)ptr; #if SIZEOF_VOID_P == 8 @@ -94,6 +94,11 @@ struct _PyTraceMalloc_Config { PyAPI_DATA(struct _PyTraceMalloc_Config) _Py_tracemalloc_config; +/* Allocate memory directly from the O/S virtual memory system, + * where supported. Otherwise fallback on malloc */ +void *_PyObject_VirtualAlloc(size_t size); +void _PyObject_VirtualFree(void *, size_t size); + #ifdef __cplusplus } diff --git a/Include/internal/pycore_pystate.h b/Include/internal/pycore_pystate.h index 4b894f3eff4967..6601ce2f80bf7e 100644 --- a/Include/internal/pycore_pystate.h +++ b/Include/internal/pycore_pystate.h @@ -147,6 +147,9 @@ PyAPI_FUNC(int) _PyState_AddModule( PyAPI_FUNC(int) _PyOS_InterruptOccurred(PyThreadState *tstate); +PyObject **_PyThreadState_PushLocals(PyThreadState *, int size); +void _PyThreadState_PopLocals(PyThreadState *, PyObject **); + #ifdef __cplusplus } #endif diff --git a/Include/internal/pycore_tuple.h b/Include/internal/pycore_tuple.h index 5353e18d08327d..d1d0d2a92e49fb 100644 --- a/Include/internal/pycore_tuple.h +++ b/Include/internal/pycore_tuple.h @@ -12,7 +12,7 @@ extern "C" { #define _PyTuple_ITEMS(op) (_PyTuple_CAST(op)->ob_item) -PyAPI_FUNC(PyObject *) _PyTuple_FromArray(PyObject *const *, Py_ssize_t); +extern PyObject *_PyTuple_FromArray(PyObject *const *, Py_ssize_t); #ifdef __cplusplus } diff --git a/Include/object.h b/Include/object.h index 4c069998574b4a..a3b5d0d29d3e7b 100644 --- a/Include/object.h +++ b/Include/object.h @@ -134,10 +134,16 @@ static inline Py_ssize_t _Py_REFCNT(const PyObject *ob) { // bpo-39573: The Py_SET_TYPE() function must be used to set an object type. -#define Py_TYPE(ob) (_PyObject_CAST(ob)->ob_type) +static inline PyTypeObject* _Py_TYPE(const PyObject *ob) { + return ob->ob_type; +} +#define Py_TYPE(ob) _Py_TYPE(_PyObject_CAST_CONST(ob)) // bpo-39573: The Py_SET_SIZE() function must be used to set an object size. -#define Py_SIZE(ob) (_PyVarObject_CAST(ob)->ob_size) +static inline Py_ssize_t _Py_SIZE(const PyVarObject *ob) { + return ob->ob_size; +} +#define Py_SIZE(ob) _Py_SIZE(_PyVarObject_CAST_CONST(ob)) static inline int _Py_IS_TYPE(const PyObject *ob, const PyTypeObject *type) { diff --git a/Include/pylifecycle.h b/Include/pylifecycle.h index 7925eafc660474..9b2dd0868eb25b 100644 --- a/Include/pylifecycle.h +++ b/Include/pylifecycle.h @@ -31,9 +31,6 @@ PyAPI_FUNC(void) _Py_NO_RETURN Py_Exit(int); /* Bootstrap __main__ (defined in Modules/main.c) */ PyAPI_FUNC(int) Py_Main(int argc, wchar_t **argv); - -PyAPI_FUNC(int) Py_FrozenMain(int argc, char **argv); - PyAPI_FUNC(int) Py_BytesMain(int argc, char **argv); /* In pathconfig.c */ diff --git a/Lib/configparser.py b/Lib/configparser.py index 3b4cb5e6b2407f..2f45e242b49426 100644 --- a/Lib/configparser.py +++ b/Lib/configparser.py @@ -908,6 +908,9 @@ def write(self, fp, space_around_delimiters=True): If `space_around_delimiters' is True (the default), delimiters between keys and values are surrounded by spaces. + + Please note that comments in the original configuration file are not + preserved when writing the configuration back. """ if space_around_delimiters: d = " {} ".format(self._delimiters[0]) @@ -1006,7 +1009,7 @@ def _read(self, fp, fpname): Configuration files may include comments, prefixed by specific characters (`#' and `;' by default). Comments may appear on their own in an otherwise empty line or may be entered in lines holding values or - section names. + section names. Please note that comments get stripped off when reading configuration files. """ elements_added = set() cursect = None # None, or a dictionary diff --git a/Lib/ctypes/test/test_values.py b/Lib/ctypes/test/test_values.py index 80f4ed026f963c..92c3c284f2359c 100644 --- a/Lib/ctypes/test/test_values.py +++ b/Lib/ctypes/test/test_values.py @@ -80,9 +80,9 @@ class struct_frozen(Structure): continue items.append((entry.name.decode("ascii"), entry.size)) - expected = [("__hello__", 142), - ("__phello__", -142), - ("__phello__.spam", 142), + expected = [("__hello__", 138), + ("__phello__", -138), + ("__phello__.spam", 138), ] self.assertEqual(items, expected, "PyImport_FrozenModules example " "in Doc/library/ctypes.rst may be out of date") diff --git a/Lib/dataclasses.py b/Lib/dataclasses.py index cbba320e01a54d..79739976d3d29b 100644 --- a/Lib/dataclasses.py +++ b/Lib/dataclasses.py @@ -1038,7 +1038,7 @@ def _process_class(cls, init, repr, eq, order, unsafe_hash, frozen, _set_new_attribute(cls, '__repr__', _repr_fn(flds, globals)) if eq: - # Create _eq__ method. There's no need for a __ne__ method, + # Create __eq__ method. There's no need for a __ne__ method, # since python will call __eq__ and negate it. flds = [f for f in field_list if f.compare] self_tuple = _tuple_str('self', flds) diff --git a/Lib/distutils/command/install.py b/Lib/distutils/command/install.py index d33a889afe4a81..26696cfb9dcf9c 100644 --- a/Lib/distutils/command/install.py +++ b/Lib/distutils/command/install.py @@ -37,12 +37,19 @@ # Copy from sysconfig._INSTALL_SCHEMES for key in SCHEME_KEYS: - sys_key = key - if key == "headers": - sys_key = "include" - INSTALL_SCHEMES["unix_prefix"][key] = sysconfig._INSTALL_SCHEMES["posix_prefix"][sys_key] - INSTALL_SCHEMES["unix_home"][key] = sysconfig._INSTALL_SCHEMES["posix_home"][sys_key] - INSTALL_SCHEMES["nt"][key] = sysconfig._INSTALL_SCHEMES["nt"][sys_key] + for distutils_scheme_name, sys_scheme_name in ( + ("unix_prefix", "posix_prefix"), ("unix_home", "posix_home"), + ("nt", "nt")): + sys_key = key + sys_scheme = sysconfig._INSTALL_SCHEMES[sys_scheme_name] + if key == "headers" and key not in sys_scheme: + # On POSIX-y platofrms, Python will: + # - Build from .h files in 'headers' (only there when + # building CPython) + # - Install .h files to 'include' + # When 'headers' is missing, fall back to 'include' + sys_key = 'include' + INSTALL_SCHEMES[distutils_scheme_name][key] = sys_scheme[sys_key] # Transformation to different template format for main_key in INSTALL_SCHEMES: @@ -316,6 +323,9 @@ def finalize_options(self): self.config_vars['userbase'] = self.install_userbase self.config_vars['usersite'] = self.install_usersite + if sysconfig.is_python_build(True): + self.config_vars['srcdir'] = sysconfig.get_config_var('srcdir') + self.expand_basedirs() self.dump_dirs("post-expand_basedirs()") diff --git a/Lib/fractions.py b/Lib/fractions.py index 64a8959d7d48e5..66e6831a5c7bcc 100644 --- a/Lib/fractions.py +++ b/Lib/fractions.py @@ -1,7 +1,7 @@ # Originally contributed by Sjoerd Mullender. # Significantly modified by Jeffrey Yasskin . -"""Fraction, infinite-precision, real numbers.""" +"""Fraction, infinite-precision, rational numbers.""" from decimal import Decimal import math diff --git a/Lib/gettext.py b/Lib/gettext.py index 77b67aef4204c9..6c5ec4e517f637 100644 --- a/Lib/gettext.py +++ b/Lib/gettext.py @@ -52,11 +52,10 @@ __all__ = ['NullTranslations', 'GNUTranslations', 'Catalog', - 'find', 'translation', 'install', 'textdomain', 'bindtextdomain', - 'bind_textdomain_codeset', - 'dgettext', 'dngettext', 'gettext', 'lgettext', 'ldgettext', - 'ldngettext', 'lngettext', 'ngettext', - 'pgettext', 'dpgettext', 'npgettext', 'dnpgettext', + 'bindtextdomain', 'find', 'translation', 'install', + 'textdomain', 'dgettext', 'dngettext', 'gettext', + 'ngettext', 'pgettext', 'dpgettext', 'npgettext', + 'dnpgettext' ] _default_localedir = os.path.join(sys.base_prefix, 'share', 'locale') @@ -83,6 +82,7 @@ (?P\w+|.) # invalid token """, re.VERBOSE|re.DOTALL) + def _tokenize(plural): for mo in re.finditer(_token_pattern, plural): kind = mo.lastgroup @@ -94,12 +94,14 @@ def _tokenize(plural): yield value yield '' + def _error(value): if value: return ValueError('unexpected token in plural form: %s' % value) else: return ValueError('unexpected end of plural form') + _binary_ops = ( ('||',), ('&&',), @@ -111,6 +113,7 @@ def _error(value): _binary_ops = {op: i for i, ops in enumerate(_binary_ops, 1) for op in ops} _c2py_ops = {'||': 'or', '&&': 'and', '/': '//'} + def _parse(tokens, priority=-1): result = '' nexttok = next(tokens) @@ -160,6 +163,7 @@ def _parse(tokens, priority=-1): return result, nexttok + def _as_int(n): try: i = round(n) @@ -172,6 +176,7 @@ def _as_int(n): DeprecationWarning, 4) return n + def c2py(plural): """Gets a C expression as used in PO files for plural forms and returns a Python function that implements an equivalent expression. @@ -250,12 +255,10 @@ def _expand_lang(loc): return ret - class NullTranslations: def __init__(self, fp=None): self._info = {} self._charset = None - self._output_charset = None self._fallback = None if fp is not None: self._parse(fp) @@ -274,20 +277,6 @@ def gettext(self, message): return self._fallback.gettext(message) return message - def lgettext(self, message): - import warnings - warnings.warn('lgettext() is deprecated, use gettext() instead', - DeprecationWarning, 2) - import locale - if self._fallback: - with warnings.catch_warnings(): - warnings.filterwarnings('ignore', r'.*\blgettext\b.*', - DeprecationWarning) - return self._fallback.lgettext(message) - if self._output_charset: - return message.encode(self._output_charset) - return message.encode(locale.getpreferredencoding()) - def ngettext(self, msgid1, msgid2, n): if self._fallback: return self._fallback.ngettext(msgid1, msgid2, n) @@ -296,24 +285,6 @@ def ngettext(self, msgid1, msgid2, n): else: return msgid2 - def lngettext(self, msgid1, msgid2, n): - import warnings - warnings.warn('lngettext() is deprecated, use ngettext() instead', - DeprecationWarning, 2) - import locale - if self._fallback: - with warnings.catch_warnings(): - warnings.filterwarnings('ignore', r'.*\blngettext\b.*', - DeprecationWarning) - return self._fallback.lngettext(msgid1, msgid2, n) - if n == 1: - tmsg = msgid1 - else: - tmsg = msgid2 - if self._output_charset: - return tmsg.encode(self._output_charset) - return tmsg.encode(locale.getpreferredencoding()) - def pgettext(self, context, message): if self._fallback: return self._fallback.pgettext(context, message) @@ -333,24 +304,11 @@ def info(self): def charset(self): return self._charset - def output_charset(self): - import warnings - warnings.warn('output_charset() is deprecated', - DeprecationWarning, 2) - return self._output_charset - - def set_output_charset(self, charset): - import warnings - warnings.warn('set_output_charset() is deprecated', - DeprecationWarning, 2) - self._output_charset = charset - def install(self, names=None): import builtins builtins.__dict__['_'] = self.gettext if names is not None: - allowed = {'gettext', 'lgettext', 'lngettext', - 'ngettext', 'npgettext', 'pgettext'} + allowed = {'gettext', 'ngettext', 'npgettext', 'pgettext'} for name in allowed & set(names): builtins.__dict__[name] = getattr(self, name) @@ -460,39 +418,6 @@ def _parse(self, fp): masteridx += 8 transidx += 8 - def lgettext(self, message): - import warnings - warnings.warn('lgettext() is deprecated, use gettext() instead', - DeprecationWarning, 2) - import locale - missing = object() - tmsg = self._catalog.get(message, missing) - if tmsg is missing: - if self._fallback: - return self._fallback.lgettext(message) - tmsg = message - if self._output_charset: - return tmsg.encode(self._output_charset) - return tmsg.encode(locale.getpreferredencoding()) - - def lngettext(self, msgid1, msgid2, n): - import warnings - warnings.warn('lngettext() is deprecated, use ngettext() instead', - DeprecationWarning, 2) - import locale - try: - tmsg = self._catalog[(msgid1, self.plural(n))] - except KeyError: - if self._fallback: - return self._fallback.lngettext(msgid1, msgid2, n) - if n == 1: - tmsg = msgid1 - else: - tmsg = msgid2 - if self._output_charset: - return tmsg.encode(self._output_charset) - return tmsg.encode(locale.getpreferredencoding()) - def gettext(self, message): missing = object() tmsg = self._catalog.get(message, missing) @@ -575,13 +500,12 @@ def find(domain, localedir=None, languages=None, all=False): return result - # a mapping between absolute .mo file path and Translation object _translations = {} -_unspecified = ['unspecified'] + def translation(domain, localedir=None, languages=None, - class_=None, fallback=False, codeset=_unspecified): + class_=None, fallback=False): if class_ is None: class_ = GNUTranslations mofiles = find(domain, localedir, languages, all=True) @@ -607,15 +531,6 @@ def translation(domain, localedir=None, languages=None, # are not used. import copy t = copy.copy(t) - if codeset is not _unspecified: - import warnings - warnings.warn('parameter codeset is deprecated', - DeprecationWarning, 2) - if codeset: - with warnings.catch_warnings(): - warnings.filterwarnings('ignore', r'.*\bset_output_charset\b.*', - DeprecationWarning) - t.set_output_charset(codeset) if result is None: result = t else: @@ -623,16 +538,13 @@ def translation(domain, localedir=None, languages=None, return result -def install(domain, localedir=None, codeset=_unspecified, names=None): - t = translation(domain, localedir, fallback=True, codeset=codeset) +def install(domain, localedir=None, *, names=None): + t = translation(domain, localedir, fallback=True) t.install(names) - # a mapping b/w domains and locale directories _localedirs = {} -# a mapping b/w domains and codesets -_localecodesets = {} # current global domain, `messages' used for compatibility w/ GNU gettext _current_domain = 'messages' @@ -651,16 +563,6 @@ def bindtextdomain(domain, localedir=None): return _localedirs.get(domain, _default_localedir) -def bind_textdomain_codeset(domain, codeset=None): - import warnings - warnings.warn('bind_textdomain_codeset() is deprecated', - DeprecationWarning, 2) - global _localecodesets - if codeset is not None: - _localecodesets[domain] = codeset - return _localecodesets.get(domain) - - def dgettext(domain, message): try: t = translation(domain, _localedirs.get(domain, None)) @@ -668,23 +570,6 @@ def dgettext(domain, message): return message return t.gettext(message) -def ldgettext(domain, message): - import warnings - warnings.warn('ldgettext() is deprecated, use dgettext() instead', - DeprecationWarning, 2) - import locale - codeset = _localecodesets.get(domain) - try: - with warnings.catch_warnings(): - warnings.filterwarnings('ignore', r'.*\bparameter codeset\b.*', - DeprecationWarning) - t = translation(domain, _localedirs.get(domain, None), codeset=codeset) - except OSError: - return message.encode(codeset or locale.getpreferredencoding()) - with warnings.catch_warnings(): - warnings.filterwarnings('ignore', r'.*\blgettext\b.*', - DeprecationWarning) - return t.lgettext(message) def dngettext(domain, msgid1, msgid2, n): try: @@ -696,28 +581,6 @@ def dngettext(domain, msgid1, msgid2, n): return msgid2 return t.ngettext(msgid1, msgid2, n) -def ldngettext(domain, msgid1, msgid2, n): - import warnings - warnings.warn('ldngettext() is deprecated, use dngettext() instead', - DeprecationWarning, 2) - import locale - codeset = _localecodesets.get(domain) - try: - with warnings.catch_warnings(): - warnings.filterwarnings('ignore', r'.*\bparameter codeset\b.*', - DeprecationWarning) - t = translation(domain, _localedirs.get(domain, None), codeset=codeset) - except OSError: - if n == 1: - tmsg = msgid1 - else: - tmsg = msgid2 - return tmsg.encode(codeset or locale.getpreferredencoding()) - with warnings.catch_warnings(): - warnings.filterwarnings('ignore', r'.*\blngettext\b.*', - DeprecationWarning) - return t.lngettext(msgid1, msgid2, n) - def dpgettext(domain, context, message): try: @@ -741,27 +604,10 @@ def dnpgettext(domain, context, msgid1, msgid2, n): def gettext(message): return dgettext(_current_domain, message) -def lgettext(message): - import warnings - warnings.warn('lgettext() is deprecated, use gettext() instead', - DeprecationWarning, 2) - with warnings.catch_warnings(): - warnings.filterwarnings('ignore', r'.*\bldgettext\b.*', - DeprecationWarning) - return ldgettext(_current_domain, message) def ngettext(msgid1, msgid2, n): return dngettext(_current_domain, msgid1, msgid2, n) -def lngettext(msgid1, msgid2, n): - import warnings - warnings.warn('lngettext() is deprecated, use ngettext() instead', - DeprecationWarning, 2) - with warnings.catch_warnings(): - warnings.filterwarnings('ignore', r'.*\bldngettext\b.*', - DeprecationWarning) - return ldngettext(_current_domain, msgid1, msgid2, n) - def pgettext(context, message): return dpgettext(_current_domain, context, message) diff --git a/Lib/idlelib/autocomplete_w.py b/Lib/idlelib/autocomplete_w.py index d3d1e6982bfb2e..2e3f9c14a19814 100644 --- a/Lib/idlelib/autocomplete_w.py +++ b/Lib/idlelib/autocomplete_w.py @@ -242,31 +242,46 @@ def winconfig_event(self, event): self.is_configuring = True if not self.is_active(): return - # Position the completion list window - text = self.widget - text.see(self.startindex) - x, y, cx, cy = text.bbox(self.startindex) - acw = self.autocompletewindow - acw.update() - acw_width, acw_height = acw.winfo_width(), acw.winfo_height() - text_width, text_height = text.winfo_width(), text.winfo_height() - new_x = text.winfo_rootx() + min(x, max(0, text_width - acw_width)) - new_y = text.winfo_rooty() + y - if (text_height - (y + cy) >= acw_height # enough height below - or y < acw_height): # not enough height above - # place acw below current line - new_y += cy - else: - # place acw above current line - new_y -= acw_height - acw.wm_geometry("+%d+%d" % (new_x, new_y)) - acw.update_idletasks() + + # Since the event may occur after the completion window is gone, + # catch potential TclError exceptions when accessing acw. See: bpo-41611. + try: + # Position the completion list window + text = self.widget + text.see(self.startindex) + x, y, cx, cy = text.bbox(self.startindex) + acw = self.autocompletewindow + if platform.system().startswith('Windows'): + # On Windows an update() call is needed for the completion + # list window to be created, so that we can fetch its width + # and height. However, this is not needed on other platforms + # (tested on Ubuntu and macOS) but at one point began + # causing freezes on macOS. See issues 37849 and 41611. + acw.update() + acw_width, acw_height = acw.winfo_width(), acw.winfo_height() + text_width, text_height = text.winfo_width(), text.winfo_height() + new_x = text.winfo_rootx() + min(x, max(0, text_width - acw_width)) + new_y = text.winfo_rooty() + y + if (text_height - (y + cy) >= acw_height # enough height below + or y < acw_height): # not enough height above + # place acw below current line + new_y += cy + else: + # place acw above current line + new_y -= acw_height + acw.wm_geometry("+%d+%d" % (new_x, new_y)) + acw.update_idletasks() + except TclError: + pass if platform.system().startswith('Windows'): - # See issue 15786. When on Windows platform, Tk will misbehave + # See issue 15786. When on Windows platform, Tk will misbehave # to call winconfig_event multiple times, we need to prevent this, # otherwise mouse button double click will not be able to used. - acw.unbind(WINCONFIG_SEQUENCE, self.winconfigid) + try: + acw.unbind(WINCONFIG_SEQUENCE, self.winconfigid) + except TclError: + pass self.winconfigid = None self.is_configuring = False diff --git a/Lib/idlelib/colorizer.py b/Lib/idlelib/colorizer.py index 3c527409731afa..e9f19c145c8673 100644 --- a/Lib/idlelib/colorizer.py +++ b/Lib/idlelib/colorizer.py @@ -16,6 +16,32 @@ def any(name, alternates): def make_pat(): kw = r"\b" + any("KEYWORD", keyword.kwlist) + r"\b" + match_softkw = ( + r"^[ \t]*" + # at beginning of line + possible indentation + r"(?Pmatch)\b" + + r"(?![ \t]*(?:" + "|".join([ # not followed by ... + r"[:,;=^&|@~)\]}]", # a character which means it can't be a + # pattern-matching statement + r"\b(?:" + r"|".join(keyword.kwlist) + r")\b", # a keyword + ]) + + r"))" + ) + case_default = ( + r"^[ \t]*" + # at beginning of line + possible indentation + r"(?Pcase)" + + r"[ \t]+(?P_\b)" + ) + case_softkw_and_pattern = ( + r"^[ \t]*" + # at beginning of line + possible indentation + r"(?Pcase)\b" + + r"(?![ \t]*(?:" + "|".join([ # not followed by ... + r"_\b", # a lone underscore + r"[:,;=^&|@~)\]}]", # a character which means it can't be a + # pattern-matching case + r"\b(?:" + r"|".join(keyword.kwlist) + r")\b", # a keyword + ]) + + r"))" + ) builtinlist = [str(name) for name in dir(builtins) if not name.startswith('_') and name not in keyword.kwlist] @@ -27,12 +53,29 @@ def make_pat(): sq3string = stringprefix + r"'''[^'\\]*((\\.|'(?!''))[^'\\]*)*(''')?" dq3string = stringprefix + r'"""[^"\\]*((\\.|"(?!""))[^"\\]*)*(""")?' string = any("STRING", [sq3string, dq3string, sqstring, dqstring]) - return (kw + "|" + builtin + "|" + comment + "|" + string + - "|" + any("SYNC", [r"\n"])) + prog = re.compile("|".join([ + builtin, comment, string, kw, + match_softkw, case_default, + case_softkw_and_pattern, + any("SYNC", [r"\n"]), + ]), + re.DOTALL | re.MULTILINE) + return prog -prog = re.compile(make_pat(), re.S) -idprog = re.compile(r"\s+(\w+)", re.S) +prog = make_pat() +idprog = re.compile(r"\s+(\w+)") +prog_group_name_to_tag = { + "MATCH_SOFTKW": "KEYWORD", + "CASE_SOFTKW": "KEYWORD", + "CASE_DEFAULT_UNDERSCORE": "KEYWORD", + "CASE_SOFTKW2": "KEYWORD", +} + + +def matched_named_groups(re_match): + "Get only the non-empty named groups from an re.Match object." + return ((k, v) for (k, v) in re_match.groupdict().items() if v) def color_config(text): @@ -231,14 +274,10 @@ def recolorize(self): def recolorize_main(self): "Evaluate text and apply colorizing tags." next = "1.0" - while True: - item = self.tag_nextrange("TODO", next) - if not item: - break - head, tail = item - self.tag_remove("SYNC", head, tail) - item = self.tag_prevrange("SYNC", head) - head = item[1] if item else "1.0" + while todo_tag_range := self.tag_nextrange("TODO", next): + self.tag_remove("SYNC", todo_tag_range[0], todo_tag_range[1]) + sync_tag_range = self.tag_prevrange("SYNC", todo_tag_range[0]) + head = sync_tag_range[1] if sync_tag_range else "1.0" chars = "" next = head @@ -256,23 +295,8 @@ def recolorize_main(self): return for tag in self.tagdefs: self.tag_remove(tag, mark, next) - chars = chars + line - m = self.prog.search(chars) - while m: - for key, value in m.groupdict().items(): - if value: - a, b = m.span(key) - self.tag_add(key, - head + "+%dc" % a, - head + "+%dc" % b) - if value in ("def", "class"): - m1 = self.idprog.match(chars, b) - if m1: - a, b = m1.span(1) - self.tag_add("DEFINITION", - head + "+%dc" % a, - head + "+%dc" % b) - m = self.prog.search(chars, m.end()) + chars += line + self._add_tags_in_section(chars, head) if "SYNC" in self.tag_names(next + "-1c"): head = next chars = "" @@ -291,6 +315,40 @@ def recolorize_main(self): if DEBUG: print("colorizing stopped") return + def _add_tag(self, start, end, head, matched_group_name): + """Add a tag to a given range in the text widget. + + This is a utility function, receiving the range as `start` and + `end` positions, each of which is a number of characters + relative to the given `head` index in the text widget. + + The tag to add is determined by `matched_group_name`, which is + the name of a regular expression "named group" as matched by + by the relevant highlighting regexps. + """ + tag = prog_group_name_to_tag.get(matched_group_name, + matched_group_name) + self.tag_add(tag, + f"{head}+{start:d}c", + f"{head}+{end:d}c") + + def _add_tags_in_section(self, chars, head): + """Parse and add highlighting tags to a given part of the text. + + `chars` is a string with the text to parse and to which + highlighting is to be applied. + + `head` is the index in the text widget where the text is found. + """ + for m in self.prog.finditer(chars): + for name, matched_text in matched_named_groups(m): + a, b = m.span(name) + self._add_tag(a, b, head, name) + if matched_text in ("def", "class"): + if m1 := self.idprog.match(chars, b): + a, b = m1.span(1) + self._add_tag(a, b, head, "DEFINITION") + def removecolors(self): "Remove all colorizing tags." for tag in self.tagdefs: @@ -299,27 +357,14 @@ def removecolors(self): def _color_delegator(parent): # htest # from tkinter import Toplevel, Text + from idlelib.idle_test.test_colorizer import source from idlelib.percolator import Percolator top = Toplevel(parent) top.title("Test ColorDelegator") x, y = map(int, parent.geometry().split('+')[1:]) - top.geometry("700x250+%d+%d" % (x + 20, y + 175)) - source = ( - "if True: int ('1') # keyword, builtin, string, comment\n" - "elif False: print(0)\n" - "else: float(None)\n" - "if iF + If + IF: 'keyword matching must respect case'\n" - "if'': x or'' # valid keyword-string no-space combinations\n" - "async def f(): await g()\n" - "# All valid prefixes for unicode and byte strings should be colored.\n" - "'x', '''x''', \"x\", \"\"\"x\"\"\"\n" - "r'x', u'x', R'x', U'x', f'x', F'x'\n" - "fr'x', Fr'x', fR'x', FR'x', rf'x', rF'x', Rf'x', RF'x'\n" - "b'x',B'x', br'x',Br'x',bR'x',BR'x', rb'x', rB'x',Rb'x',RB'x'\n" - "# Invalid combinations of legal characters should be half colored.\n" - "ur'x', ru'x', uf'x', fu'x', UR'x', ufr'x', rfu'x', xf'x', fx'x'\n" - ) + top.geometry("700x550+%d+%d" % (x + 20, y + 175)) + text = Text(top, background="white") text.pack(expand=1, fill="both") text.insert("insert", source) diff --git a/Lib/idlelib/help.html b/Lib/idlelib/help.html index e80384b7775222..19041c6054e4cc 100644 --- a/Lib/idlelib/help.html +++ b/Lib/idlelib/help.html @@ -5,7 +5,7 @@ - IDLE — Python 3.10.0a6 documentation + IDLE — Python 3.11.0a0 documentation @@ -18,7 +18,7 @@ @@ -71,7 +71,7 @@

Navigation

  • - 3.10.0a6 Documentation » + 3.11.0a0 Documentation »
  • @@ -102,7 +102,7 @@

    Navigation

    IDLE

    -

    Source code: Lib/idlelib/

    +

    Source code: Lib/idlelib/


    IDLE is Python’s Integrated Development and Learning Environment.

    IDLE has the following features:

    @@ -581,6 +581,11 @@

    Text colorsclass and def, strings, and comments. For any text window, these are the cursor (when present), found text (when possible), and selected text.

    +

    IDLE also highlights the soft keywords match, +case, and _ in +pattern-matching statements. However, this highlighting is not perfect and +will be incorrect in some rare cases, including some _-s in case +patterns.

    Text coloring is done in the background, so uncolorized text is occasionally visible. To change the color scheme, use the Configure IDLE dialog Highlighting tab. The marking of debugger breakpoint lines in the editor and @@ -685,7 +690,7 @@

    Running user codesys.modules starts with more entries, -and threading.activeCount() returns 2 instead of 1.

    +and threading.active_count() returns 2 instead of 1.

    By default, IDLE runs user code in a separate OS process rather than in the user interface process that runs the shell and editor. In the execution process, it replaces sys.stdin, sys.stdout, and sys.stderr @@ -939,7 +944,7 @@

    This Page

    • Report a Bug
    • - Show Source
    • @@ -971,7 +976,7 @@

      Navigation

    • - 3.10.0a6 Documentation » + 3.11.0a0 Documentation »
    • @@ -997,13 +1002,19 @@

      Navigation