Skip to content

Latest commit

 

History

History
1006 lines (746 loc) · 38.2 KB

3.10.rst

File metadata and controls

1006 lines (746 loc) · 38.2 KB
Raw

What's New In Python 3.10

Release:|release|
Date: |today|

This article explains the new features in Python 3.10, compared to 3.9.

For full details, see the :ref:`changelog <changelog>`.

Note

Prerelease users should be aware that this document is currently in draft form. It will be updated substantially as Python 3.10 moves towards release, so it's worth checking back even after reading earlier versions.

Summary -- Release highlights

New Features

Parenthesized context managers

Using enclosing parentheses for continuation across multiple lines in context managers is now supported. This allows formatting a long collection of context managers in multiple lines in a similar way as it was previously possible with import statements. For instance, all these examples are now valid:

with (CtxManager() as example):
    ...

with (
    CtxManager1(),
    CtxManager2()
):
    ...

with (CtxManager1() as example,
      CtxManager2()):
    ...

with (CtxManager1(),
      CtxManager2() as example):
    ...

with (
    CtxManager1() as example1,
    CtxManager2() as example2
):
    ...

it is also possible to use a trailing comma at the end of the enclosed group:

with (
    CtxManager1() as example1,
    CtxManager2() as example2,
    CtxManager3() as example3,
):
    ...

This new syntax uses the non LL(1) capacities of the new parser. Check PEP 617 for more details.

(Contributed by Guido van Rossum, Pablo Galindo and Lysandros Nikolaou in :issue:`12782` and :issue:`40334`.)

PEP 563: Postponed Evaluation of Annotations Becomes Default

In Python 3.7, postponed evaluation of annotations was added, to be enabled with a from __future__ import annotations directive. In 3.10 this became the default behavior, even without that future directive. With this being default, all annotations stored in :attr:`__annotations__` will be strings. If needed, annotations can be resolved at runtime using :func:`typing.get_type_hints`. See PEP 563 for a full description. Also, the :func:`inspect.signature` will try to resolve types from now on, and when it fails it will fall back to showing the string annotations. (Contributed by Batuhan Taskaya in :issue:`38605`.)

PEP 613: TypeAlias Annotation

PEP 484 introduced the concept of type aliases, only requiring them to be top-level unannotated assignments. This simplicity sometimes made it difficult for type checkers to distinguish between type aliases and ordinary assignments, especially when forward references or invalid types were involved. Compare:

StrCache = 'Cache[str]'  # a type alias
LOG_PREFIX = 'LOG[DEBUG]'  # a module constant

Now the :mod:`typing` module has a special annotation :data:`TypeAlias` to declare type aliases more explicitly:

StrCache: TypeAlias = 'Cache[str]'  # a type alias
LOG_PREFIX = 'LOG[DEBUG]'  # a module constant

See PEP 613 for more details.

(Contributed by Mikhail Golubev in :issue:`41923`.)

PEP 604: New Type Union Operator

A new type union operator was introduced which enables the syntax X | Y. This provides a cleaner way of expressing 'either type X or type Y' instead of using :data:`typing.Union`, especially in type hints (annotations).

In previous versions of Python, to apply a type hint for functions accepting arguments of multiple types, :data:`typing.Union` was used:

def square(number: Union[int, float]) -> Union[int, float]:
    return number ** 2

Type hints can now be written in a more succinct manner:

def square(number: int | float) -> int | float:
    return number ** 2

This new syntax is also accepted as the second argument to :func:`isinstance` and :func:`issubclass`:

>>> isinstance(1, int | str)
True

See :ref:`types-union` and PEP 604 for more details.

(Contributed by Maggie Moss and Philippe Prados in :issue:`41428`.)

PEP 612: Parameter Specification Variables

Two new options to improve the information provided to static type checkers for PEP 484's Callable have been added to the :mod:`typing` module.

The first is the parameter specification variable. They are used to forward the parameter types of one callable to another callable -- a pattern commonly found in higher order functions and decorators. Examples of usage can be found in :class:`typing.ParamSpec`. Previously, there was no easy way to type annotate dependency of parameter types in such a precise manner.

The second option is the new Concatenate operator. It's used in conjunction with parameter specification variables to type annotate a higher order callable which adds or removes parameters of another callable. Examples of usage can be found in :class:`typing.Concatenate`.

See :class:`typing.Callable`, :class:`typing.ParamSpec`, :class:`typing.Concatenate` and PEP 612 for more details.

(Contributed by Ken Jin in :issue:`41559`.)

Better error messages in the parser

When parsing code that contains unclosed parentheses or brackets the interpreter now includes the location of the unclosed bracket of parentheses instead of displaying SyntaxError: unexpected EOF while parsing or pointing to some incorrect location. For instance, consider the following code (notice the unclosed '{'):

expected = {9: 1, 18: 2, 19: 2, 27: 3, 28: 3, 29: 3, 36: 4, 37: 4,
            38: 4, 39: 4, 45: 5, 46: 5, 47: 5, 48: 5, 49: 5, 54: 6,
some_other_code = foo()

previous versions of the interpreter reported confusing places as the location of the syntax error:

File "example.py", line 3
    some_other_code = foo()
                    ^
SyntaxError: invalid syntax

but in Python3.10 a more informative error is emitted:

File "example.py", line 1
    expected = {9: 1, 18: 2, 19: 2, 27: 3, 28: 3, 29: 3, 36: 4, 37: 4,
               ^
SyntaxError: '{' was never closed

In a similar way, errors involving unclosed string literals (single and triple quoted) now point to the start of the string instead of reporting EOF/EOL.

These improvements are inspired by previous work in the PyPy interpreter.

(Contributed by Pablo Galindo in :issue:`42864` and Batuhan Taskaya in :issue:`40176`.)

Other Language Changes

  • Builtin and extension functions that take integer arguments no longer accept :class:`~decimal.Decimal`s, :class:`~fractions.Fraction`s and other objects that can be converted to integers only with a loss (e.g. that have the :meth:`~object.__int__` method but do not have the :meth:`~object.__index__` method). (Contributed by Serhiy Storchaka in :issue:`37999`.)
  • Assignment expressions can now be used unparenthesized within set literals and set comprehensions, as well as in sequence indexes (but not slices).
  • Functions have a new __builtins__ attribute which is used to look for builtin symbols when a function is executed, instead of looking into __globals__['__builtins__']. The attribute is initialized from __globals__["__builtins__"] if it exists, else from the current builtins. (Contributed by Mark Shannon in :issue:`42990`.)

New Modules

  • None yet.

Improved Modules

argparse

Misleading phrase "optional arguments" was replaced with "options" in argparse help. Some tests might require adaptation if they rely on exact output match. (Contributed by Raymond Hettinger in :issue:`9694`.)

base64

Add :func:`base64.b32hexencode` and :func:`base64.b32hexdecode` to support the Base32 Encoding with Extended Hex Alphabet.

codecs

Add a :func:`codecs.unregister` function to unregister a codec search function. (Contributed by Hai Shi in :issue:`41842`.)

collections.abc

The __args__ of the :ref:`parameterized generic <types-genericalias>` for :class:`collections.abc.Callable` are now consistent with :data:`typing.Callable`. :class:`collections.abc.Callable` generic now flattens type parameters, similar to what :data:`typing.Callable` currently does. This means that collections.abc.Callable[[int, str], str] will have __args__ of (int, str, str); previously this was ([int, str], str). To allow this change, :class:`types.GenericAlias` can now be subclassed, and a subclass will be returned when subscripting the :class:`collections.abc.Callable` type. Note that a :exc:`TypeError` may be raised for invalid forms of parameterizing :class:`collections.abc.Callable` which may have passed silently in Python 3.9. (Contributed by Ken Jin in :issue:`42195`.)

contextlib

Add a :func:`contextlib.aclosing` context manager to safely close async generators and objects representing asynchronously released resources. (Contributed by Joongi Kim and John Belmonte in :issue:`41229`.)

Add asynchronous context manager support to :func:`contextlib.nullcontext`. (Contributed by Tom Gringauz in :issue:`41543`.)

curses

The extended color functions added in ncurses 6.1 will be used transparently by :func:`curses.color_content`, :func:`curses.init_color`, :func:`curses.init_pair`, and :func:`curses.pair_content`. A new function, :func:`curses.has_extended_color_support`, indicates whether extended color support is provided by the underlying ncurses library. (Contributed by Jeffrey Kintscher and Hans Petter Jansson in :issue:`36982`.)

The BUTTON5_* constants are now exposed in the :mod:`curses` module if they are provided by the underlying curses library. (Contributed by Zackery Spytz in :issue:`39273`.)

distutils

The entire distutils package is deprecated, to be removed in Python 3.12. Its functionality for specifying package builds has already been completely replaced by third-party packages setuptools and packaging, and most other commonly used APIs are available elsewhere in the standard library (such as :mod:`platform`, :mod:`shutil`, :mod:`subprocess` or :mod:`sysconfig`). There are no plans to migrate any other functionality from distutils, and applications that are using other functions should plan to make private copies of the code. Refer to PEP 632 for discussion.

The bdist_wininst command deprecated in Python 3.8 has been removed. The bdist_wheel command is now recommended to distribute binary packages on Windows. (Contributed by Victor Stinner in :issue:`42802`.)

doctest

When a module does not define __loader__, fall back to __spec__.loader. (Contributed by Brett Cannon in :issue:`42133`.)

encodings

:func:`encodings.normalize_encoding` now ignores non-ASCII characters. (Contributed by Hai Shi in :issue:`39337`.)

glob

Added the root_dir and dir_fd parameters in :func:`~glob.glob` and :func:`~glob.iglob` which allow to specify the root directory for searching. (Contributed by Serhiy Storchaka in :issue:`38144`.)

inspect

When a module does not define __loader__, fall back to __spec__.loader. (Contributed by Brett Cannon in :issue:`42133`.)

Added globalns and localns parameters in :func:`~inspect.signature` and :meth:`inspect.Signature.from_callable` to retrieve the annotations in given local and global namespaces. (Contributed by Batuhan Taskaya in :issue:`41960`.)

linecache

When a module does not define __loader__, fall back to __spec__.loader. (Contributed by Brett Cannon in :issue:`42133`.)

os

Added :func:`os.cpu_count()` support for VxWorks RTOS. (Contributed by Peixing Xin in :issue:`41440`.)

Added a new function :func:`os.eventfd` and related helpers to wrap the eventfd2 syscall on Linux. (Contributed by Christian Heimes in :issue:`41001`.)

Added :func:`os.splice()` that allows to move data between two file descriptors without copying between kernel address space and user address space, where one of the file descriptors must refer to a pipe. (Contributed by Pablo Galindo in :issue:`41625`.)

Added :data:`~os.O_EVTONLY`, :data:`~os.O_FSYNC`, :data:`~os.O_SYMLINK` and :data:`~os.O_NOFOLLOW_ANY` for macOS. (Contributed by Dong-hee Na in :issue:`43106`.)

pathlib

Added slice support to :attr:`PurePath.parents <pathlib.PurePath.parents>`. (Contributed by Joshua Cannon in :issue:`35498`)

Added negative indexing support to :attr:`PurePath.parents <pathlib.PurePath.parents>`. (Contributed by Yaroslav Pankovych in :issue:`21041`)

platform

Added :func:`platform.freedesktop_os_release()` to retrieve operation system identification from freedesktop.org os-release standard file. (Contributed by Christian Heimes in :issue:`28468`)

py_compile

Added --quiet option to command-line interface of :mod:`py_compile`. (Contributed by Gregory Schevchenko in :issue:`38731`.)

pyclbr

Added an end_lineno attribute to the Function and Class objects in the tree returned by :func:`pyclbr.readline` and :func:`pyclbr.readline_ex`. It matches the existing (start) lineno. (Contributed by Aviral Srivastava in :issue:`38307`.)

shelve

The :mod:`shelve` module now uses :data:`pickle.DEFAULT_PROTOCOL` by default instead of :mod:`pickle` protocol 3 when creating shelves. (Contributed by Zackery Spytz in :issue:`34204`.)

site

When a module does not define __loader__, fall back to __spec__.loader. (Contributed by Brett Cannon in :issue:`42133`.)

socket

The exception :exc:`socket.timeout` is now an alias of :exc:`TimeoutError`. (Contributed by Christian Heimes in :issue:`42413`.)

sys

Add :data:`sys.orig_argv` attribute: the list of the original command line arguments passed to the Python executable. (Contributed by Victor Stinner in :issue:`23427`.)

Add :data:`sys.stdlib_module_names`, containing the list of the standard library module names. (Contributed by Victor Stinner in :issue:`42955`.)

threading

Added :func:`threading.gettrace` and :func:`threading.getprofile` to retrieve the functions set by :func:`threading.settrace` and :func:`threading.setprofile` respectively. (Contributed by Mario Corchero in :issue:`42251`.)

Add :data:`threading.__excepthook__` to allow retrieving the original value of :func:`threading.excepthook` in case it is set to a broken or a different value. (Contributed by Mario Corchero in :issue:`42308`.)

traceback

The :func:`~traceback.format_exception`, :func:`~traceback.format_exception_only`, and :func:`~traceback.print_exception` functions can now take an exception object as a positional-only argument. (Contributed by Zackery Spytz and Matthias Bussonnier in :issue:`26389`.)

types

Reintroduced the :data:`types.EllipsisType`, :data:`types.NoneType` and :data:`types.NotImplementedType` classes, providing a new set of types readily interpretable by type checkers. (Contributed by Bas van Beek in :issue:`41810`.)

typing

The behavior of :class:`typing.Literal` was changed to conform with PEP 586 and to match the behavior of static type checkers specified in the PEP.

  1. Literal now de-duplicates parameters.

  2. Equality comparisons between Literal objects are now order independent.

  3. Literal comparisons now respects types. For example, Literal[0] == Literal[False] previously evaluated to True. It is now False. To support this change, the internally used type cache now supports differentiating types.

  4. Literal objects will now raise a :exc:`TypeError` exception during equality comparisons if one of their parameters are not :term:`immutable`. Note that declaring Literal with mutable parameters will not throw an error:

    >>> from typing import Literal
>>> Literal[{0}]
>>> Literal[{0}] == Literal[{False}]
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: unhashable type: 'set'

(Contributed by Yurii Karabas in :issue:`42345`.)

unittest

Add new method :meth:`~unittest.TestCase.assertNoLogs` to complement the existing :meth:`~unittest.TestCase.assertLogs`. (Contributed by Kit Yan Choi in :issue:`39385`.)

urllib.parse

Python versions earlier than Python 3.10 allowed using both ; and & as query parameter separators in :func:`urllib.parse.parse_qs` and :func:`urllib.parse.parse_qsl`. Due to security concerns, and to conform with newer W3C recommendations, this has been changed to allow only a single separator key, with & as the default. This change also affects :func:`cgi.parse` and :func:`cgi.parse_multipart` as they use the affected functions internally. For more details, please see their respective documentation. (Contributed by Adam Goldschmidt, Senthil Kumaran and Ken Jin in :issue:`42967`.)

xml

Add a :class:`~xml.sax.handler.LexicalHandler` class to the :mod:`xml.sax.handler` module. (Contributed by Jonathan Gossage and Zackery Spytz in :issue:`35018`.)

zipimport

Add methods related to PEP 451: :meth:`~zipimport.zipimporter.find_spec`, :meth:`zipimport.zipimporter.create_module`, and :meth:`zipimport.zipimporter.exec_module`. (Contributed by Brett Cannon in :issue:`42131`.

Optimizations

  • Constructors :func:`str`, :func:`bytes` and :func:`bytearray` are now faster (around 30--40% for small objects). (Contributed by Serhiy Storchaka in :issue:`41334`.)
  • The :mod:`runpy` module now imports fewer modules. The python3 -m module-name command startup time is 1.3x faster in average. (Contributed by Victor Stinner in :issue:`41006`.)
  • The LOAD_ATTR instruction now uses new "per opcode cache" mechanism. It is about 36% faster now for regular attributes and 44% faster for slots. (Contributed by Pablo Galindo and Yury Selivanov in :issue:`42093` and Guido van Rossum in :issue:`42927`, based on ideas implemented originally in PyPy and MicroPython.)
  • When building Python with --enable-optimizations now -fno-semantic-interposition is added to both the compile and link line. This speeds builds of the Python interpreter created with --enable-shared with gcc by up to 30%. See this article for more details. (Contributed by Victor Stinner and Pablo Galindo in :issue:`38980`.)
  • Function parameters and their annotations are no longer computed at runtime, but rather at compilation time. They are stored as a tuple of strings at the bytecode level. It is now around 100% faster to create a function with parameter annotations. (Contributed by Yurii Karabas and Inada Naoki in :issue:`42202`)

Deprecated

Removed

  • Removed special methods __int__, __float__, __floordiv__, __mod__, __divmod__, __rfloordiv__, __rmod__ and __rdivmod__ of the :class:`complex` class. They always raised a :exc:`TypeError`. (Contributed by Serhiy Storchaka in :issue:`41974`.)

  • The ParserBase.error() method from the private and undocumented _markupbase module has been removed. :class:`html.parser.HTMLParser` is the only subclass of ParserBase and its error() implementation has already been removed in Python 3.5. (Contributed by Berker Peksag in :issue:`31844`.)

  • Removed the unicodedata.ucnhash_CAPI attribute which was an internal PyCapsule object. The related private _PyUnicode_Name_CAPI structure was moved to the internal C API. (Contributed by Victor Stinner in :issue:`42157`.)

  • Removed the parser module, which was deprecated in 3.9 due to the switch to the new PEG parser, as well as all the C source and header files that were only being used by the old parser, including node.h, parser.h, graminit.h and grammar.h.

  • Removed the Public C API functions :c:func:`PyParser_SimpleParseStringFlags`, :c:func:`PyParser_SimpleParseStringFlagsFilename`, :c:func:`PyParser_SimpleParseFileFlags` and :c:func:`PyNode_Compile` that were deprecated in 3.9 due to the switch to the new PEG parser.

  • Removed the formatter module, which was deprecated in Python 3.4. It is somewhat obsolete, little used, and not tested. It was originally scheduled to be removed in Python 3.6, but such removals were delayed until after Python 2.7 EOL. Existing users should copy whatever classes they use into their code. (Contributed by Dong-hee Na and Terry J. Reedy in :issue:`42299`.)

  • Removed the :c:func:`PyModule_GetWarningsModule` function that was useless now due to the _warnings module was converted to a builtin module in 2.6. (Contributed by Hai Shi in :issue:`42599`.)

  • Remove deprecated aliases to :ref:`collections-abstract-base-classes` from the :mod:`collections` module. (Contributed by Victor Stinner in :issue:`37324`.)

  • The loop parameter has been removed from most of :mod:`asyncio`'s :doc:`high-level API <../library/asyncio-api-index>` following deprecation in Python 3.8. The motivation behind this change is multifold:

    1. This simplifies the high-level API.
    2. The functions in the high-level API have been implicitly getting the current thread's running event loop since Python 3.7. There isn't a need to pass the event loop to the API in most normal use cases.
    3. Event loop passing is error-prone especially when dealing with loops running in different threads.

    Note that the low-level API will still accept loop. See Changes in the Python API for examples of how to replace existing code.

    (Contributed by Yurii Karabas, Andrew Svetlov, Yury Selivanov and Kyle Stanley in :issue:`42392`.)

Porting to Python 3.10

This section lists previously described changes and other bugfixes that may require changes to your code.

Changes in the Python API

CPython bytecode changes

  • The MAKE_FUNCTION instruction accepts tuple of strings as annotations instead of dictionary. (Contributed by Yurii Karabas and Inada Naoki in :issue:`42202`)

Build Changes

  • The C99 functions :c:func:`snprintf` and :c:func:`vsnprintf` are now required to build Python. (Contributed by Victor Stinner in :issue:`36020`.)

  • :mod:`sqlite3` requires SQLite 3.7.15 or higher. (Contributed by Sergey Fedoseev and Erlend E. Aasland :issue:`40744` and :issue:`40810`.)

  • The :mod:`atexit` module must now always be built as a built-in module. (Contributed by Victor Stinner in :issue:`42639`.)

  • Added --disable-test-modules option to the configure script: don't build nor install test modules. (Contributed by Xavier de Gaye, Thomas Petazzoni and Peixing Xin in :issue:`27640`.)

  • Add --with-wheel-pkg-dir=PATH option to the ./configure script. If specified, the :mod:`ensurepip` module looks for setuptools and pip wheel packages in this directory: if both are present, these wheel packages are used instead of ensurepip bundled wheel packages.

    Some Linux distribution packaging policies recommend against bundling dependencies. For example, Fedora installs wheel packages in the /usr/share/python-wheels/ directory and don't install the ensurepip._bundled package.

    (Contributed by Victor Stinner in :issue:`42856`.)

  • Add a new configure --without-static-libpython option to not build the libpythonMAJOR.MINOR.a static library and not install the python.o object file.

    (Contributed by Victor Stinner in :issue:`43103`.)

C API Changes

New Features

Porting to Python 3.10

Deprecated

Removed