Scheduled weekly dependency update for week 48

[pyup-bot]

Update asgiref from 3.6.0 to 3.7.2.

Changelog

3.7.2

------------------

* The type annotations for SyncToAsync and AsyncToSync have been changed to
more accurately reflect the kind of callables they return.

3.7.1

------------------

* On Python 3.10 and below, the version of the "typing_extensions" package
is now constrained to be at least version 4 (as we depend on functionality
in that version and above)

3.7.0

------------------

* Contextvars are now required for the implementation of Local as Python 3.6
is now no longer a supported version.

* sync_to_async and async_to_sync now pass-through

* Debug and Lifespan State extensions have resulted in a typing change for some
request and response types. This change should be backwards-compatible.

* ``asgiref`` frames will now be hidden in Django tracebacks by default.

* Raw performance and garbage collection improvements in Local, SyncToAsync,
and AsyncToSync.

Links

• PyPI: https://pypi.org/project/asgiref
• Changelog: https://data.safetycli.com/changelogs/asgiref/
• Repo: https://github.com/django/asgiref/

Update asttokens from 2.2.1 to 2.4.1.

The bot wasn't able to find a changelog for this release. Got an idea?

Links

• PyPI: https://pypi.org/project/asttokens
• Repo: https://github.com/gristlabs/asttokens

Update black from 22.12.0 to 23.11.0.

Changelog

23.11.0

Highlights

- Support formatting ranges of lines with the new `--line-ranges` command-line option
(4020)

Stable style

- Fix crash on formatting bytes strings that look like docstrings (4003)
- Fix crash when whitespace followed a backslash before newline in a docstring (4008)
- Fix standalone comments inside complex blocks crashing Black (4016)
- Fix crash on formatting code like `await (a ** b)` (3994)
- No longer treat leading f-strings as docstrings. This matches Python's behaviour and
fixes a crash (4019)

Preview style

- Multiline dicts and lists that are the sole argument to a function are now indented
less (3964)
- Multiline unpacked dicts and lists as the sole argument to a function are now also
indented less (3992)
- In f-string debug expressions, quote types that are visible in the final string are
now preserved (4005)
- Fix a bug where long `case` blocks were not split into multiple lines. Also enable
general trailing comma rules on `case` blocks (4024)
- Keep requiring two empty lines between module-level docstring and first function or
class definition (4028)
- Add support for single-line format skip with other comments on the same line (3959)

Configuration

- Consistently apply force exclusion logic before resolving symlinks (4015)
- Fix a bug in the matching of absolute path names in `--include` (3976)

Performance

- Fix mypyc builds on arm64 on macOS (4017)

Integrations

- Black's pre-commit integration will now run only on git hooks appropriate for a code
formatter (3940)

23.10.1

Highlights

- Maintenance release to get a fix out for GitHub Action edge case (3957)

Preview style

- Fix merging implicit multiline strings that have inline comments (3956)
- Allow empty first line after block open before a comment or compound statement (3967)

Packaging

- Change Dockerfile to hatch + compile black (3965)

Integrations

- The summary output for GitHub workflows is now suppressible using the `summary`
parameter. (3958)
- Fix the action failing when Black check doesn't pass (3957)

Documentation

- It is known Windows documentation CI is broken
https://github.com/psf/black/issues/3968

23.10.0

Stable style

- Fix comments getting removed from inside parenthesized strings (3909)

Preview style

- Fix long lines with power operators getting split before the line length (3942)
- Long type hints are now wrapped in parentheses and properly indented when split across
multiple lines (3899)
- Magic trailing commas are now respected in return types. (3916)
- Require one empty line after module-level docstrings. (3932)
- Treat raw triple-quoted strings as docstrings (3947)

Configuration

- Fix cache versioning logic when `BLACK_CACHE_DIR` is set (3937)

Parser

- Fix bug where attributes named `type` were not accepted inside `match` statements
(3950)
- Add support for PEP 695 type aliases containing lambdas and other unusual expressions
(3949)

Output

- Black no longer attempts to provide special errors for attempting to format Python 2
code (3933)
- Black will more consistently print stacktraces on internal errors in verbose mode
(3938)

Integrations

- The action output displayed in the job summary is now wrapped in Markdown (3914)

23.9.1

Due to various issues, the previous release (23.9.0) did not include compiled mypyc
wheels, which make Black significantly faster. These issues have now been fixed, and
this release should come with compiled wheels once again.

There will be no wheels for Python 3.12 due to a bug in mypyc. We will provide 3.12
wheels in a future release as soon as the mypyc bug is fixed.

Packaging

- Upgrade to mypy 1.5.1 (3864)

Performance

- Store raw tuples instead of NamedTuples in Black's cache, improving performance and
decreasing the size of the cache (3877)

23.9.0

Preview style

- More concise formatting for dummy implementations (3796)
- In stub files, add a blank line between a statement with a body (e.g an
`if sys.version_info > (3, x):`) and a function definition on the same level (3862)
- Fix a bug whereby spaces were removed from walrus operators within subscript(3823)

Configuration

- Black now applies exclusion and ignore logic before resolving symlinks (3846)

Performance

- Avoid importing `IPython` if notebook cells do not contain magics (3782)
- Improve caching by comparing file hashes as fallback for mtime and size (3821)

_Blackd_

- Fix an issue in `blackd` with single character input (3558)

Integrations

- Black now has an
[official pre-commit mirror](https://github.com/psf/black-pre-commit-mirror). Swapping
`https://github.com/psf/black` to `https://github.com/psf/black-pre-commit-mirror` in
your `.pre-commit-config.yaml` will make Black about 2x faster (3828)
- The `.black.env` folder specified by `ENV_PATH` will now be removed on the completion
of the GitHub Action (3759)

23.7.0

Highlights

- Runtime support for Python 3.7 has been removed. Formatting 3.7 code will still be
supported until further notice (3765)

Stable style

- Fix a bug where an illegal trailing comma was added to return type annotations using
PEP 604 unions (3735)
- Fix several bugs and crashes where comments in stub files were removed or mishandled
under some circumstances (3745)
- Fix a crash with multi-line magic comments like `type: ignore` within parentheses
(3740)
- Fix error in AST validation when _Black_ removes trailing whitespace in a type comment
(3773)

Preview style

- Implicitly concatenated strings used as function args are no longer wrapped inside
parentheses (3640)
- Remove blank lines between a class definition and its docstring (3692)

Configuration

- The `--workers` argument to _Black_ can now be specified via the `BLACK_NUM_WORKERS`
environment variable (3743)
- `.pytest_cache`, `.ruff_cache` and `.vscode` are now excluded by default (3691)
- Fix _Black_ not honouring `pyproject.toml` settings when running `--stdin-filename`
and the `pyproject.toml` found isn't in the current working directory (3719)
- _Black_ will now error if `exclude` and `extend-exclude` have invalid data types in
`pyproject.toml`, instead of silently doing the wrong thing (3764)

Packaging

- Upgrade mypyc from 0.991 to 1.3 (3697)
- Remove patching of Click that mitigated errors on Python 3.6 with `LANG=C` (3768)

Parser

- Add support for the new PEP 695 syntax in Python 3.12 (3703)

Performance

- Speed up _Black_ significantly when the cache is full (3751)
- Avoid importing `IPython` in a case where we wouldn't need it (3748)

Output

- Use aware UTC datetimes internally, avoids deprecation warning on Python 3.12 (3728)
- Change verbose logging to exactly mirror _Black_'s logic for source discovery (3749)

_Blackd_

- The `blackd` argument parser now shows the default values for options in their help
text (3712)

Integrations

- Black is now tested with
[`PYTHONWARNDEFAULTENCODING = 1`](https://docs.python.org/3/library/io.html#io-encoding-warning)
(3763)
- Update GitHub Action to display black output in the job summary (3688)

Documentation

- Add a CITATION.cff file to the root of the repository, containing metadata on how to
cite this software (3723)
- Update the _classes_ and _exceptions_ documentation in Developer reference to match
the latest code base (3755)

23.3.0

Highlights

This release fixes a longstanding confusing behavior in Black's GitHub action, where the
version of the action did not determine the version of Black being run (issue 3382). In
addition, there is a small bug fix around imports and a number of improvements to the
preview style.

Please try out the
[preview style](https://black.readthedocs.io/en/stable/the_black_code_style/future_style.html#preview-style)
with `black --preview` and tell us your feedback. All changes in the preview style are
expected to become part of Black's stable style in January 2024.

Stable style

- Import lines with ` fmt: skip` and ` fmt: off` no longer have an extra blank line
added when they are right after another import line (3610)

Preview style

- Add trailing commas to collection literals even if there's a comment after the last
entry (3393)
- `async def`, `async for`, and `async with` statements are now formatted consistently
compared to their non-async version. (3609)
- `with` statements that contain two context managers will be consistently wrapped in
parentheses (3589)
- Let string splitters respect [East Asian Width](https://www.unicode.org/reports/tr11/)
(3445)
- Now long string literals can be split after East Asian commas and periods (`、` U+3001
IDEOGRAPHIC COMMA, `。` U+3002 IDEOGRAPHIC FULL STOP, & `，` U+FF0C FULLWIDTH COMMA)
besides before spaces (3445)
- For stubs, enforce one blank line after a nested class with a body other than just
`...` (3564)
- Improve handling of multiline strings by changing line split behavior (1879)

Parser

- Added support for formatting files with invalid type comments (3594)

Integrations

- Update GitHub Action to use the version of Black equivalent to action's version if
version input is not specified (3543)
- Fix missing Python binary path in autoload script for vim (3508)

Documentation

- Document that only the most recent release is supported for security issues;
vulnerabilities should be reported through Tidelift (3612)

23.1.0

Highlights

This is the first release of 2023, and following our
[stability policy](https://black.readthedocs.io/en/stable/the_black_code_style/index.html#stability-policy),
it comes with a number of improvements to our stable style, including improvements to
empty line handling, removal of redundant parentheses in several contexts, and output
that highlights implicitly concatenated strings better.

There are also many changes to the preview style; try out `black --preview` and give us
feedback to help us set the stable style for next year.

In addition to style changes, Black now automatically infers the supported Python
versions from your `pyproject.toml` file, removing the need to set Black's target
versions separately.

Stable style

- Introduce the 2023 stable style, which incorporates most aspects of last year's
preview style (3418). Specific changes:
- Enforce empty lines before classes and functions with sticky leading comments
 (3302) (22.12.0)
- Reformat empty and whitespace-only files as either an empty file (if no newline is
 present) or as a single newline character (if a newline is present) (3348)
 (22.12.0)
- Implicitly concatenated strings used as function args are now wrapped inside
 parentheses (3307) (22.12.0)
- Correctly handle trailing commas that are inside a line's leading non-nested parens
 (3370) (22.12.0)
- `--skip-string-normalization` / `-S` now prevents docstring prefixes from being
 normalized as expected (3168) (since 22.8.0)
- When using `--skip-magic-trailing-comma` or `-C`, trailing commas are stripped from
 subscript expressions with more than 1 element (3209) (22.8.0)
- Implicitly concatenated strings inside a list, set, or tuple are now wrapped inside
 parentheses (3162) (22.8.0)
- Fix a string merging/split issue when a comment is present in the middle of
 implicitly concatenated strings on its own line (3227) (22.8.0)
- Docstring quotes are no longer moved if it would violate the line length limit
 (3044, 3430) (22.6.0)
- Parentheses around return annotations are now managed (2990) (22.6.0)
- Remove unnecessary parentheses around awaited objects (2991) (22.6.0)
- Remove unnecessary parentheses in `with` statements (2926) (22.6.0)
- Remove trailing newlines after code block open (3035) (22.6.0)
- Code cell separators `%%` are now standardised to ` %%` (2919) (22.3.0)
- Remove unnecessary parentheses from `except` statements (2939) (22.3.0)
- Remove unnecessary parentheses from tuple unpacking in `for` loops (2945) (22.3.0)
- Avoid magic-trailing-comma in single-element subscripts (2942) (22.3.0)
- Fix a crash when a colon line is marked between ` fmt: off` and ` fmt: on` (3439)

Preview style

- Format hex codes in unicode escape sequences in string literals (2916)
- Add parentheses around `if`-`else` expressions (2278)
- Improve performance on large expressions that contain many strings (3467)
- Fix a crash in preview style with assert + parenthesized string (3415)
- Fix crashes in preview style with walrus operators used in function return annotations
and except clauses (3423)
- Fix a crash in preview advanced string processing where mixed implicitly concatenated
regular and f-strings start with an empty span (3463)
- Fix a crash in preview advanced string processing where a standalone comment is placed
before a dict's value (3469)
- Fix an issue where extra empty lines are added when a decorator has ` fmt: skip`
applied or there is a standalone comment between decorators (3470)
- Do not put the closing quotes in a docstring on a separate line, even if the line is
too long (3430)
- Long values in dict literals are now wrapped in parentheses; correspondingly
unnecessary parentheses around short values in dict literals are now removed; long
string lambda values are now wrapped in parentheses (3440)
- Fix two crashes in preview style involving edge cases with docstrings (3451)
- Exclude string type annotations from improved string processing; fix crash when the
return type annotation is stringified and spans across multiple lines (3462)
- Wrap multiple context managers in parentheses when targeting Python 3.9+ (3489)
- Fix several crashes in preview style with walrus operators used in `with` statements
or tuples (3473)
- Fix an invalid quote escaping bug in f-string expressions where it produced invalid
code. Implicitly concatenated f-strings with different quotes can now be merged or
quote-normalized by changing the quotes used in expressions. (3509)
- Fix crash on `await (yield)` when Black is compiled with mypyc (3533)

Configuration

- Black now tries to infer its `--target-version` from the project metadata specified in
`pyproject.toml` (3219)

Packaging

- Upgrade mypyc from `0.971` to `0.991` so mypycified _Black_ can be built on armv7
(3380)
- This also fixes some crashes while using compiled Black with a debug build of
 CPython
- Drop specific support for the `tomli` requirement on 3.11 alpha releases, working
around a bug that would cause the requirement not to be installed on any non-final
Python releases (3448)
- Black now depends on `packaging` version `22.0` or later. This is required for new
functionality that needs to parse part of the project metadata (3219)

Output

- Calling `black --help` multiple times will return the same help contents each time
(3516)
- Verbose logging now shows the values of `pyproject.toml` configuration variables
(3392)
- Fix false symlink detection messages in verbose output due to using an incorrect
relative path to the project root (3385)

Integrations

- Move 3.11 CI to normal flow now that all dependencies support 3.11 (3446)
- Docker: Add new `latest_prerelease` tag automation to follow latest black alpha
release on docker images (3465)

Documentation

- Expand `vim-plug` installation instructions to offer more explicit options (3468)

Links

• PyPI: https://pypi.org/project/black
• Changelog: https://data.safetycli.com/changelogs/black/

Update click from 8.1.3 to 8.1.7.

Changelog

8.1.7

-------------

Released 2023-08-17

-   Fix issue with regex flags in shell completion. :issue:`2581`
-   Bash version detection issues a warning instead of an error. :issue:`2574`
-   Fix issue with completion script for Fish shell. :issue:`2567`

8.1.6

-------------

Released 2023-07-18

-   Fix an issue with type hints for ``click.group()``. :issue:`2558`

8.1.5

-------------

Released 2023-07-13

-   Fix an issue with type hints for ``click.command()``, ``click.option()``, and
 other decorators. Introduce typing tests. :issue:`2558`

8.1.4

-------------

Released 2023-07-06

-   Replace all ``typing.Dict`` occurrences to ``typing.MutableMapping`` for
 parameter hints. :issue:`2255`
-   Improve type hinting for decorators and give all generic types parameters.
 :issue:`2398`
-   Fix return value and type signature of `shell_completion.add_completion_class`
 function. :pr:`2421`
-   Bash version detection doesn't fail on Windows. :issue:`2461`
-   Completion works if there is a dot (``.``) in the program name. :issue:`2166`
-   Improve type annotations for pyright type checker. :issue:`2268`
-   Improve responsiveness of ``click.clear()``. :issue:`2284`
-   Improve command name detection when using Shiv or PEX. :issue:`2332`
-   Avoid showing empty lines if command help text is empty. :issue:`2368`
-   ZSH completion script works when loaded from ``fpath``. :issue:`2344`.
-   ``EOFError`` and ``KeyboardInterrupt`` tracebacks are not suppressed when
 ``standalone_mode`` is disabled. :issue:`2380`
-   ``group.command`` does not fail if the group was created with a custom
 ``command_class``. :issue:`2416`
-   ``multiple=True`` is allowed for flag options again and does not require
 setting ``default=()``. :issue:`2246, 2292, 2295`
-   Make the decorators returned by ``argument()`` and ``option()`` reusable when the
 ``cls`` parameter is used. :issue:`2294`
-   Don't fail when writing filenames to streams with strict errors. Replace invalid
 bytes with the replacement character (``�``). :issue:`2395`
-   Remove unnecessary attempt to detect MSYS2 environment. :issue:`2355`
-   Remove outdated and unnecessary detection of App Engine environment. :pr:`2554`
-   ``echo()`` does not fail when no streams are attached, such as with ``pythonw`` on
 Windows. :issue:`2415`
-   Argument with ``expose_value=False`` do not cause completion to fail. :issue:`2336`

Links

• PyPI: https://pypi.org/project/click
• Changelog: https://data.safetycli.com/changelogs/click/
• Homepage: https://palletsprojects.com/p/click/

Update configobj from 5.0.6 to 5.0.8.

Changelog

5.0.8

"""""""""""""

* fixing/test for a regression introduced in 5.0.7 that prevented ``import validate`` from working

5.0.7

"""""""""""""

* update testing to validate against python version 2.7 and 3.5-3.11
* update broken links / non-existent services and references

Older Releases
""""""""""""""

Links

• PyPI: https://pypi.org/project/configobj
• Changelog: https://data.safetycli.com/changelogs/configobj/
• Repo: https://github.com/DiffSK/configobj

Update coverage from 7.0.1 to 7.3.2.

Changelog

7.3.2

--------------------------

- The ``coverage lcov`` command ignored the ``[report] exclude_lines`` and
``[report] exclude_also`` settings (`issue 1684`_).  This is now fixed,
thanks `Jacqueline Lee <pull 1685_>`_.

- Sometimes SQLite will create journal files alongside the coverage.py database
files.  These are ephemeral, but could be mistakenly included when combining
data files.  Now they are always ignored, fixing `issue 1605`_. Thanks to
Brad Smith for suggesting fixes and providing detailed debugging.

- On Python 3.12+, we now disable SQLite writing journal files, which should be
a little faster.

- The new 3.12 soft keyword ``type`` is properly bolded in HTML reports.

- Removed the "fullcoverage" feature used by CPython to measure the coverage of
early-imported standard library modules.  CPython `stopped using it
<88054_>`_ in 2021, and it stopped working completely in Python 3.13.

.. _issue 1605: https://github.com/nedbat/coveragepy/issues/1605
.. _issue 1684: https://github.com/nedbat/coveragepy/issues/1684
.. _pull 1685: https://github.com/nedbat/coveragepy/pull/1685
.. _88054: https://github.com/python/cpython/issues/88054


.. _changes_7-3-1:

7.3.1

--------------------------

- The semantics of stars in file patterns has been clarified in the docs.  A
leading or trailing star matches any number of path components, like a double
star would.  This is different than the behavior of a star in the middle of a
pattern.  This discrepancy was `identified by Sviatoslav Sydorenko
<starbad_>`_, who `provided patient detailed diagnosis <pull 1650_>`_ and
graciously agreed to a pragmatic resolution.

- The API docs were missing from the last version. They are now `restored
<apidocs_>`_.

.. _apidocs: https://coverage.readthedocs.io/en/latest/api_coverage.html
.. _starbad: https://github.com/nedbat/coveragepy/issues/1407#issuecomment-1631085209
.. _pull 1650: https://github.com/nedbat/coveragepy/pull/1650

.. _changes_7-3-0:

7.3.0

--------------------------

- Added a :meth:`.Coverage.collect` context manager to start and stop coverage
data collection.

- Dropped support for Python 3.7.

- Fix: in unusual circumstances, SQLite cannot be set to asynchronous mode.
Coverage.py would fail with the error ``Safety level may not be changed
inside a transaction.`` This is now avoided, closing `issue 1646`_.  Thanks
to Michael Bell for the detailed bug report.

- Docs: examples of configuration files now include separate examples for the
different syntaxes: .coveragerc, pyproject.toml, setup.cfg, and tox.ini.

- Fix: added ``nosemgrep`` comments to our JavaScript code so that
semgrep-based SAST security checks won't raise false alarms about security
problems that aren't problems.

- Added a CITATION.cff file, thanks to `Ken Schackart <pull 1641_>`_.

.. _pull 1641: https://github.com/nedbat/coveragepy/pull/1641
.. _issue 1646: https://github.com/nedbat/coveragepy/issues/1646


.. _changes_7-2-7:

7.2.7

--------------------------

- Fix: reverted a `change from 6.4.3 <pull 1347_>`_ that helped Cython, but
also increased the size of data files when using dynamic contexts, as
described in the now-fixed `issue 1586`_. The problem is now avoided due to a
recent change (`issue 1538`_).  Thanks to `Anders Kaseorg <pull 1629_>`_
and David Szotten for persisting with problem reports and detailed diagnoses.

- Wheels are now provided for CPython 3.12.

.. _issue 1586: https://github.com/nedbat/coveragepy/issues/1586
.. _pull 1629: https://github.com/nedbat/coveragepy/pull/1629


.. _changes_7-2-6:

7.2.6

--------------------------

- Fix: the ``lcov`` command could raise an IndexError exception if a file is
translated to Python but then executed under its own name.  Jinja2 does this
when rendering templates.  Fixes `issue 1553`_.

- Python 3.12 beta 1 now inlines comprehensions.  Previously they were compiled
as invisible functions and coverage.py would warn you if they weren't
completely executed.  This no longer happens under Python 3.12.

- Fix: the ``coverage debug sys`` command includes some environment variables
in its output.  This could have included sensitive data.  Those values are
now hidden with asterisks, closing `issue 1628`_.

.. _issue 1553: https://github.com/nedbat/coveragepy/issues/1553
.. _issue 1628: https://github.com/nedbat/coveragepy/issues/1628


.. _changes_7-2-5:

7.2.5

--------------------------

- Fix: ``html_report()`` could fail with an AttributeError on ``isatty`` if run
in an unusual environment where sys.stdout had been replaced.  This is now
fixed.


.. _changes_7-2-4:

7.2.4

--------------------------

PyCon 2023 sprint fixes!

- Fix: with ``relative_files = true``, specifying a specific file to include or
omit wouldn't work correctly (`issue 1604`_).  This is now fixed, with
testing help by `Marc Gibbons <pull 1608_>`_.

- Fix: the XML report would have an incorrect ``<source>`` element when using
relative files and the source option ended with a slash (`issue 1541`_).
This is now fixed, thanks to `Kevin Brown-Silva <pull 1608_>`_.

- When the HTML report location is printed to the terminal, it's now a
terminal-compatible URL, so that you can click the location to open the HTML
file in your browser.  Finishes `issue 1523`_ thanks to `Ricardo Newbery
<pull 1613_>`_.

- Docs: a new :ref:`Migrating page <migrating>` with details about how to
migrate between major versions of coverage.py.  It currently covers the
wildcard changes in 7.x.  Thanks, `Brian Grohe <pull 1610_>`_.

.. _issue 1523: https://github.com/nedbat/coveragepy/issues/1523
.. _issue 1541: https://github.com/nedbat/coveragepy/issues/1541
.. _issue 1604: https://github.com/nedbat/coveragepy/issues/1604
.. _pull 1608: https://github.com/nedbat/coveragepy/pull/1608
.. _pull 1609: https://github.com/nedbat/coveragepy/pull/1609
.. _pull 1610: https://github.com/nedbat/coveragepy/pull/1610
.. _pull 1613: https://github.com/nedbat/coveragepy/pull/1613


.. _changes_7-2-3:

7.2.3

--------------------------

- Fix: the :ref:`config_run_sigterm` setting was meant to capture data if a
process was terminated with a SIGTERM signal, but it didn't always.  This was
fixed thanks to `Lewis Gaul <pull 1600_>`_, closing `issue 1599`_.

- Performance: HTML reports with context information are now much more compact.
File sizes are typically as small as one-third the previous size, but can be
dramatically smaller. This closes `issue 1584`_ thanks to `Oleh Krehel
<pull 1587_>`_.

- Development dependencies no longer use hashed pins, closing `issue 1592`_.

.. _issue 1584: https://github.com/nedbat/coveragepy/issues/1584
.. _pull 1587: https://github.com/nedbat/coveragepy/pull/1587
.. _issue 1592: https://github.com/nedbat/coveragepy/issues/1592
.. _issue 1599: https://github.com/nedbat/coveragepy/issues/1599
.. _pull 1600: https://github.com/nedbat/coveragepy/pull/1600


.. _changes_7-2-2:

7.2.2

--------------------------

- Fix: if a virtualenv was created inside a source directory, and a sourced
package was installed inside the virtualenv, then all of the third-party
packages inside the virtualenv would be measured.  This was incorrect, but
has now been fixed: only the specified packages will be measured, thanks to
`Manuel Jacob <pull 1560_>`_.

- Fix: the ``coverage lcov`` command could create a .lcov file with incorrect
LF (lines found) and LH (lines hit) totals.  This is now fixed, thanks to
`Ian Moore <pull 1583_>`_.

- Fix: the ``coverage xml`` command on Windows could create a .xml file with
duplicate ``<package>`` elements. This is now fixed, thanks to `Benjamin
Parzella <pull 1574_>`_, closing `issue 1573`_.

.. _pull 1560: https://github.com/nedbat/coveragepy/pull/1560
.. _issue 1573: https://github.com/nedbat/coveragepy/issues/1573
.. _pull 1574: https://github.com/nedbat/coveragepy/pull/1574
.. _pull 1583: https://github.com/nedbat/coveragepy/pull/1583


.. _changes_7-2-1:

7.2.1

--------------------------

- Fix: the PyPI page had broken links to documentation pages, but no longer
does, closing `issue 1566`_.

- Fix: public members of the coverage module are now properly indicated so that
mypy will find them, fixing `issue 1564`_.

.. _issue 1564: https://github.com/nedbat/coveragepy/issues/1564
.. _issue 1566: https://github.com/nedbat/coveragepy/issues/1566


.. _changes_7-2-0:

7.2.0

--------------------------

- Added a new setting ``[report] exclude_also`` to let you add more exclusions
without overwriting the defaults.  Thanks, `Alpha Chen <pull 1557_>`_,
closing `issue 1391`_.

- Added a :meth:`.CoverageData.purge_files` method to remove recorded data for
a particular file.  Contributed by `Stephan Deibel <pull 1547_>`_.

- Fix: when reporting commands fail, they will no longer congratulate
themselves with messages like "Wrote XML report to file.xml" before spewing a
traceback about their failure.

- Fix: arguments in the public API that name file paths now accept pathlib.Path
objects.  This includes the ``data_file`` and ``config_file`` arguments to
the Coverage constructor and the ``basename`` argument to CoverageData.
Closes `issue 1552`_.

- Fix: In some embedded environments, an IndexError could occur on stop() when
the originating thread exits before completion.  This is now fixed, thanks to
`Russell Keith-Magee <pull 1543_>`_, closing `issue 1542`_.

- Added a ``py.typed`` file to announce our type-hintedness.  Thanks,
`KotlinIsland <pull 1550_>`_.

.. _issue 1391: https://github.com/nedbat/coveragepy/issues/1391
.. _issue 1542: https://github.com/nedbat/coveragepy/issues/1542
.. _pull 1543: https://github.com/nedbat/coveragepy/pull/1543
.. _pull 1547: https://github.com/nedbat/coveragepy/pull/1547
.. _pull 1550: https://github.com/nedbat/coveragepy/pull/1550
.. _issue 1552: https://github.com/nedbat/coveragepy/issues/1552
.. _pull 1557: https://github.com/nedbat/coveragepy/pull/1557


.. _changes_7-1-0:

7.1.0

--------------------------

- Added: the debug output file can now be specified with ``[run] debug_file``
in the configuration file.  Closes `issue 1319`_.

- Performance: fixed a slowdown with dynamic contexts that's been around since
6.4.3.  The fix closes `issue 1538`_.  Thankfully this doesn't break the
`Cython change`_ that fixed `issue 972`_.  Thanks to Mathieu Kniewallner for
the deep investigative work and comprehensive issue report.

- Typing: all product and test code has type annotations.

.. _Cython change: https://github.com/nedbat/coveragepy/pull/1347
.. _issue 972: https://github.com/nedbat/coveragepy/issues/972
.. _issue 1319: https://github.com/nedbat/coveragepy/issues/1319
.. _issue 1538: https://github.com/nedbat/coveragepy/issues/1538


.. _changes_7-0-5:

7.0.5

--------------------------

- Fix: On Python 3.7, a file with type annotations but no ``from __future__
import annotations`` would be missing statements in the coverage report. This
is now fixed, closing `issue 1524`_.

.. _issue 1524: https://github.com/nedbat/coveragepy/issues/1524


.. _changes_7-0-4:

7.0.4

--------------------------

- Performance: an internal cache of file names was accidentally disabled,
resulting in sometimes drastic reductions in performance.  This is now fixed,
closing `issue 1527`_.   Thanks to Ivan Ciuvalschii for the reproducible test
case.

.. _issue 1527: https://github.com/nedbat/coveragepy/issues/1527


.. _changes_7-0-3:

7.0.3

--------------------------

- Fix: when using pytest-cov or pytest-xdist, or perhaps both, the combining
step could fail with ``assert row is not None`` using 7.0.2.  This was due to
a race condition that has always been possible and is still possible. In
7.0.1 and before, the error was silently swallowed by the combining code.
Now it will produce a message "Couldn't combine data file" and ignore the
data file as it used to do before 7.0.2.  Closes `issue 1522`_.

.. _issue 1522: https://github.com/nedbat/coveragepy/issues/1522


.. _changes_7-0-2:

7.0.2

--------------------------

- Fix: when using the ``[run] relative_files = True`` setting, a relative
``[paths]`` pattern was still being made absolute.  This is now fixed,
closing `issue 1519`_.

- Fix: if Python doesn't provide tomllib, then TOML configuration files can
only be read if coverage.py is installed with the ``[toml]`` extra.
Coverage.py will raise an error if TOML support is not installed when it sees
your settings are in a .toml file. But it didn't understand that
``[tools.coverage]`` was a valid section header, so the error wasn't reported
if you used that header, and settings were silently ignored.  This is now
fixed, closing `issue 1516`_.

- Fix: adjusted how decorators are traced on PyPy 7.3.10, fixing `issue 1515`_.

- Fix: the ``coverage lcov`` report did not properly implement the
``--fail-under=MIN`` option.  This has been fixed.

- Refactor: added many type annotations, including a number of refactorings.
This should not affect outward behavior, but they were a bit invasive in some
places, so keep your eyes peeled for oddities.

- Refactor: removed the vestigial and long untested support for Jython and
IronPython.

.. _issue 1515: https://github.com/nedbat/coveragepy/issues/1515
.. _issue 1516: https://github.com/nedbat/coveragepy/issues/1516
.. _issue 1519: https://github.com/nedbat/coveragepy/issues/1519


.. _changes_7-0-1:

Links

• PyPI: https://pypi.org/project/coverage
• Changelog: https://data.safetycli.com/changelogs/coverage/
• Repo: https://github.com/nedbat/coveragepy

Update django from 4.1.4 to 4.2.7.

Changelog

4.2.7

==========================

*November 1, 2023*

Django 4.2.7 fixes a security issue with severity "moderate" and several bugs
in 4.2.6.

CVE-2023-46695: Potential denial of service vulnerability in ``UsernameField`` on Windows
=========================================================================================

The :func:`NFKC normalization <python:unicodedata.normalize>` is slow on
Windows. As a consequence, ``django.contrib.auth.forms.UsernameField`` was
subject to a potential denial of service attack via certain inputs with a very
large number of Unicode characters.

In order to avoid the vulnerability, invalid values longer than
``UsernameField.max_length`` are no longer normalized, since they cannot pass
validation anyway.

Bugfixes
========

* Fixed a regression in Django 4.2 that caused a crash of
``QuerySet.aggregate()`` with aggregates referencing expressions containing
subqueries (:ticket:`34798`).

* Restored, following a regression in Django 4.2, creating
``varchar/text_pattern_ops`` indexes on ``CharField`` and ``TextField`` with
deterministic collations on PostgreSQL (:ticket:`34932`).


==========================

4.2.6

==========================

*October 4, 2023*

Django 4.2.6 fixes a security issue with severity "moderate" and several bugs
in 4.2.5.

CVE-2023-43665: Denial-of-service possibility in ``django.utils.text.Truncator``
================================================================================

Following the fix for :cve:`2019-14232`, the regular expressions used in the
implementation of ``django.utils.text.Truncator``'s ``chars()`` and ``words()``
methods (with ``html=True``) were revised and improved. However, these regular
expressions still exhibited linear backtracking complexity, so when given a
very long, potentially malformed HTML input, the evaluation would still be
slow, leading to a potential denial of service vulnerability.

The ``chars()`` and ``words()`` methods are used to implement the
:tfilter:`truncatechars_html` and :tfilter:`truncatewords_html` template
filters, which were thus also vulnerable.

The input processed by ``Truncator``, when operating in HTML mode, has been
limited to the first five million characters in order to avoid potential
performance and memory issues.

Bugfixes
========

* Fixed a regression in Django 4.2.5 where overriding the deprecated
``DEFAULT_FILE_STORAGE`` and ``STATICFILES_STORAGE`` settings in tests caused
the main ``STORAGES`` to mutate (:ticket:`34821`).

* Fixed a regression in Django 4.2 that caused unnecessary casting of string
based fields (``CharField``, ``EmailField``, ``TextField``, ``CICharField``,
``CIEmailField``, and ``CITextField``) used with the ``__isnull`` lookup on
PostgreSQL. As a consequence, indexes using an ``__isnull`` expression or
condition created before Django 4.2 wouldn't be used by the query planner,
leading to a performance regression (:ticket:`34840`).

You may need to recreate such indexes created in your database with Django
4.2 to 4.2.5, as they contain unnecessary ``::text`` casting. Find candidate
indexes with this query:

.. code-block:: sql

     SELECT indexname, indexdef
     FROM pg_indexes
     WHERE indexdef LIKE '%::text IS %NULL';


==========================

4.2.5

==========================

*September 4, 2023*

Django 4.2.5 fixes a security issue with severity "moderate" and several bugs
in 4.2.4.

CVE-2023-41164: Potential denial of service vulnerability in ``django.utils.encoding.uri_to_iri()``
===================================================================================================

``django.utils.encoding.uri_to_iri()`` was subject to potential denial of
service attack via certain inputs with a very large number of Unicode
characters.

Bugfixes
========

* Fixed a regression in Django 4.2 that caused an incorrect validation of
``CheckConstraints`` on ``__isnull`` lookups against ``JSONField``
(:ticket:`34754`).

* Fixed a bug in Django 4.2 where the deprecated ``DEFAULT_FILE_STORAGE`` and
``STATICFILES_STORAGE`` settings were not synced with ``STORAGES``
(:ticket:`34773`).

* Fixed a regression in Django 4.2.2 that caused an unnecessary selection of a
non-nullable ``ManyToManyField`` without a natural key during serialization
(:ticket:`34779`).

* Fixed a regression in Django 4.2 that caused a crash of a queryset when
filtering against deeply nested ``OuterRef()`` annotations (:ticket:`34803`).


==========================

4.2.4

==========================

*August 1, 2023*

Django 4.2.4 fixes several bugs in 4.2.3.

Bugfixes
========

* Fixed a regression in Django 4.2 that caused a crash of
``QuerySet.aggregate()`` with aggregates referencing window functions
(:ticket:`34717`).

* Fixed a regression in Django 4.2 that caused a crash when grouping by a
reference in a subquery (:ticket:`34748`).

* Fixed a regression in Django 4.2 that caused aggregation over query that
uses explicit grouping by multi-valued annotations to group against the wrong
columns (:ticket:`34750`).


==========================

4.2.3

==========================

*July 3, 2023*

Django 4.2.3 fixes a security issue with severity "moderate" and several bugs
in 4.2.2.

CVE-2023-36053: Potential regular expression denial of service vulnerability in ``EmailValidator``/``URLValidator``
===================================================================================================================

``EmailValidator`` and ``URLValidator`` were subject to potential regular
expression denial of service attack via a very large number of domain name
labels of emails and URLs.

Bugfixes
========

* Fixed a regression in Django 4.2 that caused incorrect alignment of timezone
warnings for ``DateField`` and ``TimeField`` in the admin (:ticket:`34645`).

* Fixed a regression in Django 4.2 that caused incorrect highlighting of rows
in the admin changelist view when ``ModelAdmin.list_editable`` contained a
``BooleanField`` (:ticket:`34638`).


==========================

4.2.2

==========================

*June 5, 2023*

Django 4.2.2 fixes several bugs in 4.2.1.

Bugfixes
========

* Fixed a regression in Django 4.2 that caused an unnecessary
``DBMS_LOB.SUBSTR()`` wrapping in the ``__isnull`` and ``__exact=None``
lookups for ``TextField()``/``BinaryField()`` on Oracle (:ticket:`34544`).

* Restored, following a regression in Django 4.2, ``get_prep_value()`` call in
``JSONField`` subclasses (:ticket:`34539`).

* Fixed a regression in Django 4.2 that caused a crash of ``QuerySet.defer()``
when passing a ``ManyToManyField`` or ``GenericForeignKey`` reference. While
doing so is a no-op, it was allowed in older version (:ticket:`34570`).

* Fixed a regression in Django 4.2 that caused a crash of ``QuerySet.only()``
when passing a reverse ``OneToOneField`` reference (:ticket:`34612`).

* Fixed a bug in Django 4.2 where :option:`makemigrations --update` didn't
respect the ``--name`` option (:ticket:`34568`).

* Fixed a performance regression in Django 4.2 when compiling queries without
ordering (:ticket:`34580`).

* Fixed a regression in Django 4.2 where nonexistent stylesheet was linked on a
“Congratulations!” page (:ticket:`34588`).

* Fixed a regression in Django 4.2 that caused a crash of
``QuerySet.aggregate()`` with expressions referencing other aggregates
(:ticket:`34551`).

* Fixed a regression in Django 4.2 that caused a crash of
``QuerySet.aggregate()`` with aggregates referencing subqueries
(:ticket:`34551`).

* Fixed a regression in Django 4.2 that caused a crash of querysets on SQLite
when filtering on ``DecimalField`` against values outside of the defined
range (:ticket:`34590`).

* Fixed a regression in Django 4.2 that caused a serialization crash on a
``ManyToManyField`` without a natural key when its ``Manager``’s base
``QuerySet`` used ``select_related()`` (:ticket:`34620`).


==========================

4.2.1

==========================

*May 3, 2023*

Django 4.2.1 fixes a security issue with severity "low" and several bugs in
4.2.

CVE-2023-31047: Potential bypass of validation when uploading multiple files using one form field
=================================================================================================

Uploading multiple files using one form field has never been supported by
:class:`.forms.FileField` or :class:`.forms.ImageField` as only the last
uploaded file was validated. Unfortunately, :ref:`uploading_multiple_files`
topic suggested otherwise.

In order to avoid the vulnerability, :class:`~django.forms.ClearableFileInput`
and :class:`~django.forms.FileInput` form widgets now raise ``ValueError`` when
the ``multiple`` HTML attribute is set on them. To prevent the exception and
keep the old behavior, set ``allow_multiple_selected`` to ``True``.

For more details on using the new attribute and handling of multiple files
through a single field, see :ref:`uploading_multiple_files`.

Bugfixes
========

* Fixed a regression in Django 4.2 that caused a crash of ``QuerySet.defer()``
when deferring fields by attribute names (:ticket:`34458`).

* Fixed a regression in Django 4.2 that caused a crash of
:class:`~django.contrib.postgres.search.SearchVector` function with ``%``
characters (:ticket:`34459`).

* Fixed a regression in Django 4.2 that caused aggregation over query that
uses explicit grouping to group against the wrong columns (:ticket:`34464`).

* Reallowed, following a regression in Django 4.2, setting the
``"cursor_factory"`` option in :setting:`OPTIONS` on PostgreSQL
(:ticket:`34466`).

* Enforced UTF-8 client encoding on PostgreSQL, following a regression in
Django 4.2 (:ticket:`34470`).

* Fixed a regression in Django 4.2 where ``i18n_patterns()`` didn't respect the
``prefix_default_language`` argument when a fallback language of the default
language was used (:ticket:`34455`).

* Fixed a regression in Django 4.2 where translated URLs of the default
language from ``i18n_patterns()`` with ``prefix_default_language`` set to
``False`` raised 404 errors for a request with a different language
(:ticket:`34515`).

* Fixed a regression in Django 4.2 where creating copies and deep copies of
``HttpRequest``, ``HttpResponse``, and their subclasses didn't always work
correctly (:ticket:`34482`, :ticket:`34484`).

* Fixed a regression in Django 4.2 where ``timesince`` and ``timeuntil``
template filters returned incorrect results for a datetime with a non-UTC
timezone when a time difference is less than 1 day (:ticket:`34483`).

* Fixed a regression in Django 4.2 that caused a crash of
:class:`~django.contrib.postgres.search.SearchHeadline` function with
``psycopg`` 3 (:ticket:`34486`).

* Fixed a regression in Django 4.2 that caused incorrect ``ClearableFileInput``
margins in the admin (:ticket:`34506`).

* Fixed a regression in Django 4.2 where breadcrumbs didn't appear on admin
site app index views (:ticket:`34512`).

* Made squashing migrations reduce ``AddIndex``, ``RemoveIndex``,
``RenameIndex``, and ``CreateModel`` operations which allows removing a
deprecated ``Meta.index_together`` option from historical migrations and use
``Meta.indexes`` instead (:ticket:`34525`).


========================

4.2

========================

*April 3, 2023*

Welcome to Django 4.2!

These release notes cover the :ref:`new features <whats-new-4.2>`, as well as
some :ref:`backwards incompatible changes <backwards-incompatible-4.2>` you'll
want to be aware of when upgrading from Django 4.1 or earlier. We've
:ref:`begun the deprecation process for some features
<deprecated-features-4.2>`.

See the :doc:`/howto/upgrade-version` guide if you're updating an existing
project.

Django 4.2 is designated as a :term:`long-term support release
<Long-term support release>`. It will receive security updates for at least
three years after its release. Support for the previous LTS, Django 3.2, will
end in April 2024.

Python compatibility
====================

Django 4.2 supports Python 3.8, 3.9, 3.10, 3.11, and 3.12 (as of 4.2.8). We
**highly recommend** and only officially support the latest release of each
series.

.. _whats-new-4.2:

What's new in Django 4.2
========================

Psycopg 3 support
-----------------

Django now supports `psycopg`_ version 3.1.8 or higher. To update your code,
install the :pypi:`psycopg library <psycopg>`, you don't need to change the
:setting:`ENGINE <DATABASE-ENGINE>` as ``django.db.backends.postgresql``
supports both libraries.

Support for ``psycopg2`` is likely to be deprecated and removed at some point
in the future.

Be aware that ``psycopg`` 3 introduces some breaking changes over ``psycopg2``.
As a consequence, you may need to make some changes to account for
`differences from psycopg2`_.

.. _psycopg: https://www.psycopg.org/psycopg3/
.. _differences from psycopg2: https://www.psycopg.org/psycopg3/docs/basic/from_pg2.html

Comments on columns and tables
------------------------------

The new :attr:`Field.db_comment <django.db.models.Field.db_comment>` and
:attr:`Meta.db_table_comment <django.db.models.Options.db_table_comment>`
options allow creating comments on columns and tables, respectively. For
example::

 from django.db import models


 class Question(models.Model):
     text = models.TextField(db_comment="Poll question")
     pub_date = models.DateTimeField(
         db_comment="Date and time when the question was published",
     )

     class Meta:
         db_table_comment = "Poll questions"


 class Answer(models.Model):
     question = models.ForeignKey(
         Question,
         on_delete=models.CASCADE,
         db_comment="Reference to a question",
     )
     answer = models.TextField(db_comment="Question answer")

     class Meta:
         db_table_comment = "Question answers"

Also, the new :class:`~django.db.migrations.operations.AlterModelTableComment`
operation allows changing table comments defined in the
:attr:`Meta.db_table_comment <django.db.models.Options.db_table_comment>`.

Mitigation for the BREACH attack
--------------------------------

:class:`~django.middleware.gzip.GZipMiddleware` now includes a mitigation for
the BREACH attack. It will add up to 100 random bytes to gzip responses to make
BREACH attacks harder. Read more about the mitigation technique in the `Heal
The Breach (HTB) paper`_.

.. _Heal The Breach (HTB) paper: https://ieeexplore.ieee.org/document/9754554

In-memory file storage
----------------------

The new :class:`django.core.files.storage.InMemoryStorage` class provides a
non-persistent storage useful for speeding up tests by avoiding disk access.

Custom file storages
--------------------

The new :setting:`STORAGES` setting allows configuring multiple custom file
storage backends. It also controls storage engines for managing
:doc:`files </topics/files>` (the ``"default"`` key) and :doc:`static files
</ref/contrib/staticfiles>` (the ``"staticfiles"`` key).

The old ``DEFAULT_FILE_STORAGE`` and ``STATICFILES_STORAGE`` settings are
deprecated as of this release.

Minor features
--------------

:mod:`django.contrib.admin`
~~~~~~~~~~~~~~~~~~~~~~~~~~~

* The light or dark color theme of the admin can now be toggled in the UI, as
well as being set to follow the system setting.

* The admin's font stack now prefers system UI fonts and no longer requires
downloading fonts. Additionally, CSS variables are available to more easily
override the default font families.

* The :source:`admin/delete_confirmation.html
<django/contrib/admin/templates/admin/delete_confirmation.html>` template now
has some additional blocks and scripting hooks to ease customization.

* The chosen options of
:attr:`~django.contrib.admin.ModelAdmin.filter_horizontal` and
:attr:`~django.contrib.admin.ModelAdmin.filter_vertical` widgets are now
filterable.

* The ``admin/base.html`` template now has a new block ``nav-breadcrumbs``
which contains the navigation landmark and the ``breadcrumbs`` block.

* :attr:`.ModelAdmin.list_editable` now uses atomic transactions when making
edits.

* jQuery is upgraded from version 3.6.0 to 3.6.4.

:mod:`django.contrib.auth`
~~~~~~~~~~~~~~~~~~~~~~~~~~

* The default iteration count for the PBKDF2 password hasher is increased from
390,000 to 600,000.

* :class:`~django.contrib.auth.forms.UserCreationForm` now saves many-to-many
form fields for a custom user model.

* The new :class:`~django.contrib.auth.forms.BaseUserCreationForm` is now the
recommended base class for customizing the user creation form.

:mod:`django.contrib.gis`
~~~~~~~~~~~~~~~~~~~~~~~~~

* The :doc:`GeoJSON serializer </ref/contrib/gis/serializers>` now outputs the
``id`` key for serialized features, which defaults to the primary key of
objects.

* The :class:`~django.contrib.gis.gdal.GDALRaster` class now supports
:class:`pathlib.Path`.

* The :class:`~django.contrib.gis.geoip2.GeoIP2` class now supports  ``.mmdb``
files downloaded from DB-IP.

* The OpenLayers template widget no longer includes inline CSS (which also
removes the former ``map_css`` block) to better comply with a strict Content
Security Policy.

* :class:`~django.contrib.gis.forms.widgets.OpenLayersWidget` is now based on
OpenLayers 7.2.2 (previously 4.6.5).

* The new :lookup:`isempty` lookup and
:class:`IsEmpty() <django.contrib.gis.db.models.functions.IsEmpty>`
expression allow filtering empty geometries on PostGIS.

* The new :class:`FromWKB() <django.contrib.gis.db.models.functions.FromWKB>`
and :class:`FromWKT() <django.contrib.gis.db.models.functions.FromWKT>`
functions allow creating geometries from Well-known binary (WKB) and
Well-known text (WKT) representations.

:mod:`django.contrib.postgres`
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

* The new :lookup:`trigram_strict_word_similar` lookup, and the
:class:`TrigramStrictWordSimilarity()
<django.contrib.postgres.search.TrigramStrictWordSimilarity>` and
:class:`TrigramStrictWordDistance()
<django.contrib.postgres.search.TrigramStrictWordDistance>` expressions allow
using trigram strict word similarity.

* The :lookup:`arrayfield.overlap` lookup now supports ``QuerySet.values()``
and ``values_list()`` as a right-hand side.

:mod:`django.contrib.sitemaps`
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

* The new :meth:`.Sitemap.get_languages_for_item` method allows customizing the
list of languages for which the item is displayed.

:mod:`django.contrib.staticfiles`
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

* :class:`~django.contrib.staticfiles.storage.ManifestStaticFilesStorage` now
has experimental support for replacing paths to JavaScript modules in
``import`` and ``export`` statements with their hashed counterparts. If you
want to try it, subclass ``ManifestStaticFilesStorage`` and set the
``support_js_module_import_aggregation`` attribute to ``True``.

* The new :attr:`.ManifestStaticFilesStorage.manifest_hash` attribute provides
a hash over all files in the manifest and changes whenever one of the files
changes.

Database backends
~~~~~~~~~~~~~~~~~

* The new ``"assume_role"`` option is now supported in :setting:`OPTIONS` on
PostgreSQL to allow specifying the :ref:`session role <database-role>`.

* The new ``"server_side_binding"`` option is now supported in
:setting:`OPTIONS` on PostgreSQL with ``psycopg`` 3.1.8+ to allow using
:ref:`server-side binding cursors <database-server-side-parameters-binding>`.

Error Reporting
~~~~~~~~~~~~~~~

* The debug page now shows :pep:`exception notes <678>` and
:pep:`fine-grained error locations <657>` on Python 3.11+.

* Session cookies are now treated as credentials and therefore hidden and
replaced with stars (``**********``) in error reports.

Forms
~~~~~

* :class:`~django.forms.ModelForm` now accepts the new ``Meta`` option
``formfield_callback`` to customize form fields.

* :func:`~django.forms.models.modelform_factory` now respects the
``formfield_callback`` attribute of the ``form``’s ``Meta``.

Internationalization
~~~~~~~~~~~~~~~~~~~~

* Added support and translations for the Central Kurdish (Sorani) language.

Logging
~~~~~~~

* The :ref:`django-db-logger` logger now logs transaction management queries
(``BEGIN``, ``COMMIT``, and ``ROLLBACK``) at the ``DEBUG`` level.

Management Commands
~~~~~~~~~~~~~~~~~~~

* :djadmin:`makemessages` command now supports locales with private sub-tags
such as ``nl_NL-x-informal``.

* The new :option:`makemigrations --update` option merges model changes into
the latest migration and optimizes the resulting operations.

Migrations
~~~~~~~~~~

* Migrations now support serialization of ``enum.Flag`` objects.

Models
~~~~~~

* ``QuerySet`` now extensively supports filtering against
:ref:`window-functions` with the exception of disjunctive filter lookups
against window functions when performing aggregation.

* :meth:`~.QuerySet.prefetch_related` now supports
:class:`~django.db.models.Prefetch` objects with sliced querysets.

* :ref:`Registering lookups <lookup-registration-api>` on
:class:`~django.db.models.Field` instances is now supported.

* The new ``robust`` argument for :func:`~django.db.transaction.on_commit`
allows performing actions that can fail after a database transaction is
successfully committed.

* The new :class:`KT() <django.db.models.fields.json.KT>` expression represents
the text value of a key, index, or path transform of
:class:`~django.db.models.JSONField`.

* :class:`~django.db.models.functions.Now` now supports microsecond precision
on MySQL and millisecond precision on SQLite.

* :class:`F() <django.db.models.F>` expressions that output ``BooleanField``
can now be negated using ``~F()`` (inversion operator).

* ``Model`` now provides asynchronous versions of some methods that use the
database, using an ``a`` prefix: :meth:`~.Model.adelete`,
:meth:`~.Model.arefresh_from_db`, and :meth:`~.Model.asave`.

* Related managers now provide asynchronous versions of methods that change a
set of related objects, using an ``a`` prefix: :meth:`~.RelatedManager.aadd`,
:meth:`~.RelatedManager.aclear`, :meth:`~.RelatedManager.aremove`, and
:meth:`~.RelatedManager.aset`.

* :attr:`CharField.max_length <django.db.models.CharField.max_length>` is no
longer required to be set on PostgreSQL, which supports unlimited ``VARCHAR``
columns.

Requests and Responses
~~~~~~~~~~~~~~~~~~~~~~

* :class:`~django.http.StreamingHttpResponse` now supports async iterators
when Django is served via ASGI.

Tests
~~~~~

* The :option:`test --debug-sql` option now formats SQL queries with
``sqlparse``.

* The :class:`~django.test.RequestFactory`,
:class:`~django.test.AsyncRequestFactory`, :class:`~django.test.Client`, and
:class:`~django.test.AsyncClient` classes now support the ``headers``
parameter, which accepts a dictionary of header names and values. This allows
a more natural syntax for declaring headers.

.. code-block:: python

   Before:
  self.client.get("/home/", HTTP_ACCEPT_LANGUAGE="fr")
  await self.async_client.get("/home/", ACCEPT_LANGUAGE="fr")

   After:
  self.client.get("/home/", headers={"accept-language": "fr"})
  await self.async_client.get("/home/", headers={"accept-language": "fr"})

Utilities
~~~~~~~~~

* The new ``encoder`` parameter for :meth:`django.utils.html.json_script`
function allows customizing a JSON encoder class.

* The private internal vendored copy of ``urllib.parse.urlsplit()`` now strips
``'\r'``, ``'\n'``, and ``'\t'`` (see :cve:`2022-0391` and :bpo:`43882`).
This is to protect projects that may be incorrectly using the internal
``url_has_allowed_host_and_scheme()`` function, instead of using one of the
documented functions for handling URL redirects. The Django functions were
not affected.

* The new :func:`django.utils.http.content_disposition_header` function returns
a ``Content-Disposition`` HTTP header value as specified by :rfc:`6266`.

Validators
~~~~~~~~~~

* The list of common passwords used by ``CommonPasswordValidator`` is updated
to the most recent version.

.. _backwards-incompatible-4.2:

Backwards incompatible changes in 4.2
=====================================

Database backend API
--------------------

This section describes changes that may be needed in third-party database
backends.

* ``DatabaseFeatures.allows_group_by_pk`` is removed as it only remained to
accommodate a MySQL extension that has been supplanted by proper functional
dependency detection in MySQL 5.7.15. Note that
``DatabaseFeatures.allows_group_by_selected_pks`` is still supported and
should be enabled if your backend supports functional dependency detection in
``GROUP BY`` clauses as specified by the ``SQL:1999`` standard.

* :djadmin:`inspectdb` now uses ``display_size`` from
``DatabaseIntrospection.get_table_description()`` rather than
``internal_size`` for ``CharField``.

Dropped support for MariaDB 10.3
--------------------------------

Upstream support for MariaDB 10.3 ends in May 2023. Django 4.2 supports MariaDB
10.4 and higher.

Dropped support for MySQL 5.7
-----------------------------

Upstream support for MySQL 5.7 ends in October 2023. Django 4.2 supports MySQL
8 and higher.

Dropped support for PostgreSQL 11
---------------------------------

Upstream support for PostgreSQL 11 ends in November 2023. Django 4.2 supports
PostgreSQL 12 and higher.

Setting ``update_fields`` in ``Model.save()`` may now be required
-----------------------------------------------------------------

In order to avoid updating unnecessary columns,
:meth:`.QuerySet.update_or_create` now passes ``update_fields`` to the
:meth:`Model.save() <django.db.models.Model.save>` calls. As a consequence, any
fields modified in the custom ``save()`` methods should be added to the
``update_fields`` keyword argument before calling ``super()``. See
:ref:`overriding-model-methods` for more details.

Miscellaneous
-------------

* The undocumented ``django.http.multipartparser.parse_header()`` function is
removed. Use ``django.utils.http.parse_header_parameters()`` instead.

* :ttag:`{% blocktranslate asvar … %}<blocktranslate>` result is now marked as
safe for (HTML) output purposes.

* The ``autofocus`` HTML attribute in the admin search box is removed as it can
be confusing for screen readers.

* The :option:`makemigrations --check` option no longer creates missing
migration files.

* The ``alias`` argument for :meth:`.Expression.get_group_by_cols` is removed.

* The minimum supported version of ``sqlparse`` is increased from 0.2.2 to
0.3.1.

* The undocumented ``negated`` parameter of the
:class:`~django.db.models.Exists` expression is removed.

* The ``is_summary`` argument of the undocumented ``Query.add_annotation()``
method is removed.

* The minimum supported version of SQLite is increased from 3.9.0 to 3.21.0.

* The minimum supported version of ``asgiref`` is increased from 3.5.2 to
3.6.0.

* :class:`~django.contrib.auth.forms.UserCreationForm` now rejects usernames
that differ only in case. If you need the previous behavior, use
:class:`~django.contrib.auth.forms.BaseUserCreationForm` instead.

* The minimum supported version of ``mysqlclient`` is increased from 1.4.0 to
1.4.3.

* The minimum supported version of ``argon2-cffi`` is increased  from 19.1.0 to
19.2.0.

* The minimum supported version of ``Pillow`` is increased from 6.2.0 to 6.2.1.

* The minimum supported version of ``jinja2`` is increased from 2.9.2 to
2.11.0.

* The minimum supported version of :pypi:`redis-py <redis>` is increased from
3.0.0 to 3.4.0.

* Manually instantiated ``WSGIRequest`` objects must be provided a file-like
object for ``wsgi.input``. Previously, Django was more lax than the expected
behavior as specified by the WSGI specification.

* Support for ``PROJ`` < 5 is removed.

* :class:`~django.core.mail.backends.smtp.EmailBackend` now verifies a
:py:attr:`hostname <ssl.SSLContext.check_hostname>` and
:py:attr:`certificates <ssl.SSLContext.verify_mode>`. If you need the
previous behavior that is less restrictive and not recommended, subclass
``EmailBackend`` and override the ``ssl_context`` property.

.. _deprecated-features-4.2:

Features deprecated in 4.2
==========================

``index_together`` option is deprecated in favor of ``indexes``
-----

[pyup-bot]

Closing this in favor of #198