Skip to content
Merged
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
98 changes: 52 additions & 46 deletions peps/pep-0808.rst
Original file line number Diff line number Diff line change
Expand Up @@ -17,9 +17,9 @@ Abstract
========

This PEP relaxes the constraint on dynamic metadata listed in the ``[project]``
section in ``pyproject.toml`` to allow the static portion of mixed metadata to
be defined in the normal location if the field is a table or array by having
the dynamic fields extend the static ones.
section in ``pyproject.toml``. It is now permitted to define a static portion
of a dynamic metadata field in the ``[project]`` table as long as the field is
a table or array.

This allows users to opt into allowing a backend to extend metadata while still
keeping the static portions of the metadata defined in the standard location in
Expand Down Expand Up @@ -52,9 +52,9 @@ as creating "used by" and "uses" graphs. It is used for code quality tooling to
detect the minimum supported version of Python. It is used by cibuildwheel_ to
automatically avoid building wheels that are not supported. It is not used,
however, to avoid wheel builds when the SDist is available; that was addressed
by METADATA 2.2, which a ``Dynamic`` field in the SDist metadata that lets a
tool know if the metadata can change when making a wheel - this is an easy
mistake to make due to the similarity of the names.
by METADATA 2.2, which added a ``Dynamic`` field in the SDist metadata that
lets a tool know if the metadata can change when making a wheel - this is an
easy mistake to make due to the similarity of the names.

Due to the rapidly increasing popularity of the project table, support from all
major backends, and a rise of backends supporting complex compiled extensions,
Expand All @@ -65,8 +65,8 @@ definition. For the most common use cases, this is fine; there is little
benefit to set the ``version`` statically if you are going to override it
dynamically. If you are using a custom README processor to filter or modify the
README for proper display, it's not a big deal to have to specify the
configuration in a custom ``tool.*`` section. But there is a specific class of
cases where the all-or-nothing approach is problematic: lists of items where
configuration in a custom ``[tool.*]`` section. But there is a specific class
of cases where the all-or-nothing approach is problematic: lists of items where
the backend needs to add items are currently forced to be fully dynamically
specified (that is, in a backend-specific configuration location). This causes
both of the original benefits (standard location and static tooling support) to
Expand All @@ -93,8 +93,7 @@ restrictions on the ``[project]`` table and ``project.dynamic`` list.
Every list and every table with arbitrary keys will now be allowed to be
specified both statically, in the ``[project]`` table, and in the
``project.dynamic`` list. If it is present in both places, the build backend
can extend list items and add new keys, but not modify existing list items or
strings.
can extend list items and add new keys, but not modify existing entries.


Use Cases
Expand All @@ -107,26 +106,27 @@ cases that have come up:
- Pinning dependency requirements when building the wheel. When building
PyTorch_ extensions, for example, the version you build with adds a constraint
to the wheel you create that is not present with the SDist.
- Generating extra scripts from a build system (this is a currently proposed in
- Generating extra scripts from a build system (this is currently proposed in
scikit-build-core_).
- Adding entry points dynamically (validate-pyproject-schema-store_ could have
used this to generate an entry point for each schema present in the package.)
used this to generate an entry point for each schema present in the package).
- Adding dependencies or optional dependencies based on configuration (such as
making an all dependency, or reading dependencies from dependency-groups, for
example). Adding constraints can also be useful; pybind11_ uses adds a ``global``
example). Adding constraints can also be useful; pybind11_ uses a ``global``
extra that pins ``pybind11-global==<version>``, as both packages are in the
same repository and released in sync. toga_ is a collection of packages that
currently is unable to set any static dependencies due to the same sort of
is currently unable to set any static dependencies due to the same sort of
pinning problem.
- Adding classifiers; some backends can compute classifiers from other places
and inject them (Poetry_ being the best known example).
- Adding license files to the wheel based on what libraries are linked in (this
is an active discussion in followup to :pep:`639`).
- Adding SBom's when building - :pep:`770` had to remove the ``pyproject.toml``
is an active discussion in follow-up to :pep:`639`).
- Adding SBOMs when building - :pep:`770` had to remove the ``pyproject.toml``
field specifically because you *want* the build tool to add these, so the
``[project]`` table setting would be useless, you'd almost never be able to
use it.
- Adding generated modules to ``import-names`` or ``import-namespaces``.
``[project]`` table setting would be useless, you would almost never be able
to use it.
- Adding generated modules to ``import-names`` or ``import-namespaces`` is
another example.

All of these use cases have a similar feature: they are adding something
dynamically to a fixed list (possibly a narrower pin for the dependency case).
Expand All @@ -141,7 +141,7 @@ availability of metadata for static tooling.
Example: pinning
----------------

For example, let's say you want to allow an imaginary build backend
For example, say you want to allow an imaginary build backend
(``my-build-backend``) to pin to the supported build of PyTorch_. Before this
PEP, you could do this:

Expand All @@ -162,16 +162,16 @@ Which would effectively expand to the following SDist metadata:
Requires-Dist: packaging
Requires-Dist: torch

Which would then could make a wheel with this:
Which could then make a wheel with this:

.. code-block:: text

Requires-Dist: packaging
Requires-Dist: torch
Requires-Dist: torch==2.8.0

Static tooling no longer can tell that ``torch`` and ``packaging`` are runtime
dependencies, and the build backend had to duplicate the dependency table,
Static tooling can no longer tell that ``torch`` and ``packaging`` are runtime
dependencies, and the build backend has to duplicate the dependency table,
making it harder for users to learn and read; the standardized place proposed
by :pep:`621` and adopted by all major build backends is lost.

Expand All @@ -198,8 +198,8 @@ Future Updates

Loosening this rule to allow purely additive metadata should address many of
the use cases that have been seen in practice. If further changes are needed,
this can be revisited in a future PEP; this PEP neither recommends or precludes
future updates like this.
this can be revisited in a future PEP; this PEP neither recommends nor
precludes future updates like this.

Terminology
===========
Expand Down Expand Up @@ -231,13 +231,13 @@ The fields that are arrays or tables with arbitrary entries are:
be changed or removed.
* ``keywords``: Keywords can be added to the list.
* ``license-files``: Files can be added to the list.
* ``optional-dependencies``: A new extra or new items can be added to a
* ``optional-dependencies``: A new extra or new items can be added to an
existing extra.
* ``urls``: New urls can be added. Existing ones cannot be changed or removed.
* ``import-names``, ``import-namespaces``: New import names or namespaces can
be added. Existing ones cannot be modified or removed.

To add items, users must opt-in by listing the field in ``dynamic``; without
To add items, users must opt in by listing the field in ``dynamic``; without
that, the metadata continues to be entirely static.

A backend SHOULD error if a field is specified and it does not support
Expand All @@ -250,7 +250,7 @@ Static analysis tools, when detecting a field is both specified and in the
new entries to be present when the package is built.

The ``Dynamic`` field, as specified in :pep:`643`, is unaffected by this PEP,
and backends can continue to fill it as they chose. However, a backend MUST
and backends can continue to fill it as they choose. However, a backend MUST
ensure that both the SDist and the wheel metadata include the static metadata
portion of the project table.

Expand All @@ -263,7 +263,7 @@ to do with dynamic metadata.

The pyproject-metadata_ project, which is used by
several build backends, will need to modify the correctness check to account
for the possible extensions; this is in `a draft PR <pyprojectmetdatapr>`__.
for the possible extensions; this is in `a draft PR <pyprojectmetadatapr>`__.

The dynamic-metadata_ project, which provides a plugin
system that backends can use to share dynamic metadata plugins, was designed to
Expand All @@ -276,15 +276,15 @@ Backwards Compatibility
Using metadata from SDists or wheels is unaffected. The METADATA version does
not need to be incremented.

This does not affect any existing ``pyproject.toml``'s, since this was strictly
not allowed before this PEP.
This does not affect any existing ``pyproject.toml`` files, since this was
strictly not allowed before this PEP.

When users adopt this in a ``pyproject.toml``, the backend must support it; an
error will be correctly generated if it doesn't following the previous
error will be correctly generated if it doesn't, following the previous
standard. Frontends were never required to throw an error, though some
frontends may need to be updated to benefit from the partially static metadata.
Some frontends and other tooling may need updating, such as schema
validation, just like other ``pyproject.toml`` PEPs.
validators, just like other ``pyproject.toml`` PEPs.

Static analysis tools may require updating to handle this change. Tools should
check the dynamic table first, like this:
Expand Down Expand Up @@ -317,19 +317,26 @@ a static component to existing dynamic metadata support.
How to Teach This
=================

If you currently have dynamic metadata, but some list or table entries are
known statically, you can now make that explicit by adding the static entry in
the ``[project]`` table, while also keeping the entry in the
``project.dynamic`` list to allow the dynamic portion to be added by your build
backend.

The current guides that state metadata must not be listed in both ``[project]``
and ``project.dynamic`` can be updated to say that some fields can be extended
by ``project.dynamic``. Since dynamic metadata is already an advanced concept,
this will likely not affect most existing tutorial material aimed at
introductory packaging.
and ``project.dynamic`` can be updated to say that lists and tables marked with
``project.dynamic`` can still have static entries. Since dynamic metadata is
already an advanced concept, this will likely not affect most existing tutorial
material aimed at introductory packaging.

The ``pyproject.toml`` `specification <pyprojectspec>`__ will be updated to
include the behavior of fields when specified and also listed in the dynamic
field.

It should also be noted that specifying something in dynamic will require any
tool that requires the full metadata to invoke the backend even if it is
partially statically specified, so it should not be used unless necessary.
It should also be noted that specifying something in ``dynamic`` will require
any tool that needs the full metadata to invoke the backend even if it is
partially statically specified. So, it should not be used unless necessary,
just like any other dynamic metadata.


Rejected Ideas
Expand Down Expand Up @@ -378,12 +385,11 @@ Allow simplifications
An earlier draft of this PEP had a clause allowing backends to simplify some
types of fields; most notably dependency specifiers would have allowed
"tightening", such as ``torch`` being replaced by ``torch>=1.2``, for example.
. This was removed due to it being impossible to ensure a variation will
resolve identically on all resolvers within the current specification, and to
simplify the contract with backends. Any other simplifications would be purely
cosmetic, and so were left out. The order in the current PEP is now required to
match the original static metadata, with the dynamic portion only allowing
insertions.
This was removed due to it being impossible to ensure a variation will resolve
identically on all resolvers within the current specification, and to simplify
the contract with backends. Any other simplifications would be purely cosmetic,
and so were left out. The order in the current PEP is now required to match the
original static metadata, with the dynamic portion only allowing insertions.


Add a general mechanism to specify dynamic-metadata
Expand Down