diff --git a/peps/pep-0808.rst b/peps/pep-0808.rst index fb4e0ed4725..8294a8dbf15 100644 --- a/peps/pep-0808.rst +++ b/peps/pep-0808.rst @@ -17,9 +17,9 @@ Abstract ======== This PEP relaxes the constraint on dynamic metadata listed in the ``[project]`` -section in ``pyproject.toml`` to allow the static portion of mixed metadata to -be defined in the normal location if the field is a table or array by having -the dynamic fields extend the static ones. +section in ``pyproject.toml``. It is now permitted to define a static portion +of a dynamic metadata field in the ``[project]`` table as long as the field is +a table or array. This allows users to opt into allowing a backend to extend metadata while still keeping the static portions of the metadata defined in the standard location in @@ -52,9 +52,9 @@ as creating "used by" and "uses" graphs. It is used for code quality tooling to detect the minimum supported version of Python. It is used by cibuildwheel_ to automatically avoid building wheels that are not supported. It is not used, however, to avoid wheel builds when the SDist is available; that was addressed -by METADATA 2.2, which a ``Dynamic`` field in the SDist metadata that lets a -tool know if the metadata can change when making a wheel - this is an easy -mistake to make due to the similarity of the names. +by METADATA 2.2, which added a ``Dynamic`` field in the SDist metadata that +lets a tool know if the metadata can change when making a wheel - this is an +easy mistake to make due to the similarity of the names. Due to the rapidly increasing popularity of the project table, support from all major backends, and a rise of backends supporting complex compiled extensions, @@ -65,8 +65,8 @@ definition. For the most common use cases, this is fine; there is little benefit to set the ``version`` statically if you are going to override it dynamically. If you are using a custom README processor to filter or modify the README for proper display, it's not a big deal to have to specify the -configuration in a custom ``tool.*`` section. But there is a specific class of -cases where the all-or-nothing approach is problematic: lists of items where +configuration in a custom ``[tool.*]`` section. But there is a specific class +of cases where the all-or-nothing approach is problematic: lists of items where the backend needs to add items are currently forced to be fully dynamically specified (that is, in a backend-specific configuration location). This causes both of the original benefits (standard location and static tooling support) to @@ -93,8 +93,7 @@ restrictions on the ``[project]`` table and ``project.dynamic`` list. Every list and every table with arbitrary keys will now be allowed to be specified both statically, in the ``[project]`` table, and in the ``project.dynamic`` list. If it is present in both places, the build backend -can extend list items and add new keys, but not modify existing list items or -strings. +can extend list items and add new keys, but not modify existing entries. Use Cases @@ -107,26 +106,27 @@ cases that have come up: - Pinning dependency requirements when building the wheel. When building PyTorch_ extensions, for example, the version you build with adds a constraint to the wheel you create that is not present with the SDist. -- Generating extra scripts from a build system (this is a currently proposed in +- Generating extra scripts from a build system (this is currently proposed in scikit-build-core_). - Adding entry points dynamically (validate-pyproject-schema-store_ could have - used this to generate an entry point for each schema present in the package.) + used this to generate an entry point for each schema present in the package). - Adding dependencies or optional dependencies based on configuration (such as making an all dependency, or reading dependencies from dependency-groups, for - example). Adding constraints can also be useful; pybind11_ uses adds a ``global`` + example). Adding constraints can also be useful; pybind11_ uses a ``global`` extra that pins ``pybind11-global==``, as both packages are in the same repository and released in sync. toga_ is a collection of packages that - currently is unable to set any static dependencies due to the same sort of + is currently unable to set any static dependencies due to the same sort of pinning problem. - Adding classifiers; some backends can compute classifiers from other places and inject them (Poetry_ being the best known example). - Adding license files to the wheel based on what libraries are linked in (this - is an active discussion in followup to :pep:`639`). -- Adding SBom's when building - :pep:`770` had to remove the ``pyproject.toml`` + is an active discussion in follow-up to :pep:`639`). +- Adding SBOMs when building - :pep:`770` had to remove the ``pyproject.toml`` field specifically because you *want* the build tool to add these, so the - ``[project]`` table setting would be useless, you'd almost never be able to - use it. -- Adding generated modules to ``import-names`` or ``import-namespaces``. + ``[project]`` table setting would be useless, you would almost never be able + to use it. +- Adding generated modules to ``import-names`` or ``import-namespaces`` is + another example. All of these use cases have a similar feature: they are adding something dynamically to a fixed list (possibly a narrower pin for the dependency case). @@ -141,7 +141,7 @@ availability of metadata for static tooling. Example: pinning ---------------- -For example, let's say you want to allow an imaginary build backend +For example, say you want to allow an imaginary build backend (``my-build-backend``) to pin to the supported build of PyTorch_. Before this PEP, you could do this: @@ -162,7 +162,7 @@ Which would effectively expand to the following SDist metadata: Requires-Dist: packaging Requires-Dist: torch -Which would then could make a wheel with this: +Which could then make a wheel with this: .. code-block:: text @@ -170,8 +170,8 @@ Which would then could make a wheel with this: Requires-Dist: torch Requires-Dist: torch==2.8.0 -Static tooling no longer can tell that ``torch`` and ``packaging`` are runtime -dependencies, and the build backend had to duplicate the dependency table, +Static tooling can no longer tell that ``torch`` and ``packaging`` are runtime +dependencies, and the build backend has to duplicate the dependency table, making it harder for users to learn and read; the standardized place proposed by :pep:`621` and adopted by all major build backends is lost. @@ -198,8 +198,8 @@ Future Updates Loosening this rule to allow purely additive metadata should address many of the use cases that have been seen in practice. If further changes are needed, -this can be revisited in a future PEP; this PEP neither recommends or precludes -future updates like this. +this can be revisited in a future PEP; this PEP neither recommends nor +precludes future updates like this. Terminology =========== @@ -231,13 +231,13 @@ The fields that are arrays or tables with arbitrary entries are: be changed or removed. * ``keywords``: Keywords can be added to the list. * ``license-files``: Files can be added to the list. -* ``optional-dependencies``: A new extra or new items can be added to a +* ``optional-dependencies``: A new extra or new items can be added to an existing extra. * ``urls``: New urls can be added. Existing ones cannot be changed or removed. * ``import-names``, ``import-namespaces``: New import names or namespaces can be added. Existing ones cannot be modified or removed. -To add items, users must opt-in by listing the field in ``dynamic``; without +To add items, users must opt in by listing the field in ``dynamic``; without that, the metadata continues to be entirely static. A backend SHOULD error if a field is specified and it does not support @@ -250,7 +250,7 @@ Static analysis tools, when detecting a field is both specified and in the new entries to be present when the package is built. The ``Dynamic`` field, as specified in :pep:`643`, is unaffected by this PEP, -and backends can continue to fill it as they chose. However, a backend MUST +and backends can continue to fill it as they choose. However, a backend MUST ensure that both the SDist and the wheel metadata include the static metadata portion of the project table. @@ -263,7 +263,7 @@ to do with dynamic metadata. The pyproject-metadata_ project, which is used by several build backends, will need to modify the correctness check to account -for the possible extensions; this is in `a draft PR `__. +for the possible extensions; this is in `a draft PR `__. The dynamic-metadata_ project, which provides a plugin system that backends can use to share dynamic metadata plugins, was designed to @@ -276,15 +276,15 @@ Backwards Compatibility Using metadata from SDists or wheels is unaffected. The METADATA version does not need to be incremented. -This does not affect any existing ``pyproject.toml``'s, since this was strictly -not allowed before this PEP. +This does not affect any existing ``pyproject.toml`` files, since this was +strictly not allowed before this PEP. When users adopt this in a ``pyproject.toml``, the backend must support it; an -error will be correctly generated if it doesn't following the previous +error will be correctly generated if it doesn't, following the previous standard. Frontends were never required to throw an error, though some frontends may need to be updated to benefit from the partially static metadata. Some frontends and other tooling may need updating, such as schema -validation, just like other ``pyproject.toml`` PEPs. +validators, just like other ``pyproject.toml`` PEPs. Static analysis tools may require updating to handle this change. Tools should check the dynamic table first, like this: @@ -317,19 +317,26 @@ a static component to existing dynamic metadata support. How to Teach This ================= +If you currently have dynamic metadata, but some list or table entries are +known statically, you can now make that explicit by adding the static entry in +the ``[project]`` table, while also keeping the entry in the +``project.dynamic`` list to allow the dynamic portion to be added by your build +backend. + The current guides that state metadata must not be listed in both ``[project]`` -and ``project.dynamic`` can be updated to say that some fields can be extended -by ``project.dynamic``. Since dynamic metadata is already an advanced concept, -this will likely not affect most existing tutorial material aimed at -introductory packaging. +and ``project.dynamic`` can be updated to say that lists and tables marked with +``project.dynamic`` can still have static entries. Since dynamic metadata is +already an advanced concept, this will likely not affect most existing tutorial +material aimed at introductory packaging. The ``pyproject.toml`` `specification `__ will be updated to include the behavior of fields when specified and also listed in the dynamic field. -It should also be noted that specifying something in dynamic will require any -tool that requires the full metadata to invoke the backend even if it is -partially statically specified, so it should not be used unless necessary. +It should also be noted that specifying something in ``dynamic`` will require +any tool that needs the full metadata to invoke the backend even if it is +partially statically specified. So, it should not be used unless necessary, +just like any other dynamic metadata. Rejected Ideas @@ -378,12 +385,11 @@ Allow simplifications An earlier draft of this PEP had a clause allowing backends to simplify some types of fields; most notably dependency specifiers would have allowed "tightening", such as ``torch`` being replaced by ``torch>=1.2``, for example. -. This was removed due to it being impossible to ensure a variation will -resolve identically on all resolvers within the current specification, and to -simplify the contract with backends. Any other simplifications would be purely -cosmetic, and so were left out. The order in the current PEP is now required to -match the original static metadata, with the dynamic portion only allowing -insertions. +This was removed due to it being impossible to ensure a variation will resolve +identically on all resolvers within the current specification, and to simplify +the contract with backends. Any other simplifications would be purely cosmetic, +and so were left out. The order in the current PEP is now required to match the +original static metadata, with the dynamic portion only allowing insertions. Add a general mechanism to specify dynamic-metadata