-
Notifications
You must be signed in to change notification settings - Fork 1
[variant-json] made mutable with vendoring/inlining #43
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Changes from all commits
2cfb0b5
b523558
88f4dfc
e4bea61
557fe5f
f059855
e37cd5c
1dde6c5
e498e3b
9015cdc
b83cc73
c2ab539
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -321,6 +321,9 @@ or they may carefully merge their values, provided that no conflicting | |
| information is introduced, and the resolution results within a subset of | ||
| variants do not change. | ||
|
|
||
| This file SHOULD NOT be considered immutable and MAY be updated in a | ||
| backward compatible way at any point (e.g. when adding a new variant). | ||
|
|
||
| Variant indexes MAY elect to either auto-generate the file from the | ||
| uploaded variant wheels or allow the user to manually generate it | ||
| themselves and upload it to the index. | ||
|
|
@@ -631,52 +634,67 @@ Variant wheels can be listed in ``pylock.toml`` file in the same manner | |
| as wheels with different Platform compatibility tags: either all variant | ||
| (and non-variant) wheels can be listed, or a subset of them. | ||
|
|
||
| A new ``[packages.variants-json]`` subtable is added to the file, to | ||
| permit locking the ``{name}-{version}-variants.json`` file to a specific | ||
| hash. If variant wheels are listed for a given package, the tool SHOULD | ||
| lock this file as well. | ||
| A new ``[packages.variants-json]`` subtable is added to the file. It | ||
| MUST inline the contents of the `index-level metadata`_ file, converting | ||
| the JSON structure into the respective TOML types. The ``$schema`` key | ||
| MUST be preserved to facilitate versioning. The tools MAY remove keys | ||
| that are not relevant to variant wheels present in ``pylock.toml``. | ||
|
|
||
| If variant wheels are listed, the tool SHOULD resolve variants to select | ||
| the best wheel file. | ||
|
|
||
| The proposed text for :doc:`packaging:specifications/pylock-toml` | ||
| follows: | ||
|
|
||
| .. code:: rst | ||
| Proposed specification update | ||
| ''''''''''''''''''''''''''''' | ||
|
|
||
| .. _pylock-packages-variants-json: | ||
| The proposed text for :doc:`packaging:specifications/pylock-toml` | ||
| follows: | ||
|
|
||
| ``[packages.variants-json]`` | ||
| ---------------------------- | ||
| ``[packages.variant_json]`` | ||
| ~~~~~~~~~~~~~~~~~~~~~~~~~~~ | ||
|
|
||
| - **Type**: table | ||
| - **Required?**: no; requires that :ref:`pylock-packages-wheels` is used, | ||
| mutually-exclusive with :ref:`pylock-packages-vcs`, | ||
| :ref:`pylock-packages-directory`, and :ref:`pylock-packages-archive`. | ||
| - **Inspiration**: uv_ | ||
| - The URL or path to the ``variants.json`` file. | ||
| - Only used if the project uses :ref:`wheel variants <wheel-variants>`. | ||
| - **Type**: table | ||
| - **Required?**: no | ||
| - **Functions**: | ||
|
|
||
| .. _pylock-packages-variants-json-url: | ||
| - Inline variant selection metadata for the package. | ||
| - The structure of this table MUST conform to the JSON Schema. | ||
| - Tools that support variant-aware resolution MUST validate this table | ||
| against the referenced schema. | ||
| - Tools that do not support variant-aware resolution MAY ignore this table | ||
| but SHOULD preserve it when rewriting the lock file. | ||
|
|
||
| ``packages.variants-json.url`` | ||
| '''''''''''''''''''''''''''''' | ||
|
|
||
| See :ref:`pylock-packages-archive-url`. | ||
| Example | ||
| ''''''' | ||
|
|
||
| .. _pylock-packages-variants-json-path: | ||
| .. code:: toml | ||
|
|
||
| ``packages.variants-json.path`` | ||
| ''''''''''''''''''''''''''''''' | ||
| lock-version = "1.0" | ||
| created-by = "uv" | ||
| requires-python = ">=3.14" | ||
|
|
||
| See :ref:`pylock-packages-archive-path`. | ||
| [[packages]] | ||
| name = "numpy" | ||
| version = "2.3.4" | ||
| index = "https://pypi.anaconda.org/mgorny/simple" | ||
| wheels = [ | ||
| { url = "https://pypi.anaconda.org/mgorny/simple/numpy/2.3.4/numpy-2.3.4-cp314-cp314-linux_x86_64-openblas.whl", hashes = {} }, | ||
|
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Not sure if this uses There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. It's an exact lock file generated by uv for the test packages I've built (except for variant metadata format that changes here) :-). |
||
| { url = "https://pypi.anaconda.org/mgorny/simple/numpy/2.3.4/numpy-2.3.4-cp314-cp314-linux_x86_64-x8664v4_mkl.whl", hashes = {} }, | ||
| { url = "https://pypi.anaconda.org/mgorny/simple/numpy/2.3.4/numpy-2.3.4-cp314-cp314-macosx_13_0_x86_64-accelerate.whl", hashes = {} }, | ||
| { url = "https://pypi.anaconda.org/mgorny/simple/numpy/2.3.4/numpy-2.3.4-cp314-cp314-macosx_13_0_x86_64-openblas.whl", hashes = {} }, | ||
| ] | ||
|
|
||
| .. _pylock-packages-variants-json-hashes: | ||
| [packages.variant_json] | ||
| "$schema" = "https://variants-schema.wheelnext.dev/peps/825/v0.1.0.json" | ||
|
|
||
| ``packages.variants-json.hashes`` | ||
| ''''''''''''''''''''''''''''''''' | ||
| [packages.variant_json.default-priorities] | ||
| namespace = [ "x86_64", "aarch64", "blas_lapack" ] | ||
|
|
||
| See :ref:`pylock-packages-archive-hashes`. | ||
| [packages.variant_json.variants] | ||
| null = { } | ||
| x8664v3_openblas = { "blas_lapack" = { "library" = ["openblas"]}, "x86_64" = { "level" = ["v3"]} } | ||
| x8664v4_mkl = { "blas_lapack" = { "library" = ["mkl"]}, "x86_64" = { "level" = ["v4"]} } | ||
|
|
||
|
|
||
| Suggested implementation logic for tools (non-normative) | ||
|
|
@@ -742,9 +760,7 @@ There are two possible approaches to publishing the index-level | |
| it can either be prepared and uploaded by the user, or it can be | ||
| generated automatically by the index. | ||
|
|
||
| The file should not be changed once it is published, as clients may have | ||
| already cached it or locked to the existing hash. For this reason, if | ||
| the index is responsible for generating the file, it should use some | ||
| If the index is responsible for generating the file, it should use some | ||
| mechanism to defer publishing it until the release is fully uploaded | ||
| (for example, :pep:`694`). | ||
|
|
||
|
|
@@ -813,6 +829,13 @@ wheels being unsupported altogether. For example, PyTorch could provide | |
| a much smaller null variant that is used when no GPU is supported, and a | ||
| fallback non-variant wheel built for the default CUDA version. | ||
|
|
||
| ``pylock.toml`` integration inlines the variant metadata to keep the | ||
| file standalone. This can avoid the additional network call that would | ||
| be required to fetch the file, and avoids having to pin to a specific | ||
| hash that could cause problems if the file changed on the index, either | ||
| due to the variant metadata being updated or being generated in a way | ||
| that does not guarantee stable bytewise output. | ||
|
|
||
|
|
||
| Backwards Compatibility | ||
| ======================= | ||
|
|
@@ -973,6 +996,8 @@ Change History | |
| rather than all compatible values, and add a fallback to ordering | ||
| on variant label. | ||
| - Removed the variant label length limitation. | ||
| - Changed ``pylock.toml`` integration to inline variant metadata | ||
| rather than storing a URL and a hash. | ||
|
|
||
|
|
||
| Appendices | ||
|
|
||
Uh oh!
There was an error while loading. Please reload this page.