Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Read modified conda dependencies from wheel metadata #25

Open
analog-cbarber opened this issue Aug 24, 2023 · 6 comments
Open

Read modified conda dependencies from wheel metadata #25

analog-cbarber opened this issue Aug 24, 2023 · 6 comments
Assignees
Labels
component: convert new feature Entirely new feature (e.g. new subcommand)

Comments

@analog-cbarber
Copy link
Collaborator

Support some way for the wheel to publish information about its conda dependencies. Wheels configured
in this way could be converted to conda packages with no other external information.

One way to do this would be using the Requires-External metadata key. The wheel METADATA file could contain lines like:

Rather than support the rename syntax in the metadata, we probably should just explicitly
list names to add and names to drop. E.g.

Requires-External: conda-add; pytables>=3.10
Requires-External: conda-drop; tables

There don't appear to be any pyproject fields that map to this metadata, but there are some
tools like auditwheel that can add these fields to existing wheels.

Note that PEP 725 proposes to add a optional-dependencies
whose entries are PURL strings. There is a PURL
format for conda
,e.g:

pkg:conda/absl-py@0.4.1?build=py36h06a4308_0&channel=main&subdir=linux-64&type=tar.bz2

but since we are not actually describing an actual external dependency but an alternate dependency,
we should probably add a type specifically for this package:

Requires-External: pkg:whl2conda/pytables@>=3.10
Requires-External: pkg:whl2conda/tables?drop=true

We could also provide a builder plugin to generate these entries from the pyproject file when
the wheel is built or we could have an option of this program to add those entries to an existing
wheel.

@analog-cbarber analog-cbarber added the enhancement New feature or request label Aug 24, 2023
@analog-cbarber
Copy link
Collaborator Author

If a tool is generating the meta data entries, then it probably would be simplest for the tool to just generate all of the conda dependencies directly, but if they are being maintained using the PEP 725 mechanism, only added/dropped (or perhaps renamed) entries should be listed.

We can use pkg:whl2conda for the former case. And pkg:whl2conda-add, pkg:whl2conda-drop and pkg:whl2conda-rename for the latter. If any pkg:whl2conda entries are present then the regular wheel dependencies will be ignored along with any pkg:whl2conda-drop and pkg:whl2conda-rename.

@analog-cbarber
Copy link
Collaborator Author

analog-cbarber commented Jan 28, 2024

It could be a long time before standard build tools support PEP 725 and poetry seems even less likely to bother to implement this, so it might be worth considering other mechanisms....

One possibility would be to use optional dependencies either using different keys:

[project.optional-dependencies]
whl2conda-add = [
   "pytables >=3.10"
]
whl2conda-drop = [
   "tables"
]

This would result in wheel METADATA:

Provides-Extra: whl2conda-add
Provides-Extra: whl2conda-drop
Requires-Dist: pytables >=3.10; extra == 'whl2conda-add'
Requires-Dist: tables; extra == 'whl2conda-drop'

Another option would be to just use an impossible version spec to indicate removal:

[project.optional-dependencies]
whl2conda = [
    "pytables >=3.10",
    "tables ===drop",
]

which would result in

Provides-Extra: whl2conda
Requires-Dist: pytables >=3.10; extra == 'whl2conda'
Requires-Dist: tables ===drop; extra == 'whl2conda'

Would be nice to also support some sort of rename declaration with wildcards, but I don't know that there is a good way to do that which would not violate the requirement that the dependencies be specified as valid PEP 508 strings.

@analog-cbarber
Copy link
Collaborator Author

analog-cbarber commented Feb 1, 2024

Probably should really include some sort of support for renaming. Otherwise, users will have to list package dependency versions in two different places, which is highly likely to break.

One possibility would be to once again use the arbitrary equality version operator ===
with a special key:

[project.optional-dependencies]
whl2conda-rename = [
    "tables ==pytables" 
]

which would result in the following METADATA entries:

Provides-Extra: whl2conda-rename
Requires-Dist: tables ===pytables; extra == 'whl2conda-rename'

@analog-cbarber
Copy link
Collaborator Author

Looks like PEP621 support work is at least underway for poetry
python-poetry/poetry#9136

@analog-cbarber analog-cbarber added status: wontfix This will not be worked on new feature Entirely new feature (e.g. new subcommand) and removed enhancement New feature or request status: wontfix This will not be worked on labels Apr 14, 2024
@analog-cbarber
Copy link
Collaborator Author

Note that poetry does not currently implement any syntax that maps to the "arbitrary equality clause" (===).

@analog-cbarber
Copy link
Collaborator Author

Another option would be to further abuse the extra namespace:

[project.optional-dependencies]
whl2conda-rename-tables = ["pytables"]
Provides-Extra: whl2conda-rename-tables
Requires-Dist: pytables; extra == 'whl2conda-rename-tables'

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
component: convert new feature Entirely new feature (e.g. new subcommand)
Projects
None yet
Development

No branches or pull requests

1 participant