Internal not enough values to unpack error for pex3 lock create 'pip @ https://github.com/pypa/pip/archive/22.0.2.zip' ...
#2057
Comments
|
A tighter / smaller case for the IT:
|
|
Ok, the lock code assumes any URL with a path not ending in ".whl" must be an sdist. That is not correct and this case demonstrates that. A 3rd possibility is the path portion could point to any old archive of the source and not necessarily a proper sdist. The Pip download log reveals: So there is enough information to robustly scrape the version info here to avoid a re-build of the archive to find out its version without adding special-case GitHub logic, namely: |
|
And a 4th possibility is the URL is a local file:// URL pointing to a project source directory: Creating a PEX works fine: Creating a lock does not: |
|
Noting on the above, the specific issue is the direct reference URL format. This works fine: Which nets: {
"allow_builds": true,
"allow_prereleases": false,
"allow_wheels": true,
"build_isolation": true,
"constraints": [],
"locked_resolves": [
{
"locked_requirements": [
{
"artifacts": [
{
"algorithm": "sha256",
"hash": "fca533d24ea5fc1b0fc8bc10ee146535bddda1c40c85510544c5766cef4d10b6",
"url": "file:///tmp/colors"
}
],
"project_name": "ansicolors",
"requires_dists": [],
"requires_python": null,
"version": "1.1.8"
}
],
"platform_tag": [
"cp310",
"cp310",
"manylinux_2_35_x86_64"
]
}
],
"path_mappings": {},
"pex_version": "2.1.122",
"pip_version": "20.3.4-patched",
"prefer_older_binary": false,
"requirements": [
"ansicolors==1.1.8"
],
"requires_python": [],
"resolver_version": "pip-legacy-resolver",
"style": "strict",
"target_systems": [],
"transitive": true,
"use_pep517": null
}And that can be used to build and run a PEX: |
jsirois
added a commit
to jsirois/pex
that referenced
this issue
Feb 17, 2023
Previously, it was assumed that direct reference URLs were either VCS
URLs or else wheel or sdist URLs. This neglected two remaining cases,
both of which failed to lock with better or worse error messages. The
two missed cases were:
1. URLs of source archives not conforming to the sdist quasi-standard
naming convention of `<project name>-<version>.{.tar.gz,.zip}`.
2. Local file:// URLs pointing at project directories.
A notable case of the 1st are project archives provided by GitHub.
A notable need for the 2nd case comes from Pants where Pip proprietary
requirement strings are not handled (e.g.: `/path/to/project`) and a
direct reference URL must be used instead (e.g.: `projectname @
/path/to/project`).
Fixes pex-tool#2057
|
Alright, both of these cases are now covered in #2060. |
jsirois
added a commit
to jsirois/pex
that referenced
this issue
Feb 23, 2023
Previously, it was assumed that direct reference URLs were either VCS
URLs or else wheel or sdist URLs. This neglected two remaining cases,
both of which failed to lock with better or worse error messages. The
two missed cases were:
1. URLs of source archives not conforming to the sdist quasi-standard
naming convention of `<project name>-<version>.{.tar.gz,.zip}`.
2. Local file:// URLs pointing at project directories.
A notable case of the 1st are project archives provided by GitHub.
A notable need for the 2nd case comes from Pants where Pip proprietary
requirement strings are not handled (e.g.: `/path/to/project`) and a
direct reference URL must be used instead (e.g.: `projectname @
/path/to/project`).
Fixes pex-tool#2057
jsirois
added a commit
that referenced
this issue
Feb 23, 2023
Previously, it was assumed that direct reference URLs were either VCS
URLs or else wheel or sdist URLs. This neglected two remaining cases,
both of which failed to lock with better or worse error messages. The
two missed cases were:
1. URLs of source archives not conforming to the sdist quasi-standard
naming convention of `<project name>-<version>.{.tar.gz,.zip}`.
2. Local `file://` URLs pointing at project directories.
A notable case of the 1st are project archives provided by GitHub.
A notable need for the 2nd case comes from Pants where Pip proprietary
requirement strings are not handled (e.g.: `/path/to/project`) and a
direct reference URL must be used instead (e.g.: `projectname @
file:///path/to/project`).
Fixes #2057
|
Thanks for the fix! I've tested 2.1.123, and indeed |
|
And thanks for the review. It took a good bit of work to be robust to ~random archive names and still stay performant for |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
I'm trying to create a lock file with a requirement like
pip @ https://github.com/pypa/pip/archive/22.0.2.zip. Pex fails with a very opaque error from an uncaughtValueError:not enough values to unpack.Potentially I'm doing something that's not supported. If so, I'd be hoping for a more explanatory error message than the raw exception.
Workaround
For a Github archive URL like that, it seems like
gitrequirement likepip @ git+https://github.com/pypa/pip@22.0.2works instead.Reproducer
(That requirement spec is taken from https://pip.pypa.io/en/stable/reference/requirement-specifiers/#examples, and indeed
pip install 'pip @ https://github.com/pypa/pip/archive/22.0.2.zip'seems to work.)Output (running under 3.9, on macOS):
PEX_VERBOSE) is just the exception messagenot enough values to unpack (expected 2, got 1).The text was updated successfully, but these errors were encountered: