Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support turning installed wheel chroots into .whl files with full fidelity. #2299

Open
jsirois opened this issue Dec 13, 2023 · 0 comments
Open

Comments

@jsirois
Copy link
Member

jsirois commented Dec 13, 2023

If we can round trip from a downloaded .whl to an installed wheel chroot and back with full fidelity, then we can fix a long existing latent bug and improve composability of Pex options:

  • Unreported bug: PEX_TOOLS=1 ./a.pex repository extract ... does not produce correct wheels when the original wheel contains any data dirs (<project>-<version>.data/* .whl zip file entries).
  • --pex-repository: via Add support for --no-pre-install-wheels and --max-install-jobs. #2298 a PEX repository with dependency .whl files can satisfy .whl file resolves and installed wheel chroot resolves but a PEX repository with installed wheel chroot dependencies can only satisfy resolves for installed wheel chroot dependencies and not for .whl files.
@jsirois jsirois self-assigned this Dec 16, 2023
jsirois added a commit to jsirois/pex that referenced this issue Mar 24, 2024
Pex cannot yet handle turning installed wheel chroots back into wheel
files (see pex-tool#2299). As such, until that time, it must reject attempts
to combine PEXes with different wheel packaging.

Clarifies pex-tool#2391
jsirois added a commit that referenced this issue Mar 28, 2024
Pex cannot yet handle turning installed wheel chroots back into wheel
files (see #2299). As such, until that time, it must reject attempts
to combine PEXes with different wheel packaging.

Clarifies #2391
huonw added a commit to pantsbuild/pants that referenced this issue Apr 3, 2024
This marks all local dist PEXes as internal-only, removing the ability
for them to be anything but internal. This is almost true already,
except for PEXes built via `PexFromTargetsRequest`, where the local
dists PEX used for building the "real" PEX has the same internal status
as that real PEX. In this case, the local dists PEX still isn't surfaced
to users, so it's appropriate for that one to be internal too.

This will probably be slightly faster in isolation (building a
`pex_binary` that uses in-repo `python_distribution`s will be able to
just copy them around with less zip-file manipulation, more often, by
creating packed-layout PEXes). However, the real motivation is
unblocking #20670, where having this PEX built with
`--no-pre-install-wheels` (as internal-only PEXes will, by default) is
required to support downstream PEXes using that argument, at least until
pex-tool/pex#2299 is fixed.

NB. there's still a separate consideration of changing how local dists
are incorporated, which isn't changed or considered here:
pex-tool/pex#2392 (comment)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

1 participant