Support --project locking and PEX building.
#2455
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Although Pex already supported local project requirements, they worked
like any other requirement and were included in locks. In the typical
case, a project would rather maintain a lock of its third party
requirements separate from itself. For projects that used requirement
files to record their unlocked dependencies, `-r` could be used instead
of locking the local project. For projects that recorded their
dependencies in a `setup.py`, `setup.cfg` or `pyproject.toml`, the
dependency information needed to be extracted from those files to create
a lock that did not lock the project code itself. Just as bad, when
constructing a PEX the source code layout of the project recorded in
those same project files needed to be replicated in some combination of
`-D`, `-P` and `-M` since resolving from a lock required all projects
needed by the PEX were represented in the lock.
With the addition of the new `--project` option to PEX builds and
`pex3 lock {create,sync}`, a project's dependencies and source code
layout can be referenced from their canonical home in the project
configuration files.
Fixes pex-tool#2412.
Previously timings could be dropped.
Previously, if a PEX was built and run on the same machine, later PEXes built from the same wheels would have `.pyc` from prior runs of that wheel code erroneously included, breaking the `--no-compile` default as well as reproducibility.
jsirois
commented
Jul 6, 2024
There was a problem hiding this comment.
I noticed dropped timings for the new TRACER.timed("Collecting requirements from {count} local {projects}") added to pex/cli/commands/lock.py.
jsirois
commented
Jul 6, 2024
There was a problem hiding this comment.
I had actually seen this .pyc leak before in scie-pants, but also encountered it in the integration test here comparing PEX files.
jsirois
changed the title
--project locking and PEX building.
kaos
approved these changes
Jul 6, 2024
jsirois
added a commit
to jsirois/pex
that referenced
this pull request
Jul 6, 2024
jsirois
added a commit
to jsirois/pex
that referenced
this pull request
Jul 6, 2024
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Although Pex already supported local project requirements, they worked
like any other requirement and were included in locks. In the typical
case, a project would rather maintain a lock of its third party
requirements separate from itself. For projects that used requirement
files to record their unlocked dependencies,
-rcould be used insteadof locking the local project. For projects that recorded their
dependencies in a
setup.py,setup.cfgorpyproject.toml, thedependency information needed to be extracted from those files to create
a lock that did not lock the project code itself. Just as bad, when
constructing a PEX the source code layout of the project recorded in
those same project files needed to be replicated in some combination of
-D,-Pand-Msince resolving from a lock required all projectsneeded by the PEX were represented in the lock.
With the addition of the new
--projectoption to PEX builds andpex3 lock {create,sync}, a project's dependencies and source codelayout can be referenced from their canonical home in the project
configuration files.
Fixes #2412.