Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add a venv tool. #1128

Merged
merged 8 commits into from
Dec 14, 2020
Merged

Add a venv tool. #1128

merged 8 commits into from
Dec 14, 2020

Conversation

jsirois
Copy link
Member

@jsirois jsirois commented Dec 8, 2020

Add a venv tool to create a virtual environment from a PEX file. The
virtual environment is seeded with just the PEX user code and
distributions applicable to the selected interpreter for the local
machine. The virtual environment does not have Pip installed by default
although that can be requested with --pip.

The virtual environment comes with a __main__.py at the root of the
venv to emulate a loose pex that can be run with python venv.dir just
like a loose pex. This entry point supports all the behavior of the
original PEX file not related to interpreter selection, namely support
for PEX_SCRIPT, PEX_MODULE, PEX_INTERPRETER and PEX_EXTRA_SYS_PATH.

A sibling pex script is linked to __main__.py to provide the
maximum performance entrypoint that always avoids interpreter
re-execing and thus yields equivalent performance to a pure virtual
environment.

Work towards #962 and #1115.

@jsirois
Copy link
Member Author

jsirois commented Dec 8, 2020

Commits are reviewable in sequence individually if that suits.

@jsirois jsirois requested a review from stuhood December 8, 2020 09:04
@jsirois
Copy link
Member Author

jsirois commented Dec 8, 2020

This is hefty, but also not as big as it looks. Of the +3,371 −49 delta +2,633 comes from embedding the virtualenv.py script for Python 2.7 support.

Thanks in advance for your patience on the remaining diff though.

@jsirois jsirois marked this pull request as draft December 8, 2020 20:07
@jsirois
Copy link
Member Author

jsirois commented Dec 8, 2020

Going into draft mode while I work out CI failures that do not repro on my machine.

@jsirois
Copy link
Member Author

jsirois commented Dec 11, 2020

OK - this is now ready for review. Its based on top of #1131 which form the 1st two commits here. Please focus on the 7 commits starting at 06eff63 "Automation for embedding of virtualenv 16.7.10.".

Thanks again in advance.

@jsirois jsirois changed the title Add a Venv tool. Add a venv tool. Dec 12, 2020
Copy link

@stuhood stuhood left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks!

pex/tools/commands/venv.py Show resolved Hide resolved
pex/tools/commands/venv.py Show resolved Hide resolved
exclude=(pex_info.internal_cache, pex_builder.BOOTSTRAP_DIR, "__main__.py"),
)

for dist in pex.activate():
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I had thought of "activate" as meaning "add to the pythonpath of the current process", so it's interesting to see it used like this. Would splitting that method into "iterate over what would have been activated" vs "activate" make sense?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That might make sense although it is neither here nor there for tool use where the side-effect of the distributions getting added to the sys.path here has no relevant effect:
https://github.com/pantsbuild/pex/blob/12ad6a63f988c07065cebbfb6aa4102a6d3fa094/pex/environment.py#L476

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a good deal thornier than it may look. The code paths involved are relevant in the resolve-from-pex work though so I'll defer to that work.

pex/tools/commands/venv.py Show resolved Hide resolved
@jsirois jsirois merged commit 43d1130 into pex-tool:master Dec 14, 2020
@jsirois jsirois deleted the tools/venv branch December 14, 2020 20:29
@jsirois jsirois restored the tools/venv branch December 14, 2020 20:31
@jsirois jsirois deleted the tools/venv branch December 14, 2020 20:32
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants