Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

uniquify paths in PYTHONPATH #1709

Merged
merged 3 commits into from Apr 5, 2019

Conversation

Projects
None yet
2 participants
@robnagler
Copy link
Contributor

commented Feb 26, 2019

When running in a complex environment with lots of installed
packages, PYTHONPATH gets way too long. Instead, just make sure
that paths_on_pythonpath doesn't contain duplicates

I don't expect this to be accepted. Just letting you know there's a problem here and how to fix it.

uniquify paths in PYTHONPATH
When running in a complex environment with lots of installed
packages, PYTHONPATH gets way too long. Instead, just make sure
that paths_on_pythonpath doesn't contain duplicates
@jaraco

This comment has been minimized.

Copy link
Member

commented Apr 5, 2019

After some investigation, I can see that the bug lies here, where setuptools is attempting to make sure that every package that was just installed has its path on PYTHONPATH, so that any subprocess invocation also has those packages available for import.

The fact that all of these paths are the same, however, is a little surprising. These paths would be installed by easy_install, which should provide a unique location for each package installed. If these paths are in fact always just the site-packages directory, perhaps this code does not have its intended effect.

Since this code is part of the pending-deprecated test command, I'm going to save trouble investigating the root cause and simply adopt a technique similar to the one you've proposed here.

jaraco added some commits Apr 5, 2019

@jaraco jaraco merged commit 7326bdf into pypa:master Apr 5, 2019

1 of 4 checks passed

continuous-integration/appveyor/pr Waiting for AppVeyor build to complete
Details
continuous-integration/travis-ci/pr The Travis CI build is in progress
Details
deploy/netlify Deploy preview processing.
Details
Mergify — Summary 1 potential rule
Details
@robnagler

This comment has been minimized.

Copy link
Contributor Author

commented Apr 6, 2019

Thanks @jaraco for taking a look at this.

These paths would be installed by easy_install, which should provide a unique location for each package installed.

This seems like this approach would be problematic in complex installations, like ours. Why is this necessary?

@jaraco

This comment has been minimized.

Copy link
Member

commented Apr 6, 2019

That’s just how easy_install works. Each package is meant to be an encapsulated egg, such that uninstallation is simply an rm operation (similar to how macOS installs Applications). To make those packages live on the python path, easy_install would have a .pth file to add these paths... but for these on-demand test dependencies, another technique was needed.

I think one motivation for moving away from easy_install was to avoid the essential performance hit of each package having its own path.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.