Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve detection of installed packages #1786

Merged
merged 1 commit into from Jan 5, 2020

Conversation

@sdispater
Copy link
Member

sdispater commented Dec 23, 2019

Pull Request Check List

  • Added tests for changed code.
  • Updated documentation for changed code.

In version 1.0.0, we changed the way the installed packages are detected to improve the detection of the type of packages (directory vs git dependencies for instance). However, this introduced issues where Poetry always reinstalled the packages because it was detecting the wrong type of package or because it was not retrieving it from the correct directory. This PR should fix most of these issues.

Fixes #1711
Fixes #1612

@sdispater sdispater requested a review from python-poetry/core Dec 23, 2019
if site_packages:
self._site_packages = site_packages[0]
else:
self._site_packages = dist_packages[0]
Comment on lines +775 to +778

This comment has been minimized.

Copy link
@finswimmer

finswimmer Dec 29, 2019

Member

I don't know where the site_packages property is used, so it isn't obvious to me, why only the first site_package is stored and why dist-packages are never stored when site_packages where found.

This comment has been minimized.

Copy link
@sdispater

sdispater Dec 30, 2019

Author Member

The site_packages property is used in the InstalledRepository class (see https://github.com/python-poetry/poetry/blob/master/poetry/repositories/installed_repository.py#L32).

We only need the "site-packages" directory where pip will install the dependencies which is, as far as I know, the site-packages in virtual environments and most OS and dist-packages for Debian-based distributions.

This comment has been minimized.

Copy link
@finswimmer

finswimmer Dec 30, 2019

Member

Ah, ok I got it. I wonder if something with sysconfig would be more reliable? But as far as I can see, we must be able to detect if we are inside a virtual environment and of we are on a posix or nt system, to get the correct path (purelib seems to be the one we are interested in), are we?

This comment has been minimized.

Copy link
@sdispater

sdispater Jan 2, 2020

Author Member

Thanks for pointing me to sysconfig! Do you know if there are some systems where purelib and platlib are different? If there is none we could indeed use purelib and avoid the heuristics we are currently doing.

This comment has been minimized.

Copy link
@finswimmer

finswimmer Jan 3, 2020

Member

I'm not sure. I've just discovered that module as well. There is something in PEP427 about the difference.

This comment has been minimized.

Copy link
@sdispater

sdispater Jan 3, 2020

Author Member

In the end, it seems that all installed packages will reside in purelib and/or platlib, so to get populate the InstalledRepository we can just check both, I think. I'll test it out and see where it goes.

This comment has been minimized.

Copy link
@sdispater

sdispater Jan 5, 2020

Author Member

After testing it out it seems that using sysconfig won't help us. While it works in some cases, for Debian-based systems, for instance, it won't return the directory we need. We could use site.getsitepackages() for non-virtual environments but we would need to find a way to replicate it for virtual environments which would lead to using the same heuristic this PR already uses.

I think we can leave the PR as-is for now since I tested it on Ubuntu, MacOS and Windows without any issue and it fixes the issues I mentioned above.

Copy link
Member

finswimmer left a comment

Then go ahead 😃

@sdispater sdispater merged commit 8a3d2d7 into master Jan 5, 2020
32 checks passed
32 checks passed
Linting
Details
Linting
Details
Linux (2.7)
Details
Linux (2.7)
Details
Linux (3.5)
Details
Linux (3.5)
Details
Linux (3.6)
Details
Linux (3.6)
Details
Linux (3.7)
Details
Linux (3.7)
Details
Linux (3.8)
Details
Linux (3.8)
Details
MacOS (2.7)
Details
MacOS (2.7)
Details
MacOS (3.5)
Details
MacOS (3.5)
Details
MacOS (3.6)
Details
MacOS (3.6)
Details
MacOS (3.7)
Details
MacOS (3.7)
Details
MacOS (3.8)
Details
MacOS (3.8)
Details
Windows (2.7)
Details
Windows (2.7)
Details
Windows (3.5)
Details
Windows (3.5)
Details
Windows (3.6)
Details
Windows (3.6)
Details
Windows (3.7)
Details
Windows (3.7)
Details
Windows (3.8)
Details
Windows (3.8)
Details
@sdispater sdispater deleted the improve-installed-repository-handling branch Jan 5, 2020
@StephenBrown2 StephenBrown2 mentioned this pull request Jan 10, 2020
5 of 5 tasks complete
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
2 participants
You can’t perform that action at this time.