New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Problems caused by different in package name in Pip (pyqt vs PyQt5) #1554

Closed
titusjan opened this Issue Mar 25, 2017 · 15 comments

Comments

Projects
None yet
7 participants
@titusjan

titusjan commented Mar 25, 2017

One of the things I like about Anaconda is that it works well together with Pip. If a package is not available in Anaconda I can fall back on installation from PyPi. However, this does not work for PyQt because the package name in Anaconda is pyqt, but on PyPi it is PyQt5.

For me, as a developer of packages that use PyQt, this means that I can not add PyQt5 to the install_requires parameter in the setup.py of my package. If I do, and a user installs my package with Pip, it doesn't recognize that pyqt has already been installed with Anaconda. It will then download PyQt5 from PyPi and install it over the pyqt package. This breaks the installation of other environments beyond repair. The user will get a segmentation fault if a program tries to import PyQt!

Please rename the pyqt package to PyQt5 (I don't know if case-sensitivity matters). In general please use the default package name as it is defined in the setup.py. I realize that this has a lot of impact, if it is possible at all. However, I think it should at leat be discussed (I could not find anything on this with Google).

To repeat the problem you can install PyQtChart, which automatically installs PyQt5. I accidentally did this and now my PyQt broken for all environments that use the same Python version (any help on fixing that would be appreciated as well).

# Create two environments to illustrate that multiple envs are affected.
conda create -n env1 python=3.5.1  
conda install -n env1 pyqt
conda create -n env2 python=3.5.1  
conda install -n env2 pyqt
. activate env2

# This works fine now
python -c "from PyQt5 import QtCore" 

# This will install PyQt5 from PyPi as well
pip install PyQtChart

# Now you see two installation (pyqt and PyQt5)
conda list -n env2 qt

# This still works because it uses the PyPi PyQt
python -c "from PyQt5 import QtCore" 

# However, the first environment is now broken and gives "Segmentation fault: 11"
. activate env1
python -c "from PyQt5 import QtCore"

Like I said. I haven't managed to fix the issue. Removing PyQt with pip and reinstalling with conda didn't help. Luckily my PyQt from Python 3.6 still works but all 3.5 PyQt installations are broken. I use OS-X 10.11.5. Help on how to repair it is appreciated.

@titusjan titusjan changed the title from Problems caused by different in package name in Pip (pyqt vs Pyqt5) to Problems caused by different in package name in Pip (pyqt vs PyQt5) Mar 25, 2017

@ccordoba12

This comment has been minimized.

Show comment
Hide comment
@ccordoba12

ccordoba12 Mar 25, 2017

I don't think we're going to change this package name. Some reasons:

  1. We need to maintain compatibility with pyqt 4, so it's easier to have one package name and two versions, instead of two packages.
  2. Our package and the pyqt5 wheel are very different, so it's better to keep them as separate things. The wheel has all its dependencies packaged with it while our package properly depends on other packages. Also, the wheel doesn't work on old systems (centos 5) while our package does.

ccordoba12 commented Mar 25, 2017

I don't think we're going to change this package name. Some reasons:

  1. We need to maintain compatibility with pyqt 4, so it's easier to have one package name and two versions, instead of two packages.
  2. Our package and the pyqt5 wheel are very different, so it's better to keep them as separate things. The wheel has all its dependencies packaged with it while our package properly depends on other packages. Also, the wheel doesn't work on old systems (centos 5) while our package does.
@ccordoba12

This comment has been minimized.

Show comment
Hide comment
@ccordoba12

ccordoba12 Mar 25, 2017

Solution for you: create conda packages for your projects and their dependencies instead of trying to use wheels in conda. Although conda and pip work well most of time, they are not totally compatible, specially regarding packages with compiled libraries.

You can use conda-forge for that, as most people do these days.

ccordoba12 commented Mar 25, 2017

Solution for you: create conda packages for your projects and their dependencies instead of trying to use wheels in conda. Although conda and pip work well most of time, they are not totally compatible, specially regarding packages with compiled libraries.

You can use conda-forge for that, as most people do these days.

@ccordoba12 ccordoba12 closed this Mar 25, 2017

@mingwandroid

This comment has been minimized.

Show comment
Hide comment
@mingwandroid

mingwandroid Mar 26, 2017

Member

In general please use the default package name as it is defined in the setup.py

Making this the general rule would preclude having sensible package names for packages that are not based on CPython. Personally, I would name all of those ones as py-something. We have a plan to implement 'ecosystem namespaces' to ease interactions with the default ecosystem package names and to hide these namespaces from the CLI for people who only care about one ecosystem, but as @ccordoba12 says, pyqt vs PyQt5 has its own issues beyond that.

Member

mingwandroid commented Mar 26, 2017

In general please use the default package name as it is defined in the setup.py

Making this the general rule would preclude having sensible package names for packages that are not based on CPython. Personally, I would name all of those ones as py-something. We have a plan to implement 'ecosystem namespaces' to ease interactions with the default ecosystem package names and to hide these namespaces from the CLI for people who only care about one ecosystem, but as @ccordoba12 says, pyqt vs PyQt5 has its own issues beyond that.

@BrenBarn

This comment has been minimized.

Show comment
Hide comment
@BrenBarn

BrenBarn Jun 5, 2017

Does this mean it is not possible at all to use versions of PyQt later than the version available via conda? If so, that doesn't seem like a real solution.

BrenBarn commented Jun 5, 2017

Does this mean it is not possible at all to use versions of PyQt later than the version available via conda? If so, that doesn't seem like a real solution.

@ccordoba12

This comment has been minimized.

Show comment
Hide comment
@ccordoba12

ccordoba12 Jun 5, 2017

Of course it's possible. What it's not possible is for packages to depend on conda and pip PyQt5 packages at the same time. You have to depend on one or the other.

ccordoba12 commented Jun 5, 2017

Of course it's possible. What it's not possible is for packages to depend on conda and pip PyQt5 packages at the same time. You have to depend on one or the other.

@titusjan

This comment has been minimized.

Show comment
Hide comment
@titusjan

titusjan Jun 5, 2017

Cordoba, thanks again for your response. The reason @BrenBarn asks is that I was trying to help him on Stack Overflow here. Maybe I misunderstood your replies so perhaps you can answer some questions I have.

What will happen if you install PyQt twice in one environment, once with pip and once with conda? For instance BrenBarn's output of conda list qt looks like this:

pyqt                      5.6.0                    py36_2
PyQt5                     5.8.2                     <pip>
qt                        5.6.2                    vc14_4  [vc14]
qtconsole                 4.3.0                    py36_0
qtpy                      1.2.1                    py36_0

Is this supposed to work? If so, which version of PyQt is used then?

titusjan commented Jun 5, 2017

Cordoba, thanks again for your response. The reason @BrenBarn asks is that I was trying to help him on Stack Overflow here. Maybe I misunderstood your replies so perhaps you can answer some questions I have.

What will happen if you install PyQt twice in one environment, once with pip and once with conda? For instance BrenBarn's output of conda list qt looks like this:

pyqt                      5.6.0                    py36_2
PyQt5                     5.8.2                     <pip>
qt                        5.6.2                    vc14_4  [vc14]
qtconsole                 4.3.0                    py36_0
qtpy                      1.2.1                    py36_0

Is this supposed to work? If so, which version of PyQt is used then?

@ccordoba12

This comment has been minimized.

Show comment
Hide comment
@ccordoba12

ccordoba12 Jun 5, 2017

What will happen if you install PyQt twice in one environment, once with pip and once with conda?

I haven't done it but I guess things will break horribly because conda packages and pip wheels have very different build requirements.

So, this is a situation you can't have: you can either install PyQt5 conda packages or its pip wheels, but not both at the same time.

ccordoba12 commented Jun 5, 2017

What will happen if you install PyQt twice in one environment, once with pip and once with conda?

I haven't done it but I guess things will break horribly because conda packages and pip wheels have very different build requirements.

So, this is a situation you can't have: you can either install PyQt5 conda packages or its pip wheels, but not both at the same time.

@mingwandroid

This comment has been minimized.

Show comment
Hide comment
@mingwandroid

mingwandroid Jun 5, 2017

Member

What will happen if you install PyQt twice in one environment, once with pip and once with conda?

Conda provides the ability to have separate environments due to issues of needing to use conflicting packages. You must keep them separate, otherwise to which one would you expect an import statement to refer?

Member

mingwandroid commented Jun 5, 2017

What will happen if you install PyQt twice in one environment, once with pip and once with conda?

Conda provides the ability to have separate environments due to issues of needing to use conflicting packages. You must keep them separate, otherwise to which one would you expect an import statement to refer?

@titusjan

This comment has been minimized.

Show comment
Hide comment
@titusjan

titusjan Jun 5, 2017

Indeed. So I did understand this correctly.

My concern is that, once you have made a choice for your environment (pip or conda for PyQt), you must be very careful not to mix them. You can accidentally install a package that depends on PyQt and installs it the other way (because it doesn't recognize the existing installation).

Also, I see on Stack Overflow people like @BrenBarn installing PyQt with pip and conda simultaneously, unaware that they can't do that.

My main concern is that, once you have installed PyQt with pip and conda in a single environment, it breaks other environments and there seems no way to fix it except a complete reinstall of Anaconda (see my original post at the top of this threat).

Do you share these concerns?

titusjan commented Jun 5, 2017

Indeed. So I did understand this correctly.

My concern is that, once you have made a choice for your environment (pip or conda for PyQt), you must be very careful not to mix them. You can accidentally install a package that depends on PyQt and installs it the other way (because it doesn't recognize the existing installation).

Also, I see on Stack Overflow people like @BrenBarn installing PyQt with pip and conda simultaneously, unaware that they can't do that.

My main concern is that, once you have installed PyQt with pip and conda in a single environment, it breaks other environments and there seems no way to fix it except a complete reinstall of Anaconda (see my original post at the top of this threat).

Do you share these concerns?

@mingwandroid

This comment has been minimized.

Show comment
Hide comment
@mingwandroid

mingwandroid Jun 5, 2017

Member

I do not know the full details of how conda interoperates with pip, @kalefranz are pip installs global across all conda envs?

Member

mingwandroid commented Jun 5, 2017

I do not know the full details of how conda interoperates with pip, @kalefranz are pip installs global across all conda envs?

@BrenBarn

This comment has been minimized.

Show comment
Hide comment
@BrenBarn

BrenBarn Jun 5, 2017

It sounds like my issue is slightly different than what titusjan is reporting so maybe I'll open a separate issue. For me, pip-installing PyQt5 in an environment does not appear to break other environments (although it does break the environment where you do it). But pip-installing PyQt5 in the root environment seems to break other environments.

BrenBarn commented Jun 5, 2017

It sounds like my issue is slightly different than what titusjan is reporting so maybe I'll open a separate issue. For me, pip-installing PyQt5 in an environment does not appear to break other environments (although it does break the environment where you do it). But pip-installing PyQt5 in the root environment seems to break other environments.

@ankostis

This comment has been minimized.

Show comment
Hide comment
@ankostis

ankostis Aug 30, 2017

Would it be possible to create a dummy conda-package named PyQt5 that would become a dependency of pyqt [edit: or the other way around], so as to satisfy also the pip requirement when QT5 is installed??

ankostis commented Aug 30, 2017

Would it be possible to create a dummy conda-package named PyQt5 that would become a dependency of pyqt [edit: or the other way around], so as to satisfy also the pip requirement when QT5 is installed??

@mikofski

This comment has been minimized.

Show comment
Hide comment
@mikofski

mikofski Jan 18, 2018

I'll just add a note here, that I think in general perhaps pip and non-official channels (even conda-forge) should be prohibited from working in the conda root environment. I've told my team not to install anything untrusted into the root conda env and to treat it the same way you would sudo, because if your anaconda or miniconda root environment get's corrupted, then your entire anaconda or miniconda ecosystem may be damaged, and recovery for novices may be difficult. Just my two cents

mikofski commented Jan 18, 2018

I'll just add a note here, that I think in general perhaps pip and non-official channels (even conda-forge) should be prohibited from working in the conda root environment. I've told my team not to install anything untrusted into the root conda env and to treat it the same way you would sudo, because if your anaconda or miniconda root environment get's corrupted, then your entire anaconda or miniconda ecosystem may be damaged, and recovery for novices may be difficult. Just my two cents

@kalefranz

This comment has been minimized.

Show comment
Hide comment
@kalefranz

kalefranz Jan 18, 2018

Member

I do not know the full details of how conda interoperates with pip, @kalefranz are pip installs global across all conda envs?

Sorry just seeing this. My guess at what's going on here is that pip doesn't sever inodes before clobbering paths that exist in a prefix, and in that case it affects conda's underlying mechanism of hard-linking inodes into a prefix after a package is extracted into the package cache. So pip then alters every conda environment that shares that package.

Definitely a known issue, but one that we can't fix with conda. It's on our TODO list to patch pip (the conda-installed one) to prevent pip from behaving badly here, and then submit that patch upstream. Haven't gotten there yet though.

Member

kalefranz commented Jan 18, 2018

I do not know the full details of how conda interoperates with pip, @kalefranz are pip installs global across all conda envs?

Sorry just seeing this. My guess at what's going on here is that pip doesn't sever inodes before clobbering paths that exist in a prefix, and in that case it affects conda's underlying mechanism of hard-linking inodes into a prefix after a package is extracted into the package cache. So pip then alters every conda environment that shares that package.

Definitely a known issue, but one that we can't fix with conda. It's on our TODO list to patch pip (the conda-installed one) to prevent pip from behaving badly here, and then submit that patch upstream. Haven't gotten there yet though.

@kalefranz

This comment has been minimized.

Show comment
Hide comment
@kalefranz

kalefranz Jan 18, 2018

Member

I've told my team not to install anything untrusted into the root conda env and to treat it the same way you would sudo

I also very much support that sentiment. Conda environments solve a lot of problems. They're there to use. Not a bad idea to treat the root conda environment just like your system python, i.e. "leave the system python alone" == "leave the root conda environment alone".

We hope to fix this for everyone someday with conda/conda#3912

Member

kalefranz commented Jan 18, 2018

I've told my team not to install anything untrusted into the root conda env and to treat it the same way you would sudo

I also very much support that sentiment. Conda environments solve a lot of problems. They're there to use. Not a bad idea to treat the root conda environment just like your system python, i.e. "leave the system python alone" == "leave the root conda environment alone".

We hope to fix this for everyone someday with conda/conda#3912

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment