Problems caused by different in package name in Pip (pyqt vs PyQt5)

[titusjan]

One of the things I like about Anaconda is that it works well together with Pip. If a package is not available in Anaconda I can fall back on installation from PyPi. However, this does not work for PyQt because the package name in Anaconda is pyqt, but on PyPi it is PyQt5.

For me, as a developer of packages that use PyQt, this means that I can not add PyQt5 to the install_requires parameter in the setup.py of my package. If I do, and a user installs my package with Pip, it doesn't recognize that pyqt has already been installed with Anaconda. It will then download PyQt5 from PyPi and install it over the pyqt package. This breaks the installation of other environments beyond repair. The user will get a segmentation fault if a program tries to import PyQt!

Please rename the pyqt package to PyQt5 (I don't know if case-sensitivity matters). In general please use the default package name as it is defined in the setup.py. I realize that this has a lot of impact, if it is possible at all. However, I think it should at leat be discussed (I could not find anything on this with Google).

To repeat the problem you can install PyQtChart, which automatically installs PyQt5. I accidentally did this and now my PyQt broken for all environments that use the same Python version (any help on fixing that would be appreciated as well).

# Create two environments to illustrate that multiple envs are affected.
conda create -n env1 python=3.5.1  
conda install -n env1 pyqt
conda create -n env2 python=3.5.1  
conda install -n env2 pyqt
. activate env2

# This works fine now
python -c "from PyQt5 import QtCore" 

# This will install PyQt5 from PyPi as well
pip install PyQtChart

# Now you see two installation (pyqt and PyQt5)
conda list -n env2 qt

# This still works because it uses the PyPi PyQt
python -c "from PyQt5 import QtCore" 

# However, the first environment is now broken and gives "Segmentation fault: 11"
. activate env1
python -c "from PyQt5 import QtCore"

Like I said. I haven't managed to fix the issue. Removing PyQt with pip and reinstalling with conda didn't help. Luckily my PyQt from Python 3.6 still works but all 3.5 PyQt installations are broken. I use OS-X 10.11.5. Help on how to repair it is appreciated.

[ccordoba12]

I don't think we're going to change this package name. Some reasons:

1. We need to maintain compatibility with pyqt 4, so it's easier to have one package name and two versions, instead of two packages.
2. Our package and the pyqt5 wheel are very different, so it's better to keep them as separate things. The wheel has all its dependencies packaged with it while our package properly depends on other packages. Also, the wheel doesn't work on old systems (centos 5) while our package does.

[ccordoba12]

Solution for you: create conda packages for your projects and their dependencies instead of trying to use wheels in conda. Although conda and pip work well most of time, they are not totally compatible, specially regarding packages with compiled libraries.

You can use conda-forge for that, as most people do these days.

[mingwandroid]

In general please use the default package name as it is defined in the setup.py

Making this the general rule would preclude having sensible package names for packages that are not based on CPython. Personally, I would name all of those ones as py-something. We have a plan to implement 'ecosystem namespaces' to ease interactions with the default ecosystem package names and to hide these namespaces from the CLI for people who only care about one ecosystem, but as @ccordoba12 says, pyqt vs PyQt5 has its own issues beyond that.

[BrenBarn]

Does this mean it is not possible at all to use versions of PyQt later than the version available via conda? If so, that doesn't seem like a real solution.

[ccordoba12]

Of course it's possible. What it's not possible is for packages to depend on conda and pip PyQt5 packages at the same time. You have to depend on one or the other.

[titusjan]

Cordoba, thanks again for your response. The reason @BrenBarn asks is that I was trying to help him on Stack Overflow here. Maybe I misunderstood your replies so perhaps you can answer some questions I have.

What will happen if you install PyQt twice in one environment, once with pip and once with conda? For instance BrenBarn's output of conda list qt looks like this:

pyqt                      5.6.0                    py36_2
PyQt5                     5.8.2                     <pip>
qt                        5.6.2                    vc14_4  [vc14]
qtconsole                 4.3.0                    py36_0
qtpy                      1.2.1                    py36_0

Is this supposed to work? If so, which version of PyQt is used then?

[ccordoba12]

What will happen if you install PyQt twice in one environment, once with pip and once with conda?

I haven't done it but I guess things will break horribly because conda packages and pip wheels have very different build requirements.

So, this is a situation you can't have: you can either install PyQt5 conda packages or its pip wheels, but not both at the same time.

[mingwandroid]

What will happen if you install PyQt twice in one environment, once with pip and once with conda?

Conda provides the ability to have separate environments due to issues of needing to use conflicting packages. You must keep them separate, otherwise to which one would you expect an import statement to refer?

[titusjan]

Indeed. So I did understand this correctly.

My concern is that, once you have made a choice for your environment (pip or conda for PyQt), you must be very careful not to mix them. You can accidentally install a package that depends on PyQt and installs it the other way (because it doesn't recognize the existing installation).

Also, I see on Stack Overflow people like @BrenBarn installing PyQt with pip and conda simultaneously, unaware that they can't do that.

My main concern is that, once you have installed PyQt with pip and conda in a single environment, it breaks other environments and there seems no way to fix it except a complete reinstall of Anaconda (see my original post at the top of this threat).

Do you share these concerns?

[mingwandroid]

I do not know the full details of how conda interoperates with pip, @kalefranz are pip installs global across all conda envs?

[BrenBarn]

It sounds like my issue is slightly different than what titusjan is reporting so maybe I'll open a separate issue. For me, pip-installing PyQt5 in an environment does not appear to break other environments (although it does break the environment where you do it). But pip-installing PyQt5 in the root environment seems to break other environments.

[ankostis]

Would it be possible to create a dummy conda-package named PyQt5 that would become a dependency of pyqt [edit: or the other way around], so as to satisfy also the pip requirement when QT5 is installed??

[mikofski]

I'll just add a note here, that I think in general perhaps pip and non-official channels (even conda-forge) should be prohibited from working in the conda root environment. I've told my team not to install anything untrusted into the root conda env and to treat it the same way you would sudo, because if your anaconda or miniconda root environment get's corrupted, then your entire anaconda or miniconda ecosystem may be damaged, and recovery for novices may be difficult. Just my two cents

[kalefranz]

I do not know the full details of how conda interoperates with pip, @kalefranz are pip installs global across all conda envs?

Sorry just seeing this. My guess at what's going on here is that pip doesn't sever inodes before clobbering paths that exist in a prefix, and in that case it affects conda's underlying mechanism of hard-linking inodes into a prefix after a package is extracted into the package cache. So pip then alters every conda environment that shares that package.

Definitely a known issue, but one that we can't fix with conda. It's on our TODO list to patch pip (the conda-installed one) to prevent pip from behaving badly here, and then submit that patch upstream. Haven't gotten there yet though.

[kalefranz]

I've told my team not to install anything untrusted into the root conda env and to treat it the same way you would sudo

I also very much support that sentiment. Conda environments solve a lot of problems. They're there to use. Not a bad idea to treat the root conda environment just like your system python, i.e. "leave the system python alone" == "leave the root conda environment alone".

We hope to fix this for everyone someday with conda/conda#3912