Installation problems on Windows

[mmagnuski]

Using the default command pip install mnelab on Windows I get:

Installing collected packages: PyQt5, mnelab
Could not install packages due to an EnvironmentError: [WinError 5] Access is denied: 'C:\\Users\\mmagn\\Continuum\\anaconda3\\Lib\\site-packages\\PyQt5\\Qt.pyd'
Consider using the `--user` option or check the permissions.

strangely, I get it even when I run it in command prompt with administrator privileges. When I run using pip install mnelab --user the script the mnelab.exe is put in a separate directory and does not do anything. When I then first activate my base anaconda env (it is my default python so I don't know why it makes any difference) and then do pip install mnelab - I get everything installed in one directory but mnelab.exe still does not work.
When I try to import mnelab in python I get:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "C:\Users\mmagn\Continuum\anaconda3\lib\site-packages\mnelab\__init__.py", line 1, in <module>
    from .mainwindow import MainWindow
  File "C:\Users\mmagn\Continuum\anaconda3\lib\site-packages\mnelab\mainwindow.py", line 7, in <module>
    from PyQt5.QtCore import (pyqtSlot, QStringListModel, QModelIndex, QSettings,
ModuleNotFoundError: No module named 'PyQt5.sip'

I might have been unlucky - I can test tomorrow on other Windows computers.

[hoechenberger]

Does it work if you create and activate a new environment first?

btw I don't know if you saw, but there's now also a package on conda-forge, which you could use until this issue has been sorted out :)

[mmagnuski]

I tried the conda-forge package: after conda install mnelab -c conda-forge conda was thinking for so long that I though it might be mining bitcoin instead of solving dependencies. ;) It stopped recently with the installation plan so I'm installing now.

[mmagnuski]

I get the same error with the conda package. I will try with fresh env and other computers tomorrow.

[cbrnr]

I have also seen this (or a very similar) error message with conda on Windows. It seems like the pyqt package is acting not exactly like the official PyQt5 package on PyPI. I couldn't solve the problem either so I created a clean env and it worked. However, in your case you could try to install sip and see if this solves the problem (I don't know why conda has to call packages differently than their PyPI counterparts).

[hoechenberger]

I tried the conda-forge package: after conda install mnelab -c conda-forge conda was thinking for so long that I though it might be mining bitcoin instead of solving dependencies. ;)

I would strongly suggest to install into a fresh environment, or else solving will take ages (or never finish), and chances are you will totally screw up your current (base) environment, as you‘ll end up with a mix of conda-forge and defaults packages.

[mmagnuski]

you could try to install sip and see if this solves the problem (I don't know why conda has to call packages differently than their PyPI counterparts).

Thanks, I'll try that!

I would strongly suggest to install into a fresh environment, or else solving will take ages (or never finish), and chances are you will totally screw up your current (base) environment, as you‘ll end up with a mix of conda-forge and defaults packages.

I know, I will test on a fresh env for sure - I am trying to persuade some of my friends to teach with mnelab instead of eeglab and a ready-made conda env recipe will be the simplest solution for them and their students. However, for myself I am ready to make extra effort to have everything working in one env. I already have some packages from conda and some from conda-forge and so far it is working fine. I am aware of the dangers though, thanks. :)

[mmagnuski]

@hoechenberger
It seems that installing the mnelab from conda did what you warned about: screwed up my base env. It is kind of weird because I installed multiple packages from conda-forge before and have never encoutered such problem.
Now even trying to conda uninstall mnelab suggests updating ~100 packages and removing as many (including mayavi, seaborn, statsmodels, dask and scikit-learn...).
If such problems are expected with this conda package I would suggest to change the wording in the mnelab docs: from "ideally into its own dedicated environment" to something suggesting that installing into base is not tested/supported and may screw up the base env.

[cbrnr]

I don't think this is specific for this package because this has happened to me so often that I don't want to use conda. But exactly for this reason I'm not a conda expert, maybe it's just me and I keep screwing it up for whatever reason. It'd be interesting to learn which packages from the conda-forge channel are installed as dependencies - other than MNE itself I think everything else should be present in the default channel.

[mmagnuski]

@cbrnr

I don't think this is specific for this package because this has happened to me so often that I don't want to use conda.

😄 I have very similar experience but with pip - but its most likely because I'm a windows user, and pip can sometimes be horrible on windows.
Anyway - using a separate env works well, so I still think it might be good to recommend setting up a separate env for mnelab more strongly in the docs.

[cbrnr]

😄 I have very similar experience but with pip - but its most likely because I'm a windows user, and pip can sometimes be horrible on windows.

Yep, I can only speak for macOS/Linux.

Anyway - using a separate env works well, so I still think it might be good to recommend setting up a separate env for mnelab more strongly in the docs.

The code explicitly creates a new env, but if you want we can also re-word the preceding sentence. However, I'd be interested if this is specific for this package - but this should be discussed at https://github.com/conda-forge/mnelab-feedstock/issues.

[teekuningas]

Often one might want to use defaults channel as the default and only install packages not present there from conda-forge, especially in the base environment, like:

conda install -c defaults -c conda-forge mnelab

Anyway python packaging is notably horrible, and almost always good idea to use environments.

[hoechenberger]

Generally, it's not a great idea to mix conda-forge and defaults packages. conda-forge has the policy that all dependencies of a package must be packaged on conda-forge too (I think one of the few exceptions is scipy for Windows). This is to avoid binary incompatibilities and weird interactions between the packages. I think you can report issues related to mixing conda-forge and defaults packages at conda-forge/conda-forge.github.io#434

Until recently, one would constantly run into trouble with packages depending on mathematical libraries, like numpy (see conda-forge/numpy-feedstock#108). This seems to have been mostly resolved now with the introduction of the strict channel priority feature, which also speeds up the process of solving environments dramatically in some situations. conda 4.7 added further improvements to speed up package installation; yet, the strict channel priority thing is not enabled by default, as it's creating new issues in some cases. The conda-forge documentation, however, suggests that this feature be enabled for the installation of individual packages:

conda config --add channels conda-forge 
conda config --set channel_priority strict 
conda install <package-name>

Installing conda-forge packages "regularly" into the base environment -- and even more so: installing packages with complex dependencies into the base environment -- is really asking for trouble. I would strongly suggest to never do that, it will screw things up. Similarly, each project should have their own, dedicated conda environment. mne has binary dependencies with more-or-less-strictly pinned version numbers (see e.g. the numpy pinning of mayavi). Trying to blend that in with other packages in the same environment is difficult and likely to cause trobule. I had assumed that using dedicated environments for each project was the default mode, and only in exceptional cases would one start "mixing" or installing stuff into the base environment. I will update the readme to more explicitly urge users to create a new environment for mnelab.

So to cut the long story short:

• never install complex stuff into the base environment
• try to avoid mixing conda-forge and defaults packages in one environment
• use the latest conda releases
• consider using strict channel priority

[cbrnr]

Wow, this sounds really complicated given that conda should simplify Python package management. I never had problems with installing mne and mnelab in my default Python env with pip.

What about the alternative that is already mentioned at the bottom of the README.md:

conda install numpy scipy matplotlib pyqt
pip install mne mnelab

This works and doesn't screw up the base env because no additional deps are pulled in (mne and mnelab are the only packages installed from PyPI).

[cbrnr]

Ah wait, it doesn't work because pip also installs PyQt5-sip and PyQt5 because these packages are named differently in the defaults conda channel (sip and pyqt)...

[cbrnr]

But you can do this:

pip install --no-deps mnelab

[hoechenberger]

@cbrnr If one follows the current conda installation instructions from the readme, the described issues don't arise :).

[cbrnr]

Yes I know, but should we mention the other option? I imagine that people might have reasons to install MNELAB into their base env (so it's always available without the additional activation command).

[hoechenberger]

I imagine that people might have reasons to install MNELAB into their base env

Yeah we could do that, I still believe it's... not a great thing to do. but who am I to judge :)

[hoechenberger]

@cbrnr I can do a PR on this later today

[cbrnr]

You're the conda expert, so if you say it should be done as it is currently in the README, there's no need to describe a stupid way. Also, let's make things not overly complicated - if someone uses conda there is one way to install it.

[hoechenberger]

conda install numpy scipy matplotlib pyqt

This would of course also skip mayavi, which is one of the more problematic deps

[cbrnr]

Yes, I don't need mayavi 😄 - but I get the point, my setup is very special, so let's not get into that. If someone really wants to install a barebones MNE + MNELAB in a conda base env, they will figure it out (or I will tell them upon request).

[hoechenberger]

I'd like to hear @mmagnuski's opinion on that :)

[cbrnr]

Yes, let's wait. Just to make it clear, pip install --no-deps mnelab would have worked in his case.

[mmagnuski]

Yeah we could do that, I still believe it's an almost stupid thing to do… but who am I to judge :)

I think there are many good reasons to have a good working base conda environment with everything one needs and create additional envs only when there are conflicts that one is not able to avoid. In my context for example, I was trying to simplify things for my students, who often feel uncomfortable installing packages in python from console - asking them to have separate envs for different things, or even just one env might be too much, unfortunatelly.
Also, people like me, who never had conda-forge problems before and who find constant env switching inconvenient, might be tempted to just install from conda-forge in their base or any other existing env. So if this is expected to fail - the instructions should be very clear about that.

[hoechenberger]

@mmagnuski
I don't know if you saw, but a couple minutes after I'd posted my comment, I changed the first sentence to remove the offensive phrasing I'd used initially. I'd like to apologize for that! I can see that there are potential use cases for "installing things into the base environment". :) I'm not sure regarding your point with respect to your students being unfomfortable with installing packages from the Python console and how this is related to using different environments? Environments can be created (and packages installed) from within many IDEs, and there's also this GUI thing by Anaconda (Anaconda Navigator or so? forgot the name). My IDE automatically switches to the correct environment as soon as I open a project. Using environments is, in fact, one of the earliest things I try to teach my students, and it's been working out fine so far. (btw the next major Spyder release will also support easy switching of environments!)
I wouldn't say installing into base is expected to fail, but it's likely to cause problems.

[mmagnuski]

Sure, no offence. :)
Regarding students: it's just adding to the complexity for someone who did not have much programming experience apart from some RStudio basics (where you usually don't have to worry about environments). I'm using Spyder with them (with spyder-notebook package), because without variable explorer I've found its harder for students to have a good mental model of variables and how they change. I didn't teach them Spyder projects so far, but next semester I plan to prepare a conda env recipe for the whole course so Spyder projects might be useful. And in this regard I agree that one conda env recipe might be the simplest solution.
BTW - which IDEs do you have in mind? I actually never used Anaconda Navigator for package installations, but maybe I should teach package installation this way, I will check that out. :)