problems installing mne on win10 64bit, py3.6(anaconda).

[Vincent-wq]

1. when I am trying to install mne using anaconda3, I have the following problem (seems the problem installing mayavi)
   

2. I have VC2015 installed and using the MSBuild Command Prompt for VS2015 command line tool;

3. looks like some of the dependencies has been updated since the enviorenment.yml created, what can I do, I really want to try mne as soon as possible. By the way, I am in OHBM Sigapore and has been to the hackathon just few days ago, my name is Qing Wang, hope to see you in OHBM！
   B. R.
   Vincet

[agramfort]

can you explain what you did? conda env create -f environment.yml ?

[Vincent-wq]

Yes, just follow the instructions here: https://www.martinos.org/mne/stable/install_mne_python.html#get-mne-and-its-dependencies, When I am trying to create the env for mne using conda env create -f environment.yml thanks! 发件人: Alexandre Gramfort <notifications@github.com> 发送时间: 2018年6月18日 10:01 收件人: mne-tools/mne-python <mne-python@noreply.github.com> 抄送: Wang Qing <vincent.w.qing@gmail.com>; Author <author@noreply.github.com> 主题: Re: [mne-tools/mne-python] problems installing mne on win10 64bit, py3.6(anaconda). (#5288) can you explain what you did? conda env create -f environment.yml ? — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#5288 (comment)> , or mute the thread <https://github.com/notifications/unsubscribe-auth/AKffsLgk7ztGp3m0DPP8w0WC5aiPJEGAks5t9wndgaJpZM4UrC7z> . <https://github.com/notifications/beacon/AKffsLwWwV-zH2I-ISk6f_0unwwxCGW2ks5t9wndgaJpZM4UrC7z.gif>

[agramfort]

ok let's try together. if you have the pb when doing: pip install mayavi you can ask the question on the mayavi forum / mailing list.

[Vincent-wq]

I guess it looks like some kind of general problems with installing mayavi, the problem may be the incompactable compiler version or the system did not recognize the version of complier. I will try to navigate myself in the mayavi forum, and I need to install all the following most recent version of the packages listed in the yml file, after I fix this problem, right? Great thanks. - pip: - mne - mayavi - "https://api.github.com/repos/nipy/PySurfer/zipball/master" - nitime - nibabel - nilearn - neo - pytest-sugar - pytest-faulthandler - pydocstyle - sphinx_bootstrap_theme - "https://api.github.com/repos/sphinx-gallery/sphinx-gallery/zipball/master" - python-picard - sphinx_fontawesome

[Vincent-wq]

There is problem with Traits when pip install mayavi, seems the same problem.

[mlearnx]

I also had a similar issue with Anaconda and Windows 10. Try to install the MNE without creating an environment and by directly installing dependencies one by one. This worked for me.

For instance,
pip:

• mne
• mayavi
• "https://api.github.com/repos/nipy/PySurfer/zipball/master"
• nitime
• nibabel
• nilearn
• neo
• pytest-sugar
• pytest-faulthandler
• pydocstyle
• sphinx_bootstrap_theme
• "https://api.github.com/repos/sphinx-gallery/sphinx-gallery/zipball/master"
• python-picard
• sphinx_fontawesome

[Vincent-wq]

Now I have fixed this problem, and here are my experiences:

1. intall python3.6 with anaconda (https://www.anaconda.com/download/);
2. intall visual studio2017 community(free), and check the following items and install: (https://visualstudio.microsoft.com)
   2.1 VC++ 2015.3 v14.00 (v140) toolset for desktop
   2.2 Windows 10 SDK (10.0.17134.0)
   2.3 Windows 8.1 SDK
3. following the official guidance of mne (run command in the VS2015x64 Native Tools, not Power Shell or CMD, run as administrator incase),
   3.1 curl -O https://raw.githubusercontent.com/mne-tools/mne-python/master/environment.yml
   3.2 conda env create -f environment.yml
   3.3 now it will give errors in building mayavi, and mkl needs for cython,
   3.3 activate mne
   3.4 pip install msgpack
   3.5 pip install PyQt5
   3.6 conda install cython
   3.6 copy rc.exe and rcdll.dll from "C:\Program Files (x86)\Windows Kits\8.1\bin\x64" to "C:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\bin\x86_amd64" (where the VS Native tool is)
   3.6 pip install mayavi
4. for me, problems solved at this step.
   So I guess problems are from the changes of the organization of visual c++ 14.0 complier and linker.
5. continue installing other packages listed in the environment.yml:
   5.1 cd c:\Anaconda3\envs\mne\Scripts
   5.2 pip install "https://api.github.com/repos/nipy/PySurfer/zipball/master"
   5.3 pip install nitime
   5.4 pip install nilearn
   5.5 pip install neo
   5.6 pip install pytest-sugar
   5.7 pip install pytest-faulthandler
   5.8 pip install pydocstyle
   5.9 pip install sphinx_bootstrap_theme
   5.10 pip install "https://api.github.com/repos/sphinx-gallery/sphinx-gallery/zipball/master"
   5.11 pip install python-picard
   5.12 pip install sphinx_fontawesome
6. now your brand new mne and other NIPY tools are ready, enjoy!

[agramfort]

thanks for all the details.

I am thinking that these details on how to make the mayavi install work on Windows should be on the mayavi documentation page.

What I don't fully understand is why it works fine on Appveyor via a conda env out of the box.

@prabhuramachandran any idea? Short story is: We tell MNE users do install with: http://martinos.org/mne/stable/install_mne_python.html#get-mne-and-its-dependencies
but it fails apparently on Windows due to mayavi.

[prabhuramachandran]

Hmm our appveyor tests seem to be running fine! https://ci.appveyor.com/project/EnthoughtOSS/mayavi
Of course, I am using edm and not conda here.
https://github.com/enthought/mayavi/blob/master/appveyor.yml

[agramfort]

@prabhuramachandran we install mayavi through pip :

https://github.com/mne-tools/mne-python/blob/master/environment.yml#L34

does it mean that pypi package is not enough self contained? it cannot work on a fresh win64 Python? it needs a compiler?

[prabhuramachandran]

I've uploaded VTK wheels on pypi but not Mayavi wheels. Mayavi has a tiny extension module but that apart is pure Python. It does need to compile a single extension (which was made using Pyrex!). That should work on appveyor. How do you install traits?

I can see about making the extension module optional (for a performance hit) in mayavi if it will make things easier.

[agramfort]

we install - traits>=4.6.0 - pyface>=6 - traitsui>=6 with conda. See: https://github.com/mne-tools/mne-python/blob/master/environment.yml#L29 it's working on our CIs. Is it the cause of our issue? thanks so much for being so reactive.

[prabhuramachandran]

OK, so the rest are from conda.
I've filed a ticket enthought/mayavi#658 to remind me to try and remove the array_ext extension at some point. It should be easy to do I imagine but I'll need to test it carefully.

It should be possible to install a suitable MSVC compiler relatively easily though, https://wiki.python.org/moin/WindowsCompilers
this is useful in itself for things like Cython etc. so I would advise the OP to perhaps do this.

[Vincent-wq]

Now I see new problems, when I try to test mayavi with qt5 and qt4 for my installation, they works well. But when I am trying the example (http://martinos.org/mne/stable/auto_examples/connectivity/plot_mne_inverse_psi_visual.html#sphx-glr-auto-examples-connectivity-plot-mne-inverse-psi-visual-py), I got errors, no idea what happens, it seems that the vis engine and dependencies are not well maintained:
with qt4, I have the following error:
File "C:\Users\Administrator\Desktop\01_2d_signals.py", line 96, in
clim=dict(kind='percent', pos_lims=(95, 97.5, 100)))
File "c:\Anaconda3\envs\mne\Lib\site-packages\mne\source_estimate.py", line 1463, in plot
spacing=spacing)
File "c:\Anaconda3\envs\mne\Lib\site-packages\mne\viz_3d.py", line 1635, in plot_source_estimates
from surfer import Brain, TimeViewer
File "c:\Anaconda3\envs\mne\Lib\site-packages\surfer_init_.py", line 1, in
from .viz import Brain, TimeViewer # noqa
File "c:\Anaconda3\envs\mne\Lib\site-packages\surfer\viz.py", line 13, in
from mayavi import mlab
File "c:\Anaconda3\envs\mne\Lib\site-packages\mayavi\mlab.py", line 15, in
from mayavi.tools.camera import view, roll, yaw, pitch, move
File "c:\Anaconda3\envs\mne\Lib\site-packages\mayavi\tools\camera.py", line 24, in
from .engine_manager import get_engine
File "c:\Anaconda3\envs\mne\Lib\site-packages\mayavi\tools\engine_manager.py", line 12, in
from mayavi.preferences.api import preference_manager
File "c:\Anaconda3\envs\mne\Lib\site-packages\mayavi\preferences\api.py", line 4, in
from .preference_manager import preference_manager
File "c:\Anaconda3\envs\mne\Lib\site-packages\mayavi\preferences\preference_manager.py", line 29, in
from traitsui.api import View, Group, Item
File "c:\Anaconda3\envs\mne\Lib\site-packages\traitsui\api.py", line 36, in
from .editors.api import ArrayEditor
File "c:\Anaconda3\envs\mne\Lib\site-packages\traitsui\editors_init_.py", line 23, in
from .api import ArrayEditor
File "c:\Anaconda3\envs\mne\Lib\site-packages\traitsui\editors\api.py", line 24, in
from .code_editor import CodeEditor
File "c:\Anaconda3\envs\mne\Lib\site-packages\traitsui\editors\code_editor.py", line 37, in
class ToolkitEditorFactory(EditorFactory):
File "c:\Anaconda3\envs\mne\Lib\site-packages\traitsui\editors\code_editor.py", line 49, in ToolkitEditorFactory
mark_color = Color(0xECE9D8)
File "c:\Anaconda3\envs\mne\Lib\site-packages\traits\traits.py", line 522, in call
return self.maker_function( *args, **metadata )
File "c:\Anaconda3\envs\mne\Lib\site-packages\traits\traits.py", line 1236, in Color
return ColorTrait( *args, **metadata )
File "c:\Anaconda3\envs\mne\Lib\site-packages\traitsui\toolkit_traits.py", line 8, in ColorTrait
return toolkit().color_trait(*args, **traits)
File "c:\Anaconda3\envs\mne\Lib\site-packages\traitsui\toolkit.py", line 109, in toolkit
toolkit = find_toolkit('traitsui.toolkits')
File "c:\Anaconda3\envs\mne\Lib\site-packages\pyface\base_toolkit.py", line 263, in find_toolkit
return import_toolkit(ETSConfig.toolkit, entry_point)
File "c:\Anaconda3\envs\mne\Lib\site-packages\pyface\base_toolkit.py", line 217, in import_toolkit
toolkit_object = plugin.load()
File "c:\Anaconda3\envs\mne\Lib\site-packages\pkg_resources_init.py", line 2341, in load
return self.resolve()
File "c:\Anaconda3\envs\mne\Lib\site-packages\pkg_resources_init_.py", line 2347, in resolve
module = import(self.module_name, fromlist=['name'], level=0)
File "c:\Anaconda3\envs\mne\Lib\site-packages\traitsui\qt4_init_.py", line 25, in
from . import toolkit
File "c:\Anaconda3\envs\mne\Lib\site-packages\traitsui\qt4\toolkit.py", line 25, in
from pyface.toolkit import toolkit_object as pyface_toolkit
File "c:\Anaconda3\envs\mne\Lib\site-packages\pyface\toolkit.py", line 27, in
toolkit = toolkit_object = find_toolkit('pyface.toolkits')
File "c:\Anaconda3\envs\mne\Lib\site-packages\pyface\base_toolkit.py", line 263, in find_toolkit
return import_toolkit(ETSConfig.toolkit, entry_point)
File "c:\Anaconda3\envs\mne\Lib\site-packages\pyface\base_toolkit.py", line 227, in import_toolkit
raise RuntimeError(msg)

builtins.RuntimeError: No pyface.toolkits plugin could be loaded for qt4

with qt5, I have the following error:

File "C:\Users\Administrator\Desktop\01_2d_signals.py", line 96, in
clim=dict(kind='percent', pos_lims=(95, 97.5, 100)))
File "c:\Anaconda3\envs\mne\Lib\site-packages\mne\source_estimate.py", line 1463, in plot
spacing=spacing)
File "c:\Anaconda3\envs\mne\Lib\site-packages\mne\viz_3d.py", line 1669, in plot_source_estimates
views=views)

builtins.TypeError: init() got an unexpected keyword argument 'curv'

[Vincent-wq]

I have found another problem with the jupyter notebook, it starts very slow, takes more than 5 mins on a i7, 16G DDR4 machine, it looks strange....

[agramfort]

@Vincent-wq use qt5

the error you see is due to a recent non-cautious API change in PySurfer. We need to cut a new PySurfer release. In the mean time you can edit the file c:\Anaconda3\envs\mne\Lib\site-packages\mne\viz_3d.py and remove the curv parameter. Sorry about this.

[larsoner]

No need for new PySurfer release, it should be fixed by the next maintenance release of MNE

[Vincent-wq]

Thanks for Gramfort, the problem is solved now. But the problem is, when I am testing mayavi using nosetests, it passed, I have tried several examples from mayavi, it works well with all the interations, but when runing the examples from mne, it can render successfully, then just stop responding, no idea what happen.

[agramfort]

maybe a memory usage on your machine. It's really hard to tell. can someone around you replicate the pb on his machine?

[Vincent-wq]

I have tried on 2 different win 10 machines, the problems are the same, and quite stable for several examples. Some times, the mayavi window just disappear instantly after it is finished, quit strange, maybe I need to read more blogs on mayavi

[agramfort]

@prabhuramachandran your quote from last scipy still resonates in my head. "I am also known for having produced the python package which is the hardest to install" .... I will just add "among the very useful pacakges" :)

[prabhuramachandran]

@agramfort -- thank you for the kind comment! Glad it is still useful. The dependencies and options make it tough. Making VTK pip installable is a big help already and makes things much easier. I'm working on the enthought/mayavi#658 issue to try and make the extension module optional. However, building a module optionally is yet another of those silly things that takes time to figure out and get right.

[prabhuramachandran]

I've just merged a PR to only optionally add the extension module. @Vincent-wq -- can you please try with pip installing the git master version of Mayavi like so on your machine where you had the issue:

pip install git+https://github.com/enthought/mayavi.git#egg=mayavi

Hopefully this should make Mayavi pip installable even when a compiler is not available, albeit at the expense of one slower function which may not be used heavily by an average user.

[larsoner]

@prabhuramachandran I had a quick look at set_id_type_array_py and am surprised Cython helps 4x. Is this because np.insert does an extra copy? If so I think there might be a way to avoid it so it approaches Cython does for large arrays. Do you have a gist the is slow in Python but fast in Cython?

[prabhuramachandran]

I was surprised too but I think it is because of the temporaries created. insert creates a new temporary which is assigned. I don't have a gist but can make one, I just used the following code in a notebook:

from tvtk.array_handler import set_id_type_array_py
from tvtk.array_ext import set_id_type_array
import numpy as np

n = 100000
cs = 10
a = np.arange(cs*n)
a.shape = n, cs
b = np.empty(n*(cs+1))

%timeit set_id_type_array(a, b)
%timeit set_id_type_array_py(a, b)

I tried with different values of n, cs but thats the code. I tried several other approaches and they all seemed slower. Feel free to suggest any improvements or send a PR.

[larsoner]

Took the ~2x slowdown to ~25%, views are wonderful :)

[agramfort]

@larsoner shall we update our environment.yml file?
@prabhuramachandran can you make a minor release?

[prabhuramachandran]

@larsoner -- yes they are along with the ability to add an axis. Without changing the shape, this is much harder. Thanks for the extra pair of eyes. On my machine I still get close to 50% slowdown and with smaller arrays even 2x.
@agramfort -- I will do so after a few other improvements that I want to squeeze in. Also it would be nice if this can be tested to actually work without a compiler.

[prabhuramachandran]

@agramfort -- just FYI, I will push a point release by the next weekend.

[agramfort]

great please let us know when you do thanks

[Vincent-wq]

@prabhuramachandran Great thanks, I just did as follows (uninstall mayavi and pip install git+https://github.com/enthought/mayavi.git#egg=mayavi
),

but the vis problems is still there (it stops responding), now I think the compiler is OK, and it looks more like the problem when using mayavi in the ploting. when I am trying to run independent code using mayavi, it looks ok. run the nosetests tests, and it looks like follows:

then vtkgui jump out and give an error:
ERROR: In C:\VPP\standalone-build\VTK-source\Interaction\Widgets\vtkSphereWidget.cxx, line 138
vtkSphereWidget (00000210A91D3F10): The interactor must be set prior to enabling/disabling widget

[prabhuramachandran]

@Vincent-wq -- Thank you! That is excellent. I suspect the screenshot you sent first is just to show that the right version of mayavi is installed. As regards the vis problem, this is not a Mayavi installation issue but perhaps something to do with mne-tools, I don't know. The test error popping up is an annoyance more than an actual error. I will investigate that issue and see if anything can be done about that. Thanks for testing the installation out though. Just to reconfirm, on this setup, if you uninstall mayavi and do a normal pip install it still fails right?

[prabhuramachandran]

@agramfort -- I just pushed Mayavi-4.6.1 to pypi. Enjoy.

[agramfort]

great !

@Vincent-wq can you give a try to the new release?

[Vincent-wq]

@prabhuramachandran @agramfort , sorry for the long delay, just came back from vocation.
I have tried the follows:

1. update mne to 0.16.2 version
2. update Mayavi to 4.6.1 version
   but the same visualization problem still exists, the computation works perfect all the time, the rendering of the results is fine, but once I put my cursor on the windows of mayavi, it stops responding, maybe it is the problem of the interactions in the visualization? I think mayavi can work well independently
   no idea what the problems is...

[agramfort]

can you run fine the pysurfer examples: https://pysurfer.github.io/auto_examples/index.html ?

[larsoner]

Or even simpler before that, can you interact with this window:

from mayavi import mlab
mlab.test_plot3d()

[prabhuramachandran]

But after you run %gui qt right?

[agramfort]

@Vincent-wq can I close this?

[prabhuramachandran]

@agramfort -- this is also relevant: enthought/mayavi#683 , it is possible that the earlier release did not implement the compiler check correctly. Should be fixed in master.

[schelian]

@Vincent-wq , I'm following your steps above up to 3.6. I don't have C:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\bin\x86_amd64 because you said to install Visual Studio 2017 (v. 15). Should I install v.14 instead?

[larsoner]

You should no longer need MSVC (or any complier) to install Mayavi

[Vincent-wq]

Or even simpler before that, can you interact with this window:

from mayavi import mlab
mlab.test_plot3d()

for this part, it works well.

[Vincent-wq]

@agramfort @prabhuramachandran @larsoner @mlearnx
Sorry for the year long delay, I have spend some time debugging this problems again, here are my observations:

1. install with the most recent version of environment.yml will only install the conda install pkgs (not including the pip install pkgs);
2. Then, I pip install all the rest pkgs according to the list in environment.yml, and then success;
3. The only way I can run the example is that:
   3.1 in spider: force ipython console in spider to use Qt5 or Qt4 or auto, it works well, but sometimes not that stable, qt4 is more stable, and I have tries some mayavi examples, it only runs with qt4;
   3.2 in spider: with or without the following lines (hoping to tell mayavi to use qt5 as backend), it works well for mne example and mayavi example;
   #os.environ['ETS_TOOLKIT'] = 'qt'
4. The biggest problem is when I am using other IDE (or directly in system cmd), and set python interpreter as the one in mne environment, none of the above methods work.

The good news is that, at least I can work with it, but I really want to know how to tame mayavi to work better, I may spend some time doing a visualization project, also I really do not how ipython and spider made it work, why it does not work in the system cmd console, even I am trying to tell mayavi which backend to use. Also the jupyter notebook is always not working well for me (also this time), I prefer to use wingIDE, and spider is very easy to corrupt when dealing with intensive task (lacking good default good color setting is also not that firendly).

Great thanks!

[Vincent-wq]

@Vincent-wq , I'm following your steps above up to 3.6. I don't have C:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\bin\x86_amd64 because you said to install Visual Studio 2017 (v. 15). Should I install v.14 instead?

the new version of document suppose to have all problems settled, at least it works for me now, please refer to my last reply.

[Vincent-wq]

You should no longer need MSVC (or any complier) to install Mayavi

yet, I think you already put the wheel file in the server.

[larsoner]

The biggest problem is when I am using other IDE (or directly in system cmd), and set python interpreter as the one in mne environment, none of the above methods work.

You can probably get matplotlib to make things interactive in vanilla Python (assuming it and Mayavi are both set to use PyQt5) with:

import matplotlib.pyplot as plt
plt.ion()

[Vincent-wq]

Not sure if I am doing the right thing, I have tried to set qt5 as backend for both mayavi and matplotlib as follows:
import matplotlib
matplotlib.use('Qt5Agg')
import matplotlib.pyplot as plt
plt.ion()
import os
os.environ['ETS_TOOLKIT'] = 'qt'
os.environ['QT_API'] = 'pyqt5'
it is still not responding (guessing the interaction has problems)

and if I put qt4 for both, then TVTK Scene apears and stoped responding...
And I realize if I am not using plt.ion(), TVTK will always be prefered....

The biggest problem is when I am using other IDE (or directly in system cmd), and set python interpreter as the one in mne environment, none of the above methods work.

You can probably get matplotlib to make things interactive in vanilla Python (assuming it and Mayavi are both set to use PyQt5) with:

import matplotlib.pyplot as plt
plt.ion()