MRG: Prototype of notebook viz (ipyvtk)

[GuillaumeFavelier]

This PR updates the current notebook 3d backend to use ipyvtk. It allows native support for widgets, 3d render view and intuitive camera interactions. It could also support google colab and jupyter lab? More test is needed.

It's still a work in progress:

code

import os
import mne
from mne.datasets import sample
mne.viz.set_3d_backend('notebook')
data_path = sample.data_path()
sample_dir = os.path.join(data_path, 'MEG', 'sample')
subjects_dir = os.path.join(data_path, 'subjects')
fname_stc = os.path.join(sample_dir, 'sample_audvis-meg')
stc = mne.read_source_estimate(fname_stc, subject='sample')
initial_time = 0.13
brain = stc.plot(subjects_dir=subjects_dir, initial_time=initial_time,
                 clim=dict(kind='value', pos_lims=[3, 6, 9]),
                 time_viewer=True,
                 show_traces=True,
                 hemi='split',
                 size=600,
                 background="white",
                 )

[hoechenberger]

😍😍😍

[GuillaumeFavelier]

Update after 4d9b63a:

I think we could use ipywidgets to reproduce the tool bar buttons.

I started by working directly on Brain but next I'll see how the other 3d plots behave.

[GuillaumeFavelier]

Update after dcaff44:

[larsoner]

Lots of - lines is nice, and it seems to work well! I assume this is server-side rendering piping PNGs to the client?

Anything else left to do here?

@christianhacker is this more like what you want to use?

[larsoner]

Anything else left to do here?

I guess one thing from seeing what you've shown so far would be making sure that show_traces=True adds a matplotlib plot below. Would this require a lot of hacking of the backend to get right? Maybe this is a PR we can merge early in the 0.23 cycle so people can play with it.

[GuillaumeFavelier]

I assume this is server-side rendering piping PNGs to the client?

After digging into https://github.com/Kitware/ipyvtk-simple/blob/master/ipyvtk_simple/viewer.py#L163-L171 I can confirm this indeed. JPEG though :)

Anything else left to do here?

• shortcuts
• toolbar
• matplotlib canvas

Would this require a lot of hacking of the backend to get right?

I don't think so...

[GuillaumeFavelier]

Update after ec0d265:

[larsoner]

if you set the figsize and dpi and such it will even be the right size! Not 100% sure how you get it to show up below the brain plot, probably has to do with when each is instantiated which seems like it could be a pain to manage

[GuillaumeFavelier]

I don't know where to get correct values for dpi. We were using Qt for that before

[larsoner]

I would try just setting it to 100. I don't know if it will matter. If it does we can figure out the right thing to do. One reason you need it is because there figsize is in inches, so you need to know the DPI to get from the pixel width to the figsize.

[christianhacker]

It's beautiful...

I'll test it out when I have more time, but this looks very good. Thank you for your hard work.

[larsoner]

For shortcuts I'm not sure where to look. I'd see how matplotlib handles it.

For toolbar I'd also look at how matplotlib makes theirs and use the same code

[GuillaumeFavelier]

Update after c254d79:

The matplotlib figure events are now connected correctly.

Supported shortcuts:

• i toggle slider visibility
• s apply auto scaling
• r restore scaling
• c clear glyphs

Not supported (yet):

• play (this one was based on QThread, I need a workaround)
• ? help (just does not work for me)
• ctrl+shift+s save movie (based on modifiers, I need to find a solution too)

[agramfort]

please update what's new page and then +1 for MRG when CIs are green

[agramfort]

+1 to merge when green !

[agramfort]

ok to merge for you @larsoner ?

[larsoner]

LGTM +1 for merge once CIs are happy

[agramfort]

🍻 🍾 🎉 !!!

@GuillaumeFavelier can you tweet a screenshot movie of this?

[GuillaumeFavelier]

Unfortunately, I'm not on Twitter 😅

I can still prepare a video though 👍

[GuillaumeFavelier]

Here is a simple MP4 demo :

output.mp4

[drammock]

Unfortunately, I'm not on Twitter sweat_smile

I think you mean fortunately I'm not on twitter :)

[agramfort]

that will do it thanks !

…