Skip to content
This repository has been archived by the owner on May 14, 2021. It is now read-only.

Ipysurfer #31

Merged
merged 17 commits into from Aug 12, 2018
Merged

Ipysurfer #31

merged 17 commits into from Aug 12, 2018

Conversation

OlehKSS
Copy link
Collaborator

@OlehKSS OlehKSS commented Jul 19, 2018

API changes to be consistent with PySurfer.

@OlehKSS
Copy link
Collaborator Author

OlehKSS commented Jul 19, 2018

I have started to work on API changes. I have noticed that in PySurfer there is a different dictionary that represents mesh orientation and for me it is not obvious how to go from that representation to MNE-like with azimuth and elevation dictionary. Which one should I stick with? How to go from PySurfer angles toward azimuth, elevation representation?

Another question is what should I do with MNE functions, e. g. _limits_to_control_points, that I used previously, should I create a copy of those in GSOC project?

@larsoner
Copy link
Member

Which one should I stick with? How to go from PySurfer angles toward azimuth, elevation representation?

The important thing is that each entry gives the same view of the brain in both places, so lat should look from the side (from the left or right depending on hemi). To get what this angle dict (which is for Mayavi) means in IPyvolume I don't know, but you can figure it out using trial and error if you really need to (i.e., for each view, do it in PySurfer, then figure out what you need to do in IPyvolume to make the same view).

@OlehKSS OlehKSS added this to To do in GSOC-2018 via automation Jul 24, 2018
@choldgraf
Copy link
Collaborator

@OlehKSS let us know when you'd like some 👀 on this!

@OlehKSS
Copy link
Collaborator Author

OlehKSS commented Jul 30, 2018

Thanks, I am going to publish version for the review soon

@OlehKSS
Copy link
Collaborator Author

OlehKSS commented Jul 31, 2018

Hi, I have moved almost everything from the mne_g3d to ipysurfer. For creating a plot and adding activation data ones should use snippet:

brain_plot = Brain(subject, hemi, surface, size=size,
                       subjects_dir=subjects_dir, title=title,
                       foreground=foreground)
brain_plot.add_data(data, min=ctrl_pts[0], mid=ctrl_pts[1],
                                hemi=h, max=ctrl_pts[2], center=center,
                                colormap=lim_cmap, alpha=alpha)
brain_plot.show()

You can see it in stc.plot_source_estimates sample function. How should time viewer should be added?
Should it be a part of Brain class?

@larsoner
Copy link
Member

larsoner commented Aug 1, 2018

No it should be separate:

https://github.com/nipy/PySurfer/blob/master/surfer/viz.py#L3436

Sorry it's not in the API ref, I'm working on adding it but it's a pain.

@OlehKSS
Copy link
Collaborator Author

OlehKSS commented Aug 1, 2018

Ok, thanks. What about color bar visualization, should that function be a part of Brain class or should it go to the TimeViewer as well?

@larsoner
Copy link
Member

larsoner commented Aug 1, 2018

colorbar control should be in Brain

@OlehKSS
Copy link
Collaborator Author

OlehKSS commented Aug 2, 2018

Hello, I have finished major changes to the API, now it is ipysurfer instead of mne_g3d. The plotting capabilities are the same as they were. You can see usage sample in this file. Could take a look at the proposed changes?

@OlehKSS
Copy link
Collaborator Author

OlehKSS commented Aug 3, 2018

@larsoner @choldgraf Regarding the fact that in a week and a half I will need to submit the final version of my work, what do you think are the most important things to do, besides this pull request and ipyvolume bug #156? As far as I understand, to meet GSOC requirements I should improve read.md file, so it will contain project description, use examples, maybe some screenshots and share the link to the repository. What else should I do to prepare my project for the final evaluation/submission?

@larsoner
Copy link
Member

larsoner commented Aug 4, 2018

I am currently traveling, @choldgraf do you have time to look?



def _calculate_cmap(lim_cmap, alpha, ctrl_pts, scale_pts):
def _calculate_lut(lim_cmap, alpha, min, mid, max, center=None):
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

folders probably don't need to have a _ before them, you can just call it utils

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ok

@choldgraf
Copy link
Collaborator

I'll take a look at it today!

@OlehKSS
Copy link
Collaborator Author

OlehKSS commented Aug 6, 2018

Hi, has anyone manage to take a look at this pull request?

@choldgraf
Copy link
Collaborator

I'll look at it today - haven't had much time since I've been busy prepping for my wedding in 2 weeks :-O

Color map control points.
scale_pts : tuple(float)
Data scale control points.
min : float
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

probably best not to user "min" and "max" since these would overwrite core python functions. How about c_min and c_max? or vmin and vmax to be consistent with matplaotlib?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes, that would be better

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I will change the names to fmin and fmax, since these are the ones used in MNE.

def _fast_cross_3d(x, y):
"""Compute cross product between list of 3D vectors.

Much faster than np.cross() when the number of cross products
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Are these copied over from MNE or custom for this repo?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have used PySurfer source code for these. As far as I understand, ipysurfer should be a stand-alone package, so I was trying not to depend on PySurfer or create a circular dependency with MNE.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

in general I'm fine with this package depending on pysurfer and using some of its utility functions, unless that would make "mayavi" a dependency of ipysurfer. @larsoner do you know if that's a reason not to depend on pysurfer?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I was avoiding dependency on PySurfer due to possible mayavi involvement.

floating point time values to strings, or None for no label). The
default is ``time=%0.2f ms``.
smoothing_steps : int
The amount of smoothing
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

could we make this more descriptive? "amount of smoothing" is a bit vague - what exactly does it correspond to? Alternatively if this is a copy of the docstring from the MNE version, we can leave it

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, it's a copy from MNE function. I will leave like that, ok?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yep - and if any of my other comments are equally relevant to the MNE docstrings, I don't think they need to be fixed here

ipysurfer/stc.py Outdated
smoothing_steps : int
The amount of smoothing
transparent : bool | None
If True, use a linear transparency between fmin and fmid.
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

what do "fmin" and "fmid" correspond to?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These are two control points of a colormap. I will make this statement more clear.

ipysurfer/stc.py Outdated
Alpha value to apply globally to the overlay. Has no effect with mpl
backend.
time_viewer : bool
Display ipybolume time slider.
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

*ipyvolume

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks

ipysurfer/stc.py Outdated
figure : ipyvolume.Figure | list | int | None
If None, a new figure will be created. If multiple views or a
split view is requested, this must be a list of the appropriate
length. If int is provided it will be used to identify the Mayavi
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

*ipyvolume figure, not mayavi figure, presumably

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yep

from .surface import Surface


class Brain:
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As far as I see, that depends on Python version. In Python 3 I can write class Brain and it implicitly inherit from object. On the other hand, I will need to write class Brain(object) for Python 2. I was using Python 3 for this repo, maybe it will be better to make explicit inheritance from object, in case we would want this module to work with Python 2. What do you think?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ah I see what you mean - I think we should try and keep things relatively 2.X compatible for now (though it seems like packages are starting to drop 2.X compatibility, I think MNE will lag behind this by a little bit...besides, I'm pretty sure "explicit is better than implicit" is the second line of the Zen of Python :-) )

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ok

Color of the foreground (will be used for colorbars and text).
None (default) will use black or white depending on the value
of ``background``.
figure : list of mayavi.core.scene.Scene | None | int
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ipyvolume, not mayavi

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yep

Attributes
----------
geo : dict
Cortex surface objects.
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

clarify if these are ipyvolume objects or something unique to ipysurfer

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ok

u"""Display widget."""
ipv.show()

def update_lut(self, min=None, mid=None, max=None):
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

what does "lut" stand for? if the method updates teh colormap, let's just call it "update_colors" or something

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As I know, "lut" stands for "Look-up table". I have borrowed naming from PySurfer. I will change it to something more obvious.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

if this is just copy-pasted from ipysurfer then I think it's fine...



class Surface:
"""Container for a surface.
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Add a more verbose description of the class underneath the one-liner at the top of the docstring so people can understand in more detail what it does. (same goes for the "Brain" class)

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ok

@choldgraf
Copy link
Collaborator

choldgraf commented Aug 7, 2018

Ah I just realized I was leaving all of those comments as "single comments" instead of as a review, so they've already showed up. In general the code looks good, though I'm not really able to debug guts of the STC stuff since I haven't done that work before.

I'm still trying to get the examples to run on my machine but it's taken a while to set this up. It sounds like this requires the "master" branch of ipyvolume, yeah? Once I get them running I can give some more comments on the examples themselves, and maybe the API in general.

@choldgraf
Copy link
Collaborator

finally got it working! the visualizations look really slick :-) nicely done!

a quick thought: the "time bar" works relatively well, but the 3D visualization seems to lag behind changes in time by a few seconds (e.g. if I hit a different point on the time bar, then the plot takes a few seconds to update). Is this just because I'm on a not-great computer?

Here's an example:

2018-08-06_23-48-39

@OlehKSS
Copy link
Collaborator Author

OlehKSS commented Aug 7, 2018

For me it was also lagging. I suppose it is because I do some calculations on the Python side, not on JavaScript and communication between the two happens through the server. As a result the frame rate is quite low.

@OlehKSS
Copy link
Collaborator Author

OlehKSS commented Aug 7, 2018

I have updated the pull request according to the code review.

@larsoner
Copy link
Member

larsoner commented Aug 7, 2018

I do not have time to look closely right now so I trust @choldgraf's judgment :)

I do agree that it's better to avoid dependencies and have this be standalone for now. Longer term we might try to get PySurfer to have multiple backends, which this could be one of. However, right now PySurfer requires Mayavi so it's worth copy-pasting code a bit to avoid needing to satisfy that dependency.

@choldgraf
Copy link
Collaborator

choldgraf commented Aug 7, 2018

Nice! I just pulled changes and took another look. A few more points for you @OlehKSS :-)

  • slow animation updating

    • I wonder if it's possible for tihs to happen on the javascript side so we're not passing big-ish arrays back and forth each time somebody updates the viz. What do you think Oleh? At the least, we can open a TODO issue about it.
  • Colors

    • Alpha blending looks good 👍

    • Colorbar updating is nice too!

    • Document that "auto" is an option in the docstring for 'colormap'

    • On that note, the colorbar doesn't seem to match the fmin/fmax widget:

      image

      (I'd expect the fmin/fmax to be symmetric around zero, but it isn't here.

    • To that extent it'd be useful to document how to build this particular colorbar on your own, so people can have it without using "auto".

  • Views and updating

    • One thing I noticed is that the "views" don't seem to be working the way I'd expect. Here's an image of the default output for the plot_source_estimates function:

      image

      It stays this way for maybe 5-6 seconds, and then all of the views switch to their proper "lateral / medial" view at once. Any idea why this would happen?

  • Documentation

    • We should note in the docstrings where features are not yet enabled (e.g. in the kwarg for "cortex=")
    • Can you expand on the ipysurfer.ipynb demo a little bit more? I think this one will be useful for many people who don't directly use the plot_stc functions of MNE. Think about each of the major features you like in ipysurfer, and show a step-by-step example to use and explain each one (AKA, don't just make a single plot with all of the features in it, show them off one by one). This example / demo can also be material for your blog post so we can kill 2 birds with one stone.
    • The plot_stc.ipynb demo looks pretty good! The main thing I'd address is to give a bit of context for what exactly is happening so that users can understand.

Let me know if you have any questions or thoughts - really nice job so far, now that most of the features are in place lets make sure they're polished and documented so people can appreciate them!

@OlehKSS
Copy link
Collaborator Author

OlehKSS commented Aug 8, 2018

Hi, @choldgraf,

  • about animation, previously we had all data passed to the JavaScript, and the animation was quite quick, but according to @larsoner suggestions that up-sized stc data can occupy a lot of memory I have changed it to calculate required data upon request. I have added an issue on optimization, see Animations speed-up #33.

  • the problem with fmin/fmid/fmax in divergent color map is due to the way I get 'mne' color map, I obtain it from mne._limits_to_control_points function. After I change control points I can't change the color-map returned by the mne function, since by tweaking fmin/fmid/fmax values I change only alpha blending of the colors, not the colors itself. As a result, color map keeps gray area as it was defined by mne.

  • the problem with views is because of a bug in ipyvolume, see. I have partly solved it, but the problem that you descibed about changing initial camera position is still present. I am not sure whether I will be able to solve it until the end of GSOC. I have added an issue for this, see Display of different views #34.

  • do you have any suggestions on how to do my final evaluation report? There are several ways described in the provided recomendations. I thought to extend README.md with project description. Do you think I should do a blog post also? E.g. I will update README.md file with information that will be more useful for other people that would like to use this project and I will create a blog post where I will use information from README.md and I will elaborate more on what I have done during GSOC.

@OlehKSS
Copy link
Collaborator Author

OlehKSS commented Aug 8, 2018

I have updated documentation and usage samples according to the latest code review suggestions.

Copy link
Collaborator

@choldgraf choldgraf left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good - some small nitpicks in there (and now we need a rebase to make the merge possible) but I think in general we should merge this and iterate in smaller pieces later on. The examples work for me and this is a complex enough PR that I think it's better we get it through. @larsoner what do you think?

ipysurfer/stc.py Outdated
@@ -28,9 +28,11 @@ def plot_source_estimates(stc, subject=None, surface='inflated', hemi='lh',
hemi : str, 'lh' | 'rh' | 'split' | 'both'
The hemisphere to display.
colormap : str | np.ndarray of float, shape(n_colors, 3 | 4)
Name of colormap to use or a custom look up table. If array, must
Only str type is supported.
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm a bit confused here - so you cannot input an array? If so, then we should just remove the np.ndarray ...etc from the first line of this argument. Or, clarify by saying "only str type is currently supported" to imply that it'll be supported in the future

@@ -18,12 +18,12 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"Initialize a basic visualization session.\n"
"Initialize a basic visualization sessiona and plot a brain mesh. For this purpose we need to provide subject information, e.g. `subject_id`, `subjects_dir`; which type of freesurfer surface mesh we would like to plot, e.g. 'pialed' or 'inflated'; and which hemisphere to plot or whether to plot both."
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

small typo!

ipysurfer/stc.py Outdated
@@ -138,14 +140,14 @@ def plot_source_estimates(stc, subject=None, surface='inflated', hemi='lh',
subject = _check_subject(stc.subject, subject, True)

if not isinstance(colormap, str):
raise NotImplementedError('Support for "colomap" of a type other' +
' than str is not yet implemented.')
raise ValueError('Support for "colomap" of a type other' +
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

small typo!

@choldgraf
Copy link
Collaborator

@OlehKSS regarding the final report, I'd recommend a blog post that covers the technical things that you had to do in order to make this happen. What were the core pieces of tech that made this package possible? What were some challenges that you had to overcome, and the technical decisions you made to overcome them? I think it'd be interesting and useful for you to tell a short narrative about your experience working on this problem! In addition we can link to interactive examples via Binder and make sure that the readme looks good and everything is documented.

@OlehKSS
Copy link
Collaborator Author

OlehKSS commented Aug 9, 2018

@choldgraf I have changed binder link in README.MD to lead to the ipyserfer.ipynb file instead of brain-mesh.ipynb, since the latter one is outdated and I have deleted it. Do I need to add the ipyserfer.ipynb file to binder or it will figure out which file to show automatically?

@OlehKSS OlehKSS merged commit 9ae3cea into mne-tools:master Aug 12, 2018
GSOC-2018 automation moved this from To do to Done Aug 12, 2018
This was referenced Aug 12, 2018
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
GSOC-2018
  
Done
Development

Successfully merging this pull request may close these issues.

None yet

3 participants