Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

MRG, ENH: ECoG plotting on pial #8190

Merged
merged 5 commits into from Oct 16, 2020
Merged

MRG, ENH: ECoG plotting on pial #8190

merged 5 commits into from Oct 16, 2020

Conversation

larsoner
Copy link
Member

@larsoner larsoner commented Sep 1, 2020

Closes #7787.

I don't work with ECoG data so I don't know how sensible this is. @adam2392 can you see if this is something that ECoG people would be interested in? Here is what the tutorial image looks like on my machine:

Screenshot from 2020-09-01 13-22-15

Todo:

  • Verify it's actually useful
  • Pick a better name for the function
  • Add tests

@larsoner
Copy link
Member Author

larsoner commented Sep 1, 2020

I think some combination of this and sensor-plotting-on-brain would probably be the best. But this at least is maybe an incremental step

@adam2392
Copy link
Member

adam2392 commented Sep 1, 2020

Wow this is awesome! you are a magician @larsoner.

I think for me I haven't seen these kinds of mne-visualizations yet except for MEG, so the only issue I have is interpreting the cool viz. So correct me where I'm wrong. You are visualizing the gamma band activity mapped to the closest parietal lobe on both hemispheres (in this case it's the left hemisphere), at 0.680 seconds. There is a small window where you are smoothing with a factor of 10.

I might be asking a stupid question, but what is being shown in the bottom time series plot? not sure what GFP stands for and the other things in the legend.

@larsoner
Copy link
Member Author

larsoner commented Sep 1, 2020

I think for me I haven't seen these kinds of mne-visualizations yet except for MEG,

The real question to me is, is the method used for projecting the data to the brain sensible for ECoG? It's pretty basic, doesn't use forward/inverse at all... But I imagine this is what people mentally try to do when they see electrodes on the brain anyway. Hence why I think this plus an actual sensor overlay would be potentially very useful.

You are visualizing the gamma band activity mapped to the closest parietal lobe

The Notes say this, does it clarify it for you?

This function projects the ECoG sensors to the pial surface, then the
activation at each pial surface vertex is given by the sum of the
activation from each sensor. Activation is weighted by the fractional
``distance`` from the sensor, such that the vertex where the sensor
projects gets weight 1 and a sensor at ``distance`` meters away (or
larger) gets weight 0.

There is a small window where you are smoothing with a factor of 10.

Actually the smoothing here has no effect. It's spatial upsampling which is not used for these data.

but what is being shown in the bottom time series plot? not sure what GFP stands for and the other things in the legend.

It's the time course of the vertex shown by the ball, in this case the time course of the gamma activation. GFP is global field power. You can try running the example locally to play with the options and clicking around, playing with spacebar, etc.

@adam2392
Copy link
Member

adam2392 commented Sep 1, 2020

Ah okay makes a bit more sense! So just to clarify is this the projected activity of all the channels on the brain? The time course on the bottom shows that for just one region (i.e. the blue ball plotted onto the brain). If say someone wants to only visualize the effect from a subset of channels, then one simply needs to "pick_channels" right?

Do I think this is useful?

So the heatmap on the pial surface is a weighted (by distance) linear combination of all the channel's gamma activity. I think that is pretty useful because there are many recordings that are ECoG arrays on the brain cortex, and many publications essentially produce (crappier) versions of what you have here. The cool thing I can see is that any "metric" or feature can be visualized as such following the example. I would definitely use it.

The future of iEEG though with epilepsy surgery is moving towards SEEG (so depth electrodes), which obviously then become way harder to visualize. But one step at a time :p. This looks so cool!

@larsoner
Copy link
Member Author

larsoner commented Sep 1, 2020

So just to clarify is this the projected activity of all the channels on the brain?

Each colorful point on the brain is the result of summing together channels that are (or really, the version of them projected to the nearest point on the pial surface is) within distance of that point, weighted as 1. if they have 0 distance and 0. if they have distance (or greater) distance. So any colorful point is the weighted sum of 1 or more electrodes, but generally it's not going to be more than probably a couple of channels.

The time course on the bottom shows that for just one region (i.e. the blue ball plotted onto the brain). If say someone wants to only visualize the effect from a subset of channels, then one simply needs to "pick_channels" right?

Yes indeed, do this before stc.plot and it should plot the subset. Or just set your distance less than your minimum electrode spacing, then you'll see the individual points on the brain, and can click on them to make their time-courses be plotted, etc.

The future of iEEG though with epilepsy surgery is moving towards SEEG (so depth electrodes), which obviously then become way harder to visualize. But one step at a time :p. This looks so cool!

Hopefully not too bad -- we have translucent volumetric rendering already:

https://mne.tools/dev/auto_tutorials/source-modeling/plot_beamformer_lcmv.html#volumetric-rendering-3d

sEEG should be fairly easy by comparison :) We can for example, when we add the sensor plots, make sEEG electrodes translucent spheres. And/or we figure out some sensible way to interpolate/map the sEEG data into a volumetric space, then the viz is already done actually. Again I think a combination would probably be best...

I would definitely use it.

Okay let's wait to see if @agramfort thinks it makes sense, too -- if so, I'll add tests and we can pick a name then we should be good to go.

@larsoner
Copy link
Member Author

larsoner commented Sep 1, 2020

Actually also @choldgraf if you have an opinion on this PR/viz method it would be great!

@adam2392
Copy link
Member

adam2392 commented Sep 2, 2020

  • Pick a better name for the function

For naming, I wonder if it should be explicit that this is for the pial surface of the brain? Also... I might be asking another stupid question, but what is stc stand for? I couldn't find anything from using mne documentation search. It seems to be a file type that contains source reconstructions?

@larsoner larsoner changed the title WIP: ECoG plotting on pial MRG, ENH: ECoG plotting on pial Sep 2, 2020
@larsoner larsoner added this to the 0.21 milestone Sep 2, 2020
@larsoner
Copy link
Member Author

larsoner commented Sep 2, 2020

Tests added, ready for review/merge from my end

@larsoner
Copy link
Member Author

larsoner commented Sep 2, 2020

For naming, I wonder if it should be explicit that this is for the pial surface of the brain?

In theory at some point we might want to make the surface to project to optional, so I'd rather leave it. I actually can live with stc_near_sensors I guess.

what is stc stand for? I couldn't find anything from using mne documentation search

Argh, our search needs to work a bit better! See:

https://mne.tools/stable/glossary.html#term-source-estimates-abbr.-stc

@adam2392
Copy link
Member

adam2392 commented Sep 2, 2020

For naming, I wonder if it should be explicit that this is for the pial surface of the brain?

In theory at some point we might want to make the surface to project to optional, so I'd rather leave it. I actually can live with stc_near_sensors I guess.

What about something to inform ECoG vs SEEG? Because to allude to what you said about SEEG: would we have an analagous function for doing the volumetric rendering of SEEG?

Hopefully not too bad -- we have translucent volumetric rendering already:

https://mne.tools/dev/auto_tutorials/source-modeling/plot_beamformer_lcmv.html#volumetric-rendering-3d

sEEG should be fairly easy by comparison :) We can for example, when we add the sensor plots, make sEEG electrodes translucent spheres. And/or we figure out some sensible way to interpolate/map the sEEG data into a volumetric space, then the viz is already done actually. Again I think a combination would probably be best...

Whoa, I wasn't aware of this! This looks pretty awesome too... I'm assuming there should be some way of getting the data structures to work w/ arbitrary SEEG feature signals (e.g. either the raw LFP, or some say spectral power, etc.), so that it can plot things in this volumetric rendering? If not, I can open another issue to discuss? Sorry if it's obvious and can :p. I'm just not as familiar w/ the data structures outside Raw and Epochs because the other things always seem to be in the realm of source loc.

If it can do so, perhaps I'll try it on a future patient and see if I run into any troubles.

@larsoner
Copy link
Member Author

larsoner commented Sep 2, 2020

SEEG: would we have an analagous function for doing the volumetric rendering of SEEG?

Yeah we should be able to project sEEG electrode activation into a volume without too much work. Probably about the same number of lines as here.

I'm assuming there should be some way of getting the data structures to work w/ arbitrary SEEG feature signals (e.g. either the raw LFP, or some say spectral power, etc.), so that it can plot things in this volumetric rendering

Yep, just get those signals then put them into an EvokedArray (if they aren't already in an Evoked) with a corresponding info as done in the example here.

@choldgraf
Copy link
Contributor

My 2 cents is that this is a super common visualization in ECoG / sEEG, although it has sort-of dubious physiological truth (most folks just assume there's a spherical activation around each electrode, some other kind of heuristic like that, rather than actual physiological modeling etc). However since this is so common I think it'd be a good feature for the ECoG/sEEG community!

@larsoner
Copy link
Member Author

larsoner commented Sep 2, 2020

In that case this is ready for review/merge @agramfort . Failure is just 3.8 pre segfault that's happening on master

Copy link
Member

@agramfort agramfort left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am personally uneasy advocating a projection method for ECoG data without references to literature doing it the way we do. Physics is here ignored and I am sure there is a more standard way of doing this. If not we need to write a paper ;)

I could already find some work:

https://hal.inria.fr/hal-01278377v1

although this is more ambitious.

mne/source_estimate.py Outdated Show resolved Hide resolved
@larsoner
Copy link
Member Author

larsoner commented Sep 3, 2020

this is a super common visualization in ECoG / sEEG

@choldgraf any reference papers for "each electrode just projects to a sphere of radius R"? @agramfort won't accept this viz without seeing it in use in the community

@adam2392
Copy link
Member

adam2392 commented Sep 3, 2020

this is a super common visualization in ECoG / sEEG

@choldgraf any reference papers for "each electrode just projects to a sphere of radius R"? @agramfort won't accept this viz without seeing it in use in the community

See figures:
https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5580037/

See EcoG:
http://www.fieldtriptoolbox.org/tutorial/human_ecog/

@choldgraf
Copy link
Contributor

The field trip ecog page has some visualizations like this I think, maybe they have a reference http://www.fieldtriptoolbox.org/tutorial/human_ecog/

@agramfort
Copy link
Member

agramfort commented Sep 3, 2020 via email

@choldgraf
Copy link
Contributor

@agramfort you are parisian we know you are never truly happy

@choldgraf
Copy link
Contributor

(sorry, I couldn't resist, I haven't had as much chance to troll you these days)

@agramfort
Copy link
Member

agramfort commented Sep 3, 2020 via email

@larsoner larsoner modified the milestones: 0.21, 0.22 Sep 3, 2020
@larsoner
Copy link
Member Author

larsoner commented Sep 3, 2020

Let's push this to 0.22 and hopefully we can tackle ECoG + sEEG in that release cycle

@jeschj01
Copy link

@larsoner This addition looks very useful for my current project. We are trying to show heat maps based on observed change from electrical stimulation. Is 'stc_near_sensors' is not currently available?

@larsoner
Copy link
Member Author

@jeschj01 it's not in the master branch but it's part of my branch shown at the top of this page, larsoner:ecog. If you set up a dev/editable install you can try it out. Or you can overwrite your existing 0.20 install with something like:

pip install --upgrade --no-deps https://github.com/larsoner/mne-python/archive/ecog.zip

@adam2392
Copy link
Member

@larsoner jw if you were planning on also adding the SEEG capability sometime (what you linked for source viz looks pretty neat)? No rush of course, I know you're a busy man :).

What are the steps needed to get it working on the volumetric viz? Just wondering to see if it's in my ability to even come close to implementing. I'm completely lost in mne-python's viz functions.

@larsoner
Copy link
Member Author

For sEEG, if we improve stc_near_sensors to allow returning a VolSourceEstimate object then nothing new needs to be done at the viz end. In other words, we "just" need to come up with a suitable way to interpolate the sEEG sensor activity into a volume, and once that's done, stc.plot and stc.plot_3d should just work.

Let me rebase this then we can merge it, and then @adam2392 it would be great if you could make a PR to allow interpolating into a volume!

@larsoner
Copy link
Member Author

@agramfort feel free to merge if you're happy

@agramfort
Copy link
Member

@larsoner it needs a what's new entry then +1 for MRG

@larsoner larsoner merged commit f6b58d4 into mne-tools:master Oct 16, 2020
@larsoner larsoner deleted the ecog branch October 16, 2020 18:41
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Animate ECoG activity on top of the FreeSurfer pial surface of the brain
5 participants