Pycortex flattening

Flattening cortical surfaces with pycortex and blender

This is a guide to making flattened cortical surfaces using pycortex and blender.

Index

Section
Why flattened surfaces?
Why pycortex and blender
Why does this guide exist?
Step by step guide

Why would I want to make a flattened surface?

A lot of what we do with fMRI amounts to 'mapping' the brain. When we think of maps, we naturally think of something like a 2 dimensional map of the world, which is a convenient and intuitive way of representing spatial information. Despite what some people will tell you, the world isn't actually flat, but it is convenient to us to represent it this way, because we can we can see the whole thing without needing multiple viewpoints.

However, even though we are also in the cartography business, our traditional way of viewing fMRI data is quite drastically different from this way that we prefer to view maps.

Here, we see a typical representation of 3 dimensional volumetric data:

This representation is clearly limited by the fact that we can only see small portions of the available data without scrolling through the various slices. Moreover, because the brain consists of sulci and gyri, regions that are contiguous can appear as disconnected. This is therefore an inefficient way of representing most forms of cortical data. It's kind of like writing a paper on the layout of the world and plotting multiple different angles of a 3D globe, rather than just plotting the 2D map.

One solution to this, which is facilitated by freesurfer and related software, is to instead create a model of the cortical surface, using a triangular mesh representation - as shown in this figure:

Subsequently, the functional data can be sampled at a point normal to the each one of the vertices (which are the intersections of the triangles). This allows us to view our data on the surface of the cortex.

A problem still remains though, in that it is still hard to visualize information that is buried deep in sulci.

However, since a surface model now exists for the cortex, it is possible to visualize the cortex in any number of arbitrary configurations. For instance, the surface can be inflated, so that information buried in the sulci now becomes more visible.

There are still lingering problems, because ultimately, we still require multiple multiple views of the surface to see all of the data.

Finally then, what we can do is cut our surface and unfold it. Here, we have a way of viewing our data as a 2 dimensional map that is analogous to our 2D world map atlas and it probably the most efficient way of visualising cortical data. In a configuration like this, we can view a full dataset from one viewpoint. No more fiddling around with different slices and different viewing angles.

Why blender + pycortex?

The traditional way of making flattened cortical surfaces was to use tksurfer - an application bundled with freesurfer. However, this is now defunct for 64 bit systems and is in the process of being replaced.

Blender is a free, open-source 3D modeling suite that is optimised for 3D object manipulation. It is very powerful and quite user-friendly.

Pycortex is a python based library that (amongst other things) allows you to make very pleasing surface-based visualisations, including interactive webGL viewers. Rather than re-inventing the wheel, it integrates freesurfer into its workflow and uses blender to perform cortical surface flattening. Once flatmaps are made, it is trivial to use pycortex (in conjunction with inkscape) to define regions of interest by simply drawing onto the flatmap.

Why does this guide exist?

After a lot of searching, it doesn't seem as though a formal 'guide' yet exists for this part of the pycortex functionality. It took me a while to figure all this out through searching through multiple github issue threads and my own experimentation - so I thought I would put it all together here.

Step-by-Step guide

Provisos & Pre-requisites:

Before anything else, note that this is not a guide on how to use pycortex. I will be preparing more general information on pycortex elsewhere. To get acquainted with it, you should refer to their website and test out some of the demos in their example gallery

Secondly, I am working with monkey data in this example, which is why the brain I am using looks a bit strange. The same principles exist for human data.

Third, note that I am a bit of a noob when it comes to blender. There may well be more sophisticated and accurate ways of making the cuts if you are familiar with the various fancy selection tools. I am not. I just know that what I am going to describe works.

Here is what you need to get ready, before moving to step 1:

1. You must have installed pycortex.
2. You must have installed blender. Everything here is tested using version 2.79, so you should install this legacy version.
3. You must have freesurfer installed.
4. You must have a freesurfer subject whose surfaces have been fully 'recon-alled'
5. This subject should also exist in your pycortex directory, possibly via using this command.

Step 1: Prepare some data to guide cutting.

To guide our cutting, it makes sense to show some data to guide the position of our cuts. By default, surface curvature information will be available, letting us reference things against cortical folding patterns.

We may also want to use one of the many detailed parcellations that are available.

In my case, I want to plot retinotopy data - meaning that this serves as a good guide for where I might want to position my cuts. For instance, I can make an occipital cut that separates the upper and lower hemifields.

To prepare this data, whatever it may be, we need to leverage the 'Dataset' class. Essentially, we define the a set of vertex objects and then package them into the Dataset. Here I make a Dataset object formed of some retinotopy data (eccentricity and polar angle).

import cortex # Load libraries
import numpy as np

# Define pycortex subject
mysub='NMT'

# Create empty dataset
my_retinotopy=cortex.Dataset()

# Create vertex objects,
ang_vx=cortex.Vertex(data=dat[:,1],subject=mysub, vmin=np.nanmin(dat[:,1]) \
, vmax=np.nanmax(dat[:,1]),cmap='Retinotopy_RYBCR') # Polar angle
ecc_vx=cortex.Vertex(data=dat[:,4],subject=mysub, vmin=np.nanmin(dat[:,4]) \
, vmax=np.nanmax(dat[:,4]),cmap='nipy_spectral') # eccentricity

# Make into Dataset
my_retinotopy.views['eccentricity']=ecc_vx
my_retinotopy.views['angle']=ang_vx
my_retinotopy.description='retinotopy'  

Step 2: Run the command to begin cutting in blender.

The main command to run the flattening is:

cortex.segment.cut_surface(cx_subject='NMT',hemi='lh',name='retinotopy_flat' \
,fs_subject='NMT',data=my_retinotopy)

The first argument is the name of your subject in your pycortex directory.

The second is the hemisphere you wish to flatten.

The third is the name you wish to assign to this attempt at flattening.

You also specify the name of your freesurfer subject.

The final argument is the data we just prepared.

Step 3: Familiarise yourself with the blender environment.

1. When this function is called, a blender window will open. You should see your chosen hemisphere in the main window (see below). Note that the name of the file is consistent with the name you assigned to the cut_surface call.

2. You can navigate around by moving the mouse trackpad, which will rotate the camera. You can do this in combination with shift/ control to translate/ zoom the camera.

3. To toggle your data, we first need to change the 'viewport shading' parameter positioned at the bottom of the screen to 'texture'. When we do so, we see that curvature information is shown.

4. Now, we can cycle through different data by first clicking on the 'data' tab and then selecting from the list of available vertex color schemes.

Step 4: Other preparations

1. In the panel that runs along the bottom of the main window you will see that the default is object mode. Use the dropdown to change this to Edit mode

2. You will notice that the representation of the surface has changed so that all vertices appear in an orange color.

3. This means that blender appears to select all vertices by default. Therefore, we need to use select-deselect all in the bottom panel.

4. The consequence is that we no longer see that all of the hemisphere is highlighted.

5. To toggle between inflated and smoothwm views click this button:

Step 5: Make 5 relaxation cuts.

1. Now, our job is to make a series of cuts to the surface. These are all medial. We want to make 5 relaxation cuts, followed by a final cut to take out the medial wall. For the location of these cuts, you may want to follow this guide or a slighly more detailed guide that references things more precisely against a parcellation. Both of these create a flattened cortical patch that is optimised for displaying the results of visual experiments.

2. The way the cuts are made in blender is as follows: Firstly, right click on a start vertex. This should make that vertex appear as orange. We next right click on an end vertex whilst holding the control key. This will draw a path between the start and end point. You can do this as incrementally as you like, defining many end points via control+right click to an eventual destination.

3. We can modify the way in which this path is created via the various options in the bottom left panel.

4. We can then mark this as a 'seam' by pressing control+E and selecting the relevant option.

5. When we start to make the next cut, note that the previous line is now in red - indicating that it has been designated as a seam. These red lines are not very easy to see, so you may want to switch back to curvature data to make them appear more salient.

6. We continue this process until all relaxtion cuts have been made.

Step 6: Take out the medial wall.

1. We define a final cut to take out the medial wall. This is done slightly differently. To start, we use the same steps as defined before - control+ right clicking a path until we eventually form a loop that intersects with the other 5 relaxation seams that have been made.

We also need to make sure the first and last point of the selection are the same. Here though, we do not mark our loop as a seam!.

2. Next go to select - select loop inner region The filled selection will appear in orange.

3. You can then go to Transform - delete - only edges and faces. Selecting any other deletion option will make things go horribly wrong.

4. The medial wall now disappears.

5. Go to file-save

6. Exit blender.

Step 7: Back to python

1. You should now be prompted with a message to inform you that freesurfer will now perform flattening. Type yes. This will take a while, as indicated by the message.

2. Repeat this entire process from Step 2, but specifying 'rh' instead of 'lh' in the call to cut_surface to do the right hemisphere.

3. Now, we need to import the flat surfaces (which now exist in the freesurfer directory) into the pycortex directory.

cortex.freesurfer.import_flat('NMT','retinotopy_flat',['lh','rh'],'NMT')

The first argument is the freesurfer subject. The second is the name you assigned to the attempt at flattening in cut_surface. The third indicates we want to want to import both hemispheres. The last is the name of the pycortex subject.

Step 8: Test and repeat (if necessary)

1. With the flattened surfaces imported, we should now check them. To do this, use a call to cortex.webshow to show some data - making sure that you recache.

cortex.webshow(ecc_vx,recache=True)

2. Press the F key to go from the inflated to flattened view.

\

This result seems fairly pleasing for representing retinotopy data. We can see that foveal locations (dark purple) sit roughly at the horizontal meridian of the flatmap and progress vertically into more eccentric visual field locations (red) equidistant from this point - meaning that my occipital cut has been made at a point that splits the upper and lower hemifield. This is confirmed by the polar angle data.

\

3. If things have gone badly wrong then they will shortly reveal themselves. You usually get two hemispheres that look completely different - or it will simply look nothing like the example flatmaps throughout this document and in the linked guides.

4. You are unlikely to get this right the first time. Try again if necessary, making sure to give your new attempt a different name. Here are some things to check that may have gone wrong:

• Were all the 5 relaxation cuts marked as seams?
• Did the medial wall loop intersect with all of the relaxation cuts?
• Did you delete only edges and faces ?