Create framework for explaining how a feature is calculated

[JimHokanson]

Currently many of the features that are extracted from a worm video are not easily understood. The aim is to create a framework by which a feature can be explained for a particular video with the goal of helping to give intuition to how a feature is calculated and what it means.

We are currently building support into the feature processing pipeline for a 'feature processing options' class instance. Within that instance should be placed an attribute or class that specifies what, if anything, should be explained in more detail.

The explanations themselves should most likely be branch points (if statements) within the code itself (see pseudocode below). Creation of examples outside of the code base would likely require duplication of a significant amount of code. At the same time ideally the code is minimally intrusive in terms of added computational complexity and run time.

To get the ball rolling, I think it is wise to focus on implementing code for a single feature. To start I'd like that feature to be locomotion.velocity.

One way to think of this project is as a visualization of the code itself, perhaps something that would be applicable for a methods section of a paper. That being said, some of this has already been done in a static sense (TODO: Provide links to nature methods material and Ev's thesis). Other things, as far as I know, have only been described with words, not pictures.

The actual output should be a mix of pictures and text, the form of which I am leaving unspecified. Perhaps output to a pdf or html?

To reiterate, the aim of this issue is to create framework by which others can add on more feature explanations. To do this a single issue, locomotion.velocity, should be 'explained'.

A lot of the code should probably exist inside explanation modules, with only a few calls inside the feature processing code.

Rough pseudocode:

explain_feature = explain.check(options,'locomotion_velocity')

start some processing ...

if explain_feature
explain_this

more processing ...

if explain_feature
explain that

[clinzy]

I am not familiar with Ev's thesis but I have reviewed some of the other material as follows.

I have these links to papers I have read so far related to worm movement and features:

-Nature Methods Behavioral Phenotypes Paper: http://www.ncbi.nlm.nih.gov/pmc/articles/PMC3962822/
-Eigenworms Paper: http://www.ploscompbiol.org/article/info%3Adoi%2F10.1371%2Fjournal.pcbi.1000028

I have also highlighted related articles from the sources list:

-https://docs.google.com/document/d/1wR68qwyEBXX73muhgAUeAjDyk8mXsaKrhsLaM6OK_-U/edit?usp=sharing

[JimHokanson]

@clinzy

I'll send Ev's thesis to you.

At this point I wouldn't focus too much on the papers other than the Nature Methods paper and Ev's thesis. Those combined with the code should be sufficient for now.

Let me know if you have any questions.

[MichaelCurrie]

FYI, we've transcribed Ev's supplemental notes describing how a feature is calculated here:

https://github.com/openworm/movement_validation/blob/master/documentation/Yemini%20Supplemental%20Data/ReadMe.md

Click through to the Morphology, Locomotion, etc. features.

Obviously this is just a plain-English description of how the feature is calculated, not nearly as detailed as what Jim is suggesting be done, but this might help as a starting point.

[JimHokanson]

A while back I showed an example summary of the range feature. It is a start but it is missing a bit more markup, notably how the worm centroid was calculated and how the range value is calculated from that value and the average of all centroids.

[MichaelCurrie]

Back in December @clbarnes pointed out the IPython notebook might be good for this demo code.

Also, @JimHokanson fleshed out his idea for this in his "plans" document so I thought it might be good to include here:

"Build in demo code for features to help explain how they are derived. Some of the features are a bit out there and would help from having step by step plots of how they are derived.

• add debug flag support in config (like a “--verbose” flag)
• insert “debug_verbose” calls after each processing step
• manually construct a summary document that shows the code being executed for each feature
• the final output of this could be a PDF, or just a series of plot windows"

[MichaelCurrie]

Here's an example of how iPython notebook can be used to make a package flex its muscles so to speak. In our case we'd be using it to show how a feature is calculated, but the principle is the same:

http://nbviewer.ipython.org/github/alimanfoo/petl/blob/master/notes/case_study_1.ipynb

[clinzy]

I agree that using IPython Notebook is a good approach. I am not familiar with debug_verbose but I did find information about using logging and argparse to enable --debug/--verbose flag support. I sent you a meeting request so we can discuss and review the code.

[MichaelCurrie]

Heard some good news from @clinzy about progress with this issue... here's a hint of what he's worked on, for path.range, we'll hear more about it soon: http://nbviewer.ipython.org/github/openworm/movement_validation/blob/feature_explaination/examples/PathFeatureExplaination.ipynb

[clinzy]

I have updated the code in the feature_explanation branch to perform some house keeping and also add important changes. First the code now includes a variable, defaulted to an empty list, to carry names of features to be explained from the command line down to the relevant code that produces the plots and text for the explanation. Second I have moved this code out of the init method to a separate explain method. Third I have changed the code to use the inspect module to print out the relevant source instead of retyping. Hopefully this will aid some in maintainability. I still only have the path.range feature implemented but will work on the others.