New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Create unit test that verifies correct I/V curve for ca_boyle NML2 channel #30

Open
slarson opened this Issue Jan 18, 2015 · 13 comments

Comments

Projects
None yet
5 participants
@slarson
Member

slarson commented Jan 18, 2015

We currently have a process to run the iv_analyse.py script on the output of the analyse_ca_boyle.sh script. This produces a figure.

We'd like to augment this process with an automated test that compares the output of iv_analyse with Figure 3A from the paper.

A good candidate to be done with a sciunit test @rgerkin , although anyone who can do this even in a simpler way would make a contribution.

Related to #20. @VahidGh and @rayner this would be the logical next step on the work you have both been doing related to I/V curves.

The unit test should then be instrumented to run on Travis CI in this repo.

@slarson slarson added the enhancement label Jan 18, 2015

@VahidGh

This comment has been minimized.

Member

VahidGh commented Jan 21, 2015

@slarson, Great, I was looking at the sciunit+neuronunit documentation and samples, today.
This issue also could be a good starting point for instrumenting the ion channel modeling pipeline with both sciunit+Travis CI for the verification & validation step.
I don't know if validating single ion channel via neuronunit needs a test design with training phase (as suggested in this paper) or there is some predesignated test available in existing libraries!

@rgerkin

This comment has been minimized.

Member

rgerkin commented Jan 23, 2015

I've submitted some pull requests to get some of the required tools into a better position for integration. These allow the analyse*.sh scripts to be decomposed into python functions, including NML2ChannelAnalyse and jnml.

I've also begun work on a neuronunit module to make use of these tools to run all the tests from within python without any command line invocations.

Once that is in order I can create NeuronUnit tests that recapitulate the analysis parameters themselves. As for testing/training as @VahidGh suggested, I think it depends on whether we are looking to test how good our models are, or whether we are trying to improve them by free parameter fitting. For now I think it would be good to know how close we are with the pieces we currently have, and then we can invest effort into model fitting.

@rgerkin

This comment has been minimized.

Member

rgerkin commented Jan 29, 2015

Some of what I've worked on is here. I've updated neuronunit's jnml module (which will eventually be merged into pyNeuroML to do most of what we need to in Python, although I am still relying on jNeuroML over pyLEMS for model compatibility reasons that need to be worked out.

I can reproduce the I-V curve, so in principle I am ready to devise the validation test itself, but I wanted to point out some weird behavior in the simulation in the first few time steps. See cell Out[240] near the bottom of the notebook linked above. There is some sort of oscillation of the current in the first few steps, and I'm wondering if that is causing problems with the measurement of the peak current, and whether those oscillations should even be there. Maybe it's a time step size issue.

@slarson

This comment has been minimized.

Member

slarson commented Feb 2, 2015

Pasting the image in question here for simplicity:

download

Hmm yeah that oscillation is strange. @rayner we haven't seen this right? Does it change if you change the time step?

@slarson

This comment has been minimized.

Member

slarson commented Feb 2, 2015

@VahidGh On this issue, can you remind what that digitiser software is that you found for transforming the raster I/V plot curves from papers into numbers? Can you post an example of its output? Apologies if you have already posted this somewhere and I've lost track of it.

@VahidGh

This comment has been minimized.

Member

VahidGh commented Feb 3, 2015

This is an online version, and this is a stand-alone one.
As an example, this is the digitized version for V=15 mv:
ica-v15
And this is the published Plotly version of the WebPlotDigitizer output.

@travs

This comment has been minimized.

Contributor

travs commented May 5, 2015

@rgerkin I'd like to talk about using SciUnit (and NeuronUnit) when making validation test templates for different parts of this project. Is there any chance we can have a chat about this? Thanks.

@rgerkin

This comment has been minimized.

Member

rgerkin commented May 6, 2015

@travs Yeah; chat, email, Hangouts?

@travs

This comment has been minimized.

Contributor

travs commented May 7, 2015

@rgerkin Just sent you an email.

@travs travs added this to the Validation milestone May 7, 2015

@slarson slarson added the in progress label Jun 13, 2015

@slarson slarson self-assigned this Jun 13, 2015

@slarson slarson removed the in progress label Jun 15, 2015

@travs

This comment has been minimized.

Contributor

travs commented Jun 21, 2015

This is worth looking back into again now that we have made progress in comparing simulated and observed I/V curves in ChannelWorm.

@rgerkin

This comment has been minimized.

Member

rgerkin commented Jul 28, 2015

@slarson Do you have the DOI for that paper in your first message? I want to see if we have the Figure 3A data in the ion_channel db in ChannelWorm.

@rgerkin

This comment has been minimized.

Member

rgerkin commented Jul 31, 2015

@slarson Do you know where that Fig. 3 came from in your first message? That's what I need to move forward. I thought maybe it came from this referenced in #20 but that's a different paper. I tried Google reverse image searching it but sadly that doesn't work yet for most journal figures (probably you screenshotted part of a pdf).

@VahidGh

This comment has been minimized.

Member

VahidGh commented Jul 31, 2015

@rgerkin, the fig. 3A is taken from Boyle & Cohen 2008 paper. Which originally comes from Fig. 2B of this study which I've addressed how to access to data in the ipython file.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment