Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Create unit test that verifies correct I/V curve for ca_boyle NML2 channel #30

Open
slarson opened this issue Jan 18, 2015 · 13 comments
Open

Comments

@slarson
Copy link
Member

@slarson slarson commented Jan 18, 2015

We currently have a process to run the iv_analyse.py script on the output of the analyse_ca_boyle.sh script. This produces a figure.

We'd like to augment this process with an automated test that compares the output of iv_analyse with Figure 3A from the paper.

A good candidate to be done with a sciunit test @rgerkin , although anyone who can do this even in a simpler way would make a contribution.

Related to #20. @VahidGh and @rayner this would be the logical next step on the work you have both been doing related to I/V curves.

The unit test should then be instrumented to run on Travis CI in this repo.

@slarson slarson added the enhancement label Jan 18, 2015
@VahidGh
Copy link
Member

@VahidGh VahidGh commented Jan 21, 2015

@slarson, Great, I was looking at the sciunit+neuronunit documentation and samples, today.
This issue also could be a good starting point for instrumenting the ion channel modeling pipeline with both sciunit+Travis CI for the verification & validation step.
I don't know if validating single ion channel via neuronunit needs a test design with training phase (as suggested in this paper) or there is some predesignated test available in existing libraries!

@rgerkin
Copy link
Member

@rgerkin rgerkin commented Jan 23, 2015

I've submitted some pull requests to get some of the required tools into a better position for integration. These allow the analyse*.sh scripts to be decomposed into python functions, including NML2ChannelAnalyse and jnml.

I've also begun work on a neuronunit module to make use of these tools to run all the tests from within python without any command line invocations.

Once that is in order I can create NeuronUnit tests that recapitulate the analysis parameters themselves. As for testing/training as @VahidGh suggested, I think it depends on whether we are looking to test how good our models are, or whether we are trying to improve them by free parameter fitting. For now I think it would be good to know how close we are with the pieces we currently have, and then we can invest effort into model fitting.

@rgerkin
Copy link
Member

@rgerkin rgerkin commented Jan 29, 2015

Some of what I've worked on is here. I've updated neuronunit's jnml module (which will eventually be merged into pyNeuroML to do most of what we need to in Python, although I am still relying on jNeuroML over pyLEMS for model compatibility reasons that need to be worked out.

I can reproduce the I-V curve, so in principle I am ready to devise the validation test itself, but I wanted to point out some weird behavior in the simulation in the first few time steps. See cell Out[240] near the bottom of the notebook linked above. There is some sort of oscillation of the current in the first few steps, and I'm wondering if that is causing problems with the measurement of the peak current, and whether those oscillations should even be there. Maybe it's a time step size issue.

@slarson
Copy link
Member Author

@slarson slarson commented Feb 2, 2015

Pasting the image in question here for simplicity:

download

Hmm yeah that oscillation is strange. @rayner we haven't seen this right? Does it change if you change the time step?

@slarson
Copy link
Member Author

@slarson slarson commented Feb 2, 2015

@VahidGh On this issue, can you remind what that digitiser software is that you found for transforming the raster I/V plot curves from papers into numbers? Can you post an example of its output? Apologies if you have already posted this somewhere and I've lost track of it.

@VahidGh
Copy link
Member

@VahidGh VahidGh commented Feb 3, 2015

This is an online version, and this is a stand-alone one.
As an example, this is the digitized version for V=15 mv:
ica-v15
And this is the published Plotly version of the WebPlotDigitizer output.

@travs
Copy link

@travs travs commented May 5, 2015

@rgerkin I'd like to talk about using SciUnit (and NeuronUnit) when making validation test templates for different parts of this project. Is there any chance we can have a chat about this? Thanks.

@rgerkin
Copy link
Member

@rgerkin rgerkin commented May 6, 2015

@travs Yeah; chat, email, Hangouts?

@travs
Copy link

@travs travs commented May 7, 2015

@rgerkin Just sent you an email.

@travs travs added this to the Validation milestone May 7, 2015
@slarson slarson added the in progress label Jun 13, 2015
@slarson slarson self-assigned this Jun 13, 2015
@slarson slarson removed the in progress label Jun 15, 2015
@travs
Copy link

@travs travs commented Jun 21, 2015

This is worth looking back into again now that we have made progress in comparing simulated and observed I/V curves in ChannelWorm.

@rgerkin
Copy link
Member

@rgerkin rgerkin commented Jul 28, 2015

@slarson Do you have the DOI for that paper in your first message? I want to see if we have the Figure 3A data in the ion_channel db in ChannelWorm.

@rgerkin
Copy link
Member

@rgerkin rgerkin commented Jul 31, 2015

@slarson Do you know where that Fig. 3 came from in your first message? That's what I need to move forward. I thought maybe it came from this referenced in #20 but that's a different paper. I tried Google reverse image searching it but sadly that doesn't work yet for most journal figures (probably you screenshotted part of a pdf).

@VahidGh
Copy link
Member

@VahidGh VahidGh commented Jul 31, 2015

@rgerkin, the fig. 3A is taken from Boyle & Cohen 2008 paper. Which originally comes from Fig. 2B of this study which I've addressed how to access to data in the ipython file.

@mwatts15 mwatts15 added this to Ideas in OpenWorm Top-level Apr 13, 2019
@mwatts15 mwatts15 added this to Backlog in openworm/ChannelWorm Apr 13, 2019
@mwatts15 mwatts15 added this to Backlog in muscle_model Apr 13, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
5 participants