Review request: Detorakis #35

gdetor · 2017-08-02T19:18:57Z

AUTHOR

Dear @ReScience/editors,

I request a review for the following replication:

Original article

Title: A Generalized Linear Integrate-and-Fire Neural Model Produces Diverse Spiking Behaviors
Author(s): Stefan Mihalas and Ernst Niebur
Journal (or Conference): Neural Computation
Year: 2009
DOI: 10.1162/neco.2008.12-07-680
PDF: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC2954058/
(or https://capocaccia.ethz.ch/capo/raw-attachment/wiki/2010/spinn10/Mihalas%20Niebur%20NECO.pdf)

Replication

Author(s): Georgios Detorakis
Repository: https://github.com/gdetor/ReScience-submission/tree/detorakis-2017
PDF: https://github.com/gdetor/ReScience-submission/blob/detorakis-2017/article/detorakis-2017.pdf
Keywords: linear leaky integrate-and-fire neuron, diverse spiking behavior, differential equations
Language: Python
Domain: Computational Neuroscience

Results

[ x] Article has been fully replicated
Article has been partially replicated
Article has not been replicated

Potential reviewers

EDITOR

Editor acknowledgment (@otizonaizit) 3 August 2017
Reviewer 1 (@heplesser) 3 August 2017
Reviewer 2 (@pdebuyl) 3 August 2017
Review 1 decision [accept] 18 August 2017
Review 2 decision [accept] 14 August 2017
Editor decision [accept] 18 August 2017

rougier · 2017-08-02T20:06:02Z

@gdetor Thanks for your submission. Do you have a link on the original PDF?

@otizonaizit Can you handle the submission? (I know you're busy, no problem if you can't)

gdetor · 2017-08-02T20:43:49Z

@rougier Here is the link: http://www.mitpressjournals.org/doi/pdf/10.1162/neco.2008.12-07-680

rougier · 2017-08-03T06:23:15Z

@gdetor Would you have a free preprint instead ?

heplesser · 2017-08-03T08:08:10Z

@rougier I would volunteer to review.

otizonaizit · 2017-08-03T10:37:07Z

Jep, I can do it…

…

On Wed 02 Aug, 13:06, "Nicolas P. Rougier" ***@***.***> wrote: @gdetor Thanks for your submission. Do you have a link on the original PDF? @otizonaizit Can you handle the submission? (I know you're busy, no problem if you can't) -- You are receiving this because you were mentioned. Reply to this email directly or view it on GitHub: #35 (comment)

otizonaizit · 2017-08-03T10:49:53Z

@heplesser : thank you! I assigned you as reviewer.

otizonaizit · 2017-08-03T10:50:38Z

@pdebuyl : can you review this?

pdebuyl · 2017-08-03T14:11:22Z

I can do this next week.

gdetor · 2017-08-03T17:01:54Z

@rougier Try this one: https://capocaccia.ethz.ch/capo/raw-attachment/wiki/2010/spinn10/Mihalas%20Niebur%20NECO.pdf

heplesser · 2017-08-04T11:51:29Z

@gdetor @otizonaizit I find you replication successful and your manuscript well written. I would like to request a few minor improvements to text and code before approving:

You mention only in the Conclusion how you inferred those parameters not explicitly given by Mihalas and Niebur. I think it would be better to provide this information already under Methods.
Your tables are organized slightly differently than the table in Mihalas and Niebur. It would aid the reader if you mentioned the differences briefly and stated explicitly that parameter values are identical to those in the original paper.
In your table 4, a parameter t_f appears that is not explained in the text. It appears to be the length of the trace shown in Fig 1 for each case, but needs to be explained.
In Fig 1, you should place the case labels A, B, C, ... outside the frame in black and ensure that there is a tick label at the right edge of the x-axis for each frame. To avoid confusing the reader, you should also point out that you use the same y-axis limits throughout, while the original figure used different limits for each case.
Could you add flow vectors to Fig 2?
I think your Fig 3 is more informative than the original one, but you should maybe point out that your figure is oriented differently from Fig 3 in the original paper.
You mnn_model() code mixes model simulation and plotting concerns. I would strongly suggest that mnn_model() should only simulate the model and that "prettification" is handled elsewhere.
External current is supplied via the pms dictionary as an array of current values; this array implicitly determines the duration of the simulation and assumes a certain time step. Providing simulation duration and time step as separate variables risks inconsistencies. The tidiest solution is probably to provide I_ext and dt as arguments to mnn_model() and infer time. I also do not quite understand why you build T incrementally as a list instead of creating it as a NumPy array right away.
run_all would be a bit more legible if you defined a function that creates each single plot and then called that function for each case, instead of repeating the same code over and over.

otizonaizit · 2017-08-04T20:29:46Z

@pdebuyl : thank you for accepting to review! Next week is perfectly fine, take your time…

otizonaizit · 2017-08-04T20:31:37Z

@heplesser : thank you for the thorough review!
@gdetor : you can start addressing @heplesser comments already :-)

pdebuyl · 2017-08-10T09:50:00Z

Dear @gdetor, @otizonaizit,

The reproduction by @gdetor is well executed and runs as expected on my computer. The code
dependencies are all standard, suggesting easy re-use.

I have a list of comments that I believe should be addressed before accepting the article:

In the original manuscript, as far I can read, the solutions are obtained in Eq. 3.2 with
the next firing time obtained in Eq. 3.5. I used the pdf at
https://www.ncbi.nlm.nih.gov/pmc/articles/PMC2954058/pdf/nihms211651.pdf , in the file
https://capocaccia.ethz.ch/capo/raw-attachment/wiki/2010/spinn10/Mihalas%20Niebur%20NECO.pdf
these are eqs. 4 and 7. I understand very well the rationale to choose a fixed time step
ODE solution here but this choice and the different technique with respect to the
original manuscript should be commented on explicitly.
The license for the whole repo is GPLv3. It is not suitable for the article that should
be licensed under CC-BY.
The code/README.md file does not contain any information. It should contain a brief
description of the Python files and the platform
information that is requested since several months
now by ReScience.
The first argument to the model integration routine is a mutable. In the case where one
runs the code with ipython -i or via the %run IPython magic, the defaults will be
those given by the conditional execution after the line if __name__ == '__main__'. This
is a likely source of confusion and bugs for users of the code.
The main result of the paper is figure 1. Although the layout in a 4x5 matrix is intended
to follow the original paper, it makes the figure's content unsuitably small. I suggest
to change this by using a 2x10 matrix or splitting the figure, for instance.
A check could be added in mnn_model to ensure that the length of Iext is sufficient.

gdetor · 2017-08-10T22:38:49Z

Dear @otizonaizit @heplesser @pdebuyl ,
Thank you for your corrections/comments and suggestions. All the issues have been addressed. The reviewed manuscript and the source codes have been updated accordingly.

Thank you,
Georgios

pdebuyl · 2017-08-14T15:00:08Z

Dear @gdetor,

Thank you for the update. You modified the article license to BSD, it really should be CC-BY :-)

git show master:code/LICENSE.txt > code/LICENSE.txt

will give you the original one that is totally fine. BSD is of course fine for the code
directory.

The problem of the params mutable is still of relevance. It is now avoided in the specific
cases I mentioned in my earlier report but can still bite anyone re-using params after
importing the module neuron_model. A method such as "default_params" that creates and
returns a copy of a the parameters dictionary would be much safer, along with a
params=None default-valued argument that would call this method when it is indeed
None. The default-parameters-getter could also override the values with keywords arguments
to save typing later on:

pms = neuron_model.default_parameters(a=0.005)

All my other concerns were addressed suitably.

…submission into detorakis-2017

gdetor · 2017-08-14T23:29:09Z

Dear @pdebuyl
Thank you for the comments. I changed the license to CC-BY (the license for the code though is now GPL2). Furthermore, I add a function that generates a default parameters dictionary and takes keyword arguments as you suggested.

heplesser

@gdetor Thank you for your changes. I just have a few minor language- and code-related issues left, see my inline comments.

heplesser · 2017-08-16T15:47:12Z

article/detorakis-2017.tex

-potential (spike) in a dynamic way. The ability of the MNN model to generate
+subthreshold dynamics are defined by a set of linear ordinary differential 
+equations, while an instantaneous threshold potential controls when the neuron
+firing an action potential (spike) in a dynamic way. The ability of the MNN 


Should this read "controls when the neuron fires" instead of "firing"?

heplesser · 2017-08-16T15:48:18Z

article/detorakis-2017.tex

-varies according to figure 1 of the original paper. We provide all equations
-and parameters of the model in tables as it has been suggested
-by~\cite{nordlie:2009}. 
+the dynamical system using the forward Euler's integration scheme. The time step


I think it should either be "using Euler's forward integration scheme" or "using the forward Euler integration scheme".

heplesser · 2017-08-16T15:51:30Z

article/detorakis-2017.tex

@@ -522,15 +534,27 @@ \section{Results}\label{results}
 All the different spiking behaviors of the model are illustrated in 
 Figure~\ref{fig:1}, where the black solid line indicates the membrane potential
 ($V(t)$), the red dashed line illustrates the instantaneous threshold potentials
-($\Theta(t)$), and the gray line shows the input to the neuron ($I_e/C$).
+($\Theta(t)$), and the gray line shows the input to the neuron ($I_e/C$). 
+The $x$-axis scale in all the panels are exactly the same as in the original


"The $x$-axis scales in all panels are", not "scale".

heplesser · 2017-08-16T15:51:44Z

article/detorakis-2017.tex

+The $x$-axis scale in all the panels are exactly the same as in the original
+paper (indicating the total simulation time ($t_f$), while the $y$-axis scale 
+differs from the one in the original paper. In this work the $y$-axis scale
+is the same same for all the subplots ($[-95, -25]\, \Rm{mV}$), except from


"except for"

heplesser · 2017-08-16T15:56:15Z

code/neuron_model.py

          'ThetaInf': -50.0}     # Reverse threshold


 # @profile
-def mnn_model(pms=params, time=10, dt=1.0, IC=(0.01, 0.001, -70.0, -50.0)):
+def mnn_model(pms=params, Iext=np.zeros((200,)), dt=1.0,


Mutables should not be used as default argument values (e.g. http://docs.python-guide.org/en/latest/writing/gotchas/). It is safer to use None:

def mnn_model(pms=params, Iext=None, ...): if Iext is None: Iext = np.zeros((200,))

This is weird. In my last code update, the issue with the mutable has been fixed.

heplesser · 2017-08-16T15:59:04Z

code/neuron_model.py

    """
-    sim_time = int(time / dt)   # Total simulation time
+    sim_time = Iext.shape[0]                    # Simulation time


This isn't really the simulation time, it is the number of steps. Simulation time is obtained by multiplying with dt. So sim_steps would be a better name.

gdetor · 2017-08-16T16:24:37Z

@heplesser Thank you for your corrections and comments. All the issues have beed addressed.

heplesser · 2017-08-17T05:11:05Z

@gdetor Thank you for your updates! After your last commit f6f3165, lines 70ff of neuron_model.py read

def mnn_model(pms=None, Iext=np.zeros((200,)), dt=1.0,
              IC=(0.01, 0.001, -70.0, -50.0)):

so the mutable np.zeros((200,)) is still there as default argument.
https://github.com/ReScience/ReScience-submission/pull/35/files#diff-8a081cf2e57010471dacd32d1e3805e6R70

gdetor · 2017-08-17T15:50:02Z

@heplesser Thank you for noticing that. I updated the code accordingly.

heplesser

@gdetor Thank you for your corrections!
@otizonaizit I approve the manuscript and code for publication in ReScience.

otizonaizit · 2017-08-18T13:18:34Z

@gdetor Congratulations, I hereby accept the submission for publication!
@heplesser , @pdebuyl thank you for your reviews!

pdebuyl · 2017-08-18T18:32:50Z

Before publishing, I'd just like to request that the edits are made in the markdown file and not the latex file. If ReScience decides at some point to publish html versions of the articles this will be mandatory.

otizonaizit · 2017-08-18T18:39:25Z

@pdebuyl good catch! Thanks for pointing this out. @gdetor can you take care of this before we proceed?

gdetor · 2017-08-22T16:50:25Z

@otizonaizit I fixed the markdown file accordingly. Let me know if you need anything more.
Thank you.

rougier · 2017-08-30T09:10:00Z

@otizonaizit Do you need some help for the publication (I know you're busy right now)?

otizonaizit · 2017-08-30T09:15:47Z

Thursday afternoon or Friday afternoon I'll give it a go!

…

On Wed 30 Aug, 09:10, "Nicolas P. Rougier" ***@***.***> wrote: @otizonaizit Do you need some help for the publication (I know you're busy right now)? -- You are receiving this because you were mentioned. Reply to this email directly or view it on GitHub: #35 (comment)

otizonaizit · 2017-09-06T14:57:41Z

@gdetor : we have problems compiling your file, the tex file is corrupted, some headers are wrong, the bibliography is included in the md file, etc... can you try fixing the issues and testing that you can generate a tex file with pandoc from the md file and then compile the tex to pdf?

otizonaizit · 2017-10-06T14:46:11Z

Article is published with

The article will be soon listed on http://rescience.github.io/read/

New submission

53b2bcb

rougier added 01 - Request Life Science Python labels Aug 2, 2017

rougier changed the title ~~New submission~~ Review request: Detorakis Aug 2, 2017

otizonaizit requested a review from pdebuyl August 3, 2017 10:50

otizonaizit added the 02 - Review label Aug 3, 2017

gdetor added 2 commits August 10, 2017 15:27

reviewers comments addressed

264db4c

Update README.md

59a6f15

gdetor added 2 commits August 14, 2017 16:24

Default parameters added

dbbb29e

Merge branch 'detorakis-2017' of https://github.com/gdetor/ReScience-…

42649d4

…submission into detorakis-2017

heplesser suggested changes Aug 16, 2017

View reviewed changes

more corrections

f6f3165

Iext fixed

6caa071

heplesser approved these changes Aug 18, 2017

View reviewed changes

otizonaizit added the 03 - Accepted label Aug 18, 2017

Final version -- md file fixed

11e8246

ReScience locked and limited conversation to collaborators Sep 6, 2017

otizonaizit mentioned this pull request Sep 6, 2017

Article number request ReScience/ReScience#48

Open

ReScience unlocked this conversation Sep 6, 2017

gdetor added 2 commits September 25, 2017 11:39

md file corrections

cb8e03f

markdown file fixed

5b5973d

otizonaizit added the 04 - Published label Oct 6, 2017

otizonaizit closed this Oct 6, 2017

Review request: Detorakis #35

Review request: Detorakis #35

Uh oh!

Conversation

gdetor commented Aug 2, 2017 • edited by otizonaizit Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Original article

Replication

Results

Potential reviewers

Uh oh!

rougier commented Aug 2, 2017

Uh oh!

gdetor commented Aug 2, 2017

Uh oh!

rougier commented Aug 3, 2017

Uh oh!

heplesser commented Aug 3, 2017

Uh oh!

otizonaizit commented Aug 3, 2017 via email

Uh oh!

otizonaizit commented Aug 3, 2017

Uh oh!

otizonaizit commented Aug 3, 2017

Uh oh!

pdebuyl commented Aug 3, 2017

Uh oh!

gdetor commented Aug 3, 2017

Uh oh!

heplesser commented Aug 4, 2017

Uh oh!

otizonaizit commented Aug 4, 2017

Uh oh!

otizonaizit commented Aug 4, 2017

Uh oh!

pdebuyl commented Aug 10, 2017

Uh oh!

gdetor commented Aug 10, 2017

Uh oh!

pdebuyl commented Aug 14, 2017

Uh oh!

gdetor commented Aug 14, 2017

Uh oh!

heplesser left a comment

Choose a reason for hiding this comment

Uh oh!

heplesser Aug 16, 2017

Choose a reason for hiding this comment

Uh oh!

heplesser Aug 16, 2017

Choose a reason for hiding this comment

Uh oh!

heplesser Aug 16, 2017

Choose a reason for hiding this comment

Uh oh!

heplesser Aug 16, 2017

Choose a reason for hiding this comment

Uh oh!

heplesser Aug 16, 2017

Choose a reason for hiding this comment

Uh oh!

gdetor Aug 16, 2017

Choose a reason for hiding this comment

Uh oh!

heplesser Aug 16, 2017

Choose a reason for hiding this comment

Uh oh!

gdetor commented Aug 16, 2017

Uh oh!

heplesser commented Aug 17, 2017

Uh oh!

gdetor commented Aug 17, 2017

Uh oh!

heplesser left a comment

Choose a reason for hiding this comment

Uh oh!

otizonaizit commented Aug 18, 2017

Uh oh!

pdebuyl commented Aug 18, 2017

Uh oh!

gdetor commented Aug 2, 2017 •

edited by otizonaizit

Loading