Join GitHub today
GitHub is home to over 28 million developers working together to host and review code, manage projects, and build software together.Sign up
Review Request: Metzner #30
I request a review for the following replication:
Title: Modeling GABA alterations in schizophrenia: a link between impaired inhibition and altered gamma and beta range auditory entrainment
Author(s): Christoph Metzner
@oliviaguest The one who asks
@oliviaguest Can we begin a conversation with the authors and discuss fixes as we encounter problems/imperfections, or should we wait and post a more thorough revision? I have already noticed a few small things that the author could start to fix as I work my way through the rest of the code.
Also, I cannot seem to clone the forked repo. It clones the original ReScience-submission repository, not the forked one (downloading the .zip on the other hand works fine). If anybody knows how to fix this, it would be helpful.
(REVIEWER 1) I looked through the submission and noted down some comments.
The readme instructs to run
On line 55 of
filename_ctrl_avg = directory+'/sims_ctrl_avg_'+str(int(drive_frequency))+'Hz' filename_schiz_avg = directory+'/sims_schiz_avg_'+str(int(drive_frequency))+'Hz' np.save(filename_ctrl_avg, meg_ctrl_avg) np.save(filename_schiz_avg, meg_schiz_avg)
And then changed the path to the files in
Note that in
Finally, I think the file
I now see that at least some of my previous comments are already corrected in the notebooks, but the changes still need to be transferred to the
At the expense of efficiency, the code is very readable, scoring points in article 7 of the zen of Python.
One minor flaw that I happened to notice is that lines 137 and 150 in
Not to be a PEP8 nazi (nobody likes those) but the very long equations in
Perhaps the comments on lines 167 and 171 in
Overall, the main findings are indeed replicated, however, as recognized by the author, some
If my understanding is correct, the emergence of a 20 Hz component is due to the fact that longer decay of the IPSC's hinders the 40Hz entrainment and you get responses to alternate inputs from the drive. In the absence of noise, your 40Hz-drive schizophrenic network will respond only with 20Hz, and increasing noise brings in the 40Hz component, because some cells still have enough input to fire in spite of the prolonged inhibition. All of this is to say that perhaps scaling the noise a bit lower could give you more of the 20Hz and 40Hz, making the result of the paper more marked. Have you tried anything like that?
The labels in the plots of the article are a bit too small to be readable.
The code was fairly well commented, however in some places it could do with some more comments. For example generating the spike trains (simple_model_class: line 130-140).
I'm not sure I understand _getSingleSpikeTimes - what being iterated over? Is that time and state?
I'm also unclear as to what is going on in calculatePSD. If I understand correctly meg is a time series and your slicing starting at an index that is 20% of the sampling frequency? I think this could use more comments
As @ChristophMetzner mentioned he was going for a readable over efficiency model for the code. However there are some places where the explicit nature of the code detracts from readability.
For example (line 128) just scales background_rate by 1000
As the other reviewer mentioned: the exceptionally long expressions should be broken up in some way for readability.
Out of the box I was unable to run the code as written for several reasons
I made brief changes to allow the code to run and it did so. However plot_figures could not run as the naming convention in the data folder had changed. This needs to be addressed as I have not yet confirmed the plots match the article.
Lastly not at all important the comments for simpleModel have a variable number of tabs
As a comment the cross referencing seemed to have failed. I recall a similar issue and I put in explicit latex \ref and \label tags.
In general I felt the article was fairly sparse and could do with more text and commentary everywhere. Specifically in the introduction: a more general introduction was needed, what are the results, why are they important?
For the methods tables 1 and 2 should really be written into the text - they addresses several questions that arise while reading the text and they don't lend themselves to tabular form. Also the python versions are ?.?.? I don't know if that a cross-ref error or a place holder. Also the OS this was run on should be stated.
Lastly a sentence on the numerical methods is needed - nothing much just forward Euler and a statement (if true) that the results are unaffected by a smaller time step.
There are a few things in the results that need to be addressed.
Also in figure 2 (their figure 5) the frequency ranges are different. For example in the original 30Hz (controll) has a peak ~70 Hz but your implementation stops at 50Hz.
In the original paper power is in nA m^2 Hz^-1 can you scale your power measure to match? - you mention a scaling factor - have you tried to contact the original authors?
Also to match the original paper the raster plots should be flipped so that I cells are on the bottom.
All axis and tick labels need to be much bigger.
As for your comment about about being less pronounced, you proposed that it arises from the fact that there is a random process. What happens if you change the seed - does the effect size change? In general this implementation does seem to replicate the original findings however, more work as to the nature of the discrepancy and its origin is required.
@ChristophMetzner, I agree that "Regarding the 'mistake' in the table, I think that it is not a mistake. The subharmonic component emerges at 40Hz drive and, therefore, is at 20Hz in the power spectrum.", but it used to say that it was at 40Hz before your last few updates. It is all good now.
Perhaps something will have to done about the formatting, there's many pages which now contain only one plot and are almost empty.
The article is much improved and cleaner compared to the previous version, however I still have a few issues, (in the order they appear)
The code as submitted does run with the exception of a typo (first comment)
In general while observing pep8 to the nth degree may not be necessary certain things like max line lengths are really important for readability.
@aaronshifman Regarding your other comments:
@ChristophMetzner Thank you for your last commits and for your comments. Regarding the last point you raise in your comments, on exploring the discrepancies, I agree with you, and, although interesting and welcome, I do not find it strictly necessary to investigate further possible reasons for discrepancy, given that the main results have been replicated.
I think two figures per page is nicer, if they could fit nicely and be readable. I still find it suboptimal to have several pages of figures that completely break off the text, but I don't have an immediate solution for that.
I have one more tiny aesthetic comment: in
I will be on holiday and with limited access to internet from next week. My overall recommendation at this point in time is that the replication (with a few smaller edits) should be accepted.
Hello @ChristophMetzner, I apologize for the delay in the review. I appreciate your addressing my comments and I have no further scientific issues.
My only issues are a few typos / omissions
I agree with @pietromarchesi that the disjointed figures aren't the best option however I don't have a better option and it seems to be that the style that the figure layout isn't nearly as important as the replication itself.
Given this, I move to accept this paper (conditional on these small editorial changes).