Review Request: Le Masson, Alexandre #21

Closed
wants to merge 11 commits into
from

Conversation

Projects
None yet
10 participants
@maekclena

maekclena commented Aug 9, 2016

AUTHOR

Dear @ReScience/editors,

I request a review for the replication of the following paper:

  • How Attention Can Create Synaptic Tags for the Learning of Working Memories in Sequential Tasks, J. Rombouts, M. Bohte and P. Roelfsema, PLoS Computational Biology, 11, 2015.

I believe the original results have been faithfully replicated as explained in the accompanying article.

Repository lives at https://github.com/maekclena/ReScience-submission/tree/Le_Masson-Alexandre-2016


EDITOR

  • Editor acknowledgment (@oliviaguest 09 August 2016)
  • Reviewer 1 (@vitay 12 August 2016)
  • Reviewer 2 (@eroesch 8 September 2016)
  • Review 1 decision [accept] 30 November, 2016
  • Review 2 decision [accept] 1 December, 2016
  • Editor decision [accept] 1 December, 2016
@oliviaguest

This comment has been minimized.

Show comment
Hide comment
@oliviaguest

oliviaguest Aug 9, 2016

Thanks @maekclena I will assign reviewers soon.

Thanks @maekclena I will assign reviewers soon.

@rougier rougier changed the title from Review Request to Review Request: Le Masson, Alexandre Aug 10, 2016

@oliviaguest oliviaguest added the Python label Aug 11, 2016

@rougier rougier added the 02 - Review label Aug 24, 2016

@vitay

This comment has been minimized.

Show comment
Hide comment
@vitay

vitay Aug 25, 2016

The authors reproduce successfully the results of the original paper on 2 of the 4 proposed tasks. The other tasks (category matching and vibrotactile discrimination) would not bring much to prove that the model is correctly reproduced, so this is fine. The model itself is actually quite simple, reproducing the tasks (especially the probabilistic task) seems to have been the main difficulty in the reimplementation.

There were no dependency problem to run the simulations, everything is standard and runs well (tested under linux). The code for the model is very well organized, clear and commented. There are a few docstrings missing in the classes, but the names of the methods are self-explaining anyway.

article

The article is well written and describes the difficulty of the reimplementation. A few words on the implementation itself could have been nice: why classes for the populations and projections instead of plain numpy arrays (what could have been slightly more readable)? Does reduce bring much in terms of speed, or is it just more generic (there are never more than two incoming projections to a population in the current model, but it could be beneficial for extensions)? Things like that.

In the results section, you do not provide the setup for computing the success rates and convergence speed. How many networks were used? What is the variance?

You could also describe slightly more in the text in how far Figs. 1 and 2. reproduce the original figures: it is not obvious without reading the original article that fixate dominates until the go signal, after which the correct action is chosen. Don't feel like you have to do it, but it would have been also nice to reproduce other figures, such as 2C or the upper part of 2D and 4C.

It might also be useful to clarify the nature of the feedback weights w' in equations 14 and 16: once an action is selected, only the feedback synapses leaving the corresponding selected Q-value unit are activated to update tags, more precisely: w' ij = w ij × z i . The model could also have dedicated feedback connections but the simpler method is to use the feedforward synapses' weights.

It is indeed not clear in the original article whether the feedback weights are also learned (as they seem to imply), or if they just copy the feedforward ones. What was used in the original implementation? So much for the biological plausibility: as the weights are randomly initialized, there is no chance that the feedforward and feedback weights take the same value in this learning setup. So the superiority over backpropagation is not that obvious...

Small typos:

  • Introduction: it helped to verify the correctness
  • Tasks: extra step to give the final reward
  • Tasks: A number of settings is randomized

code

  • In README.md, the usage should be described as:
python3 simulation.py -h...

as the scripts are not directly executable.

  • It could be useful to specify which version of Python, Numpy and Matplotlib are needed (at least by me Python 3.4.3, Numpy 1.8.2 and Matplotlib 1.3.1 work). As you use the new package statistics of python 3, 3.4 is likely the minimal version.
  • It would not cost much to make the code compatible with python 2.7 (still the default on popular distributions). You just need to add:
from __future__ import print_function

at the beginning of simulation.py and plot-activation.py, and to use the numpy implementation of median instead of statistics in simulation.py:

# from statistics import median
from numpy import median

vitay commented Aug 25, 2016

The authors reproduce successfully the results of the original paper on 2 of the 4 proposed tasks. The other tasks (category matching and vibrotactile discrimination) would not bring much to prove that the model is correctly reproduced, so this is fine. The model itself is actually quite simple, reproducing the tasks (especially the probabilistic task) seems to have been the main difficulty in the reimplementation.

There were no dependency problem to run the simulations, everything is standard and runs well (tested under linux). The code for the model is very well organized, clear and commented. There are a few docstrings missing in the classes, but the names of the methods are self-explaining anyway.

article

The article is well written and describes the difficulty of the reimplementation. A few words on the implementation itself could have been nice: why classes for the populations and projections instead of plain numpy arrays (what could have been slightly more readable)? Does reduce bring much in terms of speed, or is it just more generic (there are never more than two incoming projections to a population in the current model, but it could be beneficial for extensions)? Things like that.

In the results section, you do not provide the setup for computing the success rates and convergence speed. How many networks were used? What is the variance?

You could also describe slightly more in the text in how far Figs. 1 and 2. reproduce the original figures: it is not obvious without reading the original article that fixate dominates until the go signal, after which the correct action is chosen. Don't feel like you have to do it, but it would have been also nice to reproduce other figures, such as 2C or the upper part of 2D and 4C.

It might also be useful to clarify the nature of the feedback weights w' in equations 14 and 16: once an action is selected, only the feedback synapses leaving the corresponding selected Q-value unit are activated to update tags, more precisely: w' ij = w ij × z i . The model could also have dedicated feedback connections but the simpler method is to use the feedforward synapses' weights.

It is indeed not clear in the original article whether the feedback weights are also learned (as they seem to imply), or if they just copy the feedforward ones. What was used in the original implementation? So much for the biological plausibility: as the weights are randomly initialized, there is no chance that the feedforward and feedback weights take the same value in this learning setup. So the superiority over backpropagation is not that obvious...

Small typos:

  • Introduction: it helped to verify the correctness
  • Tasks: extra step to give the final reward
  • Tasks: A number of settings is randomized

code

  • In README.md, the usage should be described as:
python3 simulation.py -h...

as the scripts are not directly executable.

  • It could be useful to specify which version of Python, Numpy and Matplotlib are needed (at least by me Python 3.4.3, Numpy 1.8.2 and Matplotlib 1.3.1 work). As you use the new package statistics of python 3, 3.4 is likely the minimal version.
  • It would not cost much to make the code compatible with python 2.7 (still the default on popular distributions). You just need to add:
from __future__ import print_function

at the beginning of simulation.py and plot-activation.py, and to use the numpy implementation of median instead of statistics in simulation.py:

# from statistics import median
from numpy import median
@oliviaguest

This comment has been minimized.

Show comment
Hide comment
@oliviaguest

oliviaguest Sep 8, 2016

Many thanks to @vitay for the review. I will have to locate another 2nd reviewer, so please bear with me @maekclena while I try to track somebody down.

Many thanks to @vitay for the review. I will have to locate another 2nd reviewer, so please bear with me @maekclena while I try to track somebody down.

@oliviaguest

This comment has been minimized.

Show comment
Hide comment
@oliviaguest

oliviaguest Sep 8, 2016

EDIT: Please see below.

Would either of @MehdiKhamassi or @benoit-girard be able to take on the mantle of 2nd reviewer? 😄

oliviaguest commented Sep 8, 2016

EDIT: Please see below.

Would either of @MehdiKhamassi or @benoit-girard be able to take on the mantle of 2nd reviewer? 😄

@eroesch

This comment has been minimized.

Show comment
Hide comment
@eroesch

eroesch Sep 8, 2016

Hi @oliviaguest; I'd be interested to do the review.

eroesch commented Sep 8, 2016

Hi @oliviaguest; I'd be interested to do the review.

@eroesch eroesch closed this Sep 8, 2016

@eroesch eroesch reopened this Sep 8, 2016

@oliviaguest

This comment has been minimized.

Show comment
Hide comment
@oliviaguest

oliviaguest Sep 8, 2016

Excellent! Go ahead!

Excellent! Go ahead!

@benoit-girard

This comment has been minimized.

Show comment
Hide comment
@benoit-girard

benoit-girard Sep 8, 2016

You are too fast for me! Next time...

You are too fast for me! Next time...

@oliviaguest

This comment has been minimized.

Show comment
Hide comment
@oliviaguest

oliviaguest Sep 22, 2016

Any updates on this @eroesch?

Also have you had any time to address some of the above @maekclena?

Any updates on this @eroesch?

Also have you had any time to address some of the above @maekclena?

@maekclena

This comment has been minimized.

Show comment
Hide comment
@maekclena

maekclena Sep 23, 2016

No, I haven't had time yet. I will wait for @eroesch's review to make all necessary corrections at once.

No, I haven't had time yet. I will wait for @eroesch's review to make all necessary corrections at once.

@eroesch

This comment has been minimized.

Show comment
Hide comment
@eroesch

eroesch Sep 25, 2016

I am almost done.

eroesch commented Sep 25, 2016

I am almost done.

@oliviaguest

This comment has been minimized.

Show comment
Hide comment
@rougier

This comment has been minimized.

Show comment
Hide comment
@rougier

rougier Oct 16, 2016

Member

Don't pay attention to the conflict, I just updated the submission directory. No incidence at all on this submission. @oliviaguest @eroesch Any chance for an update ?

Member

rougier commented Oct 16, 2016

Don't pay attention to the conflict, I just updated the submission directory. No incidence at all on this submission. @oliviaguest @eroesch Any chance for an update ?

@rougier

This comment has been minimized.

Show comment
Hide comment
@rougier

rougier Oct 19, 2016

Member

@eroesch Do you think you can complete your review by the end of the week ?

Member

rougier commented Oct 19, 2016

@eroesch Do you think you can complete your review by the end of the week ?

@eroesch

This comment has been minimized.

Show comment
Hide comment
@eroesch

eroesch Oct 20, 2016

The submission presents a novel implementation of a model describing a new rule for reinforcement learning [1]. The authors review aspects of the implementation that appeared important when reproducing the work, and describe results obtained in similar setups. They reproduce 2 of the 4 tasks used in the original paper. I have managed to run the simulation (macos, Anaconda), and I am satisfied with the code provided, which is actually much shorter than I had anticipated. I thank the authors for their time—and their patience! Find a few comments below, which I believe will help making the submission clearer.

  • Please provide more information on the implementation of the original model [1]; i.e. Which programming language? What particular libraries have they used (and do not forget versions)? Are there specific aspects of the development that are worth mentioning, like GPUs, version of CUDA for ANNarchy, etc?
  • Are there comparative measures of performance between [1] and the implementation that you are presenting? For instance, you mention in the readme that computation time can be long. How does your implementation fare against that of [1]? Are there memory constraints that need to be taken into account?
  • Also, I think it would be important to review the main differences between the two implementations that are due to the choice of programming language, especially as you mention that there are performance differences; in particular, I am mindful that differences in numerical accuracy, truncation policies, etc, would yield significant differences in difference ranges of parameters.
  • Can you add information on how to get hold of the original implementation for [1]? (if on a public repo somewhere, please provide the URL)
  • Please add more information about the versions of the packages that you have used.
  • I imagine that if it would be indeed nice to add some of the above information in the text (more or less concisely), it could also feature in a dedicated table.
  • In line with my previous comments, I find the discussion about the discretisation of time events extremely interesting, for the purpose of reproduced results. I would encourage the authors to add more on that topic, and especially the consequences of such apparently innocuous behaviour for the repro of the original results.
  • Mentioning the “ambiguity” of the memory traces is important, please describe whether this is also a behaviour found in the original [1] or something due to your new implementation.
  • I agree that the original submission was a bit unclear when describing the tasks, and thank the authors for providing more information, which were helpful to me. I would encourage the authors to expand if necessary. Do not hesitate to be critical and to discuss limitations, in light of the re-appraisal work that you engaged in.
  • Ideally, you would provide enough information about [1] for a modelling-literate reader who has not yet read [1] to understand the what, why and limitations of the original work, and that of your own work. I would encourage the authors to think of their submission as the first point of entry to [1], rather than a side lined piece of evidence which someone might choose to look at after having gone through [1].
  • In the Results section, please provide information on the number of runs, etc. Incongruence between the results is to be expected because of the stochastic nature of some of the computation, and it would help the reader to know what the percentages you provide actually mean.
  • State clearly that you have followed strictly the original parameters (by referring to Table 1, 2, and 3 and adding any information that you may have found in the documentation of their code, not present in the article). E.g. in add a table with the parameters used.
  • I would personally find it very interesting to compare the two implementations in different parameter ranges, to sort of home down to the gist of the differences in results, but that may be beyond the scope of the article.

[1] J. Rombouts, S. Bohte, and P. Roelfsema. “How Attention Can Create Synaptic Tags for the Learning of Working Memories in Sequential Tasks”. In:
PLoS Computational Biology 11.3 (2015), e1004060. doi: 10.1371/journal.pcbi.1004060.

eroesch commented Oct 20, 2016

The submission presents a novel implementation of a model describing a new rule for reinforcement learning [1]. The authors review aspects of the implementation that appeared important when reproducing the work, and describe results obtained in similar setups. They reproduce 2 of the 4 tasks used in the original paper. I have managed to run the simulation (macos, Anaconda), and I am satisfied with the code provided, which is actually much shorter than I had anticipated. I thank the authors for their time—and their patience! Find a few comments below, which I believe will help making the submission clearer.

  • Please provide more information on the implementation of the original model [1]; i.e. Which programming language? What particular libraries have they used (and do not forget versions)? Are there specific aspects of the development that are worth mentioning, like GPUs, version of CUDA for ANNarchy, etc?
  • Are there comparative measures of performance between [1] and the implementation that you are presenting? For instance, you mention in the readme that computation time can be long. How does your implementation fare against that of [1]? Are there memory constraints that need to be taken into account?
  • Also, I think it would be important to review the main differences between the two implementations that are due to the choice of programming language, especially as you mention that there are performance differences; in particular, I am mindful that differences in numerical accuracy, truncation policies, etc, would yield significant differences in difference ranges of parameters.
  • Can you add information on how to get hold of the original implementation for [1]? (if on a public repo somewhere, please provide the URL)
  • Please add more information about the versions of the packages that you have used.
  • I imagine that if it would be indeed nice to add some of the above information in the text (more or less concisely), it could also feature in a dedicated table.
  • In line with my previous comments, I find the discussion about the discretisation of time events extremely interesting, for the purpose of reproduced results. I would encourage the authors to add more on that topic, and especially the consequences of such apparently innocuous behaviour for the repro of the original results.
  • Mentioning the “ambiguity” of the memory traces is important, please describe whether this is also a behaviour found in the original [1] or something due to your new implementation.
  • I agree that the original submission was a bit unclear when describing the tasks, and thank the authors for providing more information, which were helpful to me. I would encourage the authors to expand if necessary. Do not hesitate to be critical and to discuss limitations, in light of the re-appraisal work that you engaged in.
  • Ideally, you would provide enough information about [1] for a modelling-literate reader who has not yet read [1] to understand the what, why and limitations of the original work, and that of your own work. I would encourage the authors to think of their submission as the first point of entry to [1], rather than a side lined piece of evidence which someone might choose to look at after having gone through [1].
  • In the Results section, please provide information on the number of runs, etc. Incongruence between the results is to be expected because of the stochastic nature of some of the computation, and it would help the reader to know what the percentages you provide actually mean.
  • State clearly that you have followed strictly the original parameters (by referring to Table 1, 2, and 3 and adding any information that you may have found in the documentation of their code, not present in the article). E.g. in add a table with the parameters used.
  • I would personally find it very interesting to compare the two implementations in different parameter ranges, to sort of home down to the gist of the differences in results, but that may be beyond the scope of the article.

[1] J. Rombouts, S. Bohte, and P. Roelfsema. “How Attention Can Create Synaptic Tags for the Learning of Working Memories in Sequential Tasks”. In:
PLoS Computational Biology 11.3 (2015), e1004060. doi: 10.1371/journal.pcbi.1004060.

@oliviaguest

This comment has been minimized.

Show comment
Hide comment
@oliviaguest

oliviaguest Oct 26, 2016

@maekclena you probably want to read above reviews if you haven't already.

@maekclena you probably want to read above reviews if you haven't already.

@oliviaguest

This comment has been minimized.

Show comment
Hide comment
@oliviaguest

oliviaguest Nov 14, 2016

Hey @rougier do you know @maekclena's email address? Perhaps we need it to get in contact?

Hey @rougier do you know @maekclena's email address? Perhaps we need it to get in contact?

@rougier

This comment has been minimized.

Show comment
Hide comment
@rougier

rougier Nov 14, 2016

Member

On the article (PDF), there is the email of the corresponding author.

Member

rougier commented Nov 14, 2016

On the article (PDF), there is the email of the corresponding author.

@falex33

This comment has been minimized.

Show comment
Hide comment
@falex33

falex33 Nov 16, 2016

our response is being prepared... Very sorry for being late and thank you for your patience...

falex33 commented Nov 16, 2016

our response is being prepared... Very sorry for being late and thank you for your patience...

@maekclena

This comment has been minimized.

Show comment
Hide comment
@maekclena

maekclena Nov 22, 2016

Sorry for the delay and the lack of communication.
We made the necessary corrections with respect to the reviews.

Most notable changes:

  • scripts now support Python 2.7
  • added requirements' versions
  • added a link to the author's implementation
  • added an implementations comparison section
  • added more comments in the code

As well as additional details and rewritten sentences in the article.

Sorry for the delay and the lack of communication.
We made the necessary corrections with respect to the reviews.

Most notable changes:

  • scripts now support Python 2.7
  • added requirements' versions
  • added a link to the author's implementation
  • added an implementations comparison section
  • added more comments in the code

As well as additional details and rewritten sentences in the article.

@oliviaguest

This comment has been minimized.

Show comment
Hide comment
@oliviaguest

oliviaguest Nov 29, 2016

Reviewers — @vitay & @eroesch — are you satisfied with the changes that have been made? If yes, can you please formally accept the paper?

Reviewers — @vitay & @eroesch — are you satisfied with the changes that have been made? If yes, can you please formally accept the paper?

@vitay

This comment has been minimized.

Show comment
Hide comment
@vitay

vitay Nov 30, 2016

I am for accepting the paper, the changes answered my questions.

vitay commented Nov 30, 2016

I am for accepting the paper, the changes answered my questions.

@eroesch

This comment has been minimized.

Show comment
Hide comment
@eroesch

eroesch Dec 1, 2016

I am accepting the paper, the changes answered (most of) my questions.

eroesch commented Dec 1, 2016

I am accepting the paper, the changes answered (most of) my questions.

@oliviaguest

This comment has been minimized.

Show comment
Hide comment
@oliviaguest

oliviaguest Dec 1, 2016

Dear @falex33 and @maekclena — Congratulations! How Attention Can Create Synaptic Tags for the Learning of Working Memories in Sequential Tasks has been accepted! Publication and more details will soon follow.

Dear @falex33 and @maekclena — Congratulations! How Attention Can Create Synaptic Tags for the Learning of Working Memories in Sequential Tasks has been accepted! Publication and more details will soon follow.

@ReScience ReScience locked and limited conversation to collaborators Dec 1, 2016

@oliviaguest

This comment has been minimized.

Show comment
Hide comment
@oliviaguest

oliviaguest Dec 12, 2016

Hi all — I have attempted my first go at this:
DOI

Hi all — I have attempted my first go at this:
DOI

@rougier rougier closed this Dec 18, 2016

@pdebuyl

This comment has been minimized.

Show comment
Hide comment
@pdebuyl

pdebuyl Jan 7, 2017

Member

Hi @oliviaguest

The archived PDF has wrong links for the red buttons "Article repository", etc. on the first page.

The first one, for instance, points to https://github.com/ReScience-Archives/Le_Masson-Alexandre-2016/article whereas the "good" link would be https://github.com/ReScience-Archives/Le_Masson-Alexandre-2016/tree/master/article

Member

pdebuyl commented Jan 7, 2017

Hi @oliviaguest

The archived PDF has wrong links for the red buttons "Article repository", etc. on the first page.

The first one, for instance, points to https://github.com/ReScience-Archives/Le_Masson-Alexandre-2016/article whereas the "good" link would be https://github.com/ReScience-Archives/Le_Masson-Alexandre-2016/tree/master/article

@oliviaguest

This comment has been minimized.

Show comment
Hide comment
@oliviaguest

oliviaguest Jan 7, 2017

Sorry about this. No idea how to fix this problem unfortunately as this is the first time I've done this. I won't try anything until somebody who knows more advises. Perhaps @rougier knows more?

Sorry about this. No idea how to fix this problem unfortunately as this is the first time I've done this. I won't try anything until somebody who knows more advises. Perhaps @rougier knows more?

@pdebuyl

This comment has been minimized.

Show comment
Hide comment
@pdebuyl

pdebuyl Jan 7, 2017

Member

The links are in the header of the .md file for the paper. What I don't know is how to manage the update to Zenodo, if one is done at all.

Sorry @rougier , we'll need an opinion :-)

Member

pdebuyl commented Jan 7, 2017

The links are in the header of the .md file for the paper. What I don't know is how to manage the update to Zenodo, if one is done at all.

Sorry @rougier , we'll need an opinion :-)

@rougier

This comment has been minimized.

Show comment
Hide comment
@rougier

rougier Jan 7, 2017

Member

I think you need to regenerate the right PDF and upload a new version to Zenodo (while keeping the same DOI).

Member

rougier commented Jan 7, 2017

I think you need to regenerate the right PDF and upload a new version to Zenodo (while keeping the same DOI).

@oliviaguest

This comment has been minimized.

Show comment
Hide comment
@oliviaguest

oliviaguest Jan 7, 2017

Indeed. This is why I asked for your more experienced opinion. So unless I am mistaken, which is possible... I'm not sure that I can change any files without emailing Zenodo. I am of course more than happy to email them myself to fix this problem!

Note: File addition, removal or modification are not allowed after an upload has been published. This is because a Digital Object Identifier (DOI) is registered with DataCite for each upload. If you've made a mistake please contact us.

Indeed. This is why I asked for your more experienced opinion. So unless I am mistaken, which is possible... I'm not sure that I can change any files without emailing Zenodo. I am of course more than happy to email them myself to fix this problem!

Note: File addition, removal or modification are not allowed after an upload has been published. This is because a Digital Object Identifier (DOI) is registered with DataCite for each upload. If you've made a mistake please contact us.

@rougier

This comment has been minimized.

Show comment
Hide comment
@rougier

rougier Jan 7, 2017

Member

Can't you release a new version on Zenodo using the same DOI ? If not, we'll have to issue a new DOI and update the website and ReScience repository.

Member

rougier commented Jan 7, 2017

Can't you release a new version on Zenodo using the same DOI ? If not, we'll have to issue a new DOI and update the website and ReScience repository.

@oliviaguest

This comment has been minimized.

Show comment
Hide comment
@oliviaguest

oliviaguest Jan 7, 2017

I'm unfamiliar with Zenodo but I don't think so...?

I'm unfamiliar with Zenodo but I don't think so...?

@rougier

This comment has been minimized.

Show comment
Hide comment
@rougier

rougier Jan 8, 2017

Member

Just checked and you'r right. We'll have to contact Zenodo. Can you handle it ?

Member

rougier commented Jan 8, 2017

Just checked and you'r right. We'll have to contact Zenodo. Can you handle it ?

@oliviaguest

This comment has been minimized.

Show comment
Hide comment
@oliviaguest

oliviaguest Jan 8, 2017

I think before emailing we/I should fix the repo which we can edit ourselves first. So I need to edit the md file to correct the link, and recompile the pdf. Is there anything else that needs updating? Does the release on github allow changes?

I think before emailing we/I should fix the repo which we can edit ourselves first. So I need to edit the md file to correct the link, and recompile the pdf. Is there anything else that needs updating? Does the release on github allow changes?

@rougier

This comment has been minimized.

Show comment
Hide comment
@rougier

rougier Jan 8, 2017

Member

No, I think you will have to make a new release (and double check links before making the release). I can help you on checking the PDF.

We really need to automate this part to avoid future errors. I did the same for one submission but I was lucky enough to realizd something was wrong before submitting to Zenodo.

Member

rougier commented Jan 8, 2017

No, I think you will have to make a new release (and double check links before making the release). I can help you on checking the PDF.

We really need to automate this part to avoid future errors. I did the same for one submission but I was lucky enough to realizd something was wrong before submitting to Zenodo.

@oliviaguest

This comment has been minimized.

Show comment
Hide comment
@oliviaguest

oliviaguest Jan 8, 2017

OK, I'll try on Monday!

OK, I'll try on Monday!

@khinsen

This comment has been minimized.

Show comment
Hide comment
@khinsen

khinsen Jan 8, 2017

Zenodo lets you upload a new file and declare that it renders an earlier one obsolete. That's probably the best and most transparent way to handle corrections. If I remember correctly, you have to

  1. upload and publish the new version, and refer to the old one in the metadata as "replaces..."
  2. edit the metadata of the old version to add a reference to the new one.

khinsen commented Jan 8, 2017

Zenodo lets you upload a new file and declare that it renders an earlier one obsolete. That's probably the best and most transparent way to handle corrections. If I remember correctly, you have to

  1. upload and publish the new version, and refer to the old one in the metadata as "replaces..."
  2. edit the metadata of the old version to add a reference to the new one.
@pdebuyl

This comment has been minimized.

Show comment
Hide comment
@pdebuyl

pdebuyl Jan 8, 2017

Member

I see

is new version of this upload

Is this a kind of "official" replacement method? Will there be two DOIs?

Member

pdebuyl commented Jan 8, 2017

I see

is new version of this upload

Is this a kind of "official" replacement method? Will there be two DOIs?

@oliviaguest

This comment has been minimized.

Show comment
Hide comment
@oliviaguest

oliviaguest Jan 8, 2017

@khinsen could you point me to where that option is, please?

@khinsen could you point me to where that option is, please?

@khinsen

This comment has been minimized.

Show comment
Hide comment
@khinsen

khinsen Jan 9, 2017

It's under "Related/alternate identifiers". There you can enter URLs or DOIs of related things and assign a role to them. The two relevant roles are "is new version of this upload" and "is previous version of this upload".

For an example of how this is presented to the reader, see this software upload. Under "related identifiers" in the right column you can see both "previous versions" and "new versions".

@pdebuyl: Each upload has its own DOI. There are just annotated links between the two uploads. It's more of a versioning than a replacement method, but that's how the scientific record should work in my opinion.

khinsen commented Jan 9, 2017

It's under "Related/alternate identifiers". There you can enter URLs or DOIs of related things and assign a role to them. The two relevant roles are "is new version of this upload" and "is previous version of this upload".

For an example of how this is presented to the reader, see this software upload. Under "related identifiers" in the right column you can see both "previous versions" and "new versions".

@pdebuyl: Each upload has its own DOI. There are just annotated links between the two uploads. It's more of a versioning than a replacement method, but that's how the scientific record should work in my opinion.

@oliviaguest

This comment has been minimized.

Show comment
Hide comment
@oliviaguest

oliviaguest Jan 11, 2017

OK, I am attempting step one now of @khinsen's solution. Can somebody check that I do the github-side of things correctly please before I do anything on Zenodo? Thanks for the help so far. Will post a link when I have compiled the corrected .md.

oliviaguest commented Jan 11, 2017

OK, I am attempting step one now of @khinsen's solution. Can somebody check that I do the github-side of things correctly please before I do anything on Zenodo? Thanks for the help so far. Will post a link when I have compiled the corrected .md.

@oliviaguest

This comment has been minimized.

Show comment
Hide comment
@oliviaguest

oliviaguest Jan 11, 2017

OK, so I think this has now fixed the link problem: https://github.com/ReScience-Archives/Le_Masson-Alexandre-2016/tree/master/article
Is the next step a new release on github?

OK, so I think this has now fixed the link problem: https://github.com/ReScience-Archives/Le_Masson-Alexandre-2016/tree/master/article
Is the next step a new release on github?

@rougier

This comment has been minimized.

Show comment
Hide comment
@rougier

rougier Jan 11, 2017

Member

I think a link for the notebook in the PDF while there is no notebook, do you know why ? Was it the case with previous version ?

Member

rougier commented Jan 11, 2017

I think a link for the notebook in the PDF while there is no notebook, do you know why ? Was it the case with previous version ?

@oliviaguest

This comment has been minimized.

Show comment
Hide comment
@oliviaguest

oliviaguest Jan 11, 2017

Better? 😄

Better? 😄

@pdebuyl

This comment has been minimized.

Show comment
Hide comment
@pdebuyl

pdebuyl Jan 11, 2017

Member

As the article is in re-processing (okay, because of me :-/) is it useful to have the link to data? It has not been used by the authors and contains only the template's README and LICENSE.

Member

pdebuyl commented Jan 11, 2017

As the article is in re-processing (okay, because of me :-/) is it useful to have the link to data? It has not been used by the authors and contains only the template's README and LICENSE.

@oliviaguest

This comment has been minimized.

Show comment
Hide comment
@oliviaguest

oliviaguest Jan 13, 2017

Removed it. Is the repo and article looking done now? Do we need the data and notebook directories?

Removed it. Is the repo and article looking done now? Do we need the data and notebook directories?

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.