[REVIEW]: LiberTEM: Software platform for scalable multidimensional data processing in transmission electron microscopy

[whedon]

Submitting author: @uellue (Dieter Weber)
Repository: https://github.com/LiberTEM/LiberTEM
Version: 0.5.0
Editor: @majensen
Reviewers: @alvarolopez, @fedorov
Archive: 10.5281/zenodo.3763313

Status

Status badge code:

HTML: <a href="https://joss.theoj.org/papers/bab08517a6d948a24c29d86caa0bcdda"><img src="https://joss.theoj.org/papers/bab08517a6d948a24c29d86caa0bcdda/status.svg"></a>
Markdown: [![status](https://joss.theoj.org/papers/bab08517a6d948a24c29d86caa0bcdda/status.svg)](https://joss.theoj.org/papers/bab08517a6d948a24c29d86caa0bcdda)

Reviewers and authors:

Please avoid lengthy details of difficulties in the review thread. Instead, please create a new issue in the target repository and link to those issues (especially acceptance-blockers) by leaving comments in the review thread below. (For completists: if the target issue tracker is also on GitHub, linking the review thread in the issue or vice versa will create corresponding breadcrumb trails in the link target.)

Reviewer instructions & questions

@alvarolopez & @fedorov, please carry out your review in this issue by updating the checklist below. If you cannot edit the checklist please:

1. Make sure you're logged in to your GitHub account
2. Be sure to accept the invite at this URL: https://github.com/openjournals/joss-reviews/invitations

The reviewer guidelines are available here: https://joss.readthedocs.io/en/latest/reviewer_guidelines.html. Any questions/concerns please let @majensen know.

✨ Please try and complete your review in the next two weeks ✨

Review checklist for @alvarolopez

Conflict of interest

• I confirm that I have read the JOSS conflict of interest (COI) policy and that: I have no COIs with reviewing this work or that any perceived COIs have been waived by JOSS for the purpose of this review.

Code of Conduct

• I confirm that I read and will adhere to the JOSS code of conduct.

General checks

• Repository: Is the source code for this software available at the repository url?
• License: Does the repository contain a plain-text LICENSE file with the contents of an OSI approved software license?
• Contribution and authorship: Has the submitting author (@uellue) made major contributions to the software? Does the full list of paper authors seem appropriate and complete?

Functionality

• Installation: Does installation proceed as outlined in the documentation?
• Functionality: Have the functional claims of the software been confirmed?
• Performance: If there are any performance claims of the software, have they been confirmed? (If there are no claims, please check off this item.)

Documentation

• A statement of need: Do the authors clearly state what problems the software is designed to solve and who the target audience is?
• Installation instructions: Is there a clearly-stated list of dependencies? Ideally these should be handled with an automated package management solution.
• Example usage: Do the authors include examples of how to use the software (ideally to solve real-world analysis problems).
• Functionality documentation: Is the core functionality of the software documented to a satisfactory level (e.g., API method documentation)?
• Automated tests: Are there automated tests or manual steps described so that the functionality of the software can be verified?
• Community guidelines: Are there clear guidelines for third parties wishing to 1) Contribute to the software 2) Report issues or problems with the software 3) Seek support

Software paper

• Summary: Has a clear description of the high-level functionality and purpose of the software for a diverse, non-specialist audience been provided?
• A statement of need: Do the authors clearly state what problems the software is designed to solve and who the target audience is?
• State of the field: Do the authors describe how this software compares to other commonly-used packages?
• Quality of writing: Is the paper well written (i.e., it does not require editing for structure, language, or writing quality)?
• References: Is the list of references complete, and is everything cited appropriately that should be cited (e.g., papers, datasets, software)? Do references in the text use the proper citation syntax?

Review checklist for @fedorov

Conflict of interest

• I confirm that I have read the JOSS conflict of interest (COI) policy and that: I have no COIs with reviewing this work or that any perceived COIs have been waived by JOSS for the purpose of this review.

Code of Conduct

• I confirm that I read and will adhere to the JOSS code of conduct.

General checks

• Repository: Is the source code for this software available at the repository url?
• License: Does the repository contain a plain-text LICENSE file with the contents of an OSI approved software license?
• Contribution and authorship: Has the submitting author (@uellue) made major contributions to the software? Does the full list of paper authors seem appropriate and complete?

Functionality

• Installation: Does installation proceed as outlined in the documentation?
• Functionality: Have the functional claims of the software been confirmed?
• Performance: If there are any performance claims of the software, have they been confirmed? (If there are no claims, please check off this item.)

Documentation

• A statement of need: Do the authors clearly state what problems the software is designed to solve and who the target audience is?
• Installation instructions: Is there a clearly-stated list of dependencies? Ideally these should be handled with an automated package management solution.
• Example usage: Do the authors include examples of how to use the software (ideally to solve real-world analysis problems).
• Functionality documentation: Is the core functionality of the software documented to a satisfactory level (e.g., API method documentation)?
• Automated tests: Are there automated tests or manual steps described so that the functionality of the software can be verified?
• Community guidelines: Are there clear guidelines for third parties wishing to 1) Contribute to the software 2) Report issues or problems with the software 3) Seek support

Software paper

• Summary: Has a clear description of the high-level functionality and purpose of the software for a diverse, non-specialist audience been provided?
• A statement of need: Do the authors clearly state what problems the software is designed to solve and who the target audience is?
• State of the field: Do the authors describe how this software compares to other commonly-used packages?
• Quality of writing: Is the paper well written (i.e., it does not require editing for structure, language, or writing quality)?
• References: Is the list of references complete, and is everything cited appropriately that should be cited (e.g., papers, datasets, software)? Do references in the text use the proper citation syntax?

[whedon]

Hello human, I'm @whedon, a robot that can help you with some common editorial tasks. @alvarolopez, @mamcdona77 it looks like you're currently assigned to review this paper 🎉.

⭐ Important ⭐

If you haven't already, you should seriously consider unsubscribing from GitHub notifications for this (https://github.com/openjournals/joss-reviews) repository. As a reviewer, you're probably currently watching this repository which means for GitHub's default behaviour you will receive notifications (emails) for all reviews 😿

To fix this do the following two things:

1. Set yourself as 'Not watching' https://github.com/openjournals/joss-reviews:

2. You may also like to change your default settings for this watching repositories in your GitHub profile here: https://github.com/settings/notifications

For a list of things I can do to help you, just type:

@whedon commands

For example, to regenerate the paper pdf after making changes in the paper's md or bib files, type:

@whedon generate pdf

[whedon]

Reference check summary:

OK DOIs

- 10.5281/zenodo.2450624 is OK
- 10.1145/1327452.1327492 is OK
- 10.1109/nuicone.2012.6493198 is OK
- 10.1109/mcse.2011.37 is OK
- 10.1145/2934664 is OK
- 10.25080/majora-7b98e3ed-013 is OK
- 10.5281/ZENODO.3396791 is OK
- 10.5281/ZENODO.2649351 is OK
- 10.1017/s1431927619000497 is OK
- 10.5281/ZENODO.3572855 is OK

MISSING DOIs

- None

INVALID DOIs

- 10.1142/11389 is INVALID

[whedon]

👉 Check article proof 📄 👈

[alvarolopez]

@majensen I am done, the paper is ok to go in its current form.

[majensen]

@alvarolopez - very much appreciated!
@mamcdona77 - how are things progressing?

[mamcdona77]

Will update early next week Michael Michael A McDonald, MD, PhD

…

On Jan 22, 2020, at 9:52 PM, Mark Jensen ***@***.***> wrote: @alvarolopez - very much appreciated! @mamcdona77 - how are things progressing? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or unsubscribe.

[uellue]

@whedon generate pdf

[uellue]

Would it be OK to include a small wording improvement in the first sentence?

[whedon]

👉 Check article proof 📄 👈

[majensen]

@uellue of course!

[majensen]

@mamcdona77 - How is it coming along? Can I help in any way? Thanks

[majensen]

@whedon add @fedorov as reviewer

[whedon]

OK, @fedorov is now a reviewer

[majensen]

Thanks @fedorov for agreeing to review this work.

[fedorov]

@uellue very nice work! I submitted just one issue, and also have few clarification questions below, before I check off the remaining items in the checklist (am not sure if I should submit issues about those):

• I could not find (or missed) pointers to publicly available sample datasets that software users could easily access - would you be able to add some pointers to the documentation?
• there are statements claiming functionality and performance advantages over alternatives, but it is not clear if those were confirmed anywhere. Do you have any additional citations comparing performance with other packages? I am aware of the performance section of the documentation, but the paper mentions few other packages, and the performance section does not include comparison with those. Since there are statements claiming superiority in the paper, it would be good to back them with evidence.
• I think it would be helpful to the user if the documentation contained a section listing similar packages (something along the lines that you already have in the submitted paper PDF). Would it make sense to add that section to the documentation?

[uellue]

Hi @fedorov, thank you for the feedback! Good points, we'll work on it.

[majensen]

@whedon remove @mamcdona77 as reviewer

[uellue]

@labarba Is this a requirement that there's a 1:1 match with the Zenodo title? It seems that I overlooked it.

The title of the Zenodo deposition is the default one if GitHub integration for releases is activated. We can also call the paper LiberTEM/LiberTEM: 0.5.0 if you prefer. A more descriptive title would just be more appealing, I figured.

As @majensen wrote, we already have quite a number of releases on Zenodo and it would seem a bit odd to rename it just to match a paper title about it.

[labarba]

We have a (soft) policy of asking authors to manually edit the metadata of the Zenodo deposit after the release has done its thing, so the titles and author list matches the paper. Of course we don't want you to change the title of the paper to match the repo name. Look, I'm not insisting. That's what we regularly do. If you have your reasons, do as you wish.

[uellue]

@labarba Thank you for the clarification! If possible, I'd like to leave it as it is. Since both paper title and archive contain the unique name "LiberTEM", I think that readers will understand that the two are referring to each other even if the titles don't match 100%. :-)

[majensen]

@labarba Sounds like we are ready to pull the trigger

[danielskatz]

@majensen - when a submission is ready to be accepted, please tag @openjournals/joss-eics to be sure you get the on-duty AEiC, which changes week to week...

[majensen]

@openjournals/joss-eics This work is recommended for publication.

[danielskatz]

👋 @uellue - if you can merge LiberTEM/LiberTEM#823, we can finalize the publication

[danielskatz]

@whedon accept

[whedon]

Attempting dry run of processing paper acceptance...

[whedon]

Reference check summary:

OK DOIs

- 10.5281/zenodo.2450624 is OK
- 10.1145/1327452.1327492 is OK
- 10.1109/nuicone.2012.6493198 is OK
- 10.1109/mcse.2011.37 is OK
- 10.1145/2934664 is OK
- 10.25080/majora-7b98e3ed-013 is OK
- 10.5281/ZENODO.3396791 is OK
- 10.5281/ZENODO.2649351 is OK
- 10.1017/s1431927619000497 is OK
- 10.5281/ZENODO.3763313 is OK
- 10.1142/9789811204579_0005 is OK

MISSING DOIs

- None

INVALID DOIs

- None

[whedon]

👋 @openjournals/joss-eics, this paper is ready to be accepted and published.

Check final proof 👉 openjournals/joss-papers#1507

If the paper PDF and Crossref deposit XML look good in openjournals/joss-papers#1507, then you can now move forward with accepting the submission by compiling again with the flag deposit=true e.g.

@whedon accept deposit=true

[danielskatz]

👋 @uellue - sorry, I missed one - can you also merge LiberTEM/LiberTEM#824

[danielskatz]

@whedon accept deposit=true

[whedon]

Doing it live! Attempting automated processing of paper acceptance...

[whedon]

🐦🐦🐦 👉 Tweet for this paper 👈 🐦🐦🐦

[whedon]

🚨🚨🚨 THIS IS NOT A DRILL, YOU HAVE JUST ACCEPTED A PAPER INTO JOSS! 🚨🚨🚨

Here's what you must now do:

0. Check final PDF and Crossref metadata that was deposited 👉 Creating pull request for 10.21105.joss.02006 joss-papers#1508
1. Wait a couple of minutes to verify that the paper DOI resolves https://doi.org/10.21105/joss.02006
2. If everything looks good, then close this review issue.
3. Party like you just published a paper! 🎉🌈🦄💃👻🤘

Any issues? notify your editorial technical team...

[sk1p]

@danielskatz Thank you! 🥳 🎉

[danielskatz]

Thanks to @alvarolopez & @fedorov for reviewing!
And @majensen for editing!

And congratulations to @uellue (Dieter Weber) and co-authors!!

[whedon]

🎉🎉🎉 Congratulations on your paper acceptance! 🎉🎉🎉

If you would like to include a link to your paper from your README use the following code snippets:

Markdown:
[![DOI](https://joss.theoj.org/papers/10.21105/joss.02006/status.svg)](https://doi.org/10.21105/joss.02006)

HTML:
<a style="border-width:0" href="https://doi.org/10.21105/joss.02006">
  <img src="https://joss.theoj.org/papers/10.21105/joss.02006/status.svg" alt="DOI badge" >
</a>

reStructuredText:
.. image:: https://joss.theoj.org/papers/10.21105/joss.02006/status.svg
   :target: https://doi.org/10.21105/joss.02006

This is how it will look in your documentation:

We need your help!

Journal of Open Source Software is a community-run journal and relies upon volunteer effort. If you'd like to support us please consider doing either one (or both) of the the following:

• Volunteering to review for us sometime in the future. You can add your name to the reviewer list here: https://joss.theoj.org/reviewer-signup.html
• Making a small donation to support our running costs here: https://numfocus.org/donate-to-joss

[uellue]

Thank you @danielskatz @alvarolopez @fedorov @majensen for reviewing and managing the submission!

@sk1p @kruzaeva @woozey @anandbaburajan @twentyse7en @jan-car @irahulcse @sayandip18 @magnunor Knut @rafaldb Many thanks and congratulations!

[uellue]

As a general remark, open writing and open review in combination with GitHub worked really well IMO, and the whole process was a refreshing, pleasant and productive experience. I'll be sure to recommend JOSS for papers like this. 👍