Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[REVIEW]: dsBinVal: Conducting distributed ROC analysis and Calibration using DataSHIELD #4545

Closed
editorialbot opened this issue Jul 5, 2022 · 86 comments
Closed

[REVIEW]: dsBinVal: Conducting distributed ROC analysis and Calibration using DataSHIELD #4545

editorialbot opened this issue Jul 5, 2022 · 86 comments
Assignees
@csoneson @AnthonyOfSeattle
Labels
accepted published Papers published in JOSS R recommend-accept Papers recommended for acceptance in JOSS. review TeX Track: 2 (BCM) Biomedical Engineering, Biosciences, Chemistry, and Materials

Comments

@editorialbot
Copy link
Collaborator

editorialbot commented Jul 5, 2022 •
edited

Submitting author: @schalkdaniel (Daniel Schalk)
Repository: https://github.com/difuture-lmu/dsBinVal
Branch with paper.md (empty if default branch):
Version: 1.0.2
Editor: @csoneson
Reviewers: @patRyserWelch8, @brunomontezano, @AnthonyOfSeattle
Archive: 10.5281/zenodo.7634619

Status

status

Status badge code:

HTML: <a href="https://joss.theoj.org/papers/05f3c28b18afafee51848423fd5b0e99"><img src="https://joss.theoj.org/papers/05f3c28b18afafee51848423fd5b0e99/status.svg"></a>
Markdown: [![status](https://joss.theoj.org/papers/05f3c28b18afafee51848423fd5b0e99/status.svg)](https://joss.theoj.org/papers/05f3c28b18afafee51848423fd5b0e99)

Reviewers and authors:

Please avoid lengthy details of difficulties in the review thread. Instead, please create a new issue in the target repository and link to those issues (especially acceptance-blockers) by leaving comments in the review thread below. (For completists: if the target issue tracker is also on GitHub, linking the review thread in the issue or vice versa will create corresponding breadcrumb trails in the link target.)

Reviewer instructions & questions

@patRyserWelch8 & @brunomontezano, your review will be checklist based. Each of you will have a separate checklist that you should update when carrying out your review.
First of all you need to run this command in a separate comment to create the checklist:

@editorialbot generate my checklist

The reviewer guidelines are available here: https://joss.readthedocs.io/en/latest/reviewer_guidelines.html. Any questions/concerns please let @csoneson know.

✨ Please start on your review when you are able, and be sure to complete your review in the next six weeks, at the very latest ✨

Checklists

📝 Checklist for @brunomontezano

📝 Checklist for @AnthonyOfSeattle
@editorialbot
Copy link
Collaborator Author

Hello humans, I'm @editorialbot, a robot that can help you with some common editorial tasks.

For a list of things I can do to help you, just type:

@editorialbot commands

For example, to regenerate the paper pdf after making changes in the paper's md or bib files, type:

@editorialbot generate pdf

@editorialbot editorialbot mentioned this issue Jul 5, 2022
Closed
@editorialbot
Copy link
Collaborator Author 

Software report:

github.com/AlDanial/cloc v 1.88  T=0.03 s (1687.6 files/s, 126188.9 lines/s)
-------------------------------------------------------------------------------
Language                     files          blank        comment           code
-------------------------------------------------------------------------------
R                               32            400            476           1362
Markdown                         5            117              0            403
YAML                             5             33             11            179
Rmd                              1             65             83             88
TeX                              1              6              0             67
-------------------------------------------------------------------------------
SUM:                            44            621            570           2099
-------------------------------------------------------------------------------


gitinspector failed to run statistical information for the repository

@editorialbot
Copy link
Collaborator Author

Wordcount for paper.md is 649

@editorialbot
Copy link
Collaborator Author 

Reference check summary (note 'MISSING' DOIs are suggestions that need verification):

OK DOIs

- 10.1093/ije/dyu188 is OK
- 10.3414/ME17-02-0022 is OK
- 10.48550/ARXIV.2203.10828 is OK
- 10.1111/j.0006-341x.2000.00352.x is OK
- 10.1007/11681878_14 is OK
- 10.2307/2531595 is OK

MISSING DOIs

- None

INVALID DOIs

- None

@editorialbot
Copy link
Collaborator Author

👉📄 Download article proof 📄 View article proof on GitHub 📄 👈

@csoneson
Copy link
Member

csoneson commented Jul 5, 2022

👋🏼 @schalkdaniel, @patRyserWelch8, @brunomontezano - this is the review thread for the submission. All of our communications will happen here from now on.

Please check the post at the top of the issue for instructions on how to generate your own review checklist. As you go over the submission, please check any items that you feel have been satisfied. There are also links to the JOSS reviewer guidelines.

The JOSS review is different from most other journals. Our goal is to work with the authors to help them meet our criteria instead of merely passing judgment on the submission. As such, the reviewers are encouraged to submit issues directly in the software repository. If you do so, please mention this thread so that a link is created (and I can keep an eye on what is happening). Please also feel free to comment and ask questions in this thread. It is often easier to post comments/questions/suggestions as you come across them instead of waiting until you've reviewed the entire package.

Please feel free to ping me (@csoneson) if you have any questions or concerns. Thanks!

@brunomontezano
Copy link

brunomontezano commented Jul 10, 2022 •
edited

Review checklist for @brunomontezano

Conflict of interest

  • I confirm that I have read the JOSS conflict of interest (COI) policy and that: I have no COIs with reviewing this work or that any perceived COIs have been waived by JOSS for the purpose of this review.

Code of Conduct

General checks

  • Repository: Is the source code for this software available at the https://github.com/difuture-lmu/dsBinVal?
  • License: Does the repository contain a plain-text LICENSE file with the contents of an OSI approved software license?
  • Contribution and authorship: Has the submitting author (@schalkdaniel) made major contributions to the software? Does the full list of paper authors seem appropriate and complete?
  • Substantial scholarly effort: Does this submission meet the scope eligibility described in the JOSS guidelines

Functionality

  • Installation: Does installation proceed as outlined in the documentation?
  • Functionality: Have the functional claims of the software been confirmed?
  • Performance: If there are any performance claims of the software, have they been confirmed? (If there are no claims, please check off this item.)

Documentation

  • A statement of need: Do the authors clearly state what problems the software is designed to solve and who the target audience is?
  • Installation instructions: Is there a clearly-stated list of dependencies? Ideally these should be handled with an automated package management solution.
  • Example usage: Do the authors include examples of how to use the software (ideally to solve real-world analysis problems).
  • Functionality documentation: Is the core functionality of the software documented to a satisfactory level (e.g., API method documentation)?
  • Automated tests: Are there automated tests or manual steps described so that the functionality of the software can be verified?
  • Community guidelines: Are there clear guidelines for third parties wishing to 1) Contribute to the software 2) Report issues or problems with the software 3) Seek support

Software paper

  • Summary: Has a clear description of the high-level functionality and purpose of the software for a diverse, non-specialist audience been provided?
  • A statement of need: Does the paper have a section titled 'Statement of need' that clearly states what problems the software is designed to solve, who the target audience is, and its relation to other work?
  • State of the field: Do the authors describe how this software compares to other commonly-used packages?
  • Quality of writing: Is the paper well written (i.e., it does not require editing for structure, language, or writing quality)?
  • References: Is the list of references complete, and is everything cited appropriately that should be cited (e.g., papers, datasets, software)? Do references in the text use the proper citation syntax?

@csoneson
Copy link
Member

👋🏻 @patRyserWelch8, @brunomontezano - just wanted to check in how things are going with the reviews. Don't hesitate to reach out if you have questions. Thanks!

@brunomontezano
Copy link

@csoneson Hi Charlotte.

These last weeks were pretty busy to me but I'm going to take a further look and provide feedback ASAP, until middle of this week for sure (until Wednesday).

Thanks for your comprehension and support.

@brunomontezano
Copy link

@schalkdaniel Some suggestions about the paper:

  • Add State of the field section. Maybe you could briefly describe some other technologies that aim to solve the same problem and how the interaction with DataSHIELD framework in R could be different in usage than these alternatives. If it would ease or improve the current approach, etc.

About the package repository itself:

  • I'd recommend you to add some community guidelines, so, how other people could contribute to the software (code-related or not), how issues or bugs could be reported (email? issue tab on GitHub?), how people could seek help about the software usage (issue tab? email? self-hosted platform?). It could be included in the end of README if you like.

Feel free not to accept my suggestions as long as you have good arguments. Any questions, just ask me.

@csoneson
Copy link
Member

👋🏻 @patRyserWelch8 - could you let us know how your review is going? Let me know if you have any questions. Thanks!

@csoneson
Copy link
Member

Ping @patRyserWelch8

👋🏻 @patRyserWelch8 - could you let us know how your review is going? Let me know if you have any questions. Thanks!

@csoneson
Copy link
Member

I have reached out to @patRyserWelch8 offline as well.

@schalkdaniel
Copy link

@brunomontezano Thanks for your suggestions. I have added a short section for the State of Field in the paper (see this this PR). For the community guidelines, there is already a file CONTRIBUTING. Is that what you meant with community guidelines?

@brunomontezano
Copy link

@schalkdaniel Yes, the CONTRIBUTING file is enough. Thanks for pointing me out. And also the state of the field that you added is pretty good.

@editorialbot editorialbot added the Track: 2 (BCM) Biomedical Engineering, Biosciences, Chemistry, and Materials label Sep 10, 2022
@csoneson
Copy link
Member

I have not heard back from @patRyserWelch8, so at this point I will try to find an additional reviewer to move the submission forward. Apologies for the delay.

@csoneson
Copy link
Member

👋🏻 @joao-lima, @dghoshal-lbl - would one of you be available and interested in reviewing this submission for JOSS?

dsBinVal: Conducting distributed ROC analysis and Calibration using DataSHIELD

Thanks in advance!

@csoneson
Copy link
Member

👋🏻 @mschubert, @gvegayon - would one of you be available and interested in reviewing this submission for JOSS?

dsBinVal: Conducting distributed ROC analysis and Calibration using DataSHIELD

Thanks in advance!

@mschubert
Copy link

Hi @csoneson, thanks for the invite but I'm unfortunately swamped for the next couple of weeks and won't be able to review

@gvegayon
Copy link

I'm sorry, @csoneson, but I don't have enough time right now. I suggest asking Dr @zhiiiyang.

@csoneson
Copy link
Member

@zhiiiyang - I am aware that I have already invited you as a potential reviewer for this submission and you were busy at the time, but as that was a few months ago I'll take a chance and check if there's a possibility that you might be interested and available now... 🙂

@patRyserWelch8
Copy link

patRyserWelch8 commented Sep 27, 2022 via email

@csoneson
Copy link
Member

Thank you @patRyserWelch8! I'm sorry, I did not receive your email last week. Could you please generate the review checklist as indicated in the first post in this issue:

@editorialbot generate my checklist

and check the boxes that you feel are satisfied by the current state of the submission? It will make it easier to get an overview and follow up on the outstanding issues. Thanks!

@zhiiiyang
Copy link

Hi @csoneson, thank you for reaching out. Sorry, I don't have the time recently, but I have identified a few friends and colleagues who would like to take on this opportunity. @LejianLeoHe7 @Epic19mz and Dr. @EugeneHao

@zhiiiyang - I am aware that I have already invited you as a potential reviewer for this submission and you were busy at the time, but as that was a few months ago I'll take a chance and check if there's a possibility that you might be interested and available now... 🙂

@csoneson
Copy link
Member

👋🏻 @schalkdaniel - I went through the submission and made a PR with some minor suggestions for the text/bibliography. Please check and merge if you agree (you can then ask @editorialbot to generate a new proof). Then, the next steps are as follows:

  • Make a tagged release of the software, and list the version tag of the archived version here.
  • Archive the reviewed software in Zenodo or a similar service
  • Check that the archival deposit (e.g., in Zenodo) has the correct metadata. In particular, the title and author list should be identical to those of the paper. You can also add the authors' ORCID.
  • Please list the DOI of the archived version here.

I can then move forward with accepting the submission.

@csoneson
Copy link
Member

For completeness and posterity, I will also note that @patRyserWelch8 did not explicitly approve the submission, but did contribute to the review process and so will be listed in the final paper.

@patRyserWelch8
Copy link

patRyserWelch8 commented Feb 13, 2023 via email

@schalkdaniel
Copy link

@csoneson I created a tagged release, the tag is v.1.0.2. I also uploaded all files of that release to Zenodo, the DOI is 10.5281/zenodo.7634619. Please also note that I was asked by one of the authors to add another affiliation, it is listed as:

 - name: Munich Center for Machine Learning, Munich, Germany
   index: 4

@csoneson
Copy link
Member

@editorialbot generate pdf

@editorialbot
Copy link
Collaborator Author

👉📄 Download article proof 📄 View article proof on GitHub 📄 👈

@csoneson
Copy link
Member

@editorialbot check references

@editorialbot
Copy link
Collaborator Author 

Reference check summary (note 'MISSING' DOIs are suggestions that need verification):

OK DOIs

- 10.1093/ije/dyu188 is OK
- 10.3414/ME17-02-0022 is OK
- 10.48550/ARXIV.2203.10828 is OK
- 10.1111/j.0006-341x.2000.00352.x is OK
- 10.1007/11681878_14 is OK
- 10.2307/2531595 is OK
- 10.1186/1471-2105-12-77 is OK

MISSING DOIs

- None

INVALID DOIs

- None

@csoneson
Copy link
Member

@editorialbot set 10.5281/zenodo.7634619 as archive

@editorialbot
Copy link
Collaborator Author

Done! Archive is now 10.5281/zenodo.7634619

@csoneson
Copy link
Member

@editorialbot set 1.0.2 as version

@editorialbot
Copy link
Collaborator Author

Done! version is now 1.0.2

@csoneson
Copy link
Member

Thanks @schalkdaniel - I'm going to hand over to the track EiC for the last steps. Thanks for submitting to JOSS!

@csoneson
Copy link
Member

@editorialbot recommend-accept

@editorialbot
Copy link
Collaborator Author 

Attempting dry run of processing paper acceptance...

@editorialbot
Copy link
Collaborator Author 

Reference check summary (note 'MISSING' DOIs are suggestions that need verification):

OK DOIs

- 10.1093/ije/dyu188 is OK
- 10.3414/ME17-02-0022 is OK
- 10.48550/ARXIV.2203.10828 is OK
- 10.1111/j.0006-341x.2000.00352.x is OK
- 10.1007/11681878_14 is OK
- 10.2307/2531595 is OK
- 10.1186/1471-2105-12-77 is OK

MISSING DOIs

- None

INVALID DOIs

- None

@editorialbot
Copy link
Collaborator Author

👋 @openjournals/bcm-eics, this paper is ready to be accepted and published.

Check final proof 👉📄 Download article

If the paper PDF and the deposit XML files look good in openjournals/joss-papers#3965, then you can now move forward with accepting the submission by compiling again with the command @editorialbot accept

@editorialbot editorialbot added the recommend-accept Papers recommended for acceptance in JOSS. label Feb 13, 2023
@Kevin-Mattheus-Moerman
Copy link
Member

Kevin-Mattheus-Moerman commented Feb 21, 2023 •
edited

@schalkdaniel I am the AEiC for this track and here to help process this submission for acceptance in JOSS. Below are some minor points that need your attention:

On the archive:

  • Make sure the ZENODO listed license matches your software license. You may need to manually edit this.

@schalkdaniel
Copy link

@Kevin-Mattheus-Moerman, thanks for your effort. I have changed the license, it should now match the one from the submission.

@Kevin-Mattheus-Moerman
Copy link
Member

@editorialbot accept

@editorialbot
Copy link
Collaborator Author 

Doing it live! Attempting automated processing of paper acceptance...

@editorialbot
Copy link
Collaborator Author

🐦🐦🐦 👉 Tweet for this paper 👈 🐦🐦🐦

@editorialbot
Copy link
Collaborator Author

🐘🐘🐘 👉 Toot for this paper 👈 🐘🐘🐘

@editorialbot
Copy link
Collaborator Author

🚨🚨🚨 THIS IS NOT A DRILL, YOU HAVE JUST ACCEPTED A PAPER INTO JOSS! 🚨🚨🚨

Here's what you must now do:

  1. Check final PDF and Crossref metadata that was deposited 👉 Creating pull request for 10.21105.joss.04545 joss-papers#3993
  2. Wait a couple of minutes, then verify that the paper DOI resolves https://doi.org/10.21105/joss.04545
  3. If everything looks good, then close this review issue.
  4. Party like you just published a paper! 🎉🌈🦄💃👻🤘

Any issues? Notify your editorial technical team...

@editorialbot editorialbot added accepted published Papers published in JOSS labels Feb 21, 2023
@Kevin-Mattheus-Moerman
Copy link
Member

Congratulations on this publication @schalkdaniel !

Thanks for editing @csoneson.

And a special thanks to the reviewers: @patRyserWelch8, @brunomontezano, @AnthonyOfSeattle !

@editorialbot
Copy link
Collaborator Author

🎉🎉🎉 Congratulations on your paper acceptance! 🎉🎉🎉

If you would like to include a link to your paper from your README use the following code snippets:

Markdown:
[![DOI](https://joss.theoj.org/papers/10.21105/joss.04545/status.svg)](https://doi.org/10.21105/joss.04545)

HTML:
<a style="border-width:0" href="https://doi.org/10.21105/joss.04545">
  <img src="https://joss.theoj.org/papers/10.21105/joss.04545/status.svg" alt="DOI badge" >
</a>

reStructuredText:
.. image:: https://joss.theoj.org/papers/10.21105/joss.04545/status.svg
   :target: https://doi.org/10.21105/joss.04545

This is how it will look in your documentation:

DOI

We need your help!

The Journal of Open Source Software is a community-run journal and relies upon volunteer effort. If you'd like to support us please consider doing either one (or both) of the the following:

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@csoneson csoneson

@AnthonyOfSeattle AnthonyOfSeattle

Labels
accepted published Papers published in JOSS R recommend-accept Papers recommended for acceptance in JOSS. review TeX Track: 2 (BCM) Biomedical Engineering, Biosciences, Chemistry, and Materials
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests