[REVIEW]: iharm3D: Vectorized General Relativistic Magnetohydrodynamics

[whedon]

Submitting author: @bprather (Ben S. Prather)
Repository: https://github.com/AFD-Illinois/iharm3d/
Version: v3.6-alpha
Editor: @eloisabentivegna
Reviewer: @bgiacoma, @cpalenzuela
Archive: 10.5281/zenodo.5567372

⚠️ JOSS reduced service mode ⚠️

Due to the challenges of the COVID-19 pandemic, JOSS is currently operating in a "reduced service mode". You can read more about what that means in our blog post.

Status

Status badge code:

HTML: <a href="https://joss.theoj.org/papers/c1516163ae242984700ebc6e3044f3f5"><img src="https://joss.theoj.org/papers/c1516163ae242984700ebc6e3044f3f5/status.svg"></a>
Markdown: [![status](https://joss.theoj.org/papers/c1516163ae242984700ebc6e3044f3f5/status.svg)](https://joss.theoj.org/papers/c1516163ae242984700ebc6e3044f3f5)

Reviewers and authors:

Please avoid lengthy details of difficulties in the review thread. Instead, please create a new issue in the target repository and link to those issues (especially acceptance-blockers) by leaving comments in the review thread below. (For completists: if the target issue tracker is also on GitHub, linking the review thread in the issue or vice versa will create corresponding breadcrumb trails in the link target.)

Reviewer instructions & questions

@bgiacoma & @cpalenzuela, please carry out your review in this issue by updating the checklist below. If you cannot edit the checklist please:

1. Make sure you're logged in to your GitHub account
2. Be sure to accept the invite at this URL: https://github.com/openjournals/joss-reviews/invitations

The reviewer guidelines are available here: https://joss.readthedocs.io/en/latest/reviewer_guidelines.html. Any questions/concerns please let @eloisabentivegna know.

✨ Please start on your review when you are able, and be sure to complete your review in the next six weeks, at the very latest ✨

Review checklist for @bgiacoma

Conflict of interest

• I confirm that I have read the JOSS conflict of interest (COI) policy and that: I have no COIs with reviewing this work or that any perceived COIs have been waived by JOSS for the purpose of this review.

Code of Conduct

• I confirm that I read and will adhere to the JOSS code of conduct.

General checks

• Repository: Is the source code for this software available at the repository url?
• License: Does the repository contain a plain-text LICENSE file with the contents of an OSI approved software license?
• Contribution and authorship: Has the submitting author (@bprather) made major contributions to the software? Does the full list of paper authors seem appropriate and complete?
• Substantial scholarly effort: Does this submission meet the scope eligibility described in the JOSS guidelines

Functionality

• Installation: Does installation proceed as outlined in the documentation?
• Functionality: Have the functional claims of the software been confirmed?
• Performance: If there are any performance claims of the software, have they been confirmed? (If there are no claims, please check off this item.)

Documentation

• A statement of need: Do the authors clearly state what problems the software is designed to solve and who the target audience is?
• Installation instructions: Is there a clearly-stated list of dependencies? Ideally these should be handled with an automated package management solution.
• Example usage: Do the authors include examples of how to use the software (ideally to solve real-world analysis problems).
• Functionality documentation: Is the core functionality of the software documented to a satisfactory level (e.g., API method documentation)?
• Automated tests: Are there automated tests or manual steps described so that the functionality of the software can be verified?
• Community guidelines: Are there clear guidelines for third parties wishing to 1) Contribute to the software 2) Report issues or problems with the software 3) Seek support

Software paper

• Summary: Has a clear description of the high-level functionality and purpose of the software for a diverse, non-specialist audience been provided?
• A statement of need: Does the paper have a section titled 'Statement of Need' that clearly states what problems the software is designed to solve and who the target audience is?
• State of the field: Do the authors describe how this software compares to other commonly-used packages?
• Quality of writing: Is the paper well written (i.e., it does not require editing for structure, language, or writing quality)?
• References: Is the list of references complete, and is everything cited appropriately that should be cited (e.g., papers, datasets, software)? Do references in the text use the proper citation syntax?

Review checklist for @cpalenzuela

Conflict of interest

• I confirm that I have read the JOSS conflict of interest (COI) policy and that: I have no COIs with reviewing this work or that any perceived COIs have been waived by JOSS for the purpose of this review.

Code of Conduct

• I confirm that I read and will adhere to the JOSS code of conduct.

General checks

• Repository: Is the source code for this software available at the repository url?
• License: Does the repository contain a plain-text LICENSE file with the contents of an OSI approved software license?
• Contribution and authorship: Has the submitting author (@bprather) made major contributions to the software? Does the full list of paper authors seem appropriate and complete?
• Substantial scholarly effort: Does this submission meet the scope eligibility described in the JOSS guidelines

Functionality

• Installation: Does installation proceed as outlined in the documentation?
• Functionality: Have the functional claims of the software been confirmed?
• Performance: If there are any performance claims of the software, have they been confirmed? (If there are no claims, please check off this item.)

Documentation

• A statement of need: Do the authors clearly state what problems the software is designed to solve and who the target audience is?
• Installation instructions: Is there a clearly-stated list of dependencies? Ideally these should be handled with an automated package management solution.
• Example usage: Do the authors include examples of how to use the software (ideally to solve real-world analysis problems).
• Functionality documentation: Is the core functionality of the software documented to a satisfactory level (e.g., API method documentation)?
• Automated tests: Are there automated tests or manual steps described so that the functionality of the software can be verified?
• Community guidelines: Are there clear guidelines for third parties wishing to 1) Contribute to the software 2) Report issues or problems with the software 3) Seek support

Software paper

• Summary: Has a clear description of the high-level functionality and purpose of the software for a diverse, non-specialist audience been provided?
• A statement of need: Does the paper have a section titled 'Statement of Need' that clearly states what problems the software is designed to solve and who the target audience is?
• State of the field: Do the authors describe how this software compares to other commonly-used packages?
• Quality of writing: Is the paper well written (i.e., it does not require editing for structure, language, or writing quality)?
• References: Is the list of references complete, and is everything cited appropriately that should be cited (e.g., papers, datasets, software)? Do references in the text use the proper citation syntax?

[whedon]

Hello human, I'm @whedon, a robot that can help you with some common editorial tasks. @bgiacoma, @cpalenzuela it looks like you're currently assigned to review this paper 🎉.

⚠️ JOSS reduced service mode ⚠️

Due to the challenges of the COVID-19 pandemic, JOSS is currently operating in a "reduced service mode". You can read more about what that means in our blog post.

⭐ Important ⭐

If you haven't already, you should seriously consider unsubscribing from GitHub notifications for this (https://github.com/openjournals/joss-reviews) repository. As a reviewer, you're probably currently watching this repository which means for GitHub's default behaviour you will receive notifications (emails) for all reviews 😿

To fix this do the following two things:

1. Set yourself as 'Not watching' https://github.com/openjournals/joss-reviews:

2. You may also like to change your default settings for this watching repositories in your GitHub profile here: https://github.com/settings/notifications

For a list of things I can do to help you, just type:

@whedon commands

For example, to regenerate the paper pdf after making changes in the paper's md or bib files, type:

@whedon generate pdf

[whedon]

PDF failed to compile for issue #3336 with the following error:

 Can't find any papers to compile :-(

[whedon]

Software report (experimental):

github.com/AlDanial/cloc v 1.88  T=0.15 s (636.1 files/s, 98817.3 lines/s)
--------------------------------------------------------------------------------
Language                      files          blank        comment           code
--------------------------------------------------------------------------------
C                                30           1139            853           5237
Python                           31            954            756           4029
C/C++ Header                     14            185            343            643
Bourne Shell                     18            196             88            509
make                              1             32             24             91
Markdown                          1             10              0             53
Bourne Again Shell                3             28             35             19
--------------------------------------------------------------------------------
SUM:                             98           2544           2099          10581
--------------------------------------------------------------------------------


Statistical information for the repository 'd4ccdb89ce53cc87d113c86a' was
gathered on 2021/06/05.
The following historical commit information, by author, was found:

Author                     Commits    Insertions      Deletions    % of changes
Ben Prather                    474         62033          36164           80.63
Ben Ryan                        12          8992           6631           12.83
Charles F Gammie                 1             8              0            0.01
Charles F. Gammie                1          5879              0            4.83
George N. Wong                   2             6              4            0.01
George Wong                      4            71             22            0.08
Josh Dolence                     6          1075            341            1.16
Vedant Dhruv                     5            73             21            0.08
cfgammie                         1             2              1            0.00
e-petersen                       1             1              1            0.00
jdolence                         2            60            399            0.38

Below are the number of rows from each author that have survived and are still
intact in the current revision:

Author                     Rows      Stability          Age       % in comments
Ben Prather               11363           18.3         32.5               13.17
Ben Ryan                   2683           29.8         42.3               16.73
George Wong                  74          104.2          7.0                5.41
Vedant Dhruv                 53           72.6          8.3                5.66
cfgammie                      1           50.0         29.7              100.00

[whedon]

👋 @cpalenzuela, please update us on how your review is going (this is an automated reminder).

[whedon]

👋 @bgiacoma, please update us on how your review is going (this is an automated reminder).

[bgiacoma]

@whedon generate pdf

[whedon]

PDF failed to compile for issue #3336 with the following error:

 Can't find any papers to compile :-(

[bgiacoma]

@eloisabentivegna I cannot find the pdf of the paper. Sorry, I'm new to this system.

[danielskatz]

@whedon generate pdf from branch joss-paper-branch

FYI, @eloisabentivegna - when you start a review, its a good idea to check to make sure there's a paper produced, and if the source is in a branch, do this manually...

[whedon]

Attempting PDF compilation from custom branch joss-paper-branch. Reticulating splines etc...

[whedon]

👉📄 Download article proof 📄 View article proof on GitHub 📄 👈

[bgiacoma]

@danielskatz thanks!

[bgiacoma]

@bprather I added an issue about the README file on the github repository of the code
AFD-Illinois/iharm3d#30 (comment)

[eloisabentivegna]

FYI, @eloisabentivegna - when you start a review, its a good idea to check to make sure there's a paper produced, and if the source is in a branch, do this manually...

Thanks, Dan! I usually do this in pre-review (although you beat me to it there too this time... :) ). Is it better to do it (or do it again) in the review issue?

[danielskatz]

you have to do it again in the review issue, or else the reviewers won't be able to find the paper...

[eloisabentivegna]

@cpalenzuela, do you have everything you need to start your review?

[cpalenzuela]

@eloisabentivegna sorry, I was busy until today with some visitors. I started to download and compile the code, but I am not familiar with github so I am slowly getting there

[cpalenzuela]

@eloisabentivegna I cannot edit my review checklist . I made sure I am logged in, but in the step 2 (Be sure to accept the invite at this URL: https://github.com/openjournals/joss-reviews/invitations) I got the following error message "Sorry, we couldn't find that repository invitation. It is possible that the invitation was revoked or that you are not logged into the invited account."

[danielskatz]

@whedon re-invite @cpalenzuela as reviewer

the invitations expire after 14 days...

[whedon]

OK, the reviewer has been re-invited.

@cpalenzuela please accept the invite by clicking this link: https://github.com/openjournals/joss-reviews/invitations

[arfon]

@bprather – I'm going to take over editing this submission as @eloisabentivegna is heading out on maternity leave soon.

[arfon]

@whedon remind @arfon in two weeks

[whedon]

Attempting PDF compilation from custom branch joss-paper-branch. Reticulating splines etc...

[whedon]

👉📄 Download article proof 📄 View article proof on GitHub 📄 👈

[cpalenzuela]

@bprather I can now compile and run all the tests locally, but I can not still run in the Mare Nostrum cluster, so I can not check the weak/strong scaling performance. Either I am doing something very wrong, or the code is not portable easily to other clusters/infrastructures. Maybe few more details about how to compile and run in other clusters than TACC would be useful.

[bgiacoma]

@bprather thanks for sending the new version of the paper. It looks good to me and I do not have further comments.

[bprather]

@cpalenzuela Sorry it's proven so difficult to get working -- as it was written specifically for KNL originally, easy portability wasn't our top-tier concern for iharm3D, as this review has highlighted (and helped us to address somewhat).
Part of the reason we don't have a generic cluster tutorial, though, is because everywhere we've tried iharm3D has proven so different. MPI stacks seem to come in as many flavors as there are clusters, so while we've gotten it running on everything from a Raspberry Pi to OLCF Summit, every compile has proven at least a little unique. If you open an issue at the repo with some details of the problem you're seeing, I'd be happy to help try to fix this.

[cpalenzuela]

@bprather I have been able to perform the scaling test and they look reasonable. I do not have any further issues with the code or with the paper.

[bprather]

It sounds like this can conclude the review -- the remaining checklist item was discussed above I think?

@arfon is there anything further I can do to move this forward?

[arfon]

@bprather – At this point could you make a new release of this software that includes the changes that have resulted from this review. Then, please make an archive of the software in Zenodo/figshare/other service and update this thread with the DOI of the archive? For the Zenodo/figshare archive, please make sure that:

• The title of the archive is the same as the JOSS paper title
• That the authors of the archive are the same as the JOSS paper authors

I can then move forward with accepting the submission.

[bprather]

I've cut a release and uploaded the source to Zenodo here. The archive DOI is 10.5281/zenodo.5567372

[arfon]

@whedon set 10.5281/zenodo.5567372 as archive

[whedon]

OK. 10.5281/zenodo.5567372 is the archive.

[arfon]

@whedon recommend-accept from branch joss-paper-branch

[whedon]

Attempting dry run of processing paper acceptance...

[whedon]

Reference check summary (note 'MISSING' DOIs are suggestions that need verification):

OK DOIs

- 10.3847/1538-4365/ab29fd is OK
- 10.12942/lrr-2013-1 is OK
- 10.1093/mnras/sts632 is OK
- 10.1093/mnras/stt2479 is OK
- 10.1007/BF00165988 is OK
- 10.1093/mnras/stw1526 is OK
- 10.3847/2041-8213/ab1141 is OK
- 10.3847/2041-8213/ab0f43 is OK
- 10.3847/2041-8213/ab0e85 is OK
- 10.3847/2041-8213/ab0c57 is OK
- 10.3847/2041-8213/ab0c96 is OK
- 10.3847/2041-8213/ab0ec7 is OK
- 10.1146/annurev-astro-082812-141003 is OK
- 10.1093/mnras/stx3162 is OK
- 10.1086/500349 is OK
- 10.1093/mnras/208.4.721 is OK
- 10.3847/1538-4357/aa6a52 is OK
- 10.1006/jcph.2000.6519 is OK
- 10.3847/0067-0049/225/2/22 is OK
- 10.1186/s40668-017-0020-2 is OK
- 10.1093/mnras/stx364 is OK
- 10.1086/374594 is OK
- 10.1086/422244 is OK
- 10.1086/154565 is OK
- 10.1088/0004-637X/807/1/31 is OK
- 10.1111/j.1365-2966.2012.22002.x is OK
- 10.1126/sciadv.aaz1310 is OK
- 10.1111/j.1745-3933.2010.00958.x is OK
- 10.3847/1538-4357/ab86ac is OK
- 10.1088/0264-9381/32/17/175009 is OK
- 10.3847/0004-637X/822/1/34 is OK
- 10.1051/0004-6361/201526630 is OK
- 10.1088/0004-637X/752/2/123 is OK
- 10.1088/0004-637X/701/2/1175 is OK
- 10.1007/BF00642237 is OK
- 10.1088/0264-9381/24/12/S17 is OK
- 10.1086/176343 is OK
- 10.1093/mnras/staa826 is OK
- 10.1007/s41115-017-0002-8 is OK
- 10.1093/pasj/55.6.L69 is OK
- 10.1016/j.jpdc.2014.07.003 is OK
- 10.1086/170270 is OK
- 10.1146/annurev.astro.34.1.703 is OK
- 10.1146/annurev.aa.33.090195.002445 is OK
- 10.1146/annurev.aa.19.090181.001033 is OK
- 10.3847/2041-8213/aaa756 is OK
- 10.1088/0264-9381/31/6/065013 is OK
- 10.1088/0004-637X/692/1/411 is OK
- 10.1088/0004-637X/755/1/51 is OK
- 10.1016/j.jcp.2003.09.016 is OK
- 10.1086/308344 is OK
- 10.1086/497294 is OK
- 10.1088/0004-637X/796/1/22 is OK
- 10.1088/0067-0049/201/2/9 is OK
- 10.1093/mnras/stv2084 is OK
- 10.1093/mnras/staa955 is OK
- 10.1073/pnas.1812491116 is OK
- 10.3847/1538-4357/ab0c0c is OK
- 10.1093/mnras/staa2692 is OK
- 10.1109/TPDS.2020.3010016 is OK
- 10.3847/1538-4357/abee13 is OK
- 10.3847/1538-4365/ab929b is OK
- 10.1088/1361-6382/ab8be8 is OK
- 10.1088/1361-6382/abebb7 is OK
- 10.1088/0264-9381/31/1/015005 is OK
- 10.1093/mnras/stx2530 is OK
- 10.3847/1538-4357/aa9380 is OK
- 10.1086/520800 is OK
- 10.3847/2041-8213/ab0ec7 is OK
- 10.3847/2041-8213/ab0c96 is OK
- 10.3847/2041-8213/ab0c57 is OK
- 10.3847/2041-8213/ab0e85 is OK
- 10.3847/2041-8213/ab0f43 is OK
- 10.3847/2041-8213/ab1141 is OK
- 10.3847/2041-8213/abe4de is OK
- 10.3847/1538-4357/ab96c6 is OK
- 10.3847/2041-8213/abe71d is OK
- 10.3847/1538-4357/abac0d is OK

MISSING DOIs

- None

INVALID DOIs

- None

[whedon]

👋 @openjournals/joss-eics, this paper is ready to be accepted and published.

Check final proof 👉 openjournals/joss-papers#2670

If the paper PDF and Crossref deposit XML look good in openjournals/joss-papers#2670, then you can now move forward with accepting the submission by compiling again with the flag deposit=true e.g.

@whedon accept deposit=true from branch joss-paper-branch 

[arfon]

@whedon accept deposit=true from branch joss-paper-branch

[whedon]

Doing it live! Attempting automated processing of paper acceptance...

[whedon]

🐦🐦🐦 👉 Tweet for this paper 👈 🐦🐦🐦

[whedon]

🚨🚨🚨 THIS IS NOT A DRILL, YOU HAVE JUST ACCEPTED A PAPER INTO JOSS! 🚨🚨🚨

Here's what you must now do:

0. Check final PDF and Crossref metadata that was deposited 👉 Creating pull request for 10.21105.joss.03336 joss-papers#2671
1. Wait a couple of minutes, then verify that the paper DOI resolves https://doi.org/10.21105/joss.03336
2. If everything looks good, then close this review issue.
3. Party like you just published a paper! 🎉🌈🦄💃👻🤘

Any issues? Notify your editorial technical team...

[arfon]

@bgiacoma, @cpalenzuela – many thanks for your reviews here and to @eloisabentivegna for editing this submission! JOSS relies upon the volunteer effort of people like you and we simply wouldn't be able to do this without you ✨

@bprather – your paper is now accepted and published in JOSS ⚡🚀💥

[whedon]

🎉🎉🎉 Congratulations on your paper acceptance! 🎉🎉🎉

If you would like to include a link to your paper from your README use the following code snippets:

Markdown:
[![DOI](https://joss.theoj.org/papers/10.21105/joss.03336/status.svg)](https://doi.org/10.21105/joss.03336)

HTML:
<a style="border-width:0" href="https://doi.org/10.21105/joss.03336">
  <img src="https://joss.theoj.org/papers/10.21105/joss.03336/status.svg" alt="DOI badge" >
</a>

reStructuredText:
.. image:: https://joss.theoj.org/papers/10.21105/joss.03336/status.svg
   :target: https://doi.org/10.21105/joss.03336

This is how it will look in your documentation:

We need your help!

Journal of Open Source Software is a community-run journal and relies upon volunteer effort. If you'd like to support us please consider doing either one (or both) of the the following:

• Volunteering to review for us sometime in the future. You can add your name to the reviewer list here: https://joss.theoj.org/reviewer-signup.html
• Making a small donation to support our running costs here: https://numfocus.org/donate-to-joss