[REVIEW]: PorousFlow: a multiphysics simulation code for coupled problems in porous media

[whedon]

Submitting author: @WilkAndy (Andy Wilkins)
Repository: https://github.com/idaholab/moose
Version: snapshot-20-10-27
Editor: @jedbrown
Reviewer: @jbrezmorf, @rpodgorney
Archive: 10.5281/zenodo.4071026

Status

Status badge code:

HTML: <a href="https://joss.theoj.org/papers/0778dd9013e04c0ccde9fc6d2469e550"><img src="https://joss.theoj.org/papers/0778dd9013e04c0ccde9fc6d2469e550/status.svg"></a>
Markdown: [![status](https://joss.theoj.org/papers/0778dd9013e04c0ccde9fc6d2469e550/status.svg)](https://joss.theoj.org/papers/0778dd9013e04c0ccde9fc6d2469e550)

Reviewers and authors:

Please avoid lengthy details of difficulties in the review thread. Instead, please create a new issue in the target repository and link to those issues (especially acceptance-blockers) by leaving comments in the review thread below. (For completists: if the target issue tracker is also on GitHub, linking the review thread in the issue or vice versa will create corresponding breadcrumb trails in the link target.)

Reviewer instructions & questions

@jbrezmorf & @rpodgorney, please carry out your review in this issue by updating the checklist below. If you cannot edit the checklist please:

1. Make sure you're logged in to your GitHub account
2. Be sure to accept the invite at this URL: https://github.com/openjournals/joss-reviews/invitations

The reviewer guidelines are available here: https://joss.readthedocs.io/en/latest/reviewer_guidelines.html. Any questions/concerns please let @jedbrown know.

✨ Please try and complete your review in the next two weeks ✨

Review checklist for @jbrezmorf

Conflict of interest

• I confirm that I have read the JOSS conflict of interest (COI) policy and that: I have no COIs with reviewing this work or that any perceived COIs have been waived by JOSS for the purpose of this review.

Code of Conduct

• I confirm that I read and will adhere to the JOSS code of conduct.

General checks

• Repository: Is the source code for this software available at the repository url?
• License: Does the repository contain a plain-text LICENSE file with the contents of an OSI approved software license?
• Contribution and authorship: Has the submitting author (@WilkAndy) made major contributions to the software? Does the full list of paper authors seem appropriate and complete?

Functionality

• Installation: Does installation proceed as outlined in the documentation?
• Functionality: Have the functional claims of the software been confirmed?
• Performance: If there are any performance claims of the software, have they been confirmed? (If there are no claims, please check off this item.)

Documentation

• A statement of need: Do the authors clearly state what problems the software is designed to solve and who the target audience is?
• Installation instructions: Is there a clearly-stated list of dependencies? Ideally these should be handled with an automated package management solution.
• Example usage: Do the authors include examples of how to use the software (ideally to solve real-world analysis problems).
• Functionality documentation: Is the core functionality of the software documented to a satisfactory level (e.g., API method documentation)?
• Automated tests: Are there automated tests or manual steps described so that the functionality of the software can be verified?
• Community guidelines: Are there clear guidelines for third parties wishing to 1) Contribute to the software 2) Report issues or problems with the software 3) Seek support

Software paper

• Summary: Has a clear description of the high-level functionality and purpose of the software for a diverse, non-specialist audience been provided?
• A statement of need: Do the authors clearly state what problems the software is designed to solve and who the target audience is?
• State of the field: Do the authors describe how this software compares to other commonly-used packages?
• Quality of writing: Is the paper well written (i.e., it does not require editing for structure, language, or writing quality)?
• References: Is the list of references complete, and is everything cited appropriately that should be cited (e.g., papers, datasets, software)? Do references in the text use the proper citation syntax?

Review checklist for @rpodgorney

Conflict of interest

• I confirm that I have read the JOSS conflict of interest (COI) policy and that: I have no COIs with reviewing this work or that any perceived COIs have been waived by JOSS for the purpose of this review.

Code of Conduct

• I confirm that I read and will adhere to the JOSS code of conduct.

General checks

• Repository: Is the source code for this software available at the repository url?
• License: Does the repository contain a plain-text LICENSE file with the contents of an OSI approved software license?
• Contribution and authorship: Has the submitting author (@WilkAndy) made major contributions to the software? Does the full list of paper authors seem appropriate and complete?

Functionality

• Installation: Does installation proceed as outlined in the documentation?
• Functionality: Have the functional claims of the software been confirmed?
• Performance: If there are any performance claims of the software, have they been confirmed? (If there are no claims, please check off this item.)

Documentation

• A statement of need: Do the authors clearly state what problems the software is designed to solve and who the target audience is?
• Installation instructions: Is there a clearly-stated list of dependencies? Ideally these should be handled with an automated package management solution.
• Example usage: Do the authors include examples of how to use the software (ideally to solve real-world analysis problems).
• Functionality documentation: Is the core functionality of the software documented to a satisfactory level (e.g., API method documentation)?
• Automated tests: Are there automated tests or manual steps described so that the functionality of the software can be verified?
• Community guidelines: Are there clear guidelines for third parties wishing to 1) Contribute to the software 2) Report issues or problems with the software 3) Seek support

Software paper

• Summary: Has a clear description of the high-level functionality and purpose of the software for a diverse, non-specialist audience been provided?
• A statement of need: Do the authors clearly state what problems the software is designed to solve and who the target audience is?
• State of the field: Do the authors describe how this software compares to other commonly-used packages?
• Quality of writing: Is the paper well written (i.e., it does not require editing for structure, language, or writing quality)?
• References: Is the list of references complete, and is everything cited appropriately that should be cited (e.g., papers, datasets, software)? Do references in the text use the proper citation syntax?

[whedon]

Hello human, I'm @whedon, a robot that can help you with some common editorial tasks. @jbrezmorf, @rpodgorney it looks like you're currently assigned to review this paper 🎉.

⭐ Important ⭐

If you haven't already, you should seriously consider unsubscribing from GitHub notifications for this (https://github.com/openjournals/joss-reviews) repository. As a reviewer, you're probably currently watching this repository which means for GitHub's default behaviour you will receive notifications (emails) for all reviews 😿

To fix this do the following two things:

1. Set yourself as 'Not watching' https://github.com/openjournals/joss-reviews:

2. You may also like to change your default settings for this watching repositories in your GitHub profile here: https://github.com/settings/notifications

For a list of things I can do to help you, just type:

@whedon commands

For example, to regenerate the paper pdf after making changes in the paper's md or bib files, type:

@whedon generate pdf

[whedon]

👉 Check article proof 📄 👈

[jedbrown]

@jbrezmorf, @rpodgorney 👋 Welcome to JOSS and thanks for agreeing to review!

The comments from @whedon above outline the review process, which takes place in this thread (possibly with issues filed in the MOOSE repository, which is where PorousFlow resides). I'll be watching this thread if you have any questions.

The JOSS review is different from most other journals. Our goal is to work with the authors to help them meet our criteria instead of merely passing judgment on the submission. As such, the reviewers are encouraged to submit issues and pull requests on the software repository. When doing so, please mention #2176 so that a link is created to this thread (and I can keep an eye on what is happening). Please also feel free to comment and ask questions on this thread. In my experience, it is better to post comments/questions/suggestions as you come across them instead of waiting until you've reviewed the entire package.

We aim for reviews to be completed within a month or so. Please let me know if you require some more time. We can also use Whedon (our bot) to set automatic reminders if you know you'll be away for a known period of time.

Please feel free to ping me (@jedbrown) if you have any questions/concerns.

[jbrezmorf]

@jedbrown Hi Jed. Trying to proceed with the review I realized that my JOSS invitation has expired and I can not fill the checklist. Sorry, I didn't notice the invitation before.

[arfon]

@whedon re-invite @jbrezmorf as reviewer

[whedon]

OK, the reviewer has been re-invited.

@jbrezmorf please accept the invite by clicking this link: https://github.com/openjournals/joss-reviews/invitations

[arfon]

@jedbrown Hi Jed. Trying to proceed with the review I realized that my JOSS invitation has expired and I can not fill the checklist. Sorry, I didn't notice the invitation before.

☝️ You should be re-invited now.

[jbrezmorf]

Software:

no major issues, some tips fo improvements in
the issue.

• fully coupled THM multiphase
• use generic Newton method for solving the system of equations
• claimed implementation of all types of THM relationships
• supported by detailed documentation both for usage as for used numerical methods

Summary:

well written, a typographic error in Acknowledgements: "Philipp Sch{”a}dle."

[arfon]

👋 @rpodgorney - today we reopened JOSS for new submissions and are checking in on our existing reviews. Do you think you might be able to wrap up your review in the next 2-3 weeks?

[rpodgorney]

Sorry for letting this slip. All in all no issues at all, the code, tests, and documentation are al superb.

The paper is well written and contains just the right amount of details without becoming overly verbose.

[rpodgorney]

@arfon this is my first review for JOSS. I've looked through the code, documentation, and tests -- and feel it is OK to publish. Other than the checklist, is there anything else I need to do to complete/document my approval?

[arfon]

@arfon this is my first review for JOSS. I've looked through the code, documentation, and tests -- and feel it is OK to publish. Other than the checklist, is there anything else I need to do to complete/document my approval?

Nope, that's it. Over to @jedbrown to decide on next steps.

[jedbrown]

@whedon generate pdf

[whedon]

👉 Check article proof 📄 👈

[jedbrown]

Sorry about my delay. This is looking great.

1. There is a period missing after the Lime reference (end of paragraph).
2. "simply solve simplistic" -> "only solve simplistic" or some other phrasing.
3. Could you make any comments in the comparison with other software, perhaps in terms of capability, performance, and easy of use/extension?
4. You're welcome to include a figure if you'd like.
5. Would it make sense to write down the basic equations that are solved? Perhaps one or two lines for each of THMC just to get the structural gist across, or a figure representing the role and coupling. (It's sort of abrupt to launch directly into prior applications).
6. Can you double-check capitalization in your references? Our toolchain uses pandoc-citeproc, which is based on apa.csl. Its semantics are closer to biblatex than bibtex, which is required to be precisely APA-conforming. Anyway, there are extra braces needed, e.g., {I}celand or {Iceland} are needed.

[WilkAndy]

Thank you @jedbrown for all those comments. We'll get onto them next week.

[jedbrown]

@whedon check references

[whedon]

Reference check summary:

OK DOIs

- 10.1007/s12665-012-1546-x is OK
- 10.1007/s00366-006-0049-3 is OK
- 10.1080/22020586.2019.12073198 is OK

MISSING DOIs

- https://doi.org/10.1016/j.softx.2020.100430 may be missing for title: MOOSE: Enabling Massively Parallel Multiphysics Simulation
- https://doi.org/10.21203/rs.2.17644/v1 may be missing for title: Impact of Effective Normal Stress on Capillary Pressure in a Single Natural Fracture

INVALID DOIs

- None

[WilkAndy]

Hi @jedbrown . I've just submitted a PullRequest to MOOSE for the changes you request above. Perhaps you'd like to review the changes there? idaholab/moose#15540

• Regarding the equations: I've included some, although we wanted this paper to be readable by a more general audience, so have included the equations towards the end of the paper.

• Regarding comparisons with other codes: I've included another paragraph that discusses some of the benefits of using MOOSE+PorousFlow. It's really hard to write this sort of thing without appearing to denigrate other very worthy codes!

• Done the DOIs mentioned above

[WilkAndy]

The above changes have now been merged into the next branch of the MOOSE repo. Note the branch - at some stage we changed whedon's branch to joss_porous_flow , but that has not actually been used for all the editorial+reviewer changes. So the joss_porous_flow branch should not be used for our JOSS paper: instead use the next branch.

Thanks!

andy

[jedbrown]

@whedon generate pdf from branch next

[whedon]

Attempting PDF compilation from custom branch next. Reticulating splines etc...

[WilkAndy]

Yep, @arfon, the DOI is correct, and i have added to paper.bib. We are now waiting for the updates (idaholab/moose#16313) to pass through the MOOSE review process and then we should be ready for publication. Thank you everyone who contributed to this....

[arfon]

@whedon accept from branch joss_bib_16312

[whedon]

Attempting dry run of processing paper acceptance...

[whedon]

👋 @openjournals/joss-eics, this paper is ready to be accepted and published.

Check final proof 👉 openjournals/joss-papers#1943

If the paper PDF and Crossref deposit XML look good in openjournals/joss-papers#1943, then you can now move forward with accepting the submission by compiling again with the flag deposit=true e.g.

@whedon accept deposit=true from branch joss_bib_16312 

[whedon]

Reference check summary (note 'MISSING' DOIs are suggestions that need verification):

OK DOIs

- 10.1007/s12665-012-1546-x is OK
- 10.1016/j.camwa.2020.05.014 is OK
- 10.1016/j.softx.2020.100430 is OK
- 10.1007/s00366-006-0049-3 is OK
- 10.21203/rs.2.17644/v1 is OK
- 10.1080/22020586.2019.12073198 is OK

MISSING DOIs

- 10.2139/ssrn.3365603 may be a valid DOI for title: Community Code for Simulating CO_2 Storage: Modelling Multiphase Flow with Coupled Geomechanics and Geochemistry Using the Open-Source Multiphysics Framework MOOSE

INVALID DOIs

- None

[WilkAndy]

No, the above is still the old version. It is not from https://github.com/WilkAndy/moose/tree/joss_bib_16312 . I'm happy to wait till the bib changes make their way through the review process at idaholab/moose#16313

[arfon]

No, the above is still the old version. It is not from https://github.com/WilkAndy/moose/tree/joss_bib_16312 . I'm happy to wait till the bib changes make their way through the review process at idaholab/moose#16313

Ah, OK. Sorry for being slow here - @whedon doesn't know how to build from a fork branch so yes, we'll need to wait for these changes (idaholab/moose#16313) to land in https://github.com/idaholab/moose.

But then I think you already knew that 😸

[jedbrown]

@whedon generate pdf

[whedon]

👉📄 Download article proof 📄 View article proof on GitHub 📄 👈

[arfon]

@whedon accept from branch next

[whedon]

Attempting dry run of processing paper acceptance...

[whedon]

Reference check summary (note 'MISSING' DOIs are suggestions that need verification):

OK DOIs

- 10.1007/s12665-012-1546-x is OK
- 10.1016/j.camwa.2020.05.014 is OK
- 10.1016/j.softx.2020.100430 is OK
- 10.1007/s00366-006-0049-3 is OK
- 10.2139/ssrn.3365603 is OK
- 10.21203/rs.2.17644/v1 is OK
- 10.1080/22020586.2019.12073198 is OK

MISSING DOIs

- None

INVALID DOIs

- None

[whedon]

👋 @openjournals/joss-eics, this paper is ready to be accepted and published.

Check final proof 👉 openjournals/joss-papers#1944

If the paper PDF and Crossref deposit XML look good in openjournals/joss-papers#1944, then you can now move forward with accepting the submission by compiling again with the flag deposit=true e.g.

@whedon accept deposit=true from branch next 

[arfon]

@jedbrown - this looks in good shape to publish now, would you agree?

[jedbrown]

Looks good to me. Thanks, @arfon.

[WilkAndy]

I concur - the paper looks as it should. Thank you everyone who was involved in the review process

[WilkAndy]

@whedon accept deposit=true from branch next

[whedon]

I'm sorry @WilkAndy, I'm afraid I can't do that. That's something only editor-in-chiefs are allowed to do.

[arfon]

@whedon accept deposit=true from branch next

[whedon]

Doing it live! Attempting automated processing of paper acceptance...

[whedon]

🐦🐦🐦 👉 Tweet for this paper 👈 🐦🐦🐦

[whedon]

🚨🚨🚨 THIS IS NOT A DRILL, YOU HAVE JUST ACCEPTED A PAPER INTO JOSS! 🚨🚨🚨

Here's what you must now do:

0. Check final PDF and Crossref metadata that was deposited 👉 Creating pull request for 10.21105.joss.02176 joss-papers#1945
1. Wait a couple of minutes to verify that the paper DOI resolves https://doi.org/10.21105/joss.02176
2. If everything looks good, then close this review issue.
3. Party like you just published a paper! 🎉🌈🦄💃👻🤘

Any issues? Notify your editorial technical team...

[arfon]

@jbrezmorf, @rpodgorney - many thanks for your reviews here and to @jedbrown for editing ✨

@WilkAndy - your paper is now accepted into JOSS ⚡🚀💥

[whedon]

🎉🎉🎉 Congratulations on your paper acceptance! 🎉🎉🎉

If you would like to include a link to your paper from your README use the following code snippets:

Markdown:
[![DOI](https://joss.theoj.org/papers/10.21105/joss.02176/status.svg)](https://doi.org/10.21105/joss.02176)

HTML:
<a style="border-width:0" href="https://doi.org/10.21105/joss.02176">
  <img src="https://joss.theoj.org/papers/10.21105/joss.02176/status.svg" alt="DOI badge" >
</a>

reStructuredText:
.. image:: https://joss.theoj.org/papers/10.21105/joss.02176/status.svg
   :target: https://doi.org/10.21105/joss.02176

This is how it will look in your documentation:

We need your help!

Journal of Open Source Software is a community-run journal and relies upon volunteer effort. If you'd like to support us please consider doing either one (or both) of the the following:

• Volunteering to review for us sometime in the future. You can add your name to the reviewer list here: https://joss.theoj.org/reviewer-signup.html
• Making a small donation to support our running costs here: https://numfocus.org/donate-to-joss