[REVIEW]: otoole: OSeMOSYS Tools for Energy Work

[editorialbot]

Submitting author: @trevorb1 (Trevor Barnes)
Repository: https://github.com/OSeMOSYS/otoole
Branch with paper.md (empty if default branch): joss
Version: v1.1.2
Editor: @fraukewiese
Reviewers: @olejandro, @fneum
Archive: 10.5281/zenodo.10360538

Status

Status badge code:

HTML: <a href="https://joss.theoj.org/papers/e93a191ae795b171beff782a68fdc467"><img src="https://joss.theoj.org/papers/e93a191ae795b171beff782a68fdc467/status.svg"></a>
Markdown: [![status](https://joss.theoj.org/papers/e93a191ae795b171beff782a68fdc467/status.svg)](https://joss.theoj.org/papers/e93a191ae795b171beff782a68fdc467)

Reviewers and authors:

Please avoid lengthy details of difficulties in the review thread. Instead, please create a new issue in the target repository and link to those issues (especially acceptance-blockers) by leaving comments in the review thread below. (For completists: if the target issue tracker is also on GitHub, linking the review thread in the issue or vice versa will create corresponding breadcrumb trails in the link target.)

Reviewer instructions & questions

@olejandro & @fneum, your review will be checklist based. Each of you will have a separate checklist that you should update when carrying out your review.
First of all you need to run this command in a separate comment to create the checklist:

@editorialbot generate my checklist

The reviewer guidelines are available here: https://joss.readthedocs.io/en/latest/reviewer_guidelines.html. Any questions/concerns please let @fraukewiese know.

✨ Please start on your review when you are able, and be sure to complete your review in the next six weeks, at the very latest ✨

Checklists

📝 Checklist for @olejandro

📝 Checklist for @fneum

[editorialbot]

Hello humans, I'm @editorialbot, a robot that can help you with some common editorial tasks.

For a list of things I can do to help you, just type:

@editorialbot commands

For example, to regenerate the paper pdf after making changes in the paper's md or bib files, type:

@editorialbot generate pdf

[editorialbot]

Software report:

github.com/AlDanial/cloc v 1.88  T=0.15 s (413.7 files/s, 79698.9 lines/s)
-------------------------------------------------------------------------------
Language                     files          blank        comment           code
-------------------------------------------------------------------------------
Python                          29           1496           1708           5461
YAML                            11             38             36           1077
reStructuredText                12            365            183            984
TeX                              1             10              0            132
INI                              1             11              0             73
Markdown                         2             25              0             55
JSON                             1              0              0             26
Bourne Shell                     2              6              4             24
make                             1              6              8             15
TOML                             1              1              3              5
-------------------------------------------------------------------------------
SUM:                            61           1958           1942           7852
-------------------------------------------------------------------------------


gitinspector failed to run statistical information for the repository

[editorialbot]

Wordcount for paper.md is 977

[editorialbot]

Reference check summary (note 'MISSING' DOIs are suggestions that need verification):

OK DOIs

- 10.1016/j.enpol.2011.06.033 is OK
- 10.12688/openreseurope.15461.1 is OK
- 10.1038/s41597-022-01737-0 is OK
- 10.1016/j.enpol.2011.09.039 is OK
- 10.1007/s12532-017-0130-5 is OK
- 10.5281/zenodo.6522795 is OK
- 10.1016/j.esr.2021.100650 is OK
- 10.1016/j.envsci.2022.07.007 is OK

MISSING DOIs

- None

INVALID DOIs

- None

[editorialbot]

👉📄 Download article proof 📄 View article proof on GitHub 📄 👈

[olejandro]

Review checklist for @olejandro

Conflict of interest

• I confirm that I have read the JOSS conflict of interest (COI) policy and that: I have no COIs with reviewing this work or that any perceived COIs have been waived by JOSS for the purpose of this review.

Code of Conduct

• I confirm that I read and will adhere to the JOSS code of conduct.

General checks

• Repository: Is the source code for this software available at the https://github.com/OSeMOSYS/otoole?
• License: Does the repository contain a plain-text LICENSE file with the contents of an OSI approved software license?
• Contribution and authorship: Has the submitting author (@trevorb1) made major contributions to the software? Does the full list of paper authors seem appropriate and complete?
• Substantial scholarly effort: Does this submission meet the scope eligibility described in the JOSS guidelines
• Data sharing: If the paper contains original data, data are accessible to the reviewers. If the paper contains no original data, please check this item.
• Reproducibility: If the paper contains original results, results are entirely reproducible by reviewers. If the paper contains no original results, please check this item.
• Human and animal research: If the paper contains original data research on humans subjects or animals, does it comply with JOSS's human participants research policy and/or animal research policy? If the paper contains no such data, please check this item.

Functionality

• Installation: Does installation proceed as outlined in the documentation?
• Functionality: Have the functional claims of the software been confirmed?
• Performance: If there are any performance claims of the software, have they been confirmed? (If there are no claims, please check off this item.)

Documentation

• A statement of need: Do the authors clearly state what problems the software is designed to solve and who the target audience is?
• Installation instructions: Is there a clearly-stated list of dependencies? Ideally these should be handled with an automated package management solution.
• Example usage: Do the authors include examples of how to use the software (ideally to solve real-world analysis problems).
• Functionality documentation: Is the core functionality of the software documented to a satisfactory level (e.g., API method documentation)?
• Automated tests: Are there automated tests or manual steps described so that the functionality of the software can be verified?
• Community guidelines: Are there clear guidelines for third parties wishing to 1) Contribute to the software 2) Report issues or problems with the software 3) Seek support

Software paper

• Summary: Has a clear description of the high-level functionality and purpose of the software for a diverse, non-specialist audience been provided?
• A statement of need: Does the paper have a section titled 'Statement of need' that clearly states what problems the software is designed to solve, who the target audience is, and its relation to other work?
• State of the field: Do the authors describe how this software compares to other commonly-used packages?
• Quality of writing: Is the paper well written (i.e., it does not require editing for structure, language, or writing quality)?
• References: Is the list of references complete, and is everything cited appropriately that should be cited (e.g., papers, datasets, software)? Do references in the text use the proper citation syntax?

[fneum]

Review checklist for @fneum

Conflict of interest

• I confirm that I have read the JOSS conflict of interest (COI) policy and that: I have no COIs with reviewing this work or that any perceived COIs have been waived by JOSS for the purpose of this review.

Code of Conduct

• I confirm that I read and will adhere to the JOSS code of conduct.

General checks

• Repository: Is the source code for this software available at the https://github.com/OSeMOSYS/otoole?
• License: Does the repository contain a plain-text LICENSE file with the contents of an OSI approved software license?
• Contribution and authorship: Has the submitting author (@trevorb1) made major contributions to the software? Does the full list of paper authors seem appropriate and complete?
• Substantial scholarly effort: Does this submission meet the scope eligibility described in the JOSS guidelines
• Data sharing: If the paper contains original data, data are accessible to the reviewers. If the paper contains no original data, please check this item.
• Reproducibility: If the paper contains original results, results are entirely reproducible by reviewers. If the paper contains no original results, please check this item.
• Human and animal research: If the paper contains original data research on humans subjects or animals, does it comply with JOSS's human participants research policy and/or animal research policy? If the paper contains no such data, please check this item.

Functionality

• Installation: Does installation proceed as outlined in the documentation?
• Functionality: Have the functional claims of the software been confirmed?
• Performance: If there are any performance claims of the software, have they been confirmed? (If there are no claims, please check off this item.)

Documentation

• A statement of need: Do the authors clearly state what problems the software is designed to solve and who the target audience is?
• Installation instructions: Is there a clearly-stated list of dependencies? Ideally these should be handled with an automated package management solution.
• Example usage: Do the authors include examples of how to use the software (ideally to solve real-world analysis problems).
• Functionality documentation: Is the core functionality of the software documented to a satisfactory level (e.g., API method documentation)?
• Automated tests: Are there automated tests or manual steps described so that the functionality of the software can be verified?
• Community guidelines: Are there clear guidelines for third parties wishing to 1) Contribute to the software 2) Report issues or problems with the software 3) Seek support

Software paper

• Summary: Has a clear description of the high-level functionality and purpose of the software for a diverse, non-specialist audience been provided?
• A statement of need: Does the paper have a section titled 'Statement of need' that clearly states what problems the software is designed to solve, who the target audience is, and its relation to other work?
• State of the field: Do the authors describe how this software compares to other commonly-used packages?
• Quality of writing: Is the paper well written (i.e., it does not require editing for structure, language, or writing quality)?
• References: Is the list of references complete, and is everything cited appropriately that should be cited (e.g., papers, datasets, software)? Do references in the text use the proper citation syntax?

[fneum]

Excellent! This was fun reviewing.

As you can see, I already ticked all the boxes and only have a couple of questions and suggestions for you.

Paper

• The manuscript covers all the important points.

Repository

• The repository is in excellent condition!

Docs

https://otoole.readthedocs.io/en/latest/functionality.html#otoole-results

• Maybe the transformation of the raw solver output file of CPLEX could be incorporated into otoole?

https://otoole.readthedocs.io/en/latest/data.html#parmaters-foramt

• typo "Parmaters Foramt"

https://otoole.readthedocs.io/en/latest/examples.html

• Do you need both GLPK and CBC for the example or is there a way to do it with GLPK and/or CBC alone?

https://otoole.readthedocs.io/en/latest/examples.html#view-results

• Would it be a good idea to have an otoole clean function to tidy up after a run?

https://otoole.readthedocs.io/en/latest/examples.html#otoole-validate

• Maybe I am missing this, but I did not generate a file data.txt. Hence, I could not execute this part of the examples section.

https://otoole.readthedocs.io/en/latest/examples.html#install-glpk

• You could add installation instructions for Windows (https://winglpk.sourceforge.net/).
• You could add that these solvers can be installed also via conda: conda install -c conda-forge coincbc glpk (not all combinations of operating systems and solvers).

Functionality

• Are there any use cases where otoole is used within a Python script? If yes, more targeted documentation for the frontend Python API should be added besides the Module Reference.

• Is there an easy way to reduce the number of commands required to solve a model? E.g. combining building, solving and interpreting solver output?

[trevorb1]

Thank you so much for taking the time to review and leave feedback for otoole, @fneum! We will address your notes point by point once all reviewers have completed their feedback! :)

[trevorb1]

Hi @fraukewiese, I hope you are having a good summer!

I am just checking in on the review status for this paper/repository, as it has been about 2 months since the issue was created. I think one reviewer has completed their feedback, and we are just waiting on the second one.

We were hoping to do revisions and address comments once all reviews have been submitted to streamline the process. Should we continue to wait for the second reviewer to complete their feedback? Or should we begin addressing the comments from the first fully completed review?

Thanks so much!

[olejandro]

Hi @trevorb1, appologies for the delay on this. I will provide my feedback within about a week.

[fraukewiese]

@trevorb1 : You are very welcome to start adressing the comments from @fneum .

[fraukewiese]

@fneum : Thank you very much for your thorough review!

[fraukewiese]

@olejandro : Thanks for the update and your announcement to provide feedback within a week.

[fraukewiese]

@olejandro : How is the review going? :)

[trevorb1]

Hi @fneum; thank you again for the detailed review of otoole! We really appreciate it. Please find below point-by-point responses to all of your comments. If any response is unclear, needs further explanation/discussion, or requires further attention, please let us know. Thank you!

The manuscript covers all the important points.
Thank you! The manuscript has been updated to reflect all changes addressed in this review (see this commit)

The repository is in excellent condition!
Thank you :)

Maybe the transformation of the raw solver output file of CPLEX could be incorporated into otoole?
Agreed. otoole can now process raw output CPLEX solution files. The change has been implemented in this PR and documented here

typo "Parmaters Foramt"
Thank you for noticing this! This has been corrected in this PR

Do you need both GLPK and CBC for the example or is there a way to do it with GLPK and/or CBC alone?
Good point on being able to use otoole with only one solver, as this may reduce the barrier of entry for new users. Since many OSeMOSYS workflows use GLPK to build the model file, we have added functionality to process GLPK solution files. We do recognize this may be of limited use, due to the poor performance of GLPK with large models, but agree that it can reduce the learning curve for otoole.

The change has been implemented in this PR and documented here

Would it be a good idea to have an otoole clean function to tidy up after a run?
This is a great idea, which we think relates closely to your otoole run idea. We have created a new issue ticket for this, however, have not implemented it.

The current scope of otoole is just the pre and post processing of data. Bridging this gap will require serious development thought on how to implement functions that remain useful to the OSeMOSYS community. Therefore, within the current scope of the project, we leave the removal of intermediate files up to the user. Please note, as elaborated in the issue ticket, in the long term it would be great to have a central configuration file for otoole to handle a wide array of options – including the automatic removal of intermediate files.

Maybe I am missing this, but I did not generate a file data.txt. Hence, I could not execute this part of the examples section.
Thank you for flagging this! The data file was mistakenly not included with the documentation. This has been corrected in this PR, and the validation instructions have been updated here.

You could add [solver] installation instructions for Windows (https://winglpk.sourceforge.net/). You could add that these solvers can be installed also via conda: conda install -c conda-forge coincbc glpk (not all combinations of operating systems and solvers).
Agreed. Solver download instructions have been updated to reflect Windows operating systems and users who use conda. The change has been implemented in this PR and the instructions have been updated here

Are there any use cases where otoole is used within a Python script? If yes, more targeted documentation for the front end Python API should be added besides the Module Reference.
Yes, some users use otoole directly in Python scripts, rather than through the command line. We agree that documenting this process better is beneficial. A public python API has been added to otoole to allow users to convert data, read and write data, and read results without having to know the internal data structures of otoole. Moreover, dedicated examples on how to use the Python API have been added to the documentation website.

The update has been implemented in this PR and documented here

Is there an easy way to reduce the number of commands required to solve a model? E.g. combining building, solving and interpreting solver output?
This is a good thought! Similar to the otoole clean discussion, we view the current scope of otoole to be only the pre and post processing data. As further discussed in this issue, creating a central configuration file where to users can launch model runs, specify solvers, and apply external scripts, is a long-term goal that requires serious development thought outside the current scope of otoole.

[trevorb1]

Hi @fraukewiese. A couple quick questions on the review process:

1. With the changes addressing @fneum's comments, we have now released otoole version 1.1.0. We originally submitted otoole under version 1.0.3. Do we have to indicate this bump in version in anyway?
2. In the paper, we left the date as the original submission date (May 3, 2023). Does this need to be updated as we revise the paper in the review?

Thank you!

[fraukewiese]

@trevorb1 : In the final stage of the review process, before publishing, I will ask you to let me know the updated version number. For the paper, the idea of the original submission date is to indicate when the submission has been done initially, so it is fine to keep 3rd of May.

[fraukewiese]

@fneum : Are you satisfied with the response to your review by @trevorb1 ? Do you have any replies on that? And would you recommend the submission for publication?

[fraukewiese]

@olejandro : How is the review going? :)

[fneum]

I am happy with the revisions and recommend to accept this contribution.

[editorialbot]

👉📄 Download article proof 📄 View article proof on GitHub 📄 👈

[trevorb1]

@editorialbot check references

[editorialbot]

Reference check summary (note 'MISSING' DOIs are suggestions that need verification):

OK DOIs

- 10.1016/j.enpol.2011.06.033 is OK
- 10.12688/openreseurope.15461.1 is OK
- 10.1038/s41597-022-01737-0 is OK
- 10.1016/j.enpol.2011.09.039 is OK
- 10.1007/s12532-017-0130-5 is OK
- 10.5281/zenodo.6522795 is OK
- 10.1016/j.esr.2021.100650 is OK
- 10.1016/j.envsci.2022.07.007 is OK
- 10.1016/j.rser.2018.11.020 is OK
- 10.5334/jors.188 is OK
- 10.21105/joss.00825 is OK
- 10.12688/openreseurope.13633.2 is OK
- 10.1016/j.softx.2021.100967 is OK

MISSING DOIs

- None

INVALID DOIs

- None

[trevorb1]

Hi @fraukewiese! Thank you for your final edits! All points have been addressed in this commit. On your comment for line 75/76, I instead did a slight reword just to make the sentence more clear, instead of adding a comma. Please let me know if the line is still unclear and I will update. Thank you again!

[trevorb1]

Hi @fraukewiese! Please find attached all final information requested:

• Software Version: otoole version v1.1.2 has been released today. See release version on PyPI and on GitHub
• Software Archive: Please see otoole version v1.1.2 archived on Zenodo here
  • Title: "otoole: OSeMOSYS Tools for Energy Work" matches the paper title
  • Authors: Trevor Barnes and Will Usher. Same order as paper
  • DOI: otoole version v1.1.2 Zenodo DOI is 10.5281/zenodo.10360538

If you need any other information from my end, please just let me know. Thank you so much in advance! 😊

[fraukewiese]

@editorialbot set v1.1.2 as version

[editorialbot]

Done! version is now v1.1.2

[fraukewiese]

@editorialbot set 10.5281/zenodo.10360538 as archive

[editorialbot]

Done! archive is now 10.5281/zenodo.10360538

[fraukewiese]

Congratulations @trevorb1 for great work ! And thanks a lot again for the great reviews that substantially improved the submission @fneum and @olejandro

[fraukewiese]

@editorialbot recommend-accept

[editorialbot]

Attempting dry run of processing paper acceptance...

[editorialbot]

Reference check summary (note 'MISSING' DOIs are suggestions that need verification):

OK DOIs

- 10.1016/j.enpol.2011.06.033 is OK
- 10.12688/openreseurope.15461.1 is OK
- 10.1038/s41597-022-01737-0 is OK
- 10.1016/j.enpol.2011.09.039 is OK
- 10.1007/s12532-017-0130-5 is OK
- 10.5281/zenodo.6522795 is OK
- 10.1016/j.esr.2021.100650 is OK
- 10.1016/j.envsci.2022.07.007 is OK
- 10.1016/j.rser.2018.11.020 is OK
- 10.5334/jors.188 is OK
- 10.21105/joss.00825 is OK
- 10.12688/openreseurope.13633.2 is OK
- 10.1016/j.softx.2021.100967 is OK

MISSING DOIs

- None

INVALID DOIs

- None

[editorialbot]

The paper's PDF and metadata files generation produced some warnings that could prevent the final paper from being published. Please fix them before the end of the review process.

\mathsemicolon
              ^
unexpected control sequence \mathsemicolon
expecting "%", "\\label", "\\tag", "\\nonumber" or whitespace

[editorialbot]

👋 @openjournals/pe-eics, this paper is ready to be accepted and published.

Check final proof 👉📄 Download article

If the paper PDF and the deposit XML files look good in openjournals/joss-papers#4856, then you can now move forward with accepting the submission by compiling again with the command @editorialbot accept

[trevorb1]

🎉 Thank you to @fraukewiese for facilitating the review and providing feedback! Thank you to @fneum and @olejandro for reviewing the paper; your comments and suggestions were very helpful! We really appreciate everyones contributions!

[trevorb1]

@fraukewiese I have addressed the \mathsemicolon issue with this commit. That was my mistake as there was a formatting issue with a journal name in the bibliography.

[kyleniemeyer]

@editorialbot generate pdf

[editorialbot]

👉📄 Download article proof 📄 View article proof on GitHub 📄 👈

[kyleniemeyer]

@editorialbot accept

[editorialbot]

Doing it live! Attempting automated processing of paper acceptance...

[editorialbot]

Ensure proper citation by uploading a plain text CITATION.cff file to the default branch of your repository.

If using GitHub, a Cite this repository menu will appear in the About section, containing both APA and BibTeX formats. When exported to Zotero using a browser plugin, Zotero will automatically create an entry using the information contained in the .cff file.

You can copy the contents for your CITATION.cff file here:

CITATION.cff

cff-version: "1.2.0"
authors:
- family-names: Barnes
  given-names: Trevor
  orcid: "https://orcid.org/0000-0003-2458-2968"
- family-names: Usher
  given-names: Will
  orcid: "https://orcid.org/0000-0001-9367-1791"
contact:
- family-names: Barnes
  given-names: Trevor
  orcid: "https://orcid.org/0000-0003-2458-2968"
doi: 10.5281/zenodo.10360538
message: If you use this software, please cite our article in the
  Journal of Open Source Software.
preferred-citation:
  authors:
  - family-names: Barnes
    given-names: Trevor
    orcid: "https://orcid.org/0000-0003-2458-2968"
  - family-names: Usher
    given-names: Will
    orcid: "https://orcid.org/0000-0001-9367-1791"
  date-published: 2023-12-20
  doi: 10.21105/joss.05511
  issn: 2475-9066
  issue: 92
  journal: Journal of Open Source Software
  publisher:
    name: Open Journals
  start: 5511
  title: "otoole: OSeMOSYS Tools for Energy Work"
  type: article
  url: "https://joss.theoj.org/papers/10.21105/joss.05511"
  volume: 8
title: "otoole: OSeMOSYS Tools for Energy Work"

If the repository is not hosted on GitHub, a .cff file can still be uploaded to set your preferred citation. Users will be able to manually copy and paste the citation.

Find more information on .cff files here and here.

[editorialbot]

🐘🐘🐘 👉 Toot for this paper 👈 🐘🐘🐘

[editorialbot]

🚨🚨🚨 THIS IS NOT A DRILL, YOU HAVE JUST ACCEPTED A PAPER INTO JOSS! 🚨🚨🚨

Here's what you must now do:

0. Check final PDF and Crossref metadata that was deposited 👉 Creating pull request for 10.21105.joss.05511 joss-papers#4858
1. Wait five minutes, then verify that the paper DOI resolves https://doi.org/10.21105/joss.05511
2. If everything looks good, then close this review issue.
3. Party like you just published a paper! 🎉🌈🦄💃👻🤘

Any issues? Notify your editorial technical team...

[kyleniemeyer]

Congratulations @trevorb1 on your article's publication in JOSS! Please consider signing up as a reviewer if you haven't already.

Many thanks to @olejandro and @fneum for reviewing this, and @fraukewiese for editing.

[editorialbot]

🎉🎉🎉 Congratulations on your paper acceptance! 🎉🎉🎉

If you would like to include a link to your paper from your README use the following code snippets:

Markdown:
[![DOI](https://joss.theoj.org/papers/10.21105/joss.05511/status.svg)](https://doi.org/10.21105/joss.05511)

HTML:
<a style="border-width:0" href="https://doi.org/10.21105/joss.05511">
  <img src="https://joss.theoj.org/papers/10.21105/joss.05511/status.svg" alt="DOI badge" >
</a>

reStructuredText:
.. image:: https://joss.theoj.org/papers/10.21105/joss.05511/status.svg
   :target: https://doi.org/10.21105/joss.05511

This is how it will look in your documentation:

We need your help!

The Journal of Open Source Software is a community-run journal and relies upon volunteer effort. If you'd like to support us please consider doing either one (or both) of the the following:

• Volunteering to review for us sometime in the future. You can add your name to the reviewer list here: https://reviewers.joss.theoj.org/join
• Making a small donation to support our running costs here: https://numfocus.org/donate-to-joss