[REVIEW]: ODES: a high level interface to ODE and DAE solvers

[whedon]

Submitting author: bmcage (Benny Malengier)
Repository: https://github.com/bmcage/odes/
Version: v2.3.0
Editor: @kyleniemeyer
Reviewer: @arghdos
Archive: 10.5281/zenodo.1183272

Status

Status badge code:

HTML: <a href="http://joss.theoj.org/papers/59d3b2e50db759e4255736236e13193d"><img src="http://joss.theoj.org/papers/59d3b2e50db759e4255736236e13193d/status.svg"></a>
Markdown: [![status](http://joss.theoj.org/papers/59d3b2e50db759e4255736236e13193d/status.svg)](http://joss.theoj.org/papers/59d3b2e50db759e4255736236e13193d)

Reviewers and authors:

Please avoid lengthy details of difficulties in the review thread. Instead, please create a new issue in the target repository and link to those issues (especially acceptance-blockers) in the review thread below. (For completists: if the target issue tracker is also on GitHub, linking the review thread in the issue or vice versa will create corresponding breadcrumb trails in the link target.)

Reviewer questions

Conflict of interest

• As the reviewer I confirm that there are no conflicts of interest for me to review this work (such as being a major contributor to the software).

General checks

• Repository: Is the source code for this software available at the repository url?
• License: Does the repository contain a plain-text LICENSE file with the contents of an OSI approved software license?
• Version: Does the release version given match the GitHub release (v2.3.0)?
• Authorship: Has the submitting author (bmcage) made major contributions to the software?

Functionality

• Installation: Does installation proceed as outlined in the documentation?
• Functionality: Have the functional claims of the software been confirmed?
• Performance: Have any performance claims of the software been confirmed?

Documentation

• A statement of need: Do the authors clearly state what problems the software is designed to solve and who the target audience is?
• Installation instructions: Is there a clearly-stated list of dependencies? Ideally these should be handled with an automated package management solution.
• Example usage: Do the authors include examples of how to use the software (ideally to solve real-world analysis problems).
• Functionality documentation: Is the core functionality of the software documented to a satisfactory level (e.g. API method documentation)?
• Automated tests: Are there automated tests or manual steps described so that the function of the software can be verified?
• Community guidelines: Are there clear guidelines for third parties wishing to 1) Contribute to the software 2) Report issues or problems with the software 3) Seek support

Software paper

• Authors: Does the paper.md file include a list of authors with their affiliations?
• A statement of need: Do the authors clearly state what problems the software is designed to solve and who the target audience is?
• References: Do all archival references that should have a DOI list one (e.g. papers, datasets, software)?

[whedon]

Hello human, I'm @whedon. I'm here to help you with some common editorial tasks for JOSS. @arghdos it looks like you're currently assigned as the reviewer for this paper 🎉.

⭐ Important ⭐

If you haven't already, you should seriously consider unsubscribing from GitHub notifications for this (https://github.com/openjournals/joss-reviews) repository. As as reviewer, you're probably currently watching this repository which means for GitHub's default behaviour you will receive notifications (emails) for all JOSS reviews 😿

To fix this do the following two things:

1. Set yourself as 'Not watching' https://github.com/openjournals/joss-reviews:

2. You may also like to change your default settings for this watching repositories in your GitHub profile here: https://github.com/settings/notifications

For a list of things I can do to help you, just type:

@whedon commands

[skyreflectedinmirrors]

This is an important and useful piece of software, providing an easy to use interface to state-of-the art stiff ODE solvers with many examples, ample documentation and unit tests.

I recommend publishing in JOSS after a few minor revisions have been made:

1. The latest release on the target repo does not match the release number submitted to JOSS issue #45. I suspect this is just a minor error, if @bmcage confirms I think we can update the submitted version to 2.2.1 (or to 2.3.0 on the target repo)
2. As a suggestion, I would put the link to the readthedocs.io documentation more clearly in the read-me (under the Documentation section perhaps?). It is currently present as a badge, but it took a bit of hunting for me to find it.
3. Community guidelines must be added to address the JOSS requirement. issue #46
4. The release version (see above) needs a DOI, e.g. via Zenodo, issue #47
5. As a suggestion, explicit use of the references in the paper.bib (which do appear to be comprehensive) could be made in either the README.md, the paper.md, the readthedocs or all of the above

[kyleniemeyer]

Thanks for your review @arghdos! I will clarify on point 4 (bmcage/odes#47) that we only require the DOI at the end of the review process, after all necessary changes have been made.

@bmcage can you address the above points, and the associated issues @arghdos made in the software repo?

[bmcage]

@kyleniemeyer I'm working against some deadlines at the moment. I'll look at the points at the end of the week, should not be a problem, otherwise next week
We planned a release, but that got delayed, which is the reason for the version mismatch.

[bmcage]

I believe to have fixed up all needed issues.

1. release v2.3.0 done in normal way. Updated procedures to do releases so Zenodo picks them up, then did release v2.3.1 So the version here would best be changed to v2.3.1
2. readthedocs is still barebones at the moment aimed at the most common use. Nevertheless, link is now added to documentation section of the readme
3. community guidelines added
4. a release 2.3.1 done according to Zenodo procedure to have it picked up. DOI was generated

For 5. I need to look at some examples of how references are done. Only a suggestion though

[kyleniemeyer]

Hi @bmcage: thanks for making those changes!

1. Regarding mentioning the references in the paper, I agree the references should be mentioned directly in the text. You can see examples of this in a few past JOSS papers:

• r3d2
• PyGBe

2. In the other text of the paper, I noticed some inconsistent capitalization in things like ODES (vs. odes, Odes). Proper capitalization should also be used with: Python, SciPy, Cython. And, "in stead" should be replaced with "instead".

3. I noticed some of the badges at the top of the README seem to be broken (see screenshot). Can you fix these?

4. I am hesitant to approve the paper when the documentation page is in the early stages. It looks as though all the necessary information is in the README, but it would be more accessible if presented with pages on the ReadTheDocs site. (I do see the API documentation there.) Putting all of the documentation in a single file makes it somewhat difficult to read, particularly without a table of contents.

Is this something you would be willing to do?

[bmcage]

1-3 have been fixed, though a release to PyPI will be needed before the fix is visible in the tags.

For 4, @aragilar is working on cleaning up docstrings, so readthedocs comes out nicer. Start of using readthedocs was added recently and is a work in progress.

[kyleniemeyer]

@bmcage great! I think once that work is done with the docstrings, then we can accept.

[arfon]

How are we getting on here @bmcage?

[aragilar]

I've made some progress on 4, but the main blocker there is working out what to do with cython. I may end up splitting the docs between rtd for the main documentation, and using github pages or similar for the API docs.

[kyleniemeyer]

Hi @aragilar @bmcage just wanted to check in on the progress here.

[bmcage]

I merged the new sphinx doc system. Some other features, not joss related, to be fleshed out before we do a new release, but all is ok to resolve this issue in my opinion.

[kyleniemeyer]

Sorry for the delay on handling this...

@arghdos could you look and see if the changes made satisfy your concerns?

[skyreflectedinmirrors]

@kyleniemeyer @bmcage It definitely looks better, I just have a few comments on broken / missing links and I think it'll be good:

• First, the link to the API level docs in the lower level interfaces section appears to be offline.

• Similarly a number of links (presumably) to the API-docs seem to be missing, e.g.: scikits.odes.odeint.odeint in the User Guide, same with the solver links, e.g. scikits.odes.sundials.cvode in Choosing a Solver

[aragilar]

The links should work once I get doctr set up. I've got a branch where I'm working on it, but I need to work out how best to integrate it with travis, how multiple versions should work etc. I'll post a link to the PR when it's ready.

[aragilar]

The API docs are now online. See https://bmcage.github.io/odes/dev/ for the rendered docs on master.

[kyleniemeyer]

@aragilar thanks!

@arghdos do the changes address your remaining concerns about the docs?

[skyreflectedinmirrors]

@kyleniemeyer looks good to me

[kyleniemeyer]

@whedon generate pdf

[whedon]

Attempting PDF compilation. Reticulating splines etc...

[whedon]

PDF failed to compile for issue #165 with the following error: 

 /app/vendor/bundle/ruby/2.3.0/bundler/gems/whedon-8307502c3aee/lib/whedon/author.rb:36:in `block in build_affiliation_string': Problem with affiliations for Claas Abert, perhaps the affiliations index need quoting? (RuntimeError)
	from /app/vendor/bundle/ruby/2.3.0/bundler/gems/whedon-8307502c3aee/lib/whedon/author.rb:35:in `each'
	from /app/vendor/bundle/ruby/2.3.0/bundler/gems/whedon-8307502c3aee/lib/whedon/author.rb:35:in `build_affiliation_string'
	from /app/vendor/bundle/ruby/2.3.0/bundler/gems/whedon-8307502c3aee/lib/whedon/author.rb:11:in `initialize'
	from /app/vendor/bundle/ruby/2.3.0/bundler/gems/whedon-8307502c3aee/lib/whedon.rb:97:in `new'
	from /app/vendor/bundle/ruby/2.3.0/bundler/gems/whedon-8307502c3aee/lib/whedon.rb:97:in `block in parse_authors'
	from /app/vendor/bundle/ruby/2.3.0/bundler/gems/whedon-8307502c3aee/lib/whedon.rb:95:in `each'
	from /app/vendor/bundle/ruby/2.3.0/bundler/gems/whedon-8307502c3aee/lib/whedon.rb:95:in `parse_authors'
	from /app/vendor/bundle/ruby/2.3.0/bundler/gems/whedon-8307502c3aee/lib/whedon.rb:76:in `initialize'
	from /app/vendor/bundle/ruby/2.3.0/bundler/gems/whedon-8307502c3aee/lib/whedon/processor.rb:26:in `new'
	from /app/vendor/bundle/ruby/2.3.0/bundler/gems/whedon-8307502c3aee/lib/whedon/processor.rb:26:in `set_paper'
	from /app/vendor/bundle/ruby/2.3.0/bundler/gems/whedon-8307502c3aee/bin/whedon:37:in `prepare'
	from /app/vendor/bundle/ruby/2.3.0/gems/thor-0.20.0/lib/thor/command.rb:27:in `run'
	from /app/vendor/bundle/ruby/2.3.0/gems/thor-0.20.0/lib/thor/invocation.rb:126:in `invoke_command'
	from /app/vendor/bundle/ruby/2.3.0/gems/thor-0.20.0/lib/thor.rb:387:in `dispatch'
	from /app/vendor/bundle/ruby/2.3.0/gems/thor-0.20.0/lib/thor/base.rb:466:in `start'
	from /app/vendor/bundle/ruby/2.3.0/bundler/gems/whedon-8307502c3aee/bin/whedon:99:in `<top (required)>'
	from /app/vendor/bundle/ruby/2.3.0/bin/whedon:22:in `load'
	from /app/vendor/bundle/ruby/2.3.0/bin/whedon:22:in `<main>'

[arfon]

@kyleniemeyer - looks like there are some issues with the affiliations in the paper.md.

[aragilar]

@arfon @kyleniemeyer Doing some cross-referencing on github, I think I've got everyone's afflictions now, which should be fixed by bmcage/odes#64, I'm just waiting on confirmation from everyone that their details are correct, I'll ping here when I get confirmation.

[aragilar]

@arfon @kyleniemeyer The affiliations are up to date, and @bmcage is going to do a new release with all the documentation changes. Do you want any changes to the paper or anything before we do the new release?

[arfon]

@whedon generate pdf

[whedon]

Attempting PDF compilation. Reticulating splines etc...

[whedon]

PDF failed to compile for issue #165 with the following error:

% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed

0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0
100 15 0 15 0 0 50 0 --:--:-- --:--:-- --:--:-- 50
[WARNING] Could not convert image '/tmp/tex2pdf.182/43c8ef28fb100441efd0fb80244f979c7859f983.shtml': Cannot load file
Jpeg Invalid marker used
PNG Invalid PNG file, signature broken
Bitmap Invalid Bitmap magic identifier
GIF Invalid Gif signature : <!DOCT
HDR Invalid radiance file signature
Tiff Invalid endian tag value
TGA not enough bytes
Error producing PDF.
! LaTeX Error: Cannot determine size of graphic in /tmp/tex2pdf.182/43c8ef28fb1
00441efd0fb80244f979c7859f983.shtml (no BoundingBox).

See the LaTeX manual or LaTeX Companion for explanation.
Type H for immediate help.
...

l.241 ...28fb100441efd0fb80244f979c7859f983.shtml}

Looks like we failed to compile the PDF

[arfon]

Looks like the image referenced at https://github.com/bmcage/odes/blob/master/docs/ipython/Performance%20tests.png isn't working here. @aragilar Could you take a look at that please? It's probably better to have the PNG right next to the paper.md file.

[aragilar]

I've fixed links which were pointing to the wrong places, and copied across the png (I was going to symlink it, but I'm not sure whether the joss paper builder would support that).

[kyleniemeyer]

@whedon generate pdf

[whedon]

Attempting PDF compilation. Reticulating splines etc...

[whedon]

--> Check article proof 📄 <--

[kyleniemeyer]

@aragilar successful build!

I do see a few issues:

• the in-text reference to and citation for SciPy says "Jones et al. 2001–2001--"
• please capitalize Python everywhere you use it
• near the end, "For user" -> "For users"
• The Hindmarsh et al. reference should include a DOI (assuming one is available)

Please correct those things, and then we should be ready to accept. Once you do that, please archive the final version of the software & associated materials, and provide the DOI here.

[aragilar]

Fixed (the odes textbook lacks a doi apparently, even though there are ebooks...). I don't have access to make a new release, so @bmcage will make a release when he has time, and do the upload to zenodo.

[bmcage]

Release done v2.3.2, with all changes included. https://zenodo.org/record/1183272
There were some internal server errors in Zenodo originally, so had to draft the release a second time on github as v2.3.2b to have Zenodo upload working. DOI not active yet at time of writing this: should be https://doi.org/10.5281/zenodo.1183272

[kyleniemeyer]

@bmcage @aragilar thanks!

[kyleniemeyer]

@arfon this submission is ready to accept.

[arfon]

@whedon set 10.5281/zenodo.1183272 as archive

[whedon]

OK. 10.5281/zenodo.1183272 is the archive.

[arfon]

@arghdos - many thanks for your review here and to @kyleniemeyer for editing this submission ✨

@bmcage @aragilar - your paper is now accepted into JOSS and your DOI is https://doi.org/10.21105/joss.00165 ⚡ 🚀 💥

[whedon]

🎉🎉🎉 Congratulations on your paper acceptance! 🎉🎉🎉

If you would like to include a link to your paper from your README use the following code snippet:

[![DOI](http://joss.theoj.org/papers/10.21105/joss.00165/status.svg)](https://doi.org/10.21105/joss.00165)

This is how it will look in your documentation:

We need your help!

Journal of Open Source Software is a community-run journal and relies upon volunteer effort. If you'd like to support us please consider doing either one (or both) of the the following:

• Volunteering to review for us sometime in the future. You can add your name to the reviewer list here: http://joss.theoj.org/reviewer-signup.html
• Making a small donation to support our running costs here: https://www.flipcause.com/secure/cause_pdetails/Mjk3Nzk=