[REVIEW]: elapid: Species distribution modeling tools for Python

[editorialbot]

Submitting author: @earth-chris (Christopher Anderson)
Repository: https://github.com/earth-chris/elapid
Branch with paper.md (empty if default branch): joss-submission
Version: v1.0.1
Editor: @graciellehigino
Reviewers: @chrisborges, @gabrieldansereau
Archive: 10.5281/zenodo.7813017

Status

Status badge code:

HTML: <a href="https://joss.theoj.org/papers/ac415a024261efb3b397a1bad6f9cde6"><img src="https://joss.theoj.org/papers/ac415a024261efb3b397a1bad6f9cde6/status.svg"></a>
Markdown: [![status](https://joss.theoj.org/papers/ac415a024261efb3b397a1bad6f9cde6/status.svg)](https://joss.theoj.org/papers/ac415a024261efb3b397a1bad6f9cde6)

Reviewers and authors:

Please avoid lengthy details of difficulties in the review thread. Instead, please create a new issue in the target repository and link to those issues (especially acceptance-blockers) by leaving comments in the review thread below. (For completists: if the target issue tracker is also on GitHub, linking the review thread in the issue or vice versa will create corresponding breadcrumb trails in the link target.)

Reviewer instructions & questions

@chrisborges & @gabrieldansereau, your review will be checklist based. Each of you will have a separate checklist that you should update when carrying out your review.
First of all you need to run this command in a separate comment to create the checklist:

@editorialbot generate my checklist

The reviewer guidelines are available here: https://joss.readthedocs.io/en/latest/reviewer_guidelines.html. Any questions/concerns please let @graciellehigino know.

✨ Please start on your review when you are able, and be sure to complete your review in the next six weeks, at the very latest ✨

Checklists

📝 Checklist for @gabrieldansereau

📝 Checklist for @chrisborges

[editorialbot]

Hello humans, I'm @editorialbot, a robot that can help you with some common editorial tasks.

For a list of things I can do to help you, just type:

@editorialbot commands

For example, to regenerate the paper pdf after making changes in the paper's md or bib files, type:

@editorialbot generate pdf

[editorialbot]

Software report:

github.com/AlDanial/cloc v 1.88  T=0.09 s (476.1 files/s, 63280.2 lines/s)
-------------------------------------------------------------------------------
Language                     files          blank        comment           code
-------------------------------------------------------------------------------
Python                          16            794            944           1979
Markdown                        14            331              0            615
YAML                             7             25              9            254
TeX                              1             20              0            241
SVG                              1              0              0            161
CSS                              1              3              0             33
make                             1              9              5             27
-------------------------------------------------------------------------------
SUM:                            41           1182            958           3310
-------------------------------------------------------------------------------


gitinspector failed to run statistical information for the repository

[editorialbot]

Wordcount for paper.md is 1534

[editorialbot]

Reference check summary (note 'MISSING' DOIs are suggestions that need verification):

OK DOIs

- 10.2307/4072271 is OK
- 10.1073/pnas.0901639106 is OK
- 10.1111/j.1365-2699.2011.02637.x is OK
- 10.1111/ecog.03049 is OK
- 10.1111/j.0906-7590.2008.5203.x is OK
- 10.1111/ddi.12144 is OK
- 10.1111/j.1600-0587.2013.07872.x is OK
- 10.1111/j.1472-4642.2010.00725.x is OK
- 10.1214/13-aoas667 is OK
- 10.1111/geb.12684 is OK
- 10.1111/j.2041-210x.2011.00172.x is OK
- 10.1111/ddi.13442 is OK
- 10.5281/zenodo.6968622 is OK
- 10.5281/zenodo.6894736 is OK

MISSING DOIs

- None

INVALID DOIs

- None

[editorialbot]

👉📄 Download article proof 📄 View article proof on GitHub 📄 👈

[gabrieldansereau]

Review checklist for @gabrieldansereau

Conflict of interest

• I confirm that I have read the JOSS conflict of interest (COI) policy and that: I have no COIs with reviewing this work or that any perceived COIs have been waived by JOSS for the purpose of this review.

Code of Conduct

• I confirm that I read and will adhere to the JOSS code of conduct.

General checks

• Repository: Is the source code for this software available at the https://github.com/earth-chris/elapid?
• License: Does the repository contain a plain-text LICENSE file with the contents of an OSI approved software license?
• Contribution and authorship: Has the submitting author (@earth-chris) made major contributions to the software? Does the full list of paper authors seem appropriate and complete?
• Substantial scholarly effort: Does this submission meet the scope eligibility described in the JOSS guidelines
• Data sharing: If the paper contains original data, data are accessible to the reviewers. If the paper contains no original data, please check this item.
• Reproducibility: If the paper contains original results, results are entirely reproducible by reviewers. If the paper contains no original results, please check this item.
• Human and animal research: If the paper contains original data research on humans subjects or animals, does it comply with JOSS's human participants research policy and/or animal research policy? If the paper contains no such data, please check this item.

Functionality

• Installation: Does installation proceed as outlined in the documentation?
• Functionality: Have the functional claims of the software been confirmed?
• Performance: If there are any performance claims of the software, have they been confirmed? (If there are no claims, please check off this item.)

Documentation

• A statement of need: Do the authors clearly state what problems the software is designed to solve and who the target audience is?
• Installation instructions: Is there a clearly-stated list of dependencies? Ideally these should be handled with an automated package management solution.
• Example usage: Do the authors include examples of how to use the software (ideally to solve real-world analysis problems).
• Functionality documentation: Is the core functionality of the software documented to a satisfactory level (e.g., API method documentation)?
• Automated tests: Are there automated tests or manual steps described so that the functionality of the software can be verified?
• Community guidelines: Are there clear guidelines for third parties wishing to 1) Contribute to the software 2) Report issues or problems with the software 3) Seek support

Software paper

• Summary: Has a clear description of the high-level functionality and purpose of the software for a diverse, non-specialist audience been provided?
• A statement of need: Does the paper have a section titled 'Statement of need' that clearly states what problems the software is designed to solve, who the target audience is, and its relation to other work?
• State of the field: Do the authors describe how this software compares to other commonly-used packages?
• Quality of writing: Is the paper well written (i.e., it does not require editing for structure, language, or writing quality)?
• References: Is the list of references complete, and is everything cited appropriately that should be cited (e.g., papers, datasets, software)? Do references in the text use the proper citation syntax?

[gabrieldansereau]

Hi @earth-chris! Here is my review.

As a quick note, I work on SDMs as well but I am mainly a Julia and R user. @graciellehigino invited me to review knowingly as I can provide feedback on the user experience of less advanced users. I think this is especially important since many users will likely start using your software by sticking closely to the documentation. I apologize in advance if I'm missing Python obvious features.

General

I believe elapid is a very relevant package. Functionality and purpose are well described in both the manuscript and the documentation. I can definitely see these being used by researchers working on SDMs. The documentation clearly states how to install the package and has detailed explanations of the features. All JOSS criterias regarding licensing and automated tests are met.

Documentation & functionality

Unfortunately, I ran into a few issues while trying to reproduce the examples in the documentation by myself. I opened issues for each in the main repo. Given these, I could not properly review the package functionality for now. To me, these all need to be fixed before the manuscript can be accepted at JOSS.

Here is the list of issues:

• [JOSS review]: Sample data missing at installation earth-chris/elapid#74
• [JOSS review]: unexpected keyword argument with feature transformations earth-chris/elapid#75
• [JOSS review]: unexpected keyword argument in envelope suitability earth-chris/elapid#76
• [JOSS review]: data to reproduce the examples in the documentation earth-chris/elapid#77

I would like to insist on the last one regarding the reproducible examples. I really like that the package already includes sample data to run the first Maxent example. However, it would be very relevant if the users could also reproduce the "Working with Geospatial Data" and "A Simple Maxent Model" vignettes. I believe this is crucial for the users to understand how to properly use your software and adapt it to their own projects. I understand that data ownership might be a challenge, but perhaps using open sources for environmental data like WorldClim (already commonly used in SDMs) could provide a reproducible basis for your documentation?

Software paper

The software paper is very clear and well-written. All criteria are met, but I think there could be a few more elements to meet the "State of the field" criteria of comparing the software with commonly-used packages. For example, I realized that the documentation refers to maxnet, the R version of Maxent, but the software paper does not. I think it would be relevant to mention it in the software paper as well.

This also made me wonder if you should mention R packages as dismo or sdm, which offer similar species distribution modelling features. Is elapid the first package to provide similar integrated species distribution modelling features in Python? If so, this would be a great selling point for it.

Other

Finally, I spotted a few very minor typos in the manuscript. I fixed them in this PR. I added a few comments to explain them. Feel free to edit it as you wish.

[earth-chris]

Thank you @gabrieldansereau, I appreciate your feedback. These are critical issues to address, which I will do shortly. I'll report back to this thread again once complete.

@chrisborges, I suspect these issues would limit your reviewing experience, and I might recommend waiting until I address them before providing your review, which should reduce your review burden.

[chrisborges]

Review checklist for @chrisborges

Conflict of interest

• I confirm that I have read the JOSS conflict of interest (COI) policy and that: I have no COIs with reviewing this work or that any perceived COIs have been waived by JOSS for the purpose of this review.

Code of Conduct

• I confirm that I read and will adhere to the JOSS code of conduct.

General checks

• Repository: Is the source code for this software available at the https://github.com/earth-chris/elapid?
• License: Does the repository contain a plain-text LICENSE file with the contents of an OSI approved software license?
• Contribution and authorship: Has the submitting author (@earth-chris) made major contributions to the software? Does the full list of paper authors seem appropriate and complete?
• Substantial scholarly effort: Does this submission meet the scope eligibility described in the JOSS guidelines
• Data sharing: If the paper contains original data, data are accessible to the reviewers. If the paper contains no original data, please check this item.
• Reproducibility: If the paper contains original results, results are entirely reproducible by reviewers. If the paper contains no original results, please check this item.
• Human and animal research: If the paper contains original data research on humans subjects or animals, does it comply with JOSS's human participants research policy and/or animal research policy? If the paper contains no such data, please check this item.

Functionality

• Installation: Does installation proceed as outlined in the documentation?
• Functionality: Have the functional claims of the software been confirmed?
• Performance: If there are any performance claims of the software, have they been confirmed? (If there are no claims, please check off this item.)

Documentation

• A statement of need: Do the authors clearly state what problems the software is designed to solve and who the target audience is?
• Installation instructions: Is there a clearly-stated list of dependencies? Ideally these should be handled with an automated package management solution.
• Example usage: Do the authors include examples of how to use the software (ideally to solve real-world analysis problems).
• Functionality documentation: Is the core functionality of the software documented to a satisfactory level (e.g., API method documentation)?
• Automated tests: Are there automated tests or manual steps described so that the functionality of the software can be verified?
• Community guidelines: Are there clear guidelines for third parties wishing to 1) Contribute to the software 2) Report issues or problems with the software 3) Seek support

Software paper

• Summary: Has a clear description of the high-level functionality and purpose of the software for a diverse, non-specialist audience been provided?
• A statement of need: Does the paper have a section titled 'Statement of need' that clearly states what problems the software is designed to solve, who the target audience is, and its relation to other work?
• State of the field: Do the authors describe how this software compares to other commonly-used packages?
• Quality of writing: Is the paper well written (i.e., it does not require editing for structure, language, or writing quality)?
• References: Is the list of references complete, and is everything cited appropriately that should be cited (e.g., papers, datasets, software)? Do references in the text use the proper citation syntax?

[chrisborges]

@earth-chris Hey Chris, no problem. Let me know when you have a new version up.

[graciellehigino]

Hi @earth-chris 👋 How's this project going? Do you have an updated timeline and an idea of when it'll be ready for review again?

[earth-chris]

Hi @graciellehigino (and reviewers!). My apologies for the delay. I won't bore you with the details, but approximately everything happened all at once in my life. I appreciate your patience as I return to some semblance of normal.

I plan to address the most critical issues this weekend (providing / debugging sample data to demonstrate reproducible results). If I'm lucky I can get the docs updated, too. I'll be sure to ping this thread once I do so. But if this weekend proves too ambitious, I will prioritize unblocking everyone by the end of the coming week.

[earth-chris]

hello again @graciellehigino, @gabrieldansereau and @chrisborges. I've made a series of key updates in response to Gabriel's review, focused on ensuring reproducible results and clear documentation.

In short, I've added new sample data with all the fields required to run the suite of spatial and statistical analysis provided by elapid. And I've updated the documentation to include jupyter notebooks that can be run end-to-end (or the snippets could be copied and pasted to run). These changes (plus bugfixes) are now published in version 0.3.19 and on the main branch.

I'll take another pass at the manuscript to flesh out the related software/research references, but as of now I believe you all should be unblocked from moving forward with your reviews. Thanks again for your patience.

[gabrieldansereau]

Hi @earth-chris! Thank you for fixing the reproducibility issues. I was able to reproduce all the examples in the documentation. The geospatial and Maxent vignettes are very detailed and give a great idea on how to use elapid. It more than meets the JOSS requirements in my opinion.

The last comment to address is the State of the field criteria for the Software paper. I'm reposting from my earlier comment here. I know maxnet and others are well discussed in the documentation, but this is a criteria for the Software paper specifically, so I do think there should be some mention in there as well.

The software paper is very clear and well-written. All criteria are met, but I think there could be a few more elements to meet the "State of the field" criteria of comparing the software with commonly-used packages. For example, I realized that the documentation refers to maxnet, the R version of Maxent, but the software paper does not. I think it would be relevant to mention it in the software paper as well.
This also made me wonder if you should mention R packages as dismo or sdm, which offer similar species distribution modelling features. Is elapid the first package to provide similar integrated species distribution modelling features in Python? If so, this would be a great selling point for it.

[earth-chris]

@editorialbot generate pdf

[editorialbot]

👉📄 Download article proof 📄 View article proof on GitHub 📄 👈

[earth-chris]

thanks @gabrieldansereau! I've added a new paragraph pointing out a comparable package (i think ENMeval is pretty close in a few ways), mentioned the similar lineage to the R maxnet package, added an additional cross-validation reference, and highlighted the novelty of this python package (the only other established python SDM package is effectively an ArcGIS extentsion).

[gabrieldansereau]

Great, thanks for adding this.

@graciellehigino all review criteria are met for me.

[chrisborges]

Hi @earth-chris and @graciellehigino. I apologize truly for my delay in reviewing. First thing, I want to acknowledge @gabrieldansereau's amazing work in reviewing this paper, he has given amazing feedback and has surely made my work easier.

Software paper
I have no qualms with the software paper - I think it is clear, well-written and to-the-point.

Documentation
The documentation is also well-written and clear. However, on your readme ˜Why use elapid?” section, the examples link is broken (https://github.com/earth-chris/elapid/blob/main/examples/geo). I was able to find the folder with notebook examples in your repository. But this link should be fixed as it will probably be the first click for users.

Installation and reproducibility
Installation took a while, but worked as expected.

WorkingWithGeospatialData notebook is great. I was able to reproduce almost everything and enjoyed the through explanation. Unfortunately, I was not able to reproduce the histogram graph.

some things that threw errors: TypeError: zonal_stats() got an unexpected keyword argument 'quiet'

In both "WorkingWithGeospatialData" and "A simple maxent model" I was unable to run the merged command:
merged = ela.stack_geodataframes(presence, background, add_class_label=True)
Issue: AttributeError: module 'elapid' has no attribute 'stack_geodataframes'

Thus, due to this error specifically, I was unable to finish the examples and do not confirm the functional claims of the software at the moment.

Once this issues are resolved, I would love to resume with the examples.

[earth-chris]

hi @chrisborges! thanks for reviewing this.

I've fixed the broken link on the main page - thanks for pointing this out.

Regarding the remaining errors, I suspect you may be using an out-of-date branch. Following Gabriel's input, and a series of other updates, a few new versions of the software have been released since the original submission.

If you've installed from source, please pull the most recent updates from the main branch. If you installed it from pip, please try running pip install --upgrade elapid.

If you've tried this and are still stuck then please let me know and I'll dig deeper.

[chrisborges]

Hey @earth-chris

I just conda installed the software yesterday, following your Installation documentation as a new user would. If the latest version of it is not the one being installed, then I recommend this be fixed or you add the update steps to the installation documentation.

Nonetheless, I tried a “conda update elapid” and it still gives the same attribute error. What do you suggest as a workaround?

[earth-chris]

😵‍💫 ok I've created an issue to track this problem. let's work it out over there. thanks for your patience!

[kthyng]

@editorialbot set https://doi.org/10.5281/zenodo.7813017 as archive

[editorialbot]

Done! Archive is now https://doi.org/10.5281/zenodo.7813017

[kthyng]

@editorialbot check references

[kthyng]

@editorialbot generate pdf

[editorialbot]

Reference check summary (note 'MISSING' DOIs are suggestions that need verification):

OK DOIs

- 10.2307/4072271 is OK
- 10.1073/pnas.0901639106 is OK
- 10.1111/j.1365-2699.2011.02637.x is OK
- 10.1111/ecog.03049 is OK
- 10.1111/j.0906-7590.2008.5203.x is OK
- 10.1111/ddi.12144 is OK
- 10.1111/j.1600-0587.2013.07872.x is OK
- 10.1111/j.1472-4642.2010.00725.x is OK
- 10.1214/13-aoas667 is OK
- 10.1111/geb.12684 is OK
- 10.1111/j.2041-210x.2011.00172.x is OK
- 10.1111/ddi.13442 is OK
- 10.5281/zenodo.591564 is OK
- 10.5281/zenodo.6894736 is OK

MISSING DOIs

- 10.1038/s41467-020-18321-y may be a valid DOI for title: Spatial validation reveals poor predictive performance of large-scale ecological mapping models

INVALID DOIs

- https://doi.org/10.1111/2041-210X.13628 is INVALID because of 'https://doi.org/' prefix
- https://doi.org/10.1111/2041-210X.12200 is INVALID because of 'https://doi.org/' prefix

[editorialbot]

👉📄 Download article proof 📄 View article proof on GitHub 📄 👈

[kthyng]

@earth-chris Just one comment on your paper: please update your .bib file with {} around words in which you need to preserve capitalization. For example, "California" and "Australia" are not capitalized — please check the rest of the references too.

[earth-chris]

@editorialbot check references

[editorialbot]

Reference check summary (note 'MISSING' DOIs are suggestions that need verification):

OK DOIs

- 10.2307/4072271 is OK
- 10.1073/pnas.0901639106 is OK
- 10.1111/j.1365-2699.2011.02637.x is OK
- 10.1111/ecog.03049 is OK
- 10.1111/j.0906-7590.2008.5203.x is OK
- 10.1111/ddi.12144 is OK
- 10.1111/j.1600-0587.2013.07872.x is OK
- 10.1111/j.1472-4642.2010.00725.x is OK
- 10.1214/13-aoas667 is OK
- 10.1111/geb.12684 is OK
- 10.1111/j.2041-210x.2011.00172.x is OK
- 10.1111/ddi.13442 is OK
- 10.1111/2041-210X.13628 is OK
- 10.1111/2041-210X.12200 is OK
- 10.1038/s41467-020-18321-y is OK
- 10.5281/zenodo.591564 is OK
- 10.5281/zenodo.6894736 is OK

MISSING DOIs

- None

INVALID DOIs

- None

[earth-chris]

@editorialbot generate pdf

[editorialbot]

👉📄 Download article proof 📄 View article proof on GitHub 📄 👈

[earth-chris]

@kthyng thanks for catching these! I've fixed the DOIs, updated capitalization where appropriate, and fixed some author name entries.

[kthyng]

@editorialbot set 10.5281/zenodo.7813017 as archive

[editorialbot]

Done! Archive is now 10.5281/zenodo.7813017

[kthyng]

Ok everything is in place!

[kthyng]

@editorialbot accept

[editorialbot]

Doing it live! Attempting automated processing of paper acceptance...

[editorialbot]

Ensure proper citation by uploading a plain text CITATION.cff file to the default branch of your repository.

If using GitHub, a Cite this repository menu will appear in the About section, containing both APA and BibTeX formats. When exported to Zotero using a browser plugin, Zotero will automatically create an entry using the information contained in the .cff file.

You can copy the contents for your CITATION.cff file here:

CITATION.cff

cff-version: "1.2.0"
authors:
- family-names: Anderson
  given-names: Christopher B.
  orcid: "https://orcid.org/0000-0001-7392-4368"
doi: 10.5281/zenodo.7813017
message: If you use this software, please cite our article in the
  Journal of Open Source Software.
preferred-citation:
  authors:
  - family-names: Anderson
    given-names: Christopher B.
    orcid: "https://orcid.org/0000-0001-7392-4368"
  date-published: 2023-04-19
  doi: 10.21105/joss.04930
  issn: 2475-9066
  issue: 84
  journal: Journal of Open Source Software
  publisher:
    name: Open Journals
  start: 4930
  title: "elapid: Species distribution modeling tools for Python"
  type: article
  url: "https://joss.theoj.org/papers/10.21105/joss.04930"
  volume: 8
title: "elapid: Species distribution modeling tools for Python"

If the repository is not hosted on GitHub, a .cff file can still be uploaded to set your preferred citation. Users will be able to manually copy and paste the citation.

Find more information on .cff files here and here.

[editorialbot]

🐦🐦🐦 👉 Tweet for this paper 👈 🐦🐦🐦

[editorialbot]

🐘🐘🐘 👉 Toot for this paper 👈 🐘🐘🐘

[editorialbot]

🚨🚨🚨 THIS IS NOT A DRILL, YOU HAVE JUST ACCEPTED A PAPER INTO JOSS! 🚨🚨🚨

Here's what you must now do:

0. Check final PDF and Crossref metadata that was deposited 👉 Creating pull request for 10.21105.joss.04930 joss-papers#4144
1. Wait a couple of minutes, then verify that the paper DOI resolves https://doi.org/10.21105/joss.04930
2. If everything looks good, then close this review issue.
3. Party like you just published a paper! 🎉🌈🦄💃👻🤘

Any issues? Notify your editorial technical team...

[kthyng]

Congrats on your new publication to @earth-chris! Many thanks to editor @graciellehigino and reviewers @chrisborges and @gabrieldansereau for your time, hard work, and expertise!!

I will close this issue once the DOI is resolved.

[editorialbot]

🎉🎉🎉 Congratulations on your paper acceptance! 🎉🎉🎉

If you would like to include a link to your paper from your README use the following code snippets:

Markdown:
[![DOI](https://joss.theoj.org/papers/10.21105/joss.04930/status.svg)](https://doi.org/10.21105/joss.04930)

HTML:
<a style="border-width:0" href="https://doi.org/10.21105/joss.04930">
  <img src="https://joss.theoj.org/papers/10.21105/joss.04930/status.svg" alt="DOI badge" >
</a>

reStructuredText:
.. image:: https://joss.theoj.org/papers/10.21105/joss.04930/status.svg
   :target: https://doi.org/10.21105/joss.04930

This is how it will look in your documentation:

We need your help!

The Journal of Open Source Software is a community-run journal and relies upon volunteer effort. If you'd like to support us please consider doing either one (or both) of the the following:

• Volunteering to review for us sometime in the future. You can add your name to the reviewer list here: https://reviewers.joss.theoj.org/join
• Making a small donation to support our running costs here: https://numfocus.org/donate-to-joss