[REVIEW]: Jabberwocky: an ontology-aware toolkit for manipulating text

[whedon]

Submitting author: @sap218 (Samantha Pendleton)
Repository: https://github.com/sap218/jabberwocky
Version: v1.0.0.0
Editor: @majensen
Reviewer: @wdduncan, @balhoff
Archive: 10.5281/zenodo.3922261

Status

Status badge code:

HTML: <a href="https://joss.theoj.org/papers/10502055165be490feaa389d51fe99d6"><img src="https://joss.theoj.org/papers/10502055165be490feaa389d51fe99d6/status.svg"></a>
Markdown: [![status](https://joss.theoj.org/papers/10502055165be490feaa389d51fe99d6/status.svg)](https://joss.theoj.org/papers/10502055165be490feaa389d51fe99d6)

Reviewers and authors:

Please avoid lengthy details of difficulties in the review thread. Instead, please create a new issue in the target repository and link to those issues (especially acceptance-blockers) by leaving comments in the review thread below. (For completists: if the target issue tracker is also on GitHub, linking the review thread in the issue or vice versa will create corresponding breadcrumb trails in the link target.)

Reviewer instructions & questions

@wdduncan & @balhoff, please carry out your review in this issue by updating the checklist below. If you cannot edit the checklist please:

1. Make sure you're logged in to your GitHub account
2. Be sure to accept the invite at this URL: https://github.com/openjournals/joss-reviews/invitations

The reviewer guidelines are available here: https://joss.readthedocs.io/en/latest/reviewer_guidelines.html. Any questions/concerns please let @majensen know.

✨ Please try and complete your review in the next two weeks ✨

Review checklist for @wdduncan

Conflict of interest

• I confirm that I have read the JOSS conflict of interest (COI) policy and that: I have no COIs with reviewing this work or that any perceived COIs have been waived by JOSS for the purpose of this review.

Code of Conduct

• I confirm that I read and will adhere to the JOSS code of conduct.

General checks

• Repository: Is the source code for this software available at the repository url?
• License: Does the repository contain a plain-text LICENSE file with the contents of an OSI approved software license?
• Contribution and authorship: Has the submitting author (@sap218) made major contributions to the software? Does the full list of paper authors seem appropriate and complete?

Functionality

• Installation: Does installation proceed as outlined in the documentation?
• Functionality: Have the functional claims of the software been confirmed?
• Performance: If there are any performance claims of the software, have they been confirmed? (If there are no claims, please check off this item.)

Documentation

• A statement of need: Do the authors clearly state what problems the software is designed to solve and who the target audience is?
• Installation instructions: Is there a clearly-stated list of dependencies? Ideally these should be handled with an automated package management solution.
• Example usage: Do the authors include examples of how to use the software (ideally to solve real-world analysis problems).
• Functionality documentation: Is the core functionality of the software documented to a satisfactory level (e.g., API method documentation)?
• Automated tests: Are there automated tests or manual steps described so that the functionality of the software can be verified?
• Community guidelines: Are there clear guidelines for third parties wishing to 1) Contribute to the software 2) Report issues or problems with the software 3) Seek support

Software paper

• Summary: Has a clear description of the high-level functionality and purpose of the software for a diverse, non-specialist audience been provided?
• A statement of need: Do the authors clearly state what problems the software is designed to solve and who the target audience is?
• State of the field: Do the authors describe how this software compares to other commonly-used packages?
• Quality of writing: Is the paper well written (i.e., it does not require editing for structure, language, or writing quality)?
• References: Is the list of references complete, and is everything cited appropriately that should be cited (e.g., papers, datasets, software)? Do references in the text use the proper citation syntax?

Review checklist for @balhoff

Conflict of interest

• I confirm that I have read the JOSS conflict of interest (COI) policy and that: I have no COIs with reviewing this work or that any perceived COIs have been waived by JOSS for the purpose of this review.

Code of Conduct

• I confirm that I read and will adhere to the JOSS code of conduct.

General checks

• Repository: Is the source code for this software available at the repository url?
• License: Does the repository contain a plain-text LICENSE file with the contents of an OSI approved software license?
• Contribution and authorship: Has the submitting author (@sap218) made major contributions to the software? Does the full list of paper authors seem appropriate and complete?

Functionality

• Installation: Does installation proceed as outlined in the documentation?
• Functionality: Have the functional claims of the software been confirmed?
• Performance: If there are any performance claims of the software, have they been confirmed? (If there are no claims, please check off this item.)

Documentation

• A statement of need: Do the authors clearly state what problems the software is designed to solve and who the target audience is?
• Installation instructions: Is there a clearly-stated list of dependencies? Ideally these should be handled with an automated package management solution.
• Example usage: Do the authors include examples of how to use the software (ideally to solve real-world analysis problems).
• Functionality documentation: Is the core functionality of the software documented to a satisfactory level (e.g., API method documentation)?
• Automated tests: Are there automated tests or manual steps described so that the functionality of the software can be verified?
• Community guidelines: Are there clear guidelines for third parties wishing to 1) Contribute to the software 2) Report issues or problems with the software 3) Seek support

Software paper

• Summary: Has a clear description of the high-level functionality and purpose of the software for a diverse, non-specialist audience been provided?
• A statement of need: Do the authors clearly state what problems the software is designed to solve and who the target audience is?
• State of the field: Do the authors describe how this software compares to other commonly-used packages?
• Quality of writing: Is the paper well written (i.e., it does not require editing for structure, language, or writing quality)?
• References: Is the list of references complete, and is everything cited appropriately that should be cited (e.g., papers, datasets, software)? Do references in the text use the proper citation syntax?

[whedon]

Hello human, I'm @whedon, a robot that can help you with some common editorial tasks. @wdduncan, @balhoff it looks like you're currently assigned to review this paper 🎉.

⭐ Important ⭐

If you haven't already, you should seriously consider unsubscribing from GitHub notifications for this (https://github.com/openjournals/joss-reviews) repository. As a reviewer, you're probably currently watching this repository which means for GitHub's default behaviour you will receive notifications (emails) for all reviews 😿

To fix this do the following two things:

1. Set yourself as 'Not watching' https://github.com/openjournals/joss-reviews:

2. You may also like to change your default settings for this watching repositories in your GitHub profile here: https://github.com/settings/notifications

For a list of things I can do to help you, just type:

@whedon commands

For example, to regenerate the paper pdf after making changes in the paper's md or bib files, type:

@whedon generate pdf

[whedon]

👉 Check article proof 📄 👈

[whedon]

Reference check summary:

OK DOIs

- None

MISSING DOIs

- https://doi.org/10.1093/database/bau033 may be missing for title: tagtog: interactive and text-mining-assisted annotation of gene mentions in PLOS full-text articles
- https://doi.org/10.3115/v1/p14-5010 may be missing for title: The Stanford CoreNLP natural language processing toolkit
- https://doi.org/10.1093/nar/gkr972 may be missing for title: Disease Ontology: a backbone for disease semantic integration
- https://doi.org/10.1016/j.ajhg.2008.09.017 may be missing for title: The Human Phenotype Ontology: a tool for annotating and analyzing human hereditary disease
- https://doi.org/10.1093/bib/bbv011 may be missing for title: The role of ontologies in biological and biomedical research: a functional perspective

INVALID DOIs

- None

[majensen]

Quick thing if you can @sap218 - can you add the URLs that whedon-the-bot found above to your refs?

[sap218]

Currently I have (as an example for tagtog):

@ARTICLE{Cejuela2014-lv,
  title    = "tagtog: interactive and text-mining-assisted annotation of gene
              mentions in {PLOS} full-text articles",
  author = ..........,
  url      = {https://doi.org/10.1093/database/bau033}
}

and it prints as "Retrieved from: https://doi.org/10.1093/database/bau033"

is "url" the correct bibtex key?

@majensen

[arfon]

@sap218 - please change them to keys like this:

doi  = "10.1093/database/bau033"

i.e. change the url to doi and strip off the https://doi.org/ preamble.

[majensen]

Thanks @arfon

[sap218]

Thanks @arfon - @majensen I believe I have fixed the DOI issue now

[arfon]

@whedon check references

[whedon]

Reference check summary:

OK DOIs

- 10.1093/database/bau033 is OK
- 10.3115/v1/p14-5010 is OK
- 10.1093/nar/gkr972 is OK
- 10.1016/j.ajhg.2008.09.017 is OK
- 10.1093/bib/bbv011 is OK

MISSING DOIs

- None

INVALID DOIs

- None

[wdduncan]

I finished the checklist.

Comments:

1. I could not install the repo as per the instructions. When running the install script, I received error:

ERROR: TEST FAILED: /Users/wdduncan/.local/lib/python3.7/site-packages/ does NOT support .pth files 
error: bad install directory or PYTHONPATH

This could be b/c I am using OS X, or perhaps b/c I installed using venv.

2.. The examples reference a file named /ontology/pocketmonsters.owl. But this file is not in the repo.

The github documentation would benefit from having examples of expected output and descriptions of what the output means.

The paper refers to the jabberwocky-tests as a way to see the software at work. But that directory contains no instructions.

How long is the software supposed paper to be?

[majensen]

@sap218 please have a look at above issues of @wdduncan
@wdduncan - paper can be long or short, as long as it covers the requirements in the "Software paper" section of the review
thanks!

[danielskatz]

Well, actually, a goal of JOSS is that papers should be reasonably short. In https://joss.readthedocs.io/en/latest/submitting.html#what-should-my-paper-contain, we say, "the paper should be between 250-1000 words" though some papers are somewhat longer.

[majensen]

Thanks for the clarification @danielskatz!

[sap218]

@wdduncan @majensen will get on looking at the issues stated as soon as I can! Thanks.

[arfon]

👋 @balhoff - just a friendly check-in to see how things are going with your review?

[balhoff]

Thanks for the reminder, @arfon! I can work on this this week.

[sap218]

@wdduncan @majensen @balhoff I have made some fixes/changes, particularly changes to fix @wdduncan comments - paper.md has been updated too for a little more clarification. Let me know if you need any help!

edit: I can also shorten the paper if need be :-)

[sap218]

@whedon generate pdf

[whedon]

👉 Check article proof 📄 👈

[majensen]

@wdduncan @balhoff Can I get an update from you guys here? I'm hoping we're closing in on it. Thanks!

[balhoff]

Hi @majensen I ran into a blocker with sap218/jabberwocky#10. I reported it in the tool repo, but probably should have clarified here that I was waiting on that before being able to proceed further.

[majensen]

@openjournals/joss-eics really ready, thanks ever so much.

[danielskatz]

@whedon generate pdf

[whedon]

👉 Check article proof 📄 👈

[danielskatz]

👋 @sap218 - I have some comments/requests on the paper, in order from start to end:

• "gaining the key textual elements" - I don't know what this means; how does one gain textual elements? Could "gaining" be "determining"?
• "having knowledge of the synonyms for clinical terminology may increase the size of the corpus available for analysis" - how is this the case? How does having synonyms increase the size of the corpus?
• "e.g. social" <- "e.g., social"
• "their chosen ontology and" <- "their chosen ontology, and"
• "(Richardson, 2007)) and then" <- "(Richardson, 2007)), and then"
• "(e.g. lines from a corpus)" <- "(e.g., lines from a corpus)"
• "rank the important terms: using the" <- "rank the important terms by using the"
• "command arise will" <- "command, arise, will"
• "In Jabberwocky’s test repository (see Jabberwocky repo for further instructions), I show examples of each command separately. In the ‘process’ directory, I combine all three commands to demonstrate an example workflow. With 24 blog posts, the first use of catch returned 11 posts with the provided keywords. With bite I reviewed the CSV of ranked terms and curated new synonyms, simply by adding the corresponding class label from the ontology. I then used arise to add the identified synonyms into the ontology." <- "Jabberwocky’s test repository (see Jabberwocky repo for further instructions) shows examples of each command separately. The ‘process’ directory shows an example that combines all three commands to demonstrate an example workflow. With 24 blog posts, the first use of catch returned 11 posts with the provided keywords. The example uses bite to review the CSV of ranked terms and curated new synonyms, simply by adding the corresponding class label from the ontology. It then uses arise to add the identified synonyms into the ontology."
• For the reference Honnibal & Montani, "To appear" is not a journal. Please change this to include the journal name.
• For Manning et al., please protect cases with {}s in the bibtex so that the proceedings title appears in Title Case, rather than Sentence case. Also protect the {B} in the title so that it appears as Bloom, not bloom. Also Spacy should be spaCy, I think.
• In McKinney et al., do the same. Also protect the {P} in the title so that it appears as Python, not python. Also remove pdfs.semanticscholar.org
• In Pedregosa et al., fix Python in the title from python
• For Richardson, please add a URL or venue

When these are done, regenerate the pdf with @whedon generate pdf and let me know.

[sap218]

@whedon generate pdf

[whedon]

👉 Check article proof 📄 👈

[sap218]

@whedon check references

[whedon]

Reference check summary:

OK DOIs

- 10.1093/database/bau033 is OK
- 10.3115/v1/p14-5010 is OK
- 10.1093/nar/gkr972 is OK
- 10.1016/j.ajhg.2008.09.017 is OK
- 10.1093/bib/bbv011 is OK
- 10.25080/majora-92bf1922-00a is OK

MISSING DOIs

- None

INVALID DOIs

- None

[sap218]

@danielskatz I have updated the paper and addressed your comments

[danielskatz]

Thanks - I think one was missed, however: the proceedings title for Manning et al. should be in Title Case, not Sentence case.

[sap218]

@danielskatz Ah yes sorry! Just fixed that now :-)

[danielskatz]

@whedon generate pdf

[whedon]

👉 Check article proof 📄 👈

[danielskatz]

@whedon accept

[whedon]

Attempting dry run of processing paper acceptance...

[whedon]

Reference check summary:

OK DOIs

- 10.1093/database/bau033 is OK
- 10.3115/v1/p14-5010 is OK
- 10.1093/nar/gkr972 is OK
- 10.1016/j.ajhg.2008.09.017 is OK
- 10.1093/bib/bbv011 is OK
- 10.25080/majora-92bf1922-00a is OK

MISSING DOIs

- None

INVALID DOIs

- None

[whedon]

👋 @openjournals/joss-eics, this paper is ready to be accepted and published.

Check final proof 👉 openjournals/joss-papers#1526

If the paper PDF and Crossref deposit XML look good in openjournals/joss-papers#1526, then you can now move forward with accepting the submission by compiling again with the flag deposit=true e.g.

@whedon accept deposit=true

[danielskatz]

@whedon accept deposit=true

[whedon]

Doing it live! Attempting automated processing of paper acceptance...

[whedon]

🐦🐦🐦 👉 Tweet for this paper 👈 🐦🐦🐦

[whedon]

🚨🚨🚨 THIS IS NOT A DRILL, YOU HAVE JUST ACCEPTED A PAPER INTO JOSS! 🚨🚨🚨

Here's what you must now do:

0. Check final PDF and Crossref metadata that was deposited 👉 Creating pull request for 10.21105.joss.02168 joss-papers#1527
1. Wait a couple of minutes to verify that the paper DOI resolves https://doi.org/10.21105/joss.02168
2. If everything looks good, then close this review issue.
3. Party like you just published a paper! 🎉🌈🦄💃👻🤘

Any issues? Notify your editorial technical team...

[danielskatz]

Thanks to @wdduncan & @balhoff for reviewing!
And to @majensen for editing!

Congratulations to @sap218 (Samantha Pendleton) and co-author!!

[whedon]

🎉🎉🎉 Congratulations on your paper acceptance! 🎉🎉🎉

If you would like to include a link to your paper from your README use the following code snippets:

Markdown:
[![DOI](https://joss.theoj.org/papers/10.21105/joss.02168/status.svg)](https://doi.org/10.21105/joss.02168)

HTML:
<a style="border-width:0" href="https://doi.org/10.21105/joss.02168">
  <img src="https://joss.theoj.org/papers/10.21105/joss.02168/status.svg" alt="DOI badge" >
</a>

reStructuredText:
.. image:: https://joss.theoj.org/papers/10.21105/joss.02168/status.svg
   :target: https://doi.org/10.21105/joss.02168

This is how it will look in your documentation:

We need your help!

Journal of Open Source Software is a community-run journal and relies upon volunteer effort. If you'd like to support us please consider doing either one (or both) of the the following:

• Volunteering to review for us sometime in the future. You can add your name to the reviewer list here: https://joss.theoj.org/reviewer-signup.html
• Making a small donation to support our running costs here: https://numfocus.org/donate-to-joss

[majensen]

@balhoff and @wdduncan thanks very much from me personally -
@sap218 : congratulations! Go get a kebab!

[wdduncan]

@majensen Thanks Mark!
Congrats @sap218 !!

[sap218]

@majensen @danielskatz @balhoff @wdduncan thank you all! I appreciate the help!