OpenOmics: Library for integration of multi-omics, annotation, and interaction data

[JonnyTran]

Submitting Author: Jonny Tran (@JonnyTran)
All current maintainers: @JonnyTran
Package Name: openomics
One-Line Description of Package: Library for integration of multi-omics, annotation, and interaction data
Repository Link: https://github.com/JonnyTran/OpenOmics
Version submitted: 0.8.4
Editor: @NickleDave
Reviewer 1: @gawbul
Reviewer 2: @ksielemann
Archive:
JOSS DOI:
Version accepted: v 0.8.8
Date accepted (month/day/year): 04/17/2021

---

Description

OpenOmics is a Python library to assist integration of heterogeneous multi-omics bioinformatics data. By providing an API of data manipulation tools as well as a web interface (WIP), OpenOmics facilitates the common coding tasks when preparing data for bioinformatics analysis. It features support for:

• Genomics, Transcriptomics, Proteomics, and Clinical data.
• Harmonization with 20+ popular annotation, interaction, and disease-association databases (e.g. GENCODE, Ensembl, RNA Central, BioGRID, DisGeNet etc.)

OpenOmics also has an efficient data pipeline that bridges the popular data manipulation Pandas library and Dask distributed processing to address the following use cases:

• Provides a standard pipeline for dataset indexing, table joining and querying, which are transparent and customizable for end-users.
• Efficient disk storage for large multi-omics dataset with Parquet data structures.
• Multiple data types that supports both interactions and sequence data, and allows users to export to NetworkX graphs or down-stream machine learning.
• An easy-to-use API that works seamlessly with external Galaxy tool interface or the built-in Dash web interface (WIP).

Scope

• Please indicate which category or categories this package falls under:
  • Data retrieval
  • Data extraction
  • Data munging
  • Data deposition
  • Reproducibility
  • Geospatial
  • Education
  • Data visualization*

* Please fill out a pre-submission inquiry before submitting a data visualization package. For more info, see notes on categories of our guidebook.

• Explain how the and why the package falls under these categories (briefly, 1-2 sentences):

OpenOmics' core functionalities are to provide a suite of tools for data preprocessing, data integration, and public database retrieval. Its main goal is to maximize the transparency and reproducibility in the process of multi-omics data integration.

• Who is the target audience and what are scientific applications of this package?

OpenOmics' primary target audience are computational bioinformaticians, and the scientific application of this package is to provide scalable ad-hoc data-frame manipulation for multi-omics data integration in a reproducible manner. Also, we are currently developing an interactive web dashboard and interfaces to the Galaxy Tool Shed, disseminating the tool to biologists without a programming background.

• Are there other Python packages that accomplish the same thing? If so, how does yours differ?

Existing PyPI Python packages within the scope of multi-omics data analysis are "pythomics" and "omics". Their functions appear to be lacking support for manipulation of integrated multi-omics dataset, retrieval of public databases, and extensible OOP design. OpenOmics aims to follow modern software best-practices and package publishing standards.

Aside from multi-omics integration tools, several specialized Python packages exists for single omics data, such as ScanPy's "AnnData" and "Loom" files. They provide an intuitive data structure for expression arrays and side annotations, and Loom file even allows for out-of-core data-frame processing. However, they don't yet provide mechanisms for multi-omics data integration, where each omics data may have overlapping samples or varying row/column sizes.

• If you made a pre-submission enquiry, please paste the link to the corresponding issue, forum post, or other discussion, or @tag the editor you contacted:

#30

Technical checks

For details about the pyOpenSci packaging requirements, see our packaging guide. Confirm each of the following by checking the box. This package:

• does not violate the Terms of Service of any service it interacts with.
• has an OSI approved license.
• contains a README with instructions for installing the development version.
• includes documentation with examples for all functions.
• contains a vignette with examples of its essential functions and uses.
• has a test suite.
• has continuous integration, such as Travis CI, AppVeyor, CircleCI, and/or others.

Publication options

• Do you wish to automatically submit to the Journal of Open Source Software? If so:

JOSS Checks

• The package has an obvious research application according to JOSS's definition in their submission requirements. Be aware that completing the pyOpenSci review process does not guarantee acceptance to JOSS. Be sure to read their submission requirements (linked above) if you are interested in submitting to JOSS.
• The package is not a "minor utility" as defined by JOSS's submission requirements: "Minor ‘utility’ packages, including ‘thin’ API clients, are not acceptable." pyOpenSci welcomes these packages under "Data Retrieval", but JOSS has slightly different criteria.
• The package contains a paper.md matching JOSS's requirements with a high-level description in the package root or in inst/.
• The package is deposited in a long-term repository with the DOI: 10.5281/zenodo.4441167

Note: Do not submit your package separately to JOSS

Are you OK with Reviewers Submitting Issues and/or pull requests to your Repo Directly?

This option will allow reviewers to open smaller issues that can then be linked to PR's rather than submitting a more dense text based review. It will also allow you to demonstrate addressing the issue via PR links.

• Yes I am OK with reviewers submitting requested changes as issues to my repo. Reviewers will then link to the issues in their submitted review.

Code of conduct

• I agree to abide by pyOpenSci's Code of Conduct during the review process and in maintaining my package should it be accepted.

P.S. *Have feedback/comments about our review process? Leave a comment here

Editor and Review Templates

Editor and review templates can be found here

[lwasser]

welcome to pyopensci @JonnyTran someone will followup with you in the next week or so! Our progress is slow right now as we're working o n funding for this effort!! @NickleDave was this one you wanted to work on? you are doing a lot so please let me know if you have time (or if you don't that is understandable too!)

[NickleDave]

@JonnyTran this looks perfect, thank you.
Sorry for not replying when you opened this issue.

Hey @lwasser yes I have this on my to-do list and will start looking for reviewers Wednesday

[lwasser]

wonderful! @NickleDave thank you!

[NickleDave]

Hi all, just adding editor checks.

Thank you @JonnyTran for your detailed submission

Editor checks:

• Fit: The package meets criteria for fit and overlap.
  • yes, tool for working with different types of -omics data
• Automated tests: Package has a testing suite and is tested via Travis-CI or another CI service.
  • yes, Travis CI
• License: The package has an OSI accepted license
  • yes, MIT
• Repository: The repository link resolves correctly
  • yup

submitter did not check "yes" for submitting to JOSS
-- edit 2021-01-22: reviewer will submit to JOSS. Has added DOI to repository in response to my initial comments

• Archive (JOSS only, may be post-review): The repository DOI resolves correctly
• Version (JOSS only, may be post-review): Does the release version given match the GitHub release (v1.0.0)?

---

Editor comments

Overall:

• perfect fit for PyOpenSci, definitely looks like it has the potential to be an incredibly useful data retrieval / extraction / munging tool that faciliates reproducible pipelines
  • many use cases e.g. buildng datasets for machine learning from multiple omics sources

Minor comments:
things I notice on first glance

• no DOI for releases, e.g. using Zenodo integration. Good idea to have DOI to make releases citable, e.g. in papers to make explicit which version was used
  • more info on Zenodo integration with GitHub here: https://guides.github.com/activities/citable-code/
• no explicit link to docs page
  • this could be in README and / or in the "about" section -- GitHub lets you add a link there in addition to the text description
  • I did finally find the docs by clicking on the badge ... but if it were me I would have a big giant link point to the docs as obvious as possible
• some info could be added to README

This is definitely an "elements of style" question / matter of personal preference, and not a thing that is directly addressed by our developer guide, but:
I really like the readthedocs suggestions for what to include in your README:
https://www.writethedocs.org/guide/writing/beginners-guide-to-docs/#readme
I don't see a lot of those things included in yours now
Don't get me wrong, It is definitely great that you have many concrete examples built around code snippets
but there's a trade-off involved in having many of them at the expense of having all the other information, e.g. more detailed install instructions, info on where/how to raise issues, license type, etc.
Maybe have one good code snippet + then link to the other examples via a link to the docs?
You don't want to miss getting all that other info to potential contributors / other developers who are kind of most likely to look at your GitHub README, IMHO
but for sure many successful projects that don't follow this pattern (see for example the tqdm README)

---

Reviewers: @gawbul @ksielemann
Due date: February 15, 2021

[NickleDave]

started reaching out to reviewers, will update as soon as we hear back

[JonnyTran]

Minor comments:

• no DOI for releases, e.g. using Zenodo integration. Good idea to have DOI to make releases citable, e.g. in papers to make explicit which version was used
• no explicit link to docs page

Thanks so much for the great suggestions, @NickleDave. I've addressed the "releases DOI" and "docs link" issues above, and I'm planning on updating the README file to be more attractive to potential contributors.

[JonnyTran]

Hi @NickleDave and @lwasser, will it be possible for OpenOmics to make submission to JOSS at this stage of the review? I just changed my mind about this, but I already have a complete manuscript and I'm ready to provide the paper.md within a few days.

[NickleDave]

Hey @JonnyTran I think that could be okay. But first let me make sure I understand what you're asking.

Hi @NickleDave and @lwasser, will it be possible for OpenOmics to make submission to JOSS at this stage of the review?

Do you mean that you want to change "Publication Options" in your submission above, and check the box for "automatically submit to" JOSS?
If so, yes I think that's fine. @lwasser please confirm

I just changed my mind about this, but I already have a complete manuscript and I'm ready to provide the paper.md within a few days.

Do you mean that you have a complete, separate manuscript written about OpenOmics, in addition to the paper.md you would submit to JOSS?
E.g., like Physcraper
#26
which has a paper on biorxiv
https://www.biorxiv.org/content/10.1101/2020.09.15.299156v1

If that's the case, we should discuss more whether you want to submit to JOSS. We can perhaps tag some editors and ask if they can give us input. My impression is that JOSS is usually meant to provide a mechanism for getting publication credit for software in cases where the developer/maintainer can't easily publish a paper about it.
Although I think they might have changed some of the language in their submission guidelines about this.
See for example: https://joss.readthedocs.io/en/latest/submitting.html#co-publication-of-science-methods-and-software

so:
if you just want to go through PyOpenSci review and then submit to JOSS at the end, yes, totally fine assuming @lwasser agrees with me.
Anything else, we should probably discuss a little more first

[NickleDave]

Hi @JonnyTran just want to follow up on this -- please let me know what you're thinking

I think I do have one potential reviewer and can move ahead with finding another whenever you're ready