Submission: phylogram

[shaunpwilkinson]

Summary

• What does this package do? (explain in 50 words or less):

The phylogram R package is a tool for for developing phylogenetic trees as deeply-nested lists known as "dendrogram" objects. It provides functions for importing and exporting trees in parenthetic text format, as well as several tools for command-line tree manipulation and conversion between "dendrogram" and "phylo" class objects.

• Paste the full DESCRIPTION file inside a code block below:

Package: phylogram
Type: Package
Title: Dendrograms for Evolutionary Analysis
Version: 2.0.1.9000
Authors@R: person("Shaun", "Wilkinson", role = c("aut", "cre"),
                     email = "shaunpwilkinson@gmail.com")
Author: Shaun Wilkinson [aut, cre]
Maintainer: Shaun Wilkinson <shaunpwilkinson@gmail.com>
Description: Contains functions for importing and exporting 'dendrogram' 
    objects in parenthetic text format, as well as several 
    functions for command-line tree manipulation.
License: GPL-3
LazyData: TRUE
URL: http://github.com/shaunpwilkinson/phylogram
BugReports: http://github.com/shaunpwilkinson/phylogram/issues
Encoding: UTF-8
Depends:
    R (>= 3.0.0),
    methods,
    stats
Imports:
    ape (>= 4.0),
Suggests:
    knitr,
    rmarkdown,
    testthat
RoxygenNote: 6.0.1
VignetteBuilder: knitr

• URL for the package (the development repository, not a stylized html page):

https://github.com/shaunpwilkinson/phylogram

• Please indicate which category or categories from our package fit policies this package falls under *and why(? (e.g., data retrieval, reproducibility. If you are unsure, we suggest you make a pre-submission inquiry.):

[e.g., "data extraction, because the package parses a scientific data file format"]

Data extraction, the package contains a text parser for trees in the Newick parenthetic text format, and functions for converting between dendrogram and phylo object types.

•   Who is the target audience and what are scientific applications of this package?  

Biologists and bioinformaticists wishing to access the many analytical and tree-visualization functions available for dendrogram objects in R.

• Are there other R packages that accomplish the same thing? If so, how does
  yours differ or meet our criteria for best-in-category?

There are other packages that can parse Newick strings to create phylo objects (e.g. ape, ggtree) but none that we are aware of that enable two-way conversion between phylo and dendrogram objects.

•   If you made a pre-submission enquiry, please paste the link to the corresponding issue, forum post, or other discussion, or @tag the editor you contacted.

Requirements

Confirm each of the following by checking the box. This package:

• does not violate the Terms of Service of any service it interacts with.
• has a CRAN and OSI accepted license.
• contains a README with instructions for installing the development version.
• includes documentation with examples for all functions.
• contains a vignette with examples of its essential functions and uses.
• has a test suite.
• has continuous integration, including reporting of test coverage, using services such as Travis CI, Coveralls and/or CodeCov.
• I agree to abide by ROpenSci's Code of Conduct during the review process and in maintaining my package should it be accepted.

Publication options

• Do you intend for this package to go on CRAN?
• Do you wish to automatically submit to the Journal of Open Source Software? If so:
  • The package has an obvious research application according to JOSS's definition.
  • The package contains a paper.md matching JOSS's requirements with a high-level description in the package root or in inst/.
  • The package is deposited in a long-term repository with the DOI: 10.5281/zenodo.1227670
  • (Do not submit your package separately to JOSS)
• Do you wish to submit an Applications Article about your package to Methods in Ecology and Evolution? If so:
  • The package is novel and will be of interest to the broad readership of the journal.
  • The manuscript describing the package is no longer than 3000 words.
  • You intend to archive the code for the package in a long-term repository which meets the requirements of the journal (see MEE's Policy on Publishing Code)
  • (Scope: Do consider MEE's Aims and Scope for your manuscript. We make no gaurantee that your manuscript willl be within MEE scope.)
  • (Although not required, we strongly recommend having a full manuscript prepared when you submit here.)
  • (Please do not submit your package separately to Methods in Ecology and Evolution)

Detail

• Does R CMD check (or devtools::check()) succeed? Paste and describe any errors or warnings:

No errors or warnings.

• Does the package conform to rOpenSci packaging guidelines? Please describe any exceptions:

• If this is a resubmission following rejection, please explain the change in circumstances:

• If possible, please provide recommendations of reviewers - those with experience with similar packages and/or likely users of your package - and their GitHub user names:

Tal Galili (talgalili)
Zuguang Gu (jokergoo)
Michael Krabbe Borregaard (mkborregaard)
Guangchuang Yu (GuangchuangYu)

[sckott]

thanks for your submission @shaunpwilkinson - will get to editor checks tomorrow morning

[sckott]

Editor checks:

• Fit: The package meets criteria for fit and overlap
• Automated tests: Package has a testing suite and is tested via Travis-CI or another CI service.
• License: The package has a CRAN or OSI accepted license
• Repository: The repository link resolves correctly
• Archive (JOSS only, may be post-review): The repository DOI resolves correctly
• Version (JOSS only, may be post-review): Does the release version given match the GitHub release (v1.0.0)?

---

Editor comments

Thanks for your submission @shaunpwilkinson !!

• You can put a ropensci review badge in your README

[![](https://badges.ropensci.org/212_status.svg)](https://github.com/ropensci/onboarding/issues/212)

• Goodpractice output for you and reviewers to consider, for reviewers to consider:

── GP phylogram ───────────

It is good practice to

  ✖ write unit tests for all functions, and all package code in general. 82% of code lines are covered by test cases.

    R/ladder.R:33:NA
    R/ladder.R:34:NA
    R/ladder.R:35:NA
    R/ladder.R:36:NA
    R/ladder.R:37:NA
    ... and 63 more lines

  ✖ not use "Depends" in DESCRIPTION, as it can cause name clashes, and poor interaction with other packages. Use "Imports" instead.
  ✖ avoid long code lines, it is bad for readability. Also, many people prefer editor windows that are about 80 characters wide. Try make your lines
    shorter than 80 characters

    R/prune.R:66:1
    R/prune.R:96:1
    R/read.R:74:1
    R/read.R:151:1
    R/write.R:32:1
    ... and 2 more lines

  ✖ avoid sapply(), it is not type safe. It might return a vector, or a list, depending on the input data. Consider using vapply() instead.

    R/ladder.R:35:21
    R/prune.R:43:21
    R/read.R:113:25
    R/read.R:114:26
    R/read.R:158:21
    ... and 3 more lines

  ✖ not import packages as a whole, as this can cause name clashes between the imported packages. Instead, import only the specific functions you need.
  ✖ not use exportPattern in NAMESPACE. It can lead to exporting functions unintendedly. Instead, export functions that constitute the external API of
    your package.

Seeking reviewers now 🕐

---

Reviewers: @benjward @wcornwell
Due date: 2018-05-31

[shaunpwilkinson]

Thanks very much @sckott I will get onto those good-practice edits asap! S

[sckott]

sorry for delay, still trying to find 2 reviewers 🕐

[sckott]

Reviewers assigned:

Reviewers: @benjward @wcornwell
Due date: 2018-05-31

[wcornwell]

Package Review

Please check off boxes as applicable, and elaborate in comments below. Your review is not limited to these topics, as described in the reviewer guide

• As the reviewer I confirm that there are no conflicts of interest for me to review this work (If you are unsure whether you are in conflict, please speak to your editor before starting your review).

Documentation

The package includes all the following forms of documentation:

• A statement of need clearly stating problems the software is designed to solve and its target audience in README
• Installation instructions: for the development version of package and any non-standard dependencies in README
• Vignette(s) demonstrating major functionality that runs successfully locally
• Function Documentation: for all exported functions in R help
• Examples for all exported functions in R Help that run successfully locally
• Community guidelines including contribution guidelines in the README or CONTRIBUTING, and DESCRIPTION with URL, BugReports and Maintainer (which may be autogenerated via Authors@R).

Functionality

• Installation: Installation succeeds as documented.
• Functionality: Any functional claims of the software been confirmed.
• Performance: Any performance claims of the software been confirmed.
• Automated tests: Unit tests cover essential functions of the package and a reasonable range of inputs and conditions. All tests pass on the local machine.
• Packaging guidelines: The package conforms to the rOpenSci packaging guidelines

Final approval (post-review)

• The author has responded to my review and made changes to my satisfaction. I recommend approving this package.

Estimated hours spent reviewing: 3 ---

Review Comments

Thanks for this package. I didn't know anything really about the dendrogram class in R before looking into this. And it does look like a useful thing for the R-phylo-universe to have ways to go back and forth between the edge matrix style and the nested list style representation. There might be interesting applications in between taxonomy and phylogenies, and it's always good to have a full suite of tools to efficiently translate between data structures. As taxonomy moves to Non-linnaean data sets, the nested list structure might become increasingly useful.

I thought I would get to this quickly, and just do some of my normal research with this new tool and see how it goes. The tree that we have been using for the last several years is available (here)[https://datadryad.org//resource/doi:10.5061/dryad.63q27]. It's a ~30,000 tip tree for angiosperms, and typically we cut it down to a subset for whatever the task at hand is, but when I tried to load it using read.dendrogram, it still wasn't finished loading after 10 minutes. (The tree loads in less than a second using ape::read.tree). So then I did a little investigation. First I checked the size of the dendrogram objects using this code:

evolve.tree <- function(size){
  object.size(diversitree::tree.bd(pars=c(1,0),max.taxa=size))
}

evolve.dendro <- function(size){
  object.size(phylogram::as.dendrogram(diversitree::tree.bd(pars=c(1,0),max.taxa=size)))
}

x <- seq(2, 200, 10)
df <- data.frame(ape.size=purrr::map_dbl(x,evolve.tree), 
                 dendro.size=purrr::map_dbl(x,evolve.dendro))


library(ggplot2)
ggplot(df, aes(x=ape.size,y=dendro.size)) + geom_point()+theme_bw()

It turns out dendrogram objects are about 5 times as big as phylo objects, which isn't a big deal.

Then I checked read in speeds:

read_tree_dendro<-function(size){
  ape::write.tree(diversitree::tree.bd(pars=c(1,0),max.taxa=size),"twoh.tre")
  system.time({phylogram::read.dendrogram("twoh.tre")})[3]
}

read_tree_ape <- function(size){
  ape::write.tree(diversitree::tree.bd(pars=c(1,0),max.taxa=size),"twoh.tre")
  system.time({ape::read.tree("twoh.tre")})[3]
}

set.seed(42)
x <- seq(100,3000,150)
df <- data.frame(phylo_size=x,
               ape.speed=purrr::map_dbl(x, read_tree_ape),
               dendro.speed=purrr::map_dbl(x, read_tree_dendro))


ggplot(df, aes(x=ape.speed,y=dendro.speed,col=phylo_size)) + geom_point() + theme_bw()

ggplot(df, aes(x=phylo_size,y=dendro.speed,col=ape.speed)) + geom_point() + theme_bw()

So I guess there is something non-linear about the performance of read.dendrogram relative to both ape::read.tree and tree size. ape::read.tree seems to be linear and about two orders of magnitude faster (although with some mysterious variation sometimes).

I tried to read the read.dendrogram code, but I couldn't really intuit where the performance issues are. There are some magic numbers in the code, which might be one place to start. Also I have trouble reading the nested functions, but that might be because of my original training in C--I know this style is common in many languages. But for the big picture, it's worth figuring out the best way to optimise from a higher level. I afraid I'm not very knowledgable about about how R allocates memory to create deeply nested lists, but it's very likely that there are people around Ropensci who are, and who may be willing to give constructive advise on optimisation?

Phylogenies are getting bigger all the time: there was a 300,000 tip tree for plants published this week. And there are some giant bacterial trees out also. So for the functionality of this package, designing and optimising the code for at least medium sized trees is pretty crucial.

other notes:

• I'm sure there are some use cases for comparative methods where a nested list will be more useful than an edge matrix. Be nice to have a worked example in the vignette. I would also be worth comparing the performance of ape::drop.tip with phylogram::prune.

• Reading in problematic newick strings is kind of a nightmare because there are so many flavors of errors, but there are probably some cases where read.dendrogram returns a warning, when it really should give an informative error message rather than the current behavior which is to return a warning and a garbled tree.

library(phylogram)
newick <- "(A:0.1,B:0.2,C:0.3),D:0.4:0.5);"
x <- read.dendrogram(text = newick)

## Warning in read.dendrogram(text = newick): Possibly incomplete tree parse

x

## 'dendrogram' leaf 'D:0.4', at height 0

and also

newick <- "(A:0.1,(B:0.2,)C:0.3,D:0.4);"
x <- read.dendrogram(text = newick)

## Warning in read.dendrogram(text = newick): Possibly incomplete tree parse

x

## 'dendrogram' leaf 'A', at height 0

[wcornwell]

figures from above