taxlist, a package for structuring taxonomic lists and related information

[kamapu]

Summary

• What does this package do? (explain in 50 words or less):

The taxlist package structures taxonomic information into S4 objects and implements methods for the manipulation of contained information.
Such objects may or may not contain information on synonymy, taxonomic ranks, parent-child relations, taxon views (references used to establish relation between taxon usage names and taxon concepts), and taxon (functional) traits.

• Paste the full DESCRIPTION file inside a code block below:

Package: taxlist
Version: 0.1.5
Encoding: UTF-8
Date: 2018-06-18
Title: Handling Taxonomic Lists
Authors@R:
	person("Miguel", "Alvarez", email="kamapu78@gmail.com", role=c("aut", "cre"))
Depends:
    R(>= 3.0.0),
    stats,
    utils
Imports:
    foreign,
    grDevices,
    methods,
    taxize,
    stringdist,
    vegdata
Suggests:
	ape,
    devtools,
    knitr,
    stringi,
    taxa,
    rmarkdown
LazyData: true
Description: Handling taxonomic lists through objects of class 'taxlist'.
    This package provides functions to import species lists from 'Turboveg'
    (<https://www.synbiosys.alterra.nl/turboveg>) and the possibility to create
    backups from resulting R-objects.
    Also quick displays are implemented as summary-methods.
License: GPL (>= 2)
URL:
	https://cran.r-project.org/package=taxlist,
	https://github.com/kamapu/taxlist
BugReports: https://github.com/kamapu/taxlist/issues
Collate:
    'NULLing.R''auxiliary_functions.R''deprecated-functions.R''dissect_name.R'
    'taxlist-class.R''clean.R''as.list.R''taxon_views.R''add_view.R'
    'taxon_names.R''taxon_relations.R''taxon_traits.R'
    'levels.R''add_concept.R''update_concept.R''add_synonym.R'
    'accepted_name.R''synonyms.R''basionym.R''update_name.R''delete_name.R'
    'replace_view.R''get_children.R'
    'change_concept.R''Extract.R''subset.R'
    'merge_taxa.R''backup_object.R''load_last.R''summary.R'
    'df2taxlist.R''tv2taxlist.R''tnrs.R''tax2traits.R''match_names.R''print_name.R'
    'StartMessage.R'
VignetteBuilder: knitr

• URL for the package (the development repository, not a stylized html page):

https://github.com/kamapu/taxlist

• Please indicate which category or categories from our package fit policies this package falls under *and why? (e.g., data retrieval, reproducibility. If you are unsure, we suggest you make a pre-submission inquiry.):

Reproducibility, because this package makes taxonomic information available in a quasi-standard format and tests inconsistencies on the content of taxonomic lists.

•   Who is the target audience and what are scientific applications of this package?  

In general to taxonomists and biodiversity scientists, in particular to vegetation ecologists (taxlist objects are implemented in the package vegtable).

• Are there other R packages that accomplish the same thing? If so, how does
  yours differ or meet our criteria for best-in-category?

While its functionality may overlap the package taxa, the package taxlist attempts to be flexible in the degree of completeness of data (incompleteness is very frequent in vegetation-plot databases), it is meant to be integrated in objects containing diversity information (as in the mentioned package vegtable) and to import data from local storage (spreadsheets, Turboveg data sets and even PostgreSQL tables by using vegtable2).

•   If you made a pre-submission enquiry, please paste the link to the corresponding issue, forum post, or other discussion, or @tag the editor you contacted.

Requirements

Confirm each of the following by checking the box. This package:

• does not violate the Terms of Service of any service it interacts with.
• has a CRAN and OSI accepted license.
• contains a README with instructions for installing the development version.
• includes documentation with examples for all functions.
• contains a vignette with examples of its essential functions and uses.
• has a test suite.
• has continuous integration, including reporting of test coverage, using services such as Travis CI, Coveralls and/or CodeCov.
• I agree to abide by ROpenSci's Code of Conduct during the review process and in maintaining my package should it be accepted.

Publication options

• Do you intend for this package to go on CRAN?
• Do you wish to automatically submit to the Journal of Open Source Software? If so:
  • The package has an obvious research application according to JOSS's definition.
  • The package contains a paper.md matching JOSS's requirements with a high-level description in the package root or in inst/.
  • The package is deposited in a long-term repository with the DOI:
  • (Do not submit your package separately to JOSS)
• Do you wish to submit an Applications Article about your package to Methods in Ecology and Evolution? If so:
  • The package is novel and will be of interest to the broad readership of the journal.
  • The manuscript describing the package is no longer than 3000 words.
  • You intend to archive the code for the package in a long-term repository which meets the requirements of the journal (see MEE's Policy on Publishing Code)
  • (Scope: Do consider MEE's Aims and Scope for your manuscript. We make no gaurantee that your manuscript willl be within MEE scope.)
  • (Although not required, we strongly recommend having a full manuscript prepared when you submit here.)
  • (Please do not submit your package separately to Methods in Ecology and Evolution)

Detail

• Does R CMD check (or devtools::check()) succeed? Paste and describe any errors or warnings:

• Does the package conform to rOpenSci packaging guidelines? Please describe any exceptions:

• If this is a resubmission following rejection, please explain the change in circumstances:

• If possible, please provide recommendations of reviewers - those with experience with similar packages and/or likely users of your package - and their GitHub user names:

@arendsee
@zachary-foster
@sckott

[maelle]

Thanks for your submission @kamapu, we're discussing and will get back to you soon.

[maelle]

👋 again @kamapu, it's a bit of a tricky situation and we're going to need more information about your package before making a decision.

The general question is: how would an R user with a taxonomy problem at hand know how to choose one of the two packages?

Your answer could be a table as in this pre-submission inquiry or a decision tree. I know it's a bit of work that might not even warrant onboarding, but it'd be something useful to have in the docs of your package (and of taxa probably) since the existence of both packages even not under the rOpenSci's umbrella mean users might have to make a decision.

Thanks for your patience!

[maelle]

👋 @kamapu! Any update or answer on the above? Thank you!

[maelle]

👋 @kamapu! Any update or answer on the above? Thank you!

[kamapu]

Dear @maelle, there are not really updates on this issue. At the moment I am for a longer period in the field (Kenya). Though, it will be difficult for me to really document differences between taxlist and taxa. I only can tell in favour of taxlist that this package is implemented in vegtable, which is meant as a container for vegetation-plot databases in R. In fact, taxlist development runs in parallel to vegtable but they are kept in separated packages due to the complexity of taxonomic structures and the potential integration of taxonomic lists in other object classes.

Now my question is: Can I submit two packages as a bulk in ROpenSci or should I leave taxlist and try to submit vegtable alone?

Best regards!

[maelle]

One (pre-)submission by package is best.

Have a productive time in the field!

[kamapu]

Dear @maelle:

I went through most of the functions in the manual of taxa to have a better impression about differences between this package and taxlist. I have to recognize, it has been hard for me to understand the way how taxa deals with taxonomic information, since its programming style is a higher level than my primitive skills. Thus, the following comments should be interpreted as impressions rather than truth.

Basically both mentioned packages fulfil similar tasks but use different approaches, taxa uses a function-oriented approach (R6), while taxlist uses a data-oriented one (S4). Both packages attempt to provide an object class containing taxonomic information, but taxlist is strongly focusing on potential applications in vegetation-plot databases. taxlist allows diverse degrees of completeness, starting from just listing of taxonomic entities up to data sets including taxonomic ranks, synonyms, ecological traits (or further taxon attributes), and taxon views.

Contributions of taxlist

Further aspects are probably not implemented in taxa and can be considered as the contribution of taxlist to ROpenSci:

• While taxa is importing information from several on-line databases, taxlist offers alternatives of import data from local data sets. For instance data loaded in data frames can be converted by using the function df2taxlist(). There is also a function called tv2taxlist() for importing species lists included in Turboveg
• The consistency of information included in taxonomic lists is "cross-checked" by validity in S4 objects. In the case of taxlist, potential issues such as duplicated combinations, parent-child relationships that are not consistent to taxonomic ranks, etc. will generate error messages when creating new objects. It is not clear for me how those issues are handled in taxa
• I don't see any mentions to synonymy in taxa, which is a crucial feature in taxlist, assuming that all databases including data from different sources have to implement a way of linking taxon usage names (either accepted names or synonyms) with taxon concepts. There is no mention to taxon views in taxa, either
• A function backup_object() for inserting time stamps to backups produced by save()
• Some string process and comparisons through the functions dissect_name() (split names into single terms) and match_names() for comparing character values with names included in taxlist objects
• Interaction with knitr and rmarkdown for inserting taxon names in respective documents using the function print_name()

Features in Development

Additional features in development, which may not be yet included in taxa are:

• An object class for multiple taxon views (taxlist enable single taxon views per taxon concept)
• A plotting function displaying relations between taxon as dendrograms
• A function exporting the content of taxlist objects into the Veg-X standard

Final Remarks

I hope the previous comments properly support taxlist as a package rather complementary to taxa than redundant and make it eligible for ROpenSci. Disregarding the final decision, a way to make data sets exchangeable among those two applications should be strongly recommended.

[zachary-foster]

Thanks for looking through the docs/code @kamapu! Here are some thoughts:

While taxa is importing information from several on-line databases, taxlist offers alternatives of import data from local data sets.

taxa can read local datasets using parse_tax_data, but it does not have any parsers for specific local formats, since we wanted to keep everything generic.

The consistency of information included in taxonomic lists is "cross-checked" by validity in S4 objects.

Yep, taxa uses R6 classes which are not strong typed, so users can break the objects if they set fields manually instead of using our dplyr-like manipulation functions.

I don't see any mentions to synonymy in taxa, which is a crucial feature in taxlist

There is no such feature explicitly and we dont have plans for one at the moment

A plotting function displaying relations between taxon as dendrograms

taxa uses print_tree for this

A function exporting the content of taxlist objects into the Veg-X standard

I have no plans to support import or export of specific file formats in taxa, since it is primarily a data manipulation standard.

[sckott]

thanks @kamapu for the reply - we're discussing now

[sckott]

@kamapu We've decided to proceed. We do feel it is important though to have functions to convert between the two packages major data structures/classes though, in both packages if possible. your editor @maelle will follow up with more information.

[kamapu]

@sckott These are great news! I will put some effort to produce the required functions though I may require the support of @zachary-foster.

[maelle]

👋 @kamapu! I think it'd be great to add the conversion functions before the onboarding process. I'll put this submission on hold, but in the meantime, feel free to ask any questions.

[kamapu]

OK, I hope, there is no hurry (I'm already busy with the closure of 2018). You may not wander, if I use the gap between Christmas and New Year for it 😧

[maelle]

No problem, I might ping you once in a while but that's fine. Please do join the Slack as soon as you want (I sent you an invite).

[maelle]

👋 @kamapu! Happy New Year! Any update?

[kamapu]

@maelle Happy New Year! I though, the day will come: I had no time but ways thinking about every day. Now I'll start the discussion with @zachary-foster

[kamapu]

The discussion can be followed here

[maelle]

👋 @kamapu, any update? Note that I'll be unavailable from sometime between the beginning of and mid-June, until mid-October, so your editor would be @noamross after that point.

[kamapu]

Hello @maelle and @noamross
There is one recent step here
We are working in a function for converting taxlist objects into Taxmap.

[kamapu]

👋 @noamross
After a while, we managed to write a function exporting and importing taxlist objects to taxmap. That is to say, objects could move from taxlist to taxa and back (see here).
Thus, the submission can proceed.

[noamross]

Thank you @kamapu for the update! Before assigning reviewers, we still need the package to have a test suite, with test coverage reporting of at least 75%.

[kamapu]

Dear @noamross Do you mean, I should follow this instructions?
I'm a self-made programmer and now I'm feeling like someone, who is driving without driving license

[kamapu]

@maelle "unsolved issues" actually mean, it is not depending on me anymore...

[levisc8]

Before I get to this again, just want to clarify - do the unresolved issues refer to the UTF-8 strings in the example data set? Or something else?

[kamapu]

In general I consider as unresolved:

1. UTF-8 Note by goodpractice::pg() and rhub::check(platform="debian-clang-devel").
2. I think, @mcsiple got several error or warning messages, in part related to LaTeX, which I'm supposing to be relate with her local settings but it should still be clarified.

Of course, I can still receive and try suggestions.

[KevCaz]

Hi @kamapu,
As I was interested in your issue, I had a look at it. Looks like the UTF-8 note you got comes from your dataset (Easplist.dra) (see dankelley/oce#1211)

R> tools:::.check_package_datasets(".")                         
Note: found 137 marked UTF-8 strings

Something that is not checked by default by devtools::check() (I'm guessing here) but is checked if you do the combo R CMD build/R CMD check.

Hope this could be useful.

[kamapu]

Dear all, sorry for annoying you with the issue of encodings. While I am sort of developing a pathological aversion for this aspect of programming, for people dealing with taxonomy (including myself), the use of names and their authorities is quite crucial in the management of information, thus considered in taxlist as a component of the identity of a taxon concept (the accepted name of the taxon) and the taxon usage id (alternative names used to refer to a taxon), and this is the matter of my headaches: global context = special characters...

The possible solutions explored until now, just to get rid of a note, are not satisfying me in a 100%.

Now I have a request for you: The next is a block of code including the outputs in my console (I'm working in Linux with UTF-8 locale). My question is: Do you get the exact output as in my case?

library(stringi)
library(taxlist)

Names <- Easplist@taxonNames$AuthorName[c(5299, 5021, 5019)]
Names
#> [1] "Borsch, Kai Müll. & Eb.Fisch." "Ruiz & Pav."                  
#> [3] "Ség."

stri_enc_mark(Names)
#> [1] "UTF-8" "ASCII" "UTF-8"

iconv(Names, "utf8", "ascii")
#> [1] NA            "Ruiz & Pav." NA           

stri_enc_toascii(Names)
#> [1] "Borsch, Kai M\032ll. & Eb.Fisch." "Ruiz & Pav."                     
#> [3] "S\032g."                         

stri_trans_general(Names, "latin-ascii")
#> [1] "Borsch, Kai Mull. & Eb.Fisch." "Ruiz & Pav."                  
#> [3] "Seg."   

See also this discussion.

[maelle]

For reference link to Discourse thread https://discuss.ropensci.org/t/note-on-utf-8-strings-by-goodpractice-gp/2165

[levisc8]

@kamapu regarding your last question, I do get the exact same output as you do. Furthermore, stri_trans_general(Names, "latin-ascii") %>% str_enc_mark() returns 'ascii' 'ascii' 'ascii'.

[levisc8]

Review round 2:

I'm generally fine with the new form of the package. I just have some minor comments at this point.

Unresolved issues

1. UTF-8 strings: I don't have strong feelings about this, as it is pretty far beyond my area of expertise. If you can fix it with the code blurb from above, excellent!

2. The pdflatex errors could be because pdflatex is not in the PATH variable (see here). I don't think there's much you can do about this from your side though.

   • I think the qpdf warning is similar. Basically, just need to install qpdf and add it to the PATH variable to check smoothly.

Rest of the response

1. Documentation is looking good.

2. package API: I understand the point regarding backward compatibility. I also don't think it's unreasonable to make breaking changes if they result in substantial improvements to the user experience. Incrementing by a major version number (e.g. 0.2 -> 1.0) should signal to most users that they should expect major changes. If users really don't want to change their code, they can install archived versions, either from Github or from CRAN. This is something I'd defer to rOpenSci editors on though, as I don't really believe it is a deal breaker or anything.

   • Re: aggregate/count_taxa: good point - I hadn't thought about that! On the other hand, there is still an rm.na argument for a couple methods. This could change to na.rm for consistency w/ the base/stats packages. Personally, I would forget that it is rm.na every single time I tried to use the package. Again, I appreciate the point about back-compatibility, but this doesn't need to be a breaking change right away - you can retain the old form and deprecate it for the next few minor versions as well.

[mcsiple]

Hi @kamapu , @maelle et al.,

My apologies for the delay in responding to these new revisions and response. I have itemized them below mostly for my own mental organization (they match the structure of @kamapu 's response, I can keep track of what I have checked).

Note/caveat: I normally run all the checks on two different operating systems to provide better coverage in my ran everything on my other computer. This time I did my entire initial review on a Windows OS and am now checking everything again on my Mac. This could be a "good" thing because I've covered more ground, but IMO it's also suboptimal because some of the changes I've seen could potentially be caused by different software/packages instead of changes to the package. I think this is mostly the case with the LaTeX issues, but wanted to include the disclaimer.

Statement of need

This edit is great and satisfies my question(s) about the context and uses for the package.

Community guidelines

Newly added contributing guidelines look good.

Good practices checklist

UTF-8 string issues

It looks like this one is still being hashed out to some extent... I hope you find a good resolution for it. It sounds from the ropensci.org post like maybe this will remain an issue for people with a setup that doesn't have a font that renders the non-ascii characters. Maybe a note in the vignette about whether/how this would affect functionality would be good, or better yet a custom check or error message upon install... I am sure other package developers will have this issue so you're doing a service to the community by trying to figure it out!

LaTeX issue

This one is my fault; I normally run all the checks on two different operating systems to provide better coverage in my ran everything on my other computer. No LaTeX errors in goodpractice::gp() this time, so consider this part resolved.

Automated tests

devtools::check() : I found no errors with R CMD check this time.

Help files

• vignettes: the added text to the readme is useful; thank you for adding!
• Cross question: the changes to the vignette explain the structure of the dataset better - thank you.
• Codes: Your explanation is good but if you could incorporate it into the help files that would be optimal for future users!

Thanks - let me know if you have questions/comments about any of the above.

[maelle]

👋 @kamapu, would you soon be able to respond the reviewers' new comments? Thank you.

[kamapu]

Sorry for the break (many duties and a new family member arrived). I'm ready to go!

[kamapu]

Response to Reviews in Round 2

Dear @maelle @mcsiple @levisc8
I'm sorry that despite the progress on the review I had to stay out of programming for a while. Now I'm back and like to thank for all valuable comments again.

I will than refer to the comments of reviewers at round 2.

Response to @levisc8

Unresolved Issues

• UTF-8 strings
  Since the option is still open, I will take it. I would prefer to keep them as long as tolerated by CRAN.

• pdf-LaTeX errors
  It seems to be solved.

Rest

• Documentation is looking good.
  👍

• Change argument rm.na trough na.rm in function count_taxa()
  It was in fact a mistake from my side. Resolved in ropensci/taxlist@91ce67d

Response to @mcsiple

Two comments to this review, otherwise I agree with the rest.

• UTF-8
  Yes, this will be following me for a while. Since taxlist is meant to be a recipient of information that is otherwise stored in a database application, the election of Encoding will probably depend on local settings and the encoding of the data source. It is interesting to experience that while our task is to provide a way to harmonize disagreement on nomenclatures for biodiversity assessments, we stumble onto Encoding issues 🤔

• Taxon codes/identifiers
  I added the explanations in the help document of taxlist-class in ropensci/taxlist@eae874a

[maelle]

Thanks @kamapu and congrats again on the new family member!

@mcsiple @levisc8 are you ok with the response?

[mcsiple]

Yes, it looks good to me. Thank you @kamapu , @levisc8 and @maelle (and congratulations from me too!)

…

On Thu, Sep 17, 2020 at 12:02 AM Maëlle Salmon ***@***.***> wrote: Thanks @kamapu <https://github.com/kamapu> and congrats again on the new family member! @mcsiple <https://github.com/mcsiple> @levisc8 <https://github.com/levisc8> are you ok with the response? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#233 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAWM42UJRC6SOVICVT7MHM3SGGYB7ANCNFSM4FHQR3VQ> .

--

____________________________________ Margaret (Megsie) Siple James S. McDonnell Postdoctoral Fellow Department of Ecology Evolution & Marine Biology University of California Santa Barbara mobile: +1 (206) 661-8403

[levisc8]

Hi @kamapu @mcsiple @maelle ,
Sorry for the slow response - the changes look good to me as well. Congratulations on the newest addition to your family @kamapu!

[maelle]

Approved! Thanks @kamapu for submitting and @mcsiple @levisc8 for your reviews! 😸

To-dos:

• Transfer the repo to rOpenSci's "ropensci" GitHub organization under "Settings" in your repo. I have invited you to a team that should allow you to do so. You'll be made admin once you do.
• Fix all links to the GitHub repo to point to the repo under the ropensci organization.
• If you already had a pkgdown website and are ok relying only on rOpenSci central docs building and branding,
  • deactivate the automatic deployment you might have set up
  • remove styling tweaks from your pkgdown config but keep that config file
  • replace the whole current pkgdown website with a redirecting page
  • replace your package docs URL with https://docs.ropensci.org/package_name
  • In addition, in your DESCRIPTION file, include the docs link in the URL field alongside the link to the GitHub repository, e.g.: URL: https://docs.ropensci.org/foobar (website) https://github.com/ropensci/foobar
• Fix any links in badges for CI and coverage to point to the ropensci URL. We no longer transfer Appveyor projects to ropensci Appveyor account so after transfer of your repo to rOpenSci's "ropensci" GitHub organization the badge should be [![AppVeyor Build Status](https://ci.appveyor.com/api/projects/status/github/ropensci/pkgname?branch=master&svg=true)](https://ci.appveyor.com/project/individualaccount/pkgname).
• We're starting to roll out software metadata files to all ropensci packages via the Codemeta initiative, see https://github.com/ropensci/codemetar/#codemetar for how to include it in your package, after installing the package - should be easy as running codemetar::write_codemeta() in the root of your package.

Should you want to acknowledge your reviewers in your package DESCRIPTION, you can do so by making them "rev"-type contributors in the Authors@R field (with their consent). More info on this here.

Welcome aboard! We'd love to host a post about your package - either a short introduction to it with an example for a technical audience or a longer post with some narrative about its development or something you learned, and an example of its use for a broader readership. If you are interested, consult the blog guide, and tag @stefaniebutland in your reply. She will get in touch about timing and can answer any questions.

We've put together an online book with our best practice and tips, this chapter starts the 3d section that's about guidance for after onboarding. Please tell us what could be improved, the corresponding repo is here.

[kamapu]

🥳 🥳 🥳
Then, there is some work to do, still. Surely I'll come back with some questions.
Thanks to all participants in the development and review!

[maelle]

Please ping me once you've transferred the repo so that I might give you admin rights back, and feel free to ask me any question.
Note that the invitation to the ropensci GitHub organization expires after I think one week. If you try to transfer the repo too late, tell me to re-send an invite.

[kamapu]

Transfer to ropensci

• Transfer to ropensci repository
• Fixed links
  Done in ropensci/taxlist@d962046
• Redirect site to 'docs/ropensci'
  Done in ropensci/taxlist@fab0ca5
• Set links in badges
  Done by solving the second comment
• Add codemeta.json
  Done in ropensci/taxlist@89708aa

[kamapu]

@maelle I just transferred the repository to ROpenSci and tried to respond to all bullet points accordingly.
There is just one issue regarding to the new package site and the images. How can I include the figures to be displayed in docs/ropensci?

[maelle]

@kamapu I gave you admin rights again. I am not sure I follow your question. Where are those images now? What docs/ropensci deploys is a pkgdown website. I see images in https://docs.ropensci.org/taxlist/, what is missing?

[kamapu]

@maelle
In the source of the package there is a folder README-figures, which contains a PNG-image of the taxlist data schema (Figure before chapter Installing taxlist) but is also used to store figures generated by plotting functions (barplot at the very bottom, before Acknowledgements).
While these figures are visible in https://github.com/ropensci/taxlist/, they are missing in https://docs.ropensci.org/taxlist/

[maelle]

Does https://pkgdown.r-lib.org/reference/build_home.html#images-and-figures help?

[kamapu]

Oh, yes!
@maelle I used the opportunity to add some improvements:

• Function match_names() is sorting results by similarity
• I included within the package code (folder inst) all data required by the vignette.

[maelle]

Awesome! Let me plug two blog posts of mine 😁

• https://blog.r-hub.io/2020/06/03/vignettes/
• https://blog.r-hub.io/2020/05/29/distribute-data/

In any case I think I can close this issue but feel free to ask questions here / in the forum / etc.

Thanks all again!

[kamapu]

Thanks to everyone, also from my side.
@maelle interesting posts. It seems to be, that my original idea of allocating data in a data package is not that bad at all... I wonder if this may be also a good practice for their use in examples within CRAN packages, assuming that the data-package may not be a CRAN-package...