submission: suppdata

[willpearse]

Summary

• What does this package do? (explain in 50 words or less):
  Downloads supplementary materials from published papers using their DOIs as a reference. This facilitates reproducible analyses.

• Paste the full DESCRIPTION file inside a code block below:

Package: suppdata
Type: Package
Title: Downloading Supplementary Data from Published Manuscripts
Version: 0.9-0
Date: 2018-02-02
Author: William D. Pearse, Scott Chamberlain
Maintainer: William D. Pearse <will.pearse@gmail.com>
Description: Downloads data supplementary materials from manuscripts,
    using papers' DOIs as references. Includes some code to download
    from other Internet APIs (e.g., Xeno-Canto, EPMC).
License: MIT + file LICENSE
URL: https://github.com/ropensci/fulltext
BugReports: https://github.com/ropensci/fulltext/issues
VignetteBuilder: knitr
LazyLoad: yes
Suggests:
    knitr (>= 1.6),
    testthat (>= 2.0.0),
    fulltext(>= 0.1.4.9000)
Imports:
    httr (>= 1.0.0),
    xml2 (>= 1.2.0),
    jsonlite (>= 1.5),
    RCurl (>= 1.95-4.10),
    rcrossref (>= 0.8.0)
RoxygenNote: 6.0.1

• URL for the package: https://github.com/willpearse/suppdata

• Please indicate which category or categories from our package fit policies this package falls under *and why(? (e.g., data retrieval, reproducibility. If you are unsure, we suggest you make a pre-submission inquiry.):

  • data retrieval: because it downloads data from online scientific publications.
  • reproducibility: because by downloading raw data, it facilitates reproducible analysis scripts

•   Who is the target audience and what are scientific applications of this package?  

  • Audience: anyone (re-)analysing published scientific data
  • Application: Conducting reproducible analyses, database creation scripts

• Are there other R packages that accomplish the same thing? If so, how does
  yours differ or meet our criteria for best-in-category?

Originally, this code was part of fulltext, but has now been split out. That is the only package I'm aware of that does this.

•   If you made a pre-submission enquiry, please paste the link to the corresponding issue, forum post, or other discussion, or @tag the editor you contacted.
  @sckott

Requirements

Confirm each of the following by checking the box. This package:

• does not violate the Terms of Service of any service it interacts with.
  • This package does download data from journal publishers such as Wiley. I have heard stories/rumours of them getting very unhappy about people downloading articles from their websites at high speed. I'm not aware that this package violates their terms/conditions, but I want to flag this here. This code has already been part of an ROpenSci package (fulltext) without any problems, so I can't see that this is an issue.
• has a CRAN and OSI accepted license.
• contains a README with instructions for installing the development version.
• includes documentation with examples for all functions.
• contains a vignette with examples of its essential functions and uses.
• has a test suite.
• has continuous integration, including reporting of test coverage, using services such as Travis CI, Coveralls and/or CodeCov.
• I agree to abide by ROpenSci's Code of Conduct during the review process and in maintaining my package should it be accepted.

Publication options

• Do you intend for this package to go on CRAN?
• Do you wish to automatically submit to the Journal of Open Source Software? If so:
  • The package has an obvious research application according to JOSS's definition.
  • The package contains a paper.md matching JOSS's requirements with a high-level description in the package root or in inst/.
  • The package is deposited in a long-term repository with the DOI: 0.5281/zenodo.1168713
• Do you wish to submit an Applications Article about your package to Methods in Ecology and Evolution? If so:
  • The package is novel and will be of interest to the broad readership of the journal.
  • The manuscript describing the package is no longer than 3000 words.
  • You intend to archive the code for the package in a long-term repository which meets the requirements of the journal.
  • (Please do not submit your package separately to Methods in Ecology and Evolution)

Detail

• Does R CMD check (or devtools::check()) succeed? Paste and describe any errors or warnings:

• Does the package conform to rOpenSci packaging guidelines? Please describe any exceptions:
  Yes it does, and all this code was, previously, part of the ROpenSci package fulltext.

• If this is a resubmission following rejection, please explain the change in circumstances:

• If possible, please provide recommendations of reviewers - those with experience with similar packages and/or likely users of your package - and their GitHub user names:

[noamross]

Thanks @willpearse for your submission (and creation of this package at our behest!). Things look good, but I haven't been able to run all of our tests because of this build error, which I get both locally and on r-hub:

* checking for file ‘suppdata/DESCRIPTION’ ... OK
* preparing ‘suppdata’:
* checking DESCRIPTION meta-information ... OK
* installing the package to build vignettes
* creating vignettes ... ERROR
Error in texi2dvi(file = file, pdf = TRUE, clean = clean, quiet = quiet,  : 
  Running 'texi2dvi' on 'suppdata-intro.tex' failed.
LaTeX errors:
! LaTeX Error: File `ae.sty' not found.

Type X to quit or <RETURN> to proceed,
or enter new name. (Default extension: sty)

! Emergency stop.
<read *> 
         
l.30 \ifthenelse
                {\boolean{Sweave@inconsolata}}{%^^M
Calls: <Anonymous> -> texi2pdf -> texi2dvi
Execution halted

It's odd that this doesn't show up on Travis. Anything non-standard in your vignette Rnw?

Also - to get ahead of things: Please add coverage checks to your CI and the coverage badge to your README. (See usethis::use_coverage)

[willpearse]

No worries, glad to help. I've honestly no idea why this is happening (because, as you say, it doesn't trigger on Travis or on my machine), but from a quick bit of Googling I think it may be to do with your LaTeX install? Try: devtools::install_github(c('yihui/tinytex', 'rstudio/rmarkdown')) # first uninstall your existing LaTeX distribution, and then tinytex::install_tinytex() ...following from rstudio/rmarkdown#359

…

On Wed, 7 Feb 2018 at 15:28 Noam Ross ***@***.***> wrote: Thanks @willpearse <https://github.com/willpearse> for your submission (and creation of this package at our behest!). Things look good, but I haven't been able to run all of our tests because of this build error, which I get both locally and on r-hub: * checking for file ‘suppdata/DESCRIPTION’ ... OK * preparing ‘suppdata’: * checking DESCRIPTION meta-information ... OK * installing the package to build vignettes * creating vignettes ... ERROR Error in texi2dvi(file = file, pdf = TRUE, clean = clean, quiet = quiet, : Running 'texi2dvi' on 'suppdata-intro.tex' failed. LaTeX errors: ! LaTeX Error: File `ae.sty' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: sty) ! Emergency stop. <read *> l.30 \ifthenelse ***@***.***}}{%^^M Calls: <Anonymous> -> texi2pdf -> texi2dvi Execution halted It's odd that this doesn't show up on Travis. Anything non-standard in your vignette Rnw? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#195 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ABLcUuxIpeZrwEEn3zH1Cl-bSJ8qVcmvks5tSiN2gaJpZM4R9X6Y> .

[noamross]

So I tried building the vignette in a couple of fresh Docker images: rocker/verse (which has LaTeX pre-installed), and rocker/tidyverse, which does not, using tinytex to install TeX. Both gave me the same error, which means that somehow your vignette is doing something that is not in either of those standard environments, but is on Travis and your machine. Oddly the ae package, which should have the ae.sty file, is installed with tinytex.

[willpearse]

Have you followed the instructions I gave travis? Like `fulltext`, this needs `libpoppler-cpp-dev` installed. ( https://github.com/willpearse/suppdata/blob/master/.travis.yml).

…

On Wed, 7 Feb 2018 at 16:09 Noam Ross ***@***.***> wrote: So I tried building the vignette in a couple of fresh Docker images: rocker/verse (which has LaTeX pre-installed), androcker/tidyverse, which does not, using *tinytex* to install TeX. Both gave me the same error, which means that somehow your vignette is doing something that is not in either of those standard environments, but *is* on Travis and your machine. Oddly theaepackage, which should have theae.sty` file, *is* installed with tinytex. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#195 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ABLcUtHS4DL_n52D1veKMpPD4g48LCaIks5tSi0XgaJpZM4R9X6Y> .

[noamross]

OK, I've solved this by installing texinfo into my Docker container, but that doesn't seem like a long-run solution. I'll ask reviewers to see if they have issues and we'll return to it if they do. I might suggest a switch to markdown if it keeps up.

Editor checks:

• Fit: The package meets criteria for fit and overlap
• Automated tests: Package has a testing suite and is tested via Travis-CI or another CI service. Please add code coverage to your CI and a badge in your readme (see usethis::use_coverage)
• License: The package has a CRAN or OSI accepted license
• Repository: The repository link resolves correctly
• Archive (JOSS only, may be post-review): The repository DOI resolves correctly
• Version (JOSS only, may be post-review): Does the release version given match the GitHub release (v1.0.0)?

---

Editor comments

Here's the goodpractice::gp() output. Nothing to prevent moving to review but these should be straightforward fixes. I will go ahead and find reviewers.

 ✖ omit "Date" in DESCRIPTION. It is not required and it gets invalid quite
    often. A build date will be added to the package when you perform `R CMD build` on
    it.
  ✖ avoid long code lines, it is bad for readability. Also, many people prefer
    editor windows that are about 80 characters wide. Try make your lines shorter than 80
    characters

    R/journals.R:9:1
    R/journals.R:18:1
    R/journals.R:24:1
    R/journals.R:33:1
    R/journals.R:36:1
    ... and 47 more lines

  ✖ avoid sapply(), it is not type safe. It might return a vector, or a list,
    depending on the input data. Consider using vapply() instead.

    R/suppdata.R:126:17
    R/suppdata.R:135:17

  ✖ fix this R CMD check NOTE: Namespace in Imports field not imported from:
    ‘RCurl’ All declared Imports should be used.
───────────────────────────────────────────

Reviewers: @sarahsupp @rossmounce
Due date: 2018-03-16

[willpearse]

Thanks for your effort on this, and I'm sorry it gave you so much trouble to install. I'll wait to hear back from the reviewers, and then I'll make all these changes and whatever they request in one fell swoop. Thanks again!

…

On Wed, 7 Feb 2018 at 16:57 Noam Ross ***@***.***> wrote: OK, I've solved this by installing texinfo into my Docker container, but that doesn't seem like a long-run solution. I'll ask reviewers to see if they have issues and we'll return to it if they do. I might suggest a switch to markdown if it keeps up. Editor checks: - *Fit*: The package meets criteria for fit <http://policies.md#fit> and overlap <http://policies.md#fit> - *Automated tests:* Package has a testing suite and is tested via Travis-CI or another CI service. ***Please add code coverage to your CI and a badge in your readme (see usethis::use_coverage - *License:* The package has a CRAN or OSI accepted license - *Repository:* The repository link resolves correctly - *Archive* (JOSS only, may be post-review): The repository DOI resolves correctly - *Version* (JOSS only, may be post-review): Does the release version given match the GitHub release (v1.0.0)? ------------------------------ Editor comments Here's the goodpractice::gp() output. Nothing to prevent moving to review but these should be straightforward fixes. I will go ahead and find reviewers. ✖ omit "Date" in DESCRIPTION. It is not required and it gets invalid quite often. A build date will be added to the package when you perform `R CMD build` on it. ✖ avoid long code lines, it is bad for readability. Also, many people prefer editor windows that are about 80 characters wide. Try make your lines shorter than 80 characters R/journals.R:9:1 R/journals.R:18:1 R/journals.R:24:1 R/journals.R:33:1 R/journals.R:36:1 ... and 47 more lines ✖ avoid sapply(), it is not type safe. It might return a vector, or a list, depending on the input data. Consider using vapply() instead. R/suppdata.R:126:17 R/suppdata.R:135:17 ✖ fix this R CMD check NOTE: Namespace in Imports field not imported from: ‘RCurl’ All declared Imports should be used. ─────────────────────────────────────────── --- Reviewers: Due date: — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#195 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ABLcUmPDMROSOJ6g127kcsdJQoxcvF1fks5tSjh0gaJpZM4R9X6Y> .

[noamross]

Oh, one other thing, I also get the following error while testing (goodpractice runs as CRAN so it doesn't show up):

test-all.r:52: warning: suppdata fails well
404: Resource not found. - (nonsense)

I believe this is due to a breaking API change in testthat and you want expect_warning().

[willpearse]

Thanks - as I say, I'll do this when I get everything else. Thanks again for your time with this; I realise it's been a bother.

…

On Wed, 7 Feb 2018 at 17:05 Noam Ross ***@***.***> wrote: Oh, one other thing, I also get the following error while testing ( *goodpractice* runs as CRAN so it doesn't show up): test-all.r:52: warning: suppdata fails well 404: Resource not found. - (nonsense) I believe this is due to a breaking API change in *testthat* and you want expect_warning(). — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#195 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ABLcUjjJhQGHJ1UjzxY2f4WXmb_bTqVDks5tSjpDgaJpZM4R9X6Y> .