getCRUCLdata: Download and Use CRU CL2.0 Climatology Data in R

[adamhsparks]

Summary

• What does this package do? (explain in 50 words or less):
  The getCRUCLdata package provides two functions that automate downloading and importing CRU CL2.0 climatology data, facilitates the calculation of minimum temperature and maximum temperature, and formats the data into a tidy data frame or a list of raster stack objects for use in an R session or easily exports to a raster format file for use in a geographic information system (GIS).

• Paste the full DESCRIPTION file inside a code block below:

Package: getCRUCLdata
Type: Package
Title: Download and Use CRU CL2.0 Climatology Data in R
Version: 0.1.2
Authors@R: person(given = "Adam", family = "Sparks", email = "adamhsparks@gmail.com", role = c("aut", "cre"))
Description: Provides functions that automate downloading and importing
    University of East Anglia Climate Research Unit (CRU) CL2.0 climatology data
    into R, facilitates the calculation of minimum temperature and maximum
    temperature and formats the data into a tidy data frame or a list of raster
    stack objects for use in an R session.  CRU CL2.0 data are a gridded
    climatology of 1961-1990 monthly means released in 2002 and cover all land
    areas (excluding Antarctica) at 10-minute resolution.  For more
    information see the description of the data provided by the University of
    East Anglia Climate Research Unit,
    <https://crudata.uea.ac.uk/cru/data/hrg/tmc/readme.txt>.
License: MIT + file LICENSE
Depends: R (>= 3.0.0)
Imports:
    curl,
    dplyr,
    plyr,
    purrr (>= 0.2.0),
    raster,
    tidyr,
    utils
LazyData: TRUE
RoxygenNote: 5.0.1
ByteCompile: TRUE
Suggests:
    testthat,
    knitr,
    rmarkdown,
    covr
URL: https://github.com/adamhsparks/getCRUCLdata
BugReports: https://github.com/adamhsparks/getCRUCLdata/issues
VignetteBuilder: knitr

• URL for the package (the development repository, not a stylized html page):
  https://github.com/adamhsparks/getCRUCLdata

• Who is the target audience?
  Anyone interested in using climate data or generating raster files from the CRU CL2.0 data that are easy to use in a GIS

• Are there other R packages that accomplish the same thing? If so, what is different about yours?
  Not that I'm aware of

Requirements

Confirm each of the following by checking the box. This package:

• does not violate the Terms of Service of any service it interacts with.
• has a CRAN and OSI accepted license.
• contains a README with instructions for installing the development version.
• includes documentation with examples for all functions.
• contains a vignette with examples of its essential functions and uses.
• has a test suite.
• has continuous integration with Travis CI and/or another service.

Publication options

• Do you intend for this package to go on CRAN?
• Do you wish to automatically submit to the Journal of Open Source Software? If so:
  • The package contains a paper.md with a high-level description in the package root or in inst/.
  • The package is deposited in a long-term repository with the DOI: https://doi.org/10.5281/zenodo.264405
  • (Do not submit your package separately to JOSS)

Detail

• Does R CMD check (or devtools::check()) succeed? Paste and describe any errors or warnings:

• Does the package conform to rOpenSci packaging guidelines? Please describe any exceptions:

• If this is a resubmission following rejection, please explain the change in circumstances:

• If possible, please provide recommendations of reviewers - those with experience with similar packages and/or likely users of your package - and their GitHub user names:

Ivan Hanigan, https://github.com/ivanhanigan

[sckott]

Editor checks:

• Fit: The package meets criteria for fit and overlap
• Automated tests: Package has a testing suite and is tested via Travis-CI or another CI service.
• License: The package has a CRAN or OSI accepted license
• Repository: The repository link resolves correctly
• Archive (JOSS only, may be post-review): The repository DOI resolves correctly
• Version (JOSS only, may be post-review): Does the release version given match the GitHub release (v1.0.0)?

---

Editor comments

Currently seeking reviewers. It's a good fit and not overlapping.

• Tests do succeed, but take a long time, would be nice to get these to run faster somehow
• Ran goodpractice::gp() , but nothing to deal with at this time.

---

Reviewers: @ldecicco-USGS @ivanhanigan
Due date: 2017-02-22

[adamhsparks]

@sckott, I'm working on getting the test time down. I've managed to reduce it by 1/2 already and keep the code coverage the same. I'll see what I can do further.

[sckott]

thanks @adamhsparks - thanks for trying!

[sckott]

reviewers assigned, see #96 (comment)

[ldecicco-USGS]

Package Review

Please check off boxes as applicable, and elaborate in comments below. Your review is not limited to these topics, as described in the reviewer guide

• As the reviewer I confirm that there are no conflicts of interest for me to review this work (such as being a major contributor to the software).

Documentation

The package includes all the following forms of documentation:

• A statement of need clearly stating problems the software is designed to solve and its target audience in README
• Installation instructions: for the development version of package and any non-standard dependencies in README
• Vignette(s) demonstrating major functionality that runs successfully locally
• Function Documentation: for all exported functions in R help
• Examples for all exported functions in R Help that run successfully locally
• Community guidelines including contribution guidelines in the README or CONTRIBUTING, and URL, Maintainer and BugReports fields in DESCRIPTION

Paper (for packages co-submitting to JOSS)

The package contains a paper.md with:

• A short summary describing the high-level functionality of the software
• Authors: A list of authors with their affiliations
• A statement of need clearly stating problems the software is designed to solve and its target audience.
• References: with DOIs for all those that have one (e.g. papers, datasets, software).

Functionality

• Installation: Installation succeeds as documented.
• Functionality: Any functional claims of the software been confirmed.
• Performance: Any performance claims of the software been confirmed.
• Automated tests: Unit tests cover essential functions of the package
  and a reasonable range of inputs and conditions. All tests pass on the local machine.
• Packaging guidelines: The package conforms to the rOpenSci packaging guidelines

Final approval (post-review)

• The author has responded to my review and made changes to my satisfaction. I recommend approving this package.

Estimated hours spent reviewing:

---

Review Comments

Installation

I'm not sure if this is actually an issue with the package, but when I tried to install via CRAN, there was an error via the Windows binary:

> install.packages("getCRUCLdata")
Installing package into ‘C:/Users/ldecicco/Documents/R/win-library/3.3’
(as ‘lib’ is unspecified)
trying URL 'https://cloud.r-project.org/bin/windows/contrib/3.3/getCRUCLdata_0.1.4.zip'
Content type 'application/zip' length 160600 bytes (156 KB)
downloaded 0 bytes

Warning in install.packages :
  downloaded length 0 != reported length 160600
Warning in install.packages :
  error 1 in extracting from zip file
Warning in install.packages :
  cannot open compressed file 'getCRUCLdata/DESCRIPTION', probable reason 'No such file or directory'
Error in install.packages : cannot open the connection

I could install via source just fine. This could be an issue with my CRAN mirror or something else...but I haven't experienced it before.

Vignette

The first thing I tried to do was run the first example in the vignette, and was unsuccessful:

library(getCRUCLdata)
CRU_data <- create_CRU_df(pre = TRUE,
                          pre_cv = TRUE,
                          rd0 = TRUE,
                          tmp = TRUE,
                          dtr = TRUE,
                          reh = TRUE,
                          tmn = TRUE,
                          tmx = TRUE,
                          sunp = TRUE,
                          frs = TRUE,
                          wnd = TRUE,
                          elv = TRUE)
 
Downloading requested data files.
 
Error in .f(.x[[i]], ...) : Too many retries...server may be under load

I then tried setting a single parameter to TRUE, and still could not get data to return:

CRU_data <- create_CRU_df(pre = TRUE)
 
Downloading requested data files.
 
Error in .f(.x[[i]], ...) : Too many retries...server may be under load

Examples

I had the same problem running the examples from the help files as running the examples from the vignette.

Tests

Similarly, the tests via testthat didn't pass with the same error "Too many retries...server may be under load". However, looking at the tests, they looked like appropriate tests. According to Coveralls, there's a 59% coverage on code. This isn't great, but it's not too bad.

Troubleshooting

When I went to the URLs to test the service, they worked fine. I use httr for the package dataRetrieval (so, not as familiar with purr). I tried this:

library(httr)

temp <- tempfile()
temp <- paste0(temp,".dat.gz")
obs_url <- "https://crudata.uea.ac.uk/cru/data/hrg/tmc/grid_10min_dtr.dat.gz"
doc <- GET(obs_url, write_disk(temp))

headerInfo <- headers(doc)
doc2 <- read.table(gzfile(temp))

unlink(doc)

doc2 came back as a table with 14 columns and 566262 rows. So, maybe give httr a try? As it is, I can't get anything back from the package, and I don't think it's the underlying services fault.

[sckott]

Thanks for your review @ldecicco-USGS !