Add use_circleci()

[jdblischak]

I wrote a function use_circleci() to setup continuous integration with CircleCI. I modeled it after use_travis().

Features

The configuration file it creates enables the following features:

• Uses the Docker image rocker/verse:latest (DockerHub) as the environment for building and checking the package. This image contains TeX Live, pandoc, and the tidyverse packages, so it should be sufficient for many R packages. However, the user has the option of specifying the Docker image to use via the argument image.

• Very simple caching of packages. The CircleCI platform provides a complex caching mechanism, but I chose not to use this since devtools::install_deps() can update packages as needed. Also, the default is for the cache to expire after 30 days. If desired, a more sophisticated option would be to invalidate the cache whenever a specific file is changed, e.g. using a key like cache-{{ checksum "DESCRIPTION" }}.

• The *.Rcheck/ directory is uploaded as an artifact at the end of build.

Other notes

• The tests I added passed on my local machine and on the AppVeyor and Travis builds of my forked repository
• I couldn't find the function ui_ that is used in check_uses_travis() (line 67), so I used ui_field() in check_uses_circleci()
• If this new function is accepted, I'll add a bullet point to NEWS.md

[jennybc]

I'm releasing usethis today, so it's about to rest for a little while. But we anticipate another release before useR!

[jennybc]

Hi @jdblischak I used this PR as a demo of pr_*() functions in usethis during our group meeting today, which is why you see this odd little commit from me.

[jdblischak]

I used this PR as a demo of pr_*() functions in usethis during our group meeting today, which is why you see this odd little commit from me.

@jennybc Understood. Thanks!

[jdblischak]

Some potential ideas to improve the default configuration (with the downside of increased complexity):

Report installation information

Similar to the current setup for Travis or Appveyor, we could add a step that reports the installation information, e.g.

      - run:
          name: Session information and installed package versions
          command: |
            Rscript -e 'sessionInfo()'
            Rscript -e 'installed.packages()[, c("Package", "Version")]'
            Rscript -e 'rmarkdown::pandoc_version()'

Separate cache for each job

If a user tests multiple versions of R in different jobs, they won't want to use the same cache. We could make the default caching mechanism slightly more complicated by default to anticipate this use case. The following will store a separate cache for each job:

    steps:
      - restore_cache:
          keys:
            - cache-{{ .Environment.CIRCLE_JOB }}

      - save_cache:
          key: cache-{{ .Environment.CIRCLE_JOB }}
          paths:
            - "/usr/local/lib/R/site-library"

[jimhester]

Probably best to use the remotes packages for this, something like

R -e 'if (!require("remotes")) install.packages("remotes")' -e 'remotes::install_deps(dependencies = TRUE)'

We might even just want to unconditionally call install.packages("remotes") if for no other reason that to ensure it is installed and is the latest version from CRAN.

[jdblischak]

To ensure it was always up-to-date, I think we'd need to unconditionally install it. The rocker image verse already has remotes/devtools installed.

[jimhester]

At least the rocker containers set the R_LIBS_USER environment variable (to this location), maybe this should be $R_LIBS_USER ? Although I am not sure if circleci will expand the environment variable here...

[jimhester]

Also IIRC because this is the default library location for the verse docker container, the CircleCI cache will contain all of the package already installed in the library. It might be better to do something like R_LIBS_USER=/foo/bar:${R_LIBS_USER} and setup new library that is only dependencies installed by CircleCI.

[jdblischak]

This might be possible using parameters.

But I do think that most users will use a rocker image, and any user that can create an R environment from a more minimal Docker image (e.g. a standard Ubuntu image) should be able to handle updating the library path.

[jdblischak]

R_LIBS_USER has to be set in .Renviron, right? Do you want me to add a step to create this file?

[jimhester]

No, it is just a normal shell environment variable, it can be set like you would normally in the CI.

[jdblischak]

Starting a Docker container with docker run --rm -it rocker/verse:3.5.2 bash

export R_LIBS_USER=~/R/Library
echo $R_LIBS_USER
## /root/R/Library/
mkdir -p $R_LIBS_USER
Rscript -e 'Sys.getenv("R_LIBS_USER")'
## [1] "/usr/local/lib/R/site-library"

[jdblischak]

OK. I confirmed that the problem is that the rocker image is setting R_LIBS_USER in a system Renviron (line). When I delete that line, then it respects the setting of the environment variable.

[jdblischak]

@jimhester Since setting R_LIBS_USER as a standard environment variable didn't work in the rocker image, I tried defining it in .Renviron. This was at least recognized by R, but it was added to the end of the of the library path.

mkdir -p ~/R/Library
echo R_LIBS_USER=~/R/Library > ~/.Renviron
Rscript -e '.libPaths()'
## [1] "/usr/local/lib/R/site-library" "/usr/local/lib/R/library"     
## [3] "/root/R/Library"              

Thus I then tried using .Rprofile. I've had to resort to this hack before in order to configure R properly on my HPC. Note that only the call to .libPaths() is actually required. Including the call to Sys.setenv() is more for an end user trying to understand how the library paths were setup.

mkdir -p ~/R/Library
echo 'Sys.setenv(R_LIBS_USER = "~/R/Library")' > ~/.Rprofile
echo '.libPaths("~/R/Library")' >> ~/.Rprofile
cat ~/.Rprofile 
## Sys.setenv(R_LIBS_USER = "~/R/Library")
## .libPaths("~/R/Library")
Rscript -e '.libPaths()'
## [1] "/root/R/Library"               "/usr/local/lib/R/site-library"
## [3] "/usr/local/lib/R/library" 

Another alternative would be to set lib = "~/R/Library" when calling install.packages() and remotes::install_deps().

Thoughts on which solution I should implement?

[jimhester]

The problem is the rocker container is also setting R_LIBS and R_LIBS takes precedence over R_LIBS_USER, so you have to use that in this case.

[jdblischak]

Setting R_LIBS worked. Thanks! By setting R_LIBS=~/R/library and creating this directory, .libPaths() returns the desired order:

export R_LIBS=~/R/library
mkdir -p ~/R/library
Rscript -e '.libPaths()'
## [1] "/root/R/library"               "/usr/local/lib/R/site-library"
## [3] "/usr/local/lib/R/library" 

[jimhester]

From reading https://circleci.com/docs/2.0/caching/#using-keys-and-templates I think we would want something like

 - r-pkg-cache-{{ arch }}-{{ .Branch }}
 - r-pkg-cache-{{ arch }}-

This would give you specific caches for the architecture and branch, but would fall back to caches if they are available not available.

[jdblischak]

OK. I can update the caching logic.

[jdblischak]

@jimhester This PR is ready for another round of review. Thanks!

[jimhester]

looks like you have some unresolved conflict markers here

[jdblischak]

Strange. I didn't think Git would allow me to add the file and git rebase --continue if the file still contained the conflict markers. I'll fix that real quick.

[jimhester]

No by default, but you can use git diff --check to warn about them, or put this in a pre-commit hook if you want to automate it.

[jimhester]

LGTM other than the conflict marker issue.

[jennybc]

Thanks to @jdblischak for the PR and @jimhester for reviewing. I've taken another pass, with a different hat on. Just a few small changes.

[jennybc]

Can you run styler over any code that you're touching? I think we want this in curly braces, with newlines.

[jennybc]

Can you add skip_if_no_git_config()?

[jennybc]

Please switch over to usethis::use_git_remote(). We will probably switch over to gert at some point, so I don't want to gain any new git2r calls.

[jennybc]

I think styler would re-format this too.

[jennybc]

Can you add skip_if_no_git_config()?

[jennybc]

Please switch over to usethis::use_git_remote().

[jennybc]

styler might change this

[jennybc]

Same tweaks as above

[jdblischak]

@jennybc Thanks for the review! I had clearly overlooked the bullet point about the code style. I think I've addressed everything, so please let me know if I missed anything.

[jennybc]

Thanks!