A few suggestions & ideas

[lcolladotor]

Hi,

I just started checking rhub and it looks very cool! I just have a few comments.

I find https://builder.r-hub.io/ a bit unintuitive. You have to upload a tar ball created by R CMD build, right? That might not be so obvious to a new developer.

Second, the man pages of check() and similar functions say that path can be the tarball built with R CMD check. Isn't that R CMD build? I see that rhub:::build_package() runs build.

Third, lets say I'm interested in running checks on a bunch of OS. With any of the check* functions used on a directory containing an R package, you end up running rhub:::build_package(). It might be nice to export this function or make it more evident that you can run:

library('rhub')
pkgtar <- build_package('my_pkg_dir')
check_on_linux(pkgtar)
check_on_windows(pkgtar)
## Or any other checks

Finally, I'm wondering if rhub can be tied into Travis in the following way. With Travis you can set up matrix environments, so you can imagine setting a 'platform' environment for several platforms you are interested in checking. Then you configure Travis to install rhub and run:

library('rhub')
check('.', platform = Sys.getenv('platform'))

That way you use Travis to run R CMD build but then use rhub for testing on environments beyond linux. To save time, the user would have to setup Travis in such a way that R CMD check is disabled.

Best,
Leo

[lcolladotor]

Regarding the Travis idea, I should mention that the rhub TOS mention that the number of tests might be limited in the future.

Edit: Anyhow, using Travis for R CMD build would be nice in case you have a package that takes a few minutes to build (like several of mine :P).

[lcolladotor]

Hi again,

My Travis + rhub test worked as shown at https://builder.r-hub.io/status/jaffelab_0.99.0.tar.gz-9aa0b96898ae45408696555217a5ce71. The actual R CMD check on windows-x86_64-devel failed due to #38 but that's ok.

Best,
Leo

[gaborcsardi]

Thanks for the comments!

I find https://builder.r-hub.io/ a bit unintuitive. You have to upload a tar ball created by R CMD build, right? That might not be so obvious to a new developer.

Agreed, that page needs improvements.

Second, the man pages of check() and similar functions say that path can be the tarball built with R CMD check. Isn't that R CMD build? I see that rhub:::build_package() runs build.

Yes, typo, should be build. Do you want to submit a PR?

Third, lets say I'm interested in running checks on a bunch of OS. With any of the check* functions used on a directory containing an R package, you end up running rhub:::build_package(). It might be nice to export this function or make it more evident that you can run:

You can just submit a vector of platforms to check():

check(..., platform = c("platform1", "platform2"))

You'll get multiple URLs and multiple emails, which is not ideal, this will be improved in the future, by providing "meta"-jobs, similar to matrix jobs on Travis.

Finally, I'm wondering if rhub can be tied into Travis in the following way. With Travis you can set up matrix environments, so you can imagine setting a 'platform' environment for several platforms you are interested in checking.

Interesting approach, very creative! rhub will have it's own CI soon, so ideally you would not need to call it from Travis. Also, you can just submit a vector of platforms to check(), as above. So you can just build() once, and then run the check on multiple platforms.

[lcolladotor]

Hi,

Thanks for the clarifications! From the docs I didn't realize that platform could handle a vector of platforms.

I just wrote the PR that fixes the build typo, links to build() on the docs and mentions that platform can be a vector argument.

Travis + rhub

In relation to Travis + rhub, with a .travis.yml config file that includes these lines:

r_github_packages:
  - r-hub/rhub

after_success:
  - Rscript -e "library(rhub); validate_email('email', 'token'); check(Sys.getenv('PKG_TARBALL'), c('windows-x86_64-devel', 'debian-gcc-devel'))"

basically you (the pkg developer) choose a main platform (Ubuntu with a R-version of your choice by default with Travis) to work with. Only if the check is successful in that main platform, you then invoke rhub to test on other platforms. The advantage I see is that you get less emails/build reports for each commit you make and in a sense you don't waste resources (rhub's Azure credits) for commits you aren't sure if they are supposed to be error-free.

With osx as the main Travis platform

With the following on .travis.yml I tried setting OSX as the main platform since it's the one missing in rhub::platforms().

os: osx

However it failed because Travis was missing some packages for rhub to work as shown at https://travis-ci.org/LieberInstitute/jaffelab/builds/173073156#L1717. I probably just need to specify these packages in .travis.yml. But well, the point is that it might be possible to replicate the Bioconductor nightly builds in Windows, OSX and linux with R-devel. Right now, I test a lot on linux (thanks to Travis) but have to wait for the Windows and OSX reports if something goes wrong in those platforms.

Anyhow, I'm definitely excited to see what the rhub CI will look like =) Thanks again for all your work!

Best,
Leo

PS I'm assuming that there's no problem with making the rhub token public.

[lcolladotor]

Hi,

Regarding using osx as the main Travis platform, it took me a while, but I finally got it to work. I basically had to write my Travis config file in such a way that all dependencies got installed (see https://github.com/LieberInstitute/jaffelab/blob/54f0621816f423a7bc33f374b1a9a09ded8cad55/.travis.yml#L10-L25 for a simple package).

Best,
Leo

[gaborcsardi]

I think I can close this now, right?

[lcolladotor]

Yup, that's right =) Updating https://builder.r-hub.io/ might have to be it's own issue and in any case I bet that it's low priority.

Best,
Leo

[gaborcsardi]

Not low priority at all...., thanks for reminding me.