Feature/travis docker

[eddelbuettel]

This is still somewhat experimental, but it is now the third GitHub repo where I tried this with success.

In essence, we convert Travis to 'bring your own container' which is quite powerful as the CI runs now become portable. By relying on a Docker Hub container, the same test could run at GitLab, or on your computer, or anywhere else.

Plus, by relying on containers we open up for easy / easier test matrices with r-devel, r-olrel, r-rchk, r-ubsan, r-with-different-compilers, ... Truly endless possibilities.

[ The old system worked well for us too, but I need to update it as well to at least get us to using R 3.5.x (and it is a bug in my r-travis repo that we're not yet ... but I have some setups still relying on PPAs built against R 3.4.x -- but different bug in different repo. ]

Thoughts or comments on this?

[eddelbuettel]

Related: I also created a Docker organisation for us.

I will move the container build there if/when this docker/ directory is in master. For now it is still off my handle, you can see the docker pull invocation in the .travis.yml (behind an env.var.).

[kevinushey]

Thanks for putting this together!

Does this also imply improved build / test times? (Presumedly this means we can reuse a prebuilt Docker image for testing?)

Is there any worry about the Docker images becoming 'stale' as e.g. apt repositories are updated? How do those updates get incorporated into the image?

[eddelbuettel]

I wondered about the timing too! It seems rather comparable -- but it is a small sample so far. As of right now we have three different repos using this approach:

• for rvw we have no comparison (as we had no old setup; it always needed a too-new library for old Ubuntu)
• for RQuantLib the times seem comparable
• for Rcpp it seems comparable but even smaller sample.

You should be able to find the Travis histories from thre repos to look in more detail, else I can add links here.

The one metric I looked at was docker pull time compared to installing all the .deb packages, which for Rcpp we bear twice (given that covr needs additional packages). And oddly enough ... docker wins. So if we're slower the compilations may be slower (which I don't believe ex ante as Docker is pretty lightweight on Linux). So no concern here yet AFAICT.

Next, the staleness could be a concern but a) the "public" (to us, in the same PR) Dockerfile coupled with b) the Docker org I set up should alleviate that. You can have Docker containers rebuilt on each commit (that is even the default). That is probably overkill here as we depend only on R and little else. We can always trigger rebuilds, or have cron trigger them weekly (something we do for Rocker at different frequencies, or, ...)

And ... you could argue that it is better because master still checks via my r-travis script which is (cough, cough) still at R 3.4.4. Whereas this now uses R 3.5.1 on Debian testing to boot. To me, the old Travis 14.04 really blows.

And there are a few other possible upsides to going Docker: easier build matrix across R releases, compiler versions, maybe ubsan / rchk if we care etc pp.

I think this has a lot of potential but I don't want to rush this.

So let's try to find some other reasons "not to do this" so that we're sure. What else can we think, and of course which of my arguments above do you find unconvincing?

[kevinushey]

I would agree that the upsides far outweigh the potential downsides.

The only other possible concern I can think of is whether we could bump into some kind of issue that reproduces within a Docker environment but not in a plain old virtual machine, but it seems like those sorts of issues are effectively non-existent nowadays unless you're doing really low-level Linux-y stuff (something Rcpp doesn't do).

Either way, I think it's worth sleeping on this for a couple days in case some compelling blocker does come to mind but overall I think this will be a net positive.

[eddelbuettel]

So having slept over this a few more days, any concerns?

[kevinushey]

Nope! I think we can merge this.

[eddelbuettel]

@kevinushey I still wonder about the timing though. We can't control for load at their end, networking etc so a quick local comparison:

time R CMD check --no-manual --no-vignettes                   3min 49sec    3min 53sec
time docker run ... R CMD check --no-manual --no-vignettes    3min 51sec    3min 46sec

and I alternated (ie normal/Docker/normal/Docker). So I guess it is a tie, and Docker has no real discernible cost.