New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feature/travis docker #894

Merged
merged 6 commits into from Sep 2, 2018

Conversation

Projects
None yet
2 participants
@eddelbuettel
Member

eddelbuettel commented Aug 29, 2018

This is still somewhat experimental, but it is now the third GitHub repo where I tried this with success.

In essence, we convert Travis to 'bring your own container' which is quite powerful as the CI runs now become portable. By relying on a Docker Hub container, the same test could run at GitLab, or on your computer, or anywhere else.

Plus, by relying on containers we open up for easy / easier test matrices with r-devel, r-olrel, r-rchk, r-ubsan, r-with-different-compilers, ... Truly endless possibilities.

[ The old system worked well for us too, but I need to update it as well to at least get us to using R 3.5.x (and it is a bug in my r-travis repo that we're not yet ... but I have some setups still relying on PPAs built against R 3.4.x -- but different bug in different repo. ]

Thoughts or comments on this?

@eddelbuettel

This comment has been minimized.

Show comment
Hide comment
@eddelbuettel

eddelbuettel Aug 29, 2018

Member

Related: I also created a Docker organisation for us.

I will move the container build there if/when this docker/ directory is in master. For now it is still off my handle, you can see the docker pull invocation in the .travis.yml (behind an env.var.).

Member

eddelbuettel commented Aug 29, 2018

Related: I also created a Docker organisation for us.

I will move the container build there if/when this docker/ directory is in master. For now it is still off my handle, you can see the docker pull invocation in the .travis.yml (behind an env.var.).

@kevinushey

This comment has been minimized.

Show comment
Hide comment
@kevinushey

kevinushey Aug 29, 2018

Contributor

Thanks for putting this together!

Does this also imply improved build / test times? (Presumedly this means we can reuse a prebuilt Docker image for testing?)

Is there any worry about the Docker images becoming 'stale' as e.g. apt repositories are updated? How do those updates get incorporated into the image?

Contributor

kevinushey commented Aug 29, 2018

Thanks for putting this together!

Does this also imply improved build / test times? (Presumedly this means we can reuse a prebuilt Docker image for testing?)

Is there any worry about the Docker images becoming 'stale' as e.g. apt repositories are updated? How do those updates get incorporated into the image?

@eddelbuettel

This comment has been minimized.

Show comment
Hide comment
@eddelbuettel

eddelbuettel Aug 29, 2018

Member

I wondered about the timing too! It seems rather comparable -- but it is a small sample so far. As of right now we have three different repos using this approach:

  • for rvw we have no comparison (as we had no old setup; it always needed a too-new library for old Ubuntu)
  • for RQuantLib the times seem comparable
  • for Rcpp it seems comparable but even smaller sample.

You should be able to find the Travis histories from thre repos to look in more detail, else I can add links here.

The one metric I looked at was docker pull time compared to installing all the .deb packages, which for Rcpp we bear twice (given that covr needs additional packages). And oddly enough ... docker wins. So if we're slower the compilations may be slower (which I don't believe ex ante as Docker is pretty lightweight on Linux). So no concern here yet AFAICT.

Next, the staleness could be a concern but a) the "public" (to us, in the same PR) Dockerfile coupled with b) the Docker org I set up should alleviate that. You can have Docker containers rebuilt on each commit (that is even the default). That is probably overkill here as we depend only on R and little else. We can always trigger rebuilds, or have cron trigger them weekly (something we do for Rocker at different frequencies, or, ...)

And ... you could argue that it is better because master still checks via my r-travis script which is (cough, cough) still at R 3.4.4. Whereas this now uses R 3.5.1 on Debian testing to boot. To me, the old Travis 14.04 really blows.

And there are a few other possible upsides to going Docker: easier build matrix across R releases, compiler versions, maybe ubsan / rchk if we care etc pp.

I think this has a lot of potential but I don't want to rush this.

So let's try to find some other reasons "not to do this" so that we're sure. What else can we think, and of course which of my arguments above do you find unconvincing?

Member

eddelbuettel commented Aug 29, 2018

I wondered about the timing too! It seems rather comparable -- but it is a small sample so far. As of right now we have three different repos using this approach:

  • for rvw we have no comparison (as we had no old setup; it always needed a too-new library for old Ubuntu)
  • for RQuantLib the times seem comparable
  • for Rcpp it seems comparable but even smaller sample.

You should be able to find the Travis histories from thre repos to look in more detail, else I can add links here.

The one metric I looked at was docker pull time compared to installing all the .deb packages, which for Rcpp we bear twice (given that covr needs additional packages). And oddly enough ... docker wins. So if we're slower the compilations may be slower (which I don't believe ex ante as Docker is pretty lightweight on Linux). So no concern here yet AFAICT.

Next, the staleness could be a concern but a) the "public" (to us, in the same PR) Dockerfile coupled with b) the Docker org I set up should alleviate that. You can have Docker containers rebuilt on each commit (that is even the default). That is probably overkill here as we depend only on R and little else. We can always trigger rebuilds, or have cron trigger them weekly (something we do for Rocker at different frequencies, or, ...)

And ... you could argue that it is better because master still checks via my r-travis script which is (cough, cough) still at R 3.4.4. Whereas this now uses R 3.5.1 on Debian testing to boot. To me, the old Travis 14.04 really blows.

And there are a few other possible upsides to going Docker: easier build matrix across R releases, compiler versions, maybe ubsan / rchk if we care etc pp.

I think this has a lot of potential but I don't want to rush this.

So let's try to find some other reasons "not to do this" so that we're sure. What else can we think, and of course which of my arguments above do you find unconvincing?

@kevinushey

This comment has been minimized.

Show comment
Hide comment
@kevinushey

kevinushey Aug 29, 2018

Contributor

I would agree that the upsides far outweigh the potential downsides.

The only other possible concern I can think of is whether we could bump into some kind of issue that reproduces within a Docker environment but not in a plain old virtual machine, but it seems like those sorts of issues are effectively non-existent nowadays unless you're doing really low-level Linux-y stuff (something Rcpp doesn't do).

Either way, I think it's worth sleeping on this for a couple days in case some compelling blocker does come to mind but overall I think this will be a net positive.

Contributor

kevinushey commented Aug 29, 2018

I would agree that the upsides far outweigh the potential downsides.

The only other possible concern I can think of is whether we could bump into some kind of issue that reproduces within a Docker environment but not in a plain old virtual machine, but it seems like those sorts of issues are effectively non-existent nowadays unless you're doing really low-level Linux-y stuff (something Rcpp doesn't do).

Either way, I think it's worth sleeping on this for a couple days in case some compelling blocker does come to mind but overall I think this will be a net positive.

@eddelbuettel

This comment has been minimized.

Show comment
Hide comment
@eddelbuettel

eddelbuettel Sep 2, 2018

Member

So having slept over this a few more days, any concerns?

Member

eddelbuettel commented Sep 2, 2018

So having slept over this a few more days, any concerns?

@kevinushey

This comment has been minimized.

Show comment
Hide comment
@kevinushey

kevinushey Sep 2, 2018

Contributor

Nope! I think we can merge this.

Contributor

kevinushey commented Sep 2, 2018

Nope! I think we can merge this.

@eddelbuettel eddelbuettel merged commit d2e2f55 into master Sep 2, 2018

2 checks passed

continuous-integration/travis-ci/pr The Travis CI build passed
Details
continuous-integration/travis-ci/push The Travis CI build passed
Details

@eddelbuettel eddelbuettel deleted the feature/travis_docker branch Sep 2, 2018

@eddelbuettel

This comment has been minimized.

Show comment
Hide comment
@eddelbuettel

eddelbuettel Sep 2, 2018

Member

@kevinushey I still wonder about the timing though. We can't control for load at their end, networking etc so a quick local comparison:

time R CMD check --no-manual --no-vignettes                   3min 49sec    3min 53sec
time docker run ... R CMD check --no-manual --no-vignettes    3min 51sec    3min 46sec

and I alternated (ie normal/Docker/normal/Docker). So I guess it is a tie, and Docker has no real discernible cost.

Member

eddelbuettel commented Sep 2, 2018

@kevinushey I still wonder about the timing though. We can't control for load at their end, networking etc so a quick local comparison:

time R CMD check --no-manual --no-vignettes                   3min 49sec    3min 53sec
time docker run ... R CMD check --no-manual --no-vignettes    3min 51sec    3min 46sec

and I alternated (ie normal/Docker/normal/Docker). So I guess it is a tie, and Docker has no real discernible cost.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment