Setup daily docker builds of the GAP repository, for use in package's Travis tests

[fingolfin]

It would be great if we could setup a Travis cron job, which builds Docker images of GAP every 24 hours or so, for use by packages.

More specifically, the job would upload docker images to some Docker registry which

1. contain a fully compiled GAP,
2. contain the bootstrap package distro,
3. and in which as many packages as possible have been compiled (e.g. copy the logic we use for the testpackages Travis job).

I guess we'd actually generate more than one image here: GAP vs. HPC-GAP, and 32 vs. 64 bit? (And what about debug mode? hum).

These docker images could then be used by Travis tests of GAP packages elsewhere here on GitHub: these packages right now always have to clone a fresh GAP and compile it, which adds 1-2 minutes to their job times (and for some packages, this is easily 90+% of their whole test time). If they could just access a prebuilt docker image, that would speed up the tests, and also simplify the test setup considerably.

[fingolfin]

See also https://docs.travis-ci.com/user/docker/ for how to build and use Docker images with Travis.

[olexandr-konovalov]

I've set up two Docker images:

• https://github.com/gap-system/gap-docker-master
• https://github.com/gap-system/gap-docker-stable

Now I am waiting for the images to appear in DockerHub respectively at

• https://hub.docker.com/r/gapsystem/gap-docker-master/
• https://hub.docker.com/r/gapsystem/gap-docker-stable/

Both are using the gap-docker-base image (https://hub.docker.com/r/gapsystem/gap-docker-base/) which has some dependencies need by some packages.

[olexandr-konovalov]

Docker images are available - they are called

• gapsystem/gap-docker-master
• gapsystem/gap-docker-stable

To test, I suggest to start with gapsystem/gap-docker-stable on some package that should work with GAP 4.8. gapsystem/gap-docker-stable contains the tip of the stable-4.8 branch and packages from GAP 4.8.8 distribution which should provide a robust platform for testing.

• gapsystem/gap-docker-master contains PrimGrp and TransGrp already, but they are not loaded by default - therefore first load GAP with -A option and then these packages, if needed.

Example:

$ docker pull gapsystem/gap-docker-master
Using default tag: latest
latest: Pulling from gapsystem/gap-docker-master
Digest: sha256:dae49a4e8a7dd4b4efe2ea04cecfbc7def926d380ab9d6ac6e7afd05d7d9db37
Status: Image is up to date for gapsystem/gap-docker-master:latest
$ docker run --rm -i -t gapsystem/gap-docker-master
gap@bf59398a0281:~$ ~/inst/gap-master/bin/gap.sh -r -A
 *********   GAP 4.dev of today
 *  GAP  *   https://www.gap-system.org
 *********   Architecture: x86_64-pc-linux-gnu-gcc-default64
 Configuration:  gmp 5.1.3, readline
 Loading the library and packages ...
#I  Overwriting already installed help book 'matgrp not loaded'.
 Components: small* 1.0, id* 1.0
 Packages:   GAPDoc 1.6
 Try '??help' for help. See also '?copyright', '?cite' and '?authors'
gap>

It should be now possible to follow https://docs.travis-ci.com/user/docker/ in "Using a Docker Image from a Repository in a Build". I may try this, but not today.

[olexandr-konovalov]

P.S. You can see an example of using Docker containers in Travis builds here: https://github.com/gap-system/gap-docker-pkg-tests/blob/master/.travis.yml

[olexandr-konovalov]

Next: I've set up two Travis builds

• https://travis-ci.org/gap-system/gap-docker-pkg-tests-stable
• https://travis-ci.org/gap-system/gap-docker-pkg-tests-master

which run standard tests for GAP packages in gapsystem/gap-docker-stable and gapsystem/gap-docker-master containers respectively.

I see that for packages that don't have time consuming tests, the whole step of pulling and starting GAP container and then running a test takes under 3 minutes.

While package authors using Travis CI will be able to use Docker container for continuous integration to test every commit, these two travis builds will be testing published releases of GAP the packages, and will also cover packages which do not using Travis CI and possibly developed elsewhere outside of GitHub.

(builds at https://travis-ci.org/gap-system/gap-docker-pkg-tests-master are failing now because it has already PrimGrp and TransGrp packages, but #1714 is not yet done)

[olexandr-konovalov]

Remains to set up a nightly build of the Docker image. Remote build triggers look exactly like what we need.

[olexandr-konovalov]

All done. To see a collection of Travis tests for packages, click on appropriate badges in the README from https://github.com/gap-system/gap-distribution

[olexandr-konovalov]

Update: all docker-based tests are now in the new organisation https://github.com/gap-infra. GAP issue tracker may still contain URLs pointing to their old location. For the up to date list of all publicly available tests from the GAP test suite, see https://github.com/gap-system/gap-distribution/blob/master/README.md.