.travis.yml: Build docker images in parallel

[knatsakis]

Summary

Build docker images for the four supported architectures in parallel, in order to minimize the time needed.

Component Name

.travis.yml

Additional Information

You can find test builds here:
https://travis-ci.org/knatsakis/netdata/builds/572437353
and here:
https://travis-ci.org/knatsakis/netdata/builds/572439557

Please, also find test docker images and manifests here:
https://cloud.docker.com/repository/registry-1.docker.io/knatsakis/netdata/tags

Also, let me mention that, I had to change the manifest updating logic a bit in order to accommodate a parallel build. Instead of always creating a manifest list of the four architectures, the packaging/docker/publish.sh script now:

1. Checks the currently uploaded list of tags for the current version
2. Creates a manifest list containing the discovered tags only

The packaging/docker/{build,publish}.sh scripts now accepts a list of architectures to work on, from the environment via the $ARCHS variable. This can be a space separated list of one, two or more architectures.

[paulkatsoulakis]

I believe this is too much travis work for something that could probably be done easier by utilising docker compose or something similar that we can then easily transfer to a different CI facility

[knatsakis]

Hi @paulkatsoulakis,

Please allow me to respectfully try and change your mind :) :)

The changes in the build.sh and publish.sh scripts mean that the scripts can build one or more targets instead of always building all targets. I would argue that this makes the scripts more flexible and suitable to be re-used in the future under a different CI environment.

Could you please elaborate on why you think that this is a Travis CI specific change?

I agree with you that there may be better solutions to this problem, but I don't see how docker compose could help in this specific case. I don't think that docker compose allows you to build many different images in parallel (on different VMs) instead of serially as it is done now. Perhaps I am missing something?

[paulkatsoulakis]

Hi @paulkatsoulakis,

Please allow me to respectfully try and change your mind :) :)

The changes in the build.sh and publish.sh scripts mean that the scripts can build one or more targets instead of always building all targets. I would argue that this makes the scripts more flexible and suitable to be re-used in the future under a different CI environment.

Yep, the script changes are fine indeed.
I was more concerned on the CI side changes.

Could you please elaborate on why you think that this is a Travis CI specific change?

Well, we are introducing more templates in travis.yml and then extra entries to invoke the build of the different architectures

I agree with you that there may be better solutions to this problem, but I don't see how docker compose could help in this specific case. I don't think that docker compose allows you to build many different images in parallel (on different VMs) instead of serially as it is done now. Perhaps I am missing something?

There is a parallelisation flag for the build process from what i recall, but i won't lie to you i haven't experimented with it. In general i would prefer to avoid on solutions that rely heavily on travis because every change like this means more work for moving out of Travis later.

[cakrit]

There is a parallelisation flag for the build process from what i recall, but i won't lie to you i haven't experimented with it. In general i would prefer to avoid on solutions that rely heavily on travis because every change like this means more work for moving out of Travis later.

If we can get better parallelism today, let's do it. We may need to redesign the entire process in whatever CI we'll use next. Please just test the PR to see if it brings improvements. Travis is taking too long already and we need to add many more OSs and tests to the process.