Kubernetes/Cluster deployment

[sk1p]

• create a Docker image Make a docker image #484
• create automation for installing libertem on a k8s cluster (helm?)
• Optimize handling of result tiles #106 Optimize handling of result tiles (added by @uellue)

See also #67 for auth related questions.

Edit: Removed HDFS aspects

[sk1p]

For docker, see also: https://simonwillison.net/2018/Nov/19/smaller-python-docker-images/

[irahulcse]

https://stackoverflow.com/questions/26753030/how-to-build-docker-image-from-github-repository/39194765

https://docs.docker.com/docker-hub/builds/
Can @sk1p , we can think about creating the docker image directly through Github?

[sk1p]

We would love to have a docker build and publish at the end of our CI pipeline. Currently, we are building (not publishing) a docker image as a step of the dist job, but in addition, we should publish it if all other jobs have succeeded, in the packaging stage (see .travis.yml).

I think using the automated building from docker hub means the docker image will be built independently from the CI fail/success status, right?

[irahulcse]

Yes, I also think so the docker hub build will be independent of CI fail/success status. Not quite sure about it but might we need to update the docker hub after every new build?.

@sk1p and @uellue I am moving forward with this https://docs.docker.com/docker-hub/builds/ and hope everything would be fine at the end.

[sk1p]

Hmm, I'm not convinced that's the way to go. How do you prevent broken docker images from being pushed? If your answer is using autotest, how do you prevent having a completely separate test run from the rest of our pipeline? Feel free to play around with the build service, but you'll have to convince me to make it the official image 😁

On our road map, we have a migration to azure pipelines for CI, to consolidate our travis and appveyor builds into a single service. We already have the basics implemented in #565, it's mostly missing replacements for some travis specifics (basically travis-sphinx and updating the release script to migrate from the TRAVIS_* environment variables). Do you think that it may be possible to build the docker image from an azure pipeline job?