Provide caching of docker images across VM instances

[jstrachan]

this might be a bit tricky to implement, but it'd be awesome if we could use a http proxy with a persistent cache in the docker daemon in minishift thats stored on the host volume somewhere, so that if you do:

minishift start
# download and run lots of images...
minishift delete
...
minishift start
# download and run lots of images - using the previous image cache!

when trying out new versions of minishift/openshift and apps its pretty common to trash your VM and recreate it again. I'm on kinda slow wifi as am sure most folks are when at conferences or with customers; it'd be awesome to be able to cache docker images on our host operating system.

[hferentschik]

We are already working on an approach for caching images. See also #143, even though this might initially just cache the OpenShift related images. The long term plan is to use the libraries used under the hood of skopeo to cache all images.

I leave this issue open, even though it is partly a duplicate of #143. As said, the latter might end up as an intermediate solution first until we can work around issues with containers/storage and containers/image. We can use #143 to implement a simple solution targeting mainly at seedup of the bootstrap process and take this issue as continuation to provide a full image chache.

[hferentschik]

We are already working on an approach for caching images. See also #143, even though this might initially just cache the OpenShift related images. The long term plan is to use the libraries used under the hood of skopeo to cache all images.

I leave this issue open, even though it is partly a duplicate of #143. As said, the latter might end up as an intermediate solution first until we can work around issues with containers/storage and containers/image. We can use #143 to implement a simple solution targeting mainly at seedup of the bootstrap process and take this issue as continuation to provide a full image chache.

[jorgemoralespou]

I ultimately want caching of all images in order to make minishift environment transportable #397 (targeted for 1.4).

[jorgemoralespou]

I ultimately want caching of all images in order to make minishift environment transportable #397 (targeted for 1.4).

[hferentschik]

I ultimately want caching of all images in order to make minishift environment transportable

+1

[hferentschik]

I ultimately want caching of all images in order to make minishift environment transportable

+1

[jorgemoralespou]

I would want to be able to have in configuration a whitelist/blacklist of images to be cached so the developer can have greater granularity on what to save/restore.

Ideally, any image built by the VM should not be same, anything else could potentially be saved, being all the openshift related images the most important, although there is a big chance that all the technology related images would need to be cached. I wouldn't be happy to need to pull down openjdk or EAP or other images every time I create a VM.

[jorgemoralespou]

I would want to be able to have in configuration a whitelist/blacklist of images to be cached so the developer can have greater granularity on what to save/restore.

Ideally, any image built by the VM should not be same, anything else could potentially be saved, being all the openshift related images the most important, although there is a big chance that all the technology related images would need to be cached. I wouldn't be happy to need to pull down openjdk or EAP or other images every time I create a VM.

[budhram]

I would want to be able to have in configuration a whitelist/blacklist of images to be cached so the developer can have greater granularity on what to save/restore.

Seems to be the advanced step for #397 and can be taken after it.

[budhram]

I would want to be able to have in configuration a whitelist/blacklist of images to be cached so the developer can have greater granularity on what to save/restore.

Seems to be the advanced step for #397 and can be taken after it.

[paoloantinori]

Hi, I'd also love to see this implemented at minishift level, but in the meanwhile I can suggest @jstrachan and anyone else the old workflow that we used for a little while in Fabric8 Vagrantfile ancient time: https://github.com/paoloantinori/fabric8-installer/blob/94b5a7c26bf2563c79d20e73671ea8c55f1ffa26/vagrant/kubernetes/Vagrantfile

The idea is to deploy on you host a (containerized) instance of a Docker Registry, and to configure it to mirror Docker Hub (or any other registry that you want).

It's a fairly simple workflow:

# unset old docker env vars, in case they pointed to a minishift instance:
unset DOCKER_HOST
unset DOCKER_API_VERSION
unset DOCKER_TLS_VERIFY
unset DOCKER_CERT_PATH
# run on your host a docker registry, mirroring docker hub. Pass it a long lasting volume
echo "Starting Docker Registry container..."
docker run -p 5000:5000 -d --restart=always --name registry_cache   \
  -e REGISTRY_PROXY_REMOTEURL=http://registry-1.docker.io \
  -v /500GB/fabric8_vm_cache/docker_registry_cache:/var/lib/registry \
  registry:2

# eventually pre-populate your mirror cache ( but you can even let it happen at the first run):
#     docker pull openshift/origin:v1.5.0
#     docker pull openshift/origin-pod:v1.5.0
#     docker pull openshift/origin-haproxy-router:v1.5.0
#     docker pull openshift/origin-haproxy-registry:v1.5.0
#     docker pull openshift/origin-docker-registry:v1.5.0
#     docker pull openshift/origin-deployer:v1.5.0


# this is the hardest part, you have to pick a good ipaddress for your host, since minishift does not like localhost
#     hostname -I
#     192.168.1.67 192.168.42.1 192.168.122.1 192.168.121.1 172.28.128.1 172.17.0.1
minishift start --registry-mirror http://192.168.121.1:5000

# enjoy your local cache
# companion blogpost: http://giallone.blogspot.it/2016/04/dockercachingproxy.html

[paoloantinori]

Hi, I'd also love to see this implemented at minishift level, but in the meanwhile I can suggest @jstrachan and anyone else the old workflow that we used for a little while in Fabric8 Vagrantfile ancient time: https://github.com/paoloantinori/fabric8-installer/blob/94b5a7c26bf2563c79d20e73671ea8c55f1ffa26/vagrant/kubernetes/Vagrantfile

The idea is to deploy on you host a (containerized) instance of a Docker Registry, and to configure it to mirror Docker Hub (or any other registry that you want).

It's a fairly simple workflow:

# unset old docker env vars, in case they pointed to a minishift instance:
unset DOCKER_HOST
unset DOCKER_API_VERSION
unset DOCKER_TLS_VERIFY
unset DOCKER_CERT_PATH
# run on your host a docker registry, mirroring docker hub. Pass it a long lasting volume
echo "Starting Docker Registry container..."
docker run -p 5000:5000 -d --restart=always --name registry_cache   \
  -e REGISTRY_PROXY_REMOTEURL=http://registry-1.docker.io \
  -v /500GB/fabric8_vm_cache/docker_registry_cache:/var/lib/registry \
  registry:2

# eventually pre-populate your mirror cache ( but you can even let it happen at the first run):
#     docker pull openshift/origin:v1.5.0
#     docker pull openshift/origin-pod:v1.5.0
#     docker pull openshift/origin-haproxy-router:v1.5.0
#     docker pull openshift/origin-haproxy-registry:v1.5.0
#     docker pull openshift/origin-docker-registry:v1.5.0
#     docker pull openshift/origin-deployer:v1.5.0


# this is the hardest part, you have to pick a good ipaddress for your host, since minishift does not like localhost
#     hostname -I
#     192.168.1.67 192.168.42.1 192.168.122.1 192.168.121.1 172.28.128.1 172.17.0.1
minishift start --registry-mirror http://192.168.121.1:5000

# enjoy your local cache
# companion blogpost: http://giallone.blogspot.it/2016/04/dockercachingproxy.html

[budhram]

Hi @paoloantinori ,

The idea is to deploy on you host a (containerized) instance of a Docker Registry, and to configure it to mirror Docker Hub (or any other registry that you want).

Nice idea 👍

It's a fairly simple workflow:

Looks promising.

[budhram]

Hi @paoloantinori ,

The idea is to deploy on you host a (containerized) instance of a Docker Registry, and to configure it to mirror Docker Hub (or any other registry that you want).

Nice idea 👍

It's a fairly simple workflow:

Looks promising.

[jorgemoralespou]

@paoloantinori @hferentschik @budhrg keep in mind that this solution AFAICT only caches one external registry. In our case we might be pulling images down from multiple. Initially docker.io and registry.access.redhat.com.

Although I have to say that it's a workaround that we used before when we developed the all-in-one, as provided some easy benefits.

Also, we used some other workarounds: http://unpoucode.blogspot.com.es/2014/11/use-proxy-for-speeding-up-docker-images.html

[jorgemoralespou]

@paoloantinori @hferentschik @budhrg keep in mind that this solution AFAICT only caches one external registry. In our case we might be pulling images down from multiple. Initially docker.io and registry.access.redhat.com.

Although I have to say that it's a workaround that we used before when we developed the all-in-one, as provided some easy benefits.

Also, we used some other workarounds: http://unpoucode.blogspot.com.es/2014/11/use-proxy-for-speeding-up-docker-images.html

[jhcook]

I ultimately want caching of all images in order to make minishift environment transportable

+1

[jhcook]

I ultimately want caching of all images in order to make minishift environment transportable

+1

[hferentschik]

So turns our containers/image is missing the functionality we need. It allows to store single images in OCI format, but not creating a multi image OCI image repository - containers/image#155.

We can evaluate how much effort it would take to implement this in containers/image.

[hferentschik]

So turns our containers/image is missing the functionality we need. It allows to store single images in OCI format, but not creating a multi image OCI image repository - containers/image#155.

We can evaluate how much effort it would take to implement this in containers/image.

[LalatenduMohanty]

Resolved via #1685

[LalatenduMohanty]

Resolved via #1685