mkcloud should cache downloads in /var/cache

[aspiers]

Currently stuff like the SUSE-SLE12-CLOUD-5-COMPUTE .iso get downloaded into the admin node every run. They should get downloaded into /var/cache/mkcloud on the mkcloud host to avoid this.

[bmwiedemann]

You still would need to make sure that it is the latest version...
IMHO, the better alternative is to use the localdirs mechanism with rsync from clouddata (or NFS in the well-connected case)

[aspiers]

Agreed on both points. But as long as the wget remains, it should use caching ...

[aspiers]

Also looks like SP3-64up.qcow2 is being downloaded to /tmp. Again, /var/cache is a better place for this.

[Itxaka]

+1 This is terrible for remote workers. My local fixes right now:

• Have SLES12-SP1.qcow2 locally stored and cp to /tmp in the wrapper script that launches mkcloud
• Have SUSE-OPENSTACK-CLOUD-6-x86_64-GM-DVD1.iso locally stored and scp to the admin node on the sshrun() function

While this works, this is a bad quick patch to reduce my local dev deployment by about 35 minutes. We should support a timesaver like this out of the box

[AbelNavarro]

@Itxaka notice that the Devel Cloud ISO is generated every night, so every day you should preemptively download it. I tried at some point to have a local squid proxy (I'm remote too), but since I didn't deploy every day I always needed the time for squid to cache the images.

What worked for me is to have an allocated slot in one of the mkcloud hosts. This way if caching is not working, the speed to copy the images is really fast since it remains in the local network.

[dirkmueller]

is anyone interested in converting this into a pull request? adding a -do-not-download option should be relatively easy.

[Itxaka]

If I understand it correctly there is at least 2 images downloaded:

• The image for the admin node (SLES12-SP1.qcow2 if you use want_sles12=1)
• The image that is mounted on the admin node and served over tftp for node install (depends on cloudsource)

On the first case, we could download them to any dir but /tmp
On the second, Im not sure a good solution for it. Download it to a local dir and scp it to the admin node? This does not account for the develcloud image, as it changes, but a simple check for differences + delta paching would be nice to have.

At least with those 2 small fixes we would be covering some time gaps.

[bmwiedemann]

To sync files, you could use

rsync -a clouddata.nue.suse.com::cloud/images/x86_64/SLES12-SP1.qcow2 /var/cache/mkcloud/

[aspiers]

We should extend dl-ibs to take care of this.