"Remote" local volumes and HTTP volumes

[hooksie1]

Trying to wrap my head around everything in here. What would it take to implement using a backing store on a remote server (qemu+ssh:///) or having it download the image over HTTP from the remote machine instead of downloading locally and copying?

[MalloZup]

@hooksie1 you can use the remote uri. ( i dunno if i understood your question correctly)

The remote machine need to have enabled tcp

Machine one 1

provider "libvirt" {
  uri = "qemu+tcp://remote-system.com/system"
}


# We fetch the latest ubuntu release image from their mirrors
resource "libvirt_volume" "ubuntu-qcow2" {
  name = "ubuntu-qcow2"
  pool = "default"
  source = "https://cloud-images.ubuntu.com/releases/xenial/release/ubuntu-16.04-server-cloudimg-amd64-disk1.img"
  format = "qcow2"
}

Remote server KVM will perform operation

Hope it helps !

[hooksie1]

Sorry I guess I didn't word it correctly. If I'm running Terraform on my laptop and I'm connected to a remote KVM server, the HTTP file is downloaded through my laptop to the remote system. I figured this out when the download was taking a looong time. Here's a sample:

provider "libvirt" {
        uri = "qemu+ssh://jhooks@10.1.30.30/system"
}

resource "libvirt_volume" "fedora-28" {
        name = "fedora-28.qcow2"
        source = "http://web-server.com/fed-28-template.qcow2"                                                                         
        pool = "VMs"
}

If I change my hosts file on my laptop and point web-server.com to 127.0.0.1 the download fails. What I wanted to implement was a way to clone an image on the remote server that's local to that server (if that sentence makes sense) or have the remote server download the image directly instead of through my laptop (or whatever system Terraform is on).

I'll try to explain a different way just to make sure I can explain this correctly. My workflow is usually, I SSH into a KVM server and do a virt-clone. I essentially want to replicate that with Terraform but all of the images be on the remote filesystem and not have anything other than the .tf files locally.

Sorry for the long winded response!

[MalloZup]

The source image like "http://web-server.com/fed-28-template.qcow2" are downloaded to a golang stream and saved directly to libvirt volume which is in the KVM server.
So there is no local-copy and then transfer happens to remote uri kvm server.
Everything happens on the remote-kvm server with the libvirt-api, if you specify "
uri = "qemu+ssh://jhooks@10.1.30.30/system" libvirt will use that system and your laptop is only a scheduler.
So taking your example

provider "libvirt" {
        uri = "qemu+ssh://jhooks@10.1.30.30/system"
}

resource "libvirt_volume" "fedora-28" {
        name = "fedora-28.qcow2"
        source = "http://web-server.com/fed-28-template.qcow2"                                                                         
        pool = "VMs"
}

THe images/volume are uploaded on this server 10.1.30.30. and not were the provider is runnig. ( if you specify the remote URI as you did.
I just checked the code and also in PROD we have a configuration like this,

A server(only terraform-libvirt here) ----> remote-uri ---> B-KVM server
main.tf  ---> schedule => images, disk and domain are here

hopte it helps. 🎴

Imho we can close it, or i still don't get what is the question then .. 🤔 😄
Ciao have nice morning or day depending where you are 🌞

[MalloZup]

@hooksie1 for any other question feel free to ask 👍 or reopen the issue if you feel that is nor answered. thx

[hooksie1]

Ok I uploaded a video to the youtubes https://youtu.be/Os1h29n4n2E (I apologize for the typing ahead of time). Sorry it's so long, but I was trying to get everything I'm seeing. If I run terraform from my laptop over wireless to the KVM server I see much slower transfer speeds than if I run terraform directly on the KVM server. If I lose connection to the remote web server holding the image the apply fails even though the KVM server can get to the image (did this by editing the hosts file for a fake failure). However if I lose the connection (by editing the hosts file) on the KVM server and running terraform on my laptop it still works?

I admit these don't mean that the laptop is copying the file, I made that assumption. But it does seem strange that the performance/behaviour would change that much from running on one system vs the other when only one should be doing the work.

Top left is my laptop. Bottom left and right are the KVM server. Using the same test.tf file I posted above, but with a different template URL so I didn't have to run the web server internally again (since I'm lazy).

[hooksie1]

Also sorry it's taken so long to respond. Started a new job and it's been busy.

[inercia]

@MalloZup I think images are imported using the image interface implemented here for local and http images, and called from the "source" for volumes. So I think the download is performed in the machine where the provider is running...

Regarding a possible solution for this problem, unless libvirt could do the download for us (and AFAIK it cannot do that), there is not solution for this...

[hooksie1]

Ah that makes sense @inercia, thanks.