New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Qemu builder #385

Merged
merged 10 commits into from Nov 5, 2013

Conversation

Projects
None yet
4 participants
@tdhite
Contributor

tdhite commented Sep 3, 2013

I added an issue for a pure qemu builder. This pull request provides an initial implementation for that issue: Issue #384.

I will support the testing and requests against this builder. As it operates qemu-system-x86_64 directly, it has to perform some management usually handled by external apps, for instance VBoxManage or similar. In any event, it tests out ok and has a working sample set of configurations in the documentation (website) markdown.

Tom Hite added some commits Sep 3, 2013

Tom Hite
Initial checkin to GitHub -- has extensive changes to conform to the …
…latest API model to match the 0.3.6 (Sept. 2, 2013) release.
Tom Hite
added network and disk driver options, also a source comment on the k…
…ickstart file in the docs (I can't find the original source).
@mitchellh

This comment has been minimized.

Member

mitchellh commented Sep 3, 2013

/cc @qur thoughts?

@qur

This comment has been minimized.

Contributor

qur commented Sep 3, 2013

I actually started working on a direct qemu builder a while ago (https://github.com/qur/packer/tree/kvm), though I connected to the monitor channel and used a shell script to persist the configuration in the output dir. However the difference between using qemu directly and using libvirt is quite large, so my libvirt builder is not related.

Personally, I want to use packer to build boxes for the vagrant kvm plugin - which uses libvirt so direct qemu support isn't really that interesting (do many people actually use qemu without libvirt?). However, I note that in the issue Tom mentions converting to libvirt in the future. So perhaps he would be interested on helping get a libvirt builder going instead?

If Tom wants to get his qemu builder in, then he's welcome to anything from my kvm branch that may be helpful ...

@tdhite

This comment has been minimized.

Contributor

tdhite commented Sep 3, 2013

Hi all,

Regarding helping on a libvirt 'instead' of pure qemu -- let's go! I'm
happy to help and we can start immediately as far as I'm concerned.

However, I'd like to see the qemu contribution added as it is. My logic
here isn't so much about the deploying an image after creation (e.g.,
Vagrant/KVM, libvirt, or other VM managers) -- the output of the builder
is an image (qcow2 or raw) for use in deploying wherever a hypervisor
may exist and by any reasonable means (Vagrant, libvirt, straight
command line, etc.). We create a lot of images for clients, and having
an ability to do so with the minimum installation requirements is, in my
opinion, important. An install of qemu / kvm is all one needs to 'mass
produce' a host of images for later transfer, for instance, to Glance,
or other image managers. From there, the provisioning is libvirt (if
OpenStack) or otherwise. I wonder if we're blurring lines between
deployment and image creation in requiring libvirt for an addition to
Packer?

My point is, I think the qemu builder makes sense as is -- really very
lightweight and relatively easy to deal with.

Then on to libvirt -- if there's starter code, I can take that and
augment, or I can start from scratch in a new project. Let me know
what's best.

On 09/03/2013 01:18 PM, Julian Phillips wrote:

So perhaps he would be interested on helping get a libvirt builder
going instead?

If Tom wants to get his qemu builder in, then he's welcome to anything
from my kvm branch that may be helpful ...


Tom Hite
phone: (312) 834-4833
twitter: @tdhite
gtalk: tdhite; tdhite.msi
skype: tdhite

@mitchellh

This comment has been minimized.

Member

mitchellh commented Sep 3, 2013

I agree with you @tdhite. I want to see both. But it is a big PR so it'll take time to review and properly merge. :) But I'll get it in.

@qur

This comment has been minimized.

Contributor

qur commented Sep 3, 2013

There is one thing I prefer of my own kvm builder is that it produces a script in addition to the .img file. A VM is not just the disk image, but also the configuration that defines the hardware environment, and I think it makes sense that packer should record that config. In some environments only the disk image may be relevant, but I don't think that's true in the general case, and a qemu builder does not give you the configuration to run with libvirt (and vice-versa).

In terms of interacting with qemu, I was connecting to the monitor to control the process rather than trying to manage the process itself which seems closer to how other builders work (and less confusing frankly - though that's probably just a familiarity thing).

If this gets merged, then I can try and pull out some of those bits as pull requests based on this builder?

More directly relevant to this pull request, I would note that there's quite a lot of copy paste that still needs updating. References to virtualbox, ovf etc in comments/errors ...

In terms of the libvirt builder, probably top of the list of reasons not to merge yet is a lack of testing ... I've added Tom as a contributor to my fork if he wants to help get my libvirt branch into shape for a pull request ...

@qur

This comment has been minimized.

Contributor

qur commented Sep 3, 2013

I also find myself wondering if all of the builders based on local tools could be sharing more code ...

@tdhite

This comment has been minimized.

Contributor

tdhite commented Sep 3, 2013

In fact, I was thinking the same thing. There's a lot of dup'ed code
because it's very easy to start with an existing builder, fix it up, and
test. Seems I left some stray references to VB due to that (eyes
scouring code didn't work this time). Hmmmmm..... Where there's copy,
there's better reuse ...

I'll put some thoughts into this.

On 09/03/2013 02:42 PM, Julian Phillips wrote:

I also find myself wondering if all of the builders based on local
tools could be sharing more code ...


Reply to this email directly or view it on GitHub
#385 (comment).


Tom Hite
phone: (312) 834-4833
twitter: @tdhite
gtalk: tdhite; tdhite.msi
skype: tdhite

@tdhite

This comment has been minimized.

Contributor

tdhite commented Sep 3, 2013

A couple more thoughts (good reading from @qur):

"I think it makes sense that packer should record that config." That's interesting, I always considered the input (JSON) as the record of truth, but the back-side might be that instead? The thing is -- single sources of truth are indeed important and you just hit one where we may be creating two by dumping config files. Dunno -- just thinking through it quickly.

"If this gets merged, then I can try and pull out some of those bits as pull requests based on this builder?"
Seems good to me -- particularly related to the above, though. As I went through Packer, I honestly never did consider where that single source of truth was regarding the image, and the VM instance that might one day get generated by using it. For instance, in the case OpenStack, you have absolutely no ability (absent mucking in the back-end python code) to specify the libvirt configuration -- OpenStack creates that on the fly (as does Eucalyptus, et al). The question is -- hinting? I'll look into it and see if there's not a way to 'hint' to the cloud providers what you want in the libvirt.xml files they create.

Tom Hite added some commits Sep 3, 2013

Tom Hite
changed error string referring to 'ova' and 'ovf' to refer to 'qcow2'…
… and 'img' as the former were stray leftovers from the virtualbox code used as a basis for this plugin.
@tdhite

This comment has been minimized.

Contributor

tdhite commented Sep 4, 2013

Just a quick fyi -- I fixed the stray VirtualBox and file format strings. There's not too many -- if you want I can just reissue the PR with the latest, otherwise if you merge it, I'll send a follow-on PR for this (and any other issues I can find).

@qur -- I pulled the libvirt code, have some ideas, but I'll communicate those over e-mail rather than pollute this comment set.

@ramereth

This comment has been minimized.

ramereth commented Oct 2, 2013

I've been using this branch over the past week and have been fairly happy with it. If I have features/bugs/requests should I submit them here or on your repo Tom? Specifically I'd like to be able to set the memory size of the VM. I also ran into an issue with qemuargs not working and giving me an error. Thanks for the great patch set!

@tdhite

This comment has been minimized.

Contributor

tdhite commented Oct 2, 2013

Hi,

until the merge happens, you can just apply an Issue against: https://github.com/TranscendComputing/packer and I'll fix it up. I'll make sure @mitchellh gets the updated code.

@ramereth

This comment has been minimized.

ramereth commented Oct 3, 2013

Would you mind please enabling the issue tracker for your fork?

@tdhite

This comment has been minimized.

Contributor

tdhite commented Oct 4, 2013

Just set the flag, @ramereth. Apologies for the oversight.

@mitchellh

This comment has been minimized.

Member

mitchellh commented Oct 4, 2013

@ramereth @tdhite I'm glad to hear everything is working! I am very excited to merge this code I just haven't had sufficient time to test it out. :) It will come soon.

@ramereth

This comment has been minimized.

ramereth commented Oct 4, 2013

I still need to do a little more testing but overall its working nicely. I'm planning on trying to find time to submit some documentation pull requests for @tdhite that I found a little confusing.

Tom Hite and others added some commits Oct 8, 2013

Tom Hite
Fixes #3 via minor documentation fix and setting default properly (in…
… the net_device template value, virtio is incorrect -- must be virtio-net).
@mitchellh

This comment has been minimized.

Member

mitchellh commented Nov 5, 2013

This looks good enough to merge, but I did run into some issues I'll look into. Thanks!

mitchellh added a commit that referenced this pull request Nov 5, 2013

Merge pull request #385 from TranscendComputing/master
builder/qemu: Qemu builder

@mitchellh mitchellh merged commit 701f31c into hashicorp:master Nov 5, 2013

1 check passed

default The Travis CI build passed
Details

@mitchellh mitchellh referenced this pull request Nov 6, 2013

Closed

Qemu (KVM / Xen) builder #384

@mitchellh

This comment has been minimized.

Member

mitchellh commented Nov 6, 2013

@tdhite I've made some early modifications to make things a bit more Go-like, but things appear to be working great! Great job.

@ramereth Would love any docs assistance or any assistance otherwise. :)

mitchellh added a commit that referenced this pull request Apr 28, 2014

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment