New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add images build capability to vsphere #1183
Add images build capability to vsphere #1183
Conversation
[test] |
Thanks for the contribution @mynameisrufus, this will be much appreciated by our users. I'm worried about the approach though, since it's a new workflow for something that already exists in Foreman. There are currently two provisioning methods for new hosts on compute resources (the code refers to these as capabilities): build and images, so the VMware CR currently only implements the "build" method (PXE). I was imagining that implementing hosts from templates would use the existing images mechanism, rather than adding a new VMware-specific workflow. Have you had a look at this method - what do you think? |
#1184 is a really good example of adding image provisioning to oVirt - many of the general UI and JS fixes that @jimmidyson's added in the process would be useful here. |
Thanks @domcleal, I was looking for feedback on this as I have only just started looking at the codebase. I will update this PR over the coming week. |
Thanks @mynameisrufus. Please just ping me when you'd like another look, or if you have any questions. |
@domcleal I think I have this working at some level, I am successfully cloning vms through hosts but my request times out and a I also needed to make this patch to fog https://github.com/fog/fog/pull/2642/files |
@mynameisrufus You can possibly try https://github.com/iNecas/foreman-tasks to get your cloning (or host creation) to run asynchronously. |
@elobato that approach seems to be solid however this PR will be used by everybody so adding that dependency does not seem like an option? |
Yeah, relying on foreman-tasks isn't an option at the moment. Maybe in future releases. I'm surprised it's so slow - how long are you talking, minutes just for VM creation? Is there any sort of COW functionality we can use instead? |
@domcleal There are multiple places where timeouts can affect the creation of systems. We hit timeouts in DHCP reservations for example, this was limited to 60 seconds (hardcoded in lib/proxy_api/resource.rb), but due to the configuration of our DHCP server managing multiple subnets this was taking a long time. Also, it would be have the option to request a host creation asynchronously, especially through the API. Kick off host creation, keep checking for progress, or use the tasks API to keep track. |
@jimmidyson indeed, we're working on foreman-tasks etc to fix these very problems. We're not going to add it as a dependency today though. |
Successfully cloning machines based on images in the hosts ui.
@domcleal we've won! Please review. The Fog adaptor is a little beta, I stayed away from altering it so this PR can actually get merged. |
Nice, let's [test]. |
# +searchIndex.FindChild("Resources")+ in RbVmomi that then returns nil | ||
# because it has no children. | ||
def clone_vm args | ||
vm = client.servers.get(args["image_id"], datacenter) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'm having issues with this line, and the one below for looking up the template by the image_id, as neither of them return the template "VM". Which identifier are you using when creating your images?
Related is that I tried implementing #available_images
(so it shows a list of templates in the "new image" form) for Foreman::Model::Vmware
by simply returning client.templates
(with fog/fog#2702) and it returns this:
=> [ <Fog::Compute::Vsphere::Template id="5030c304-48b1-b2ce-867e-5ab4a8889637", name="dcleal-tmpl-rhel6", uuid="4230b500-017b-e739-4cf4-cec65f389c78" >,
Though none of the three IDs work for either client.servers.get or client.templates.get. Any ideas how this should work?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Right now, it looks like the 'template' has actually to be a real VM in the list of servers. So the ID must be the UUID of an actual VM. Though, if we could get the list of actual VmWare template through client.templates, it would be great
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Our use case is currently to build a VM with packer and clone off that, in this case the VM is not a template but we are treating as such.
@domcleal due to some lack of implementation around the clone code in fog
(network device defaults to VIM::VirtualE1000
but we need VIM::VirtualVmxnet3
) we have rolled our own microservice using rbvmomi
directly. A pull request exists on fog
that has been open for a year that addresses many of these things but it has diverged too much and I don't think it will ever make it in. The fog
chaps highest priority is to create a core gem with adapter gems to make the codebase not so monolithic, I would like to work on the fog-vshpere
gem when it drops but until then you might want to close this PR.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Right, that makes sense, thanks. The more I look at the fog code, the less likely it looks to work with templates anyway - I just can't see any capability for cloning or creating VMs from templates, so I think this'll have to do for now.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Or not, it looks like calling client.vm_clone() with "template_path" => "/foo" should work, but I can't get it to work for my template that's in a few folders deep, having tried lots of permutations.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Ugh, it does actually work, when you spell the template_path correctly. But I don't think we have the relative path to templates available from vSphere for this to work end to end, as Fog's template object only stores the name.
I also changed app/views/images/_form.html.erb:9 to read:
(so you could set a password for SSH finish provisioning) |
Any News about this, because I was hoping this would become available in foreman 1.4.2 :) |
We won't include new features like this in a point release (.2), but if we can get it into a mergeable state by the end of March, then it could be released in 1.5.0. |
Ok, thanks for the info. Thought It was panned for 1.4.2 because it is an item in sprint 21 as Feature #1966 |
"network_adapter_device_key" => network_adapter_device_key | ||
} | ||
|
||
vm.relative_path = vm.folder.path.gsub(path_replace, '') |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This is not working for me. Folders are present only in relative_path attribute (which starts with dc name), and this line is overridding it. As such this can work only if the VM to be cloned is in /.
This is working for me :
dc_replace = /#{datacenter}//
vm.relative_path = vm.folder.path.gsub(path_replace, '')+vm.relative_path.gsub(dc_replace,'')
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@couniojc I'm not quite sure how your fix works, it produces a string like this for me, doubling up the name:
2.0.0-p247 :018 > vm.folder.path.gsub(path_replace, '')+vm.relative_path.gsub(dc_replace,'') => "Scratch/Training - SA3Scratch/Training - SA3"
The original seems to do something sensible to me:
2.0.0-p247 :019 > vm.folder.path.gsub(path_replace, '') => "Stratch/Training - SA3"
Or even just vm.relative_path.gsub(dc_replace,'')
has the same result.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@domcleal path is '/' and vm.folder.path is '/Datacenters/{dcname}/vm/' for all the VMs on my side. So relative_path is the only variable that has the folders.
I suppose yours are correct then
Strangely, the folders (path attribute) are correct when listed in 'virtual machines' list but in this case data is coming from MiniServers.
If it works for you, then that's fine, you can drop that one. If it still happens in 1.5.0, at worse we can live with VMs at root for a while (or I will patch our install)
I've created this branch which changes the behaviour to enable cloning of templates by entering the path to the template on the new image form (I think the same will work for a VM too, actually). What do you think? https://github.com/domcleal/foreman/compare/vmware-clone-template I agree with @couniojc's comments on the issue, this is useful enough to get into 1.5 in time. |
@domcleal Thanks ! I tested with the new branch and it's working on my side ! Excellent ! |
Thanks for testing @couniojc, much appreciated! Merged as 3304049 for Foreman 1.5.0. Thanks @mynameisrufus - you've solved our highest-voted issue! |
Successfully cloning machines based on images in the hosts ui.
Cluster option currently not working because of the way the fog provider is attempting to retrieve the resourcePool.
http://projects.theforeman.org/issues/2438