Hooks for Different Actions in XO

[ebartz]

I want to be able to announce different actions to external tools.

For Example, once a VM is created, I want to tell my DHCP Server to provide a lease only for that new MAC address. Also I would like to add some information into our CMDB or automatically call the Monitoring Server so that the host ist monitored up until the creation of a VM.

The easiest way for me would be a Plugin, that would allow me to call scripts.

I know something like this from TheForeman.

If you would adopt this, it would be like this:

There would be a plugin called xo-hooks. In that directory is a directory called hooks.
In hooks you would finde subfolders for each action that you could add hooks to.

For example:

xo-hooks/hooks/vm.create
xo-hooks/hooks/vm.delete
xo-hooks/hooks/user.create
xo-hooks/hooks/user.delete
...

In those folders you can then place scripts which get executed maybe using shell_exec.

It would on every of such events read the directory and execute the scripts which are in there with some arguments.

Example:

xo-hooks/hooks/vm.create/01_add_dhcp.sh
xo-hooks/hooks/vm.create/02_add_monitoring.sh
xo-hooks/hooks/vm.create/03_call_cmdb.sh

Those scripts will then be called with available arguments for example:

01_add_dhcp.sh <VM Name> <VM UUID> <VM MAC> ...

It is also possible that it has not all the information. It should just be enough to query the API to get the rest of the information.

Also the execution should maybe run in background so that it does not delay UI actions.

What do you think about that? Maybe you can point me to the file where most of those calls happen currently to create VM's so I can try to hack something like that into the code?

Best Regards,
Enrico

[dezmodue]

Hi,

I am also interested in this feature - mainly to hook up VM creation/deletion to our configuration management, DNS, alerting, messaging

Another possible approach is to have the ability to call a webhook on specific events either configured globally (e.g. on each vm creation) or per object (e.g. to a specific VM lifecycle)

We have a multi tenant environment where teams have operator access to their resources - they do not have the ability to drop hooks on the XOA server but they would be able to associate a webhook to their own VMs to trigger notifications to their messaging platform or hook into documentation/alerting etc

Regards
Simone

[olivierlambert]

Associate a web hook during VM creation in the "advanced section" should be doable without too much trouble (speaking without checking with @julien-f but sounds the easiest thing to do).

Associate a web hook to multiple actions in an existing VM (like start/stop I assume) would be far more complicated, I mean in terms of UI, without clogging the existing interface with a lot of new fields to manage that.

So we need to find the first and easiest path to do that. @dezmodue maybe we could list a limited set of action that you would like to have "webhook compatible"?

[ebartz]

For the beginning I also would be fine with VM creation and deletion. Webhooks would be no problem :)

[olivierlambert]

VM creation should be doable I think (pinging @julien-f ), especially that it's totally temporary while creating the VM and easy to put in "Advanced" in the UI.

Deletion is a bit trickier, we'll have to store it, then where and how.

[julien-f]

Question: what happens if the VM is created/deleted by another XS client (e.g. XenCenter)?

[ebartz]

Well I guess in that case just nothing happens. For my use case I just want to add automatic actions after a user created / deleted a VM in XOA.

[diamatrix]

Like ebartz mentioned in the first post, have a hooks select under the settings in XOA where admin's can upload bash hook files under each hook event and save them in folders on the XOA.

So create the hook directories like (ebartz mentioned) for pre and post events

xo-hooks/hooks/vm.create/pre/
xo-hooks/hooks/vm.create/post/

Then create a section under "Settings" that allow admins to upload bash scripts into that location in the format of "priority_name.sh"
Example being 01_add_dhcp.sh

Install jq on XOA (apt-get install jq).

The (for example) before a VM is created, you run the bash scripts in xo-hooks/hooks/vm.create/pre/
Then create the VM
Then run the bash scripts in xo-hooks/hooks/vm.create/post/

Pass all information about the VM in a single argument to the bash script in a json string.
Example: xo-hooks/hooks/vm.create/post/01_add_dhcp.sh

We as admins can use JQ in our bash scripts to decode the json string in $argv[1] and then perform whatever actions we need using this data.

The code from your side shouldn't be difficult in that:

• Before VM creating, get a list of files in xo-hooks/hooks/vm.create/pre/
• Run them 1 at a time using the node.js exec function
• Do the VM Creation
• Get a list of files in xo-hooks/hooks/vm.create/post/
• Run them 1 at a time

VM Delete shouldn't be hard at all.

• Get the information of the VM (xo-cli --list-objects id=) and hold it in a variable.
• run scripts in xo-hooks/hooks/vm.remove/pre/
• Remove the VM
• run scripts in xo-hooks/hooks/vm.delete/post/
• Destory the variable

This format could be extended for each event in XOA (VM Suspend, VM Restart, etc) with vm.create and vm.delete being the priority

-Dave

[olivierlambert]

@julien-f I speak on your behalf (so please comment to confirm or not), but IIRC our last discussion about it was: if we introduce this, it will be web hooks, so you could create whatever stuff you want on a server that listen for these hooks, and we'll keep it simple as possible in XOA.

Also the question I'm working on is now around the UI:

• hooks should be configured during VM creation and while editing a VM, so where exactly? (advanced section for creating VM I suppose, collapsing section in "Advanced" tab in VM view)
• we should provide hooks examples and document it

[diamatrix]

The reason I mentioned the bash hooks is code execution can happen on xoa directly and perform the task of ssh'ing into a machine and setting up dhcp for the vm (as an example). However we will take what we can get.

With the web hooks, could we have "global" hooks set under Settings and local vm creation hook in the advanced section? With the global hooks under settings, actions can be called before VM delete
The global hooks help with things like automatic ip addressing for the VM's.
As a cloud provider, you don't want to have to inform all your customers that they need to run a specific hook in the advanced settings each time they create a VM in order for their VM to boot with an IP address.

[olivierlambert]

I don't know yet how to organize that: do we want global hooks? If yes, if we doesn't want for specific VMs?

The hard stuff (as always) is in the specification. What do we really want to achieve (at functional level) and how to configure it.

Regarding bash: I would like to avoid it, because that will become messy and impossible to debug.

[diamatrix]

Why not have both? Run the global one first and then anything else that is specified in the advanced section. I do however believe that a business will have standard actions that they want performed on all VM's created. It would make sense for them to be able to set & forget in global rules, rather than specifying the web hooks on each vm creation.

What are your thought on how the data is going to be passed to the web hook? Json encoded POST ?

[olivierlambert]

I'm not against global hooks, but it means we need to think about a dedicated UI to create them, and maybe a "set as default for all VMs" or something like that (and the ability to de-activate a webhook in VM create I suppose).

About the technical details: that's a question for @julien-f

[ebartz]

In my case A Hook is nothing that the user has to know. It is needed setting up things for new VM's automatically. Also it is useful for reporting and other integration stuff. I would like to have a plugin that you could disable or enable globally for your needs. If it really has to be a wekhook ( i definitly prefer a folder with scripts as theforeman guys do it) you should just configure a server and a port. And weather it is https and weather you trust selfsigned certs. Then in the end you will just have a specification of events that will be sent to that service. Cloud be json. The post data should contain enougj data to identify the object for which this notification has been sent and the type of event. For instance it cloud contain: vm.post.create + vmuuid Storage.post.delete + uuid Snapshot.post.create + uuid If you would have pre create hooks and the service returns an error code, the vm should not be created. At least thats what others do so far.

…

Am 04.06.2017 um 20:14 schrieb Olivier Lambert ***@***.***>: I'm not against global hooks, but it means we need to think about a dedicated UI to create them, and maybe a "set as default for all VMs" or something like that (and the ability to de-activate a webhook in VM create I suppose). About the technical details: that's a question for @julien-f — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub, or mute the thread.

[diamatrix]

I'm not against global hooks, but it means we need to think about a dedicated UI to create them

You mean a dedicated interface like this: https://ibb.co/f2MWTn

EDIT: dropbox changed the link address

[olivierlambert]

Exactly, that's a great mockup! (we need to have a list of actions possible in a dropdown too)

[diamatrix]

I would start with some basic actions and extend it as & when there is a demand.

vm.create
vm.delete
vm.suspend
vm.stop
vm.start
user.create
user.delete

[olivierlambert]

Noted. We are in the middle of improving backups, so right now it's not totally in the top of the TODO list due to our limited manpower.

[ebartz]

Any progress on that request?

[olivierlambert]

Not at the moment (especially in the middle of the summer with less than half people at the office). We'll have reinforcements coming mid-September, so it will be one of the topic :)

[ebartz]

Any news? :)

[olivierlambert]

Still training the new devs, @julien-f do you think this is something we could delegate to them?

[julien-f]

@olivierlambert Nope, not for the moment.

[ebartz]

Are there any updates?

[julien-f]

@ebartz Not yet, but we have started talking about putting it on the roadmap :)

[olivierlambert]

After our next-release, we'll enter in a specification phase.

[olivierlambert]

@julien-f what do you think to let @badrAZ working on something (draft/specs?) this month?

[diamatrix]

Is this feature going to be added soon?
It is a real deal breaker for ourselves.

[julien-f]

We also need to add contextual data like the user doing the action.

[pdonias]

The PR implements web hooks, which is not the exact request of the issue. We're still closing the issue but feel free to keep discussing it if it doesn't match your use cases.