Add a guestfs connexion plugin #13612

mscherer · 2015-12-20T11:20:33Z

This permit to edit a base image in a similar fashion to packer,
thus permitting for example to customize base image before uploading
them directly to Glance or similar services for openstack, enabling
immutable server deployment.

It can be used like this:

$ cat hosts
fedora_22 ansible_ssh_host=Fedora-Cloud-Base-22-20150521.x86_64.qcow2 ansible_connection=guestfs

$ cat deploy.yml

---
- hosts: fedora_22
  roles:
  - somerole

$ ansible-playbook -i hosts deploy.yml
[....]

For now, it only support local VM with single OS, but it should be trivial
to add support for remote disks and multiple OS.

jimi-c · 2016-06-21T15:18:03Z

lib/ansible/plugins/connection/guestfs.py

+
+        self.guestfs.add_drive_opts(self.disk)
+
+    def _find_python(self):


Shouldn't this use the value of ansible_python_interpreter? There is a mechanism (winrm.py uses it) to pass in some host variables to connection plugins, so this should probably implement that.

I didn't knew about set_host_overrides, I will take a look.

So, we need autodetection for Fedora and Ubuntu (since they come with python 3 pre installed, so I need to detect them for running raw to install python2). Now, I am not sure how should the plugin react if the user give a explicit python_interpreter, and if we can assume to be installed if given (especially for something like raw module, which should work without python).

In fact, I wonder if I shouldn't also have a wrapper in shell, since you can't use it on freebsd image with the current system of using a python wrapper if python is not installed (which is the case on freebsd among others). And I might need it soon for netbsd images.

connection plugins should not have their own python logic. If distros are using different python/missing python it is up to user to add (plenty of examples with raw and ansible_python_interpreter).

The python detection is used by the connexion plugin for internal matter, ie be able to run the script whose code is in the EXECUTION_SCRIPT constant, which is a wrapper that do encode stdin/stdout as json, because guestfish do not return the 2 streams separate.

This is separate and not related to the fact that ansible also use python. If perl was installed by default everywhere, I would use it, or if I had a good reason to believe that I could write a portable shell script doing the same without losing sanity points, I would do it.

I guess I could replace that part by a script that do execute the binary, dump stdout and stderr in 2 separate file and then get them, but this look more ugly than the current way (albeit more portable). And if I remember correctly, the python detection is needed so I can run raw or the various shell fragment in the first place. I can take a look again, because that code is not so fresh, but I am confident that if I could have avoided that, I would.

this is why we have shell plugins .... but I do agree on the sanity part ...

mattclay · 2016-06-21T16:16:47Z

@mscherer: Could you add this to the integration tests? Specifically, add a section to test_connection.inventory for guestfs and then make sure the tests pass by running:

TEST_FLAGS='-l guestfs' make test_connection

alikins · 2016-06-21T17:22:42Z

Seems to work for me, I was able to point it to a qcow2 image and run ansible playbook against it. I only tested fedora images, so didn't try windows/ntfs/etc.

For now, it only support local VM with single OS, but it should be trivial
to add support for remote disks and multiple OS.

Would this include specifying a (local) libvirt/qemu url? ie, user vs system ?

pilou- · 2016-07-03T09:41:48Z

lib/ansible/plugins/connection/guestfs.py

+
+import os
+import os.path
+import guestfs


You should raise AnsibleError when guestfs python module isn't available.

Good point, fixed in the pushed branch

mscherer · 2016-07-04T12:08:20Z

@mattclay I added a first attempt at the test, I am not sure how to run them the correct way (and I rather avoid running disruptive test on my work laptop). It requires to download a cloud image to test, so I guess I should commit that file in git :)

mscherer · 2016-07-04T12:19:17Z

@alikins Yeah, I think so. It should be able to do all that you can do with guestfs command line.

So we can use http://libguestfs.org/guestfish.1.html#adding-remote-storage for remote disks, and or also a different libvirt url if needed. Of course, people need to make sure that remote VM are shutdown for obvious concurency issue.

I guess we could pass ansible_libvirt_uri variable or something, provided the connexion plugin can access that information (but I didn't dig that direction yet).

mscherer · 2016-07-04T12:30:21Z

Ok so we could pass more variable using the set_host_overrides system that @jimi-c spoke earlier, but I rather add the features later.

mattclay · 2016-07-05T15:51:56Z

@mscherer No need to include the image in git, just instructions on what is necessary to run the test, as it appears you've already done. If you can run TEST_FLAGS='-l guestfs' make test_connection and have the tests pass, that's all that is needed for testing.

mscherer · 2016-10-08T13:12:27Z

So I did started to look at it (after Robyn talk on ansible container), and I did it a issue, and I think that's because we have test using UTF 8 string, but the Fedora cloud image I have do not seems to support utf-8.

I will try to dig a bit, but if test requires a less featurefull image than the usual image for testing utf8, that might be a issue.

mscherer · 2016-10-20T20:41:03Z

Ok so I fixed the first issue regarding utf8, now I hit another issue (so still trying to get it merged)

mscherer · 2016-10-22T18:01:49Z

So I fixed the issue I had, and the tests are working (and are documented) on my laptop on RHEL 7.

This permit to edit a base image in a similar fashion to packer, thus permitting for example to customize base image before uploading them directly to Glance or similar services for openstack, enabling immutable server deployment. It can be used like this: $ cat hosts fedora_22 ansible_ssh_host=Fedora-Cloud-Base-22-20150521.x86_64.qcow2 ansible_connection=guestfs $ cat deploy.yml --- - hosts: fedora_22 roles: - somerole $ ansible-playbook -i hosts deploy.yml [....] For now, it only support local VM with single OS, but it should be trivial to add support for remote disks and multiple OS.

bcoca · 2017-01-05T17:04:29Z

lib/ansible/plugins/connection/guestfs.py

+        ''' return the filename for the command output '''
+        return '%s/cmd.out' % self._tmp
+
+    def exec_command(self, cmd, in_data=None, sudoable=False):


if not handling become methods update the class variable to indicate this

I am not sure to understand. I suspect I need to change sudoable to True rather than False, becuase that seems to be the default everywhere else, but what class variable do I need to update ?

not that, i'm talking about become_methods list class variable.

example that supports all defaults except 'su'

become_methods = frozenset(C.BECOME_METHODS).difference(('su',))

I would need to test if sudo and su work I guess, as I didn't need them (but I can see why it would be required, at least 'su').

su requires a tty, that is why most 'subprocess' plugins don't support it.

ansibot · 2017-04-11T19:10:18Z

@mscherer Greetings! Thanks for taking the time to open this pullrequest. In order for the community to handle your pullrequest effectively, we need a bit more information.

Here are the items we could not find in your description:

issue type

Please set the description of this pullrequest with this template:
https://raw.githubusercontent.com/ansible/ansible/devel/.github/PULL_REQUEST_TEMPLATE.md

click here for bot help

gundalow · 2019-09-19T12:55:39Z

@mscherer
Thank you for your PR.
As part of reviewing the backlog of PRs we are looking at PRs older PRs that haven't been updated in a while

Given that:

PR has needed a rebase since April June 2017

Therefore I'm going to close this.

If you or anyone else wants to continue with this work then please do feel free to create a fresh PR and @mention the previous reviewers in here.

mscherer force-pushed the guestfish_pr branch from d90746c to 84457a6 Compare December 20, 2015 11:26

bcoca added the new_plugin This PR includes a new plugin. label Dec 21, 2015

bcoca added this to the next milestone Dec 21, 2015

jimi-c added the feature_pull_request label Jun 21, 2016

jimi-c modified the milestones: 2.2.0, stable-2.1 Jun 21, 2016

jimi-c reviewed Jun 21, 2016
View reviewed changes

pilou- reviewed Jul 3, 2016
View reviewed changes

mscherer force-pushed the guestfish_pr branch 2 times, most recently from af1d478 to 15521fc Compare July 4, 2016 12:06

mscherer force-pushed the guestfish_pr branch from 15521fc to 9e6bb4c Compare October 7, 2016 09:14

mscherer force-pushed the guestfish_pr branch from 9e6bb4c to 514d51d Compare October 20, 2016 17:56

mscherer force-pushed the guestfish_pr branch 2 times, most recently from c5ef5ec to 76efc29 Compare October 22, 2016 17:54

ansibot added the test_pull_requests label Oct 25, 2016

ansibot added the needs_revision This PR fails CI tests or a maintainer has requested a review/revision of the PR. label Dec 24, 2016

mscherer force-pushed the guestfish_pr branch from 76efc29 to d8a6291 Compare December 28, 2016 21:05

ansibot added needs_revision This PR fails CI tests or a maintainer has requested a review/revision of the PR. and removed needs_revision This PR fails CI tests or a maintainer has requested a review/revision of the PR. labels Dec 28, 2016

ansibot removed the needs_revision This PR fails CI tests or a maintainer has requested a review/revision of the PR. label Jan 5, 2017

mattclay approved these changes Jan 5, 2017

View reviewed changes

bcoca requested changes Jan 5, 2017

View reviewed changes

ansibot added needs_revision This PR fails CI tests or a maintainer has requested a review/revision of the PR. c:plugins/connection labels Jan 6, 2017

gundalow modified the milestone: 2.2.0 Apr 13, 2017

ansibot added support:core This issue/PR relates to code supported by the Ansible Engineering Team. needs_rebase https://docs.ansible.com/ansible/devel/dev_guide/developing_rebasing.html labels Jun 29, 2017

ansibot added plugins/connection test This PR relates to tests. labels Sep 7, 2017

ansibot added feature This issue/PR relates to a feature request. and removed feature_pull_request labels Mar 2, 2018

dagwieers removed the test_pull_requests label Jun 22, 2018

gundalow added the pr_day Has been reviewed during a PR review Day label Sep 19, 2019

gundalow closed this Sep 19, 2019

ansible locked and limited conversation to collaborators Oct 22, 2019

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add a guestfs connexion plugin #13612

Add a guestfs connexion plugin #13612

mscherer commented Dec 20, 2015

jimi-c Jun 21, 2016

mscherer Jun 21, 2016

mscherer Jul 4, 2016

bcoca Jan 5, 2017

mscherer Jan 5, 2017

bcoca Jan 5, 2017

mattclay commented Jun 21, 2016

alikins commented Jun 21, 2016

pilou- Jul 3, 2016

mscherer Jul 4, 2016

mscherer commented Jul 4, 2016

mscherer commented Jul 4, 2016

mscherer commented Jul 4, 2016

mattclay commented Jul 5, 2016

mscherer commented Oct 8, 2016

mscherer commented Oct 20, 2016

mscherer commented Oct 22, 2016

bcoca Jan 5, 2017

mscherer Jan 5, 2017

bcoca Jan 5, 2017

bcoca Jan 5, 2017

mscherer Jan 6, 2017

bcoca Jan 6, 2017

ansibot commented Apr 11, 2017

gundalow commented Sep 19, 2019


		self.guestfs.add_drive_opts(self.disk)

		def _find_python(self):

Add a guestfs connexion plugin #13612

Add a guestfs connexion plugin #13612

Conversation

mscherer commented Dec 20, 2015

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

mattclay commented Jun 21, 2016

alikins commented Jun 21, 2016

Choose a reason for hiding this comment

Choose a reason for hiding this comment

mscherer commented Jul 4, 2016

mscherer commented Jul 4, 2016

mscherer commented Jul 4, 2016

mattclay commented Jul 5, 2016

mscherer commented Oct 8, 2016

mscherer commented Oct 20, 2016

mscherer commented Oct 22, 2016

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

ansibot commented Apr 11, 2017

gundalow commented Sep 19, 2019