add connection plugin for buildah #26170

TomasTomecek · 2017-06-28T09:18:50Z

Signed-off-by: Tomas Tomecek ttomecek@redhat.com

SUMMARY

This PR introduces a new connection plugin for buildah.

The connection plugin should be utilized in Ansible Container to perform builds of container images without the need to use a dedicated daemon (read, dockerd).

ISSUE TYPE

Feature Pull Request

COMPONENT NAME

plugin/connection/buildah

ANSIBLE VERSION

2.3.1

ADDITIONAL INFORMATION

If you look at the code, you can see a bunch of TODOs -- should I resolve all of them? The biggest one is: copying files from the container -- buildah is expected to create new container images, hence it doesn't support copying files from the container now -- shall I request such feature?

CC @chouseknecht @gregdek @j00bar @nalind

ansibot · 2017-06-28T13:18:25Z

The test ansible-test sanity --test boilerplate failed with the following error:

Command "test/sanity/code-smell/boilerplate.sh" returned exit status 2.
>>> Standard Output
== Missing __metaclass__ = type ==
./lib/ansible/plugins/connection/buildah.py

The test ansible-test sanity --test pep8 failed with the following error:

lib/ansible/plugins/connection/buildah.py:109:5: E301 expected 1 blank line, found 0

The test ansible-test sanity --test pylint failed with the following error:

lib/ansible/plugins/connection/buildah.py:107:8: notimplemented-raised NotImplemented raised - should raise NotImplementedError

click here for bot help

chouseknecht

One minor nit. Outside of that, looks great!

chouseknecht · 2017-06-29T15:44:31Z

lib/ansible/plugins/connection/buildah.py

+
+    @ensure_connect
+    def exec_command(self, cmd, in_data=None, sudoable=False):
+        """ Run a command on the docker host """


I think this comment is misleading. It's actually running a command inside a buildah container.

I forgot to update all the comments, these come from the docker connection plugin.

chouseknecht · 2017-06-29T16:14:33Z

This is a thought, and not a blocker...

Not being able to copy a file from the container is interesting, and problematic potentially from an Ansible Container standpoint.

During a build, Ansible Container mounts /usr from conductor to /_usr on the target container. This is to enable sharing the Python runtime, package managers, and the like, without having to install them into the target container. I'm not sure you can do that with buildah.

I notice there is a buildah mount command, which allows mounting the container file system. Could it be that's Dan's intention is that you mount the container's filesystem, and perform operations against it directly, using something like chroot.

TomasTomecek · 2017-06-30T09:12:00Z

Chris, that is a very good point -- how would we utilize conductor with buildah?

I'm assuming that mount operation should be sufficient for the /_usr operation. Here's a RFE for buildah: containers/buildah#177

I will try to finish the PR today.

bcoca · 2017-06-30T11:44:41Z

lib/ansible/plugins/connection/buildah.py

+        cmd_bytes = to_bytes(cmd, errors='surrogate_or_strict')
+        cmd_args_list = shlex.split(cmd_bytes)
+
+        local_cmd = ['buildah', 'run', '--', self._container_id]


assumes that buildah is in path, i would check first and return a fatal exception if this is missing informing the user the need for it to exist.

bcoca · 2017-06-30T11:45:30Z

lib/ansible/plugins/connection/buildah.py

+        stderr = to_bytes(stderr, errors='surrogate_or_strict')
+        display.vvvvv("STDOUT %s STDERR %s" % (stdout, stderr))
+
+    def fetch_file(self, in_path, out_path):


see the 'dd' workaround we use in chroot and other 'local plugins' to get around this limitation

I did it with cat >$out_path, is it okay?

that has many implications, starting by requiring specific shells and issues with encodings

I don't understand how shells have anything to do with this: it's not even invoked via shell. The sample I posted above is a shortcut to:

local_cmd = ['buildah', 'run', '--', self._container_id, 'cat', to_bytes(in_path, errors='surrogate_or_strict')] with open(to_bytes(out_path, errors='surrogate_or_strict'), 'wb') as out_file: p = subprocess.Popen(local_cmd, shell=False, stdin=subprocess.PIPE, stdout=out_file, stderr=subprocess.PIPE)

which I hoped would be obvious, sorry for not being precisely specific.

I understand that encodings might get messy. I'll take a look on the other workarounds. I fear that doing it with dd would not be possible since dd would get executed in the container and we need to get the file to host (some mounting magic could do it).

> is a redirection, which ONLY works inside shells, and some shells don't use it the same way.

bcoca · 2017-06-30T11:46:41Z

lib/ansible/plugins/connection/buildah.py

+
+    def close(self):
+        super(Connection, self).close()
+        # TODO: we should probably get rid of ~/.ansible directory in the container


no to the TODO, as this assumes exclusive access, which is not guaranteed

okay, I will remove the TODO then

bcoca · 2017-06-30T11:48:28Z

lib/ansible/plugins/connection/buildah.py

+
+    def _connect(self):
+        """ Create a container from specified container image, via host """
+        super(Connection, self)._connect()


should this check that the instance actually exists?

you mean the container exists?

bcoca · 2017-06-30T11:49:30Z

lib/ansible/plugins/connection/buildah.py

+#
+# You should have received a copy of the GNU General Public License
+# along with Ansible.  If not, see <http://www.gnu.org/licenses/>.
+from __future__ import (absolute_import, division, print_function)


not a requirement, but it would be nice to have docs here, in 2.4 we are adding docs to connection plugins also (see ssh plugin)

will definitely provide documentation

bcoca · 2017-06-30T11:50:45Z

lib/ansible/plugins/connection/buildah.py

+        # TODO: save actual user and print it
+
+    def _connect(self):
+        """ Create a container from specified container image, via host """


comment does not match code ... also connections should not be creating hosts, just connecting to them

Signed-off-by: Tomas Tomecek <ttomecek@redhat.com>

bcoca · 2017-06-30T14:23:58Z

lib/ansible/plugins/connection/buildah.py

+        stderr = to_bytes(stderr, errors='surrogate_or_strict')
+        display.vvvvv("STDOUT %s STDERR %s" % (stdout, stderr))
+
+    def fetch_file(self, in_path, out_path):


that has many implications, starting by requiring specific shells and issues with encodings

TomasTomecek · 2017-06-30T14:25:46Z

@bcoca thanks for comments, I will try to resolve all of them on Monday.

I have one question: how should I implement integration tests? Originally I did it the same way other connection plugins were doing it -- after the rebase, I realized it's the old way so I turned it into a role by copying the playbook of generic connection test -- please review.

mattclay · 2017-06-30T18:00:52Z

@TomasTomecek You should be able to re-use the existing generic connection test without copying the playbook. Look at connection_lxc for an example.

You'll need to test locally with ansible-test integration connection_buildah since it won't run in CI.

* create a method to invoke buildah * mount container filesystem persistently so we can access it during put and fetch * use copyfile function for copying files Signed-off-by: Tomas Tomecek <ttomecek@redhat.com>

Signed-off-by: Tomas Tomecek <ttomecek@redhat.com>

TomasTomecek · 2017-07-01T09:29:43Z

@mattclay thanks for the tip! Unfortunately I'm running into an issue which I have no idea how to resolve:

root@laptop $ ansible-test integration --local connection_buildah
...
TASK [remove remote file with unicode filename and content] ******************************************************************************************************************
task path: /home/tt/g/ansible/test/integration/targets/connection/test_connection.yml:21
Using module_utils file /home/tt/g/ansible/lib/ansible/module_utils/_text.py
Using module_utils file /home/tt/g/ansible/lib/ansible/module_utils/basic.py
Using module_utils file /home/tt/g/ansible/lib/ansible/module_utils/pycompat24.py
Using module_utils file /home/tt/g/ansible/lib/ansible/module_utils/six/__init__.py
Using module_utils file /home/tt/g/ansible/lib/ansible/module_utils/six/_six.py
Using module file /home/tt/g/ansible/lib/ansible/modules/files/file.py
<buildah-container> RUN ['buildah', 'mount', '--', 'buildah-container']
MOUNTPOINT /var/lib/containers/storage/overlay/9b20ff75c09d5a38c2fa4690790b4f972e57655621cade4dddd15d03433a7e40/merged RC 0 STDERR ''
<buildah-container> RUN ['buildah', 'run', '--', 'buildah-container', '/bin/sh', '-c', 'echo ~ && sleep 0']
STDOUT '' STDERR ''
<buildah-container> RUN ['buildah', 'run', '--', 'buildah-container', '/bin/sh', '-c', '( umask 77 && mkdir -p "` echo /root/.ansible/tmp/ansible-tmp-1498856124.09-5712721270
452 `" && echo ansible-tmp-1498856124.09-5712721270452="` echo /root/.ansible/tmp/ansible-tmp-1498856124.09-5712721270452 `" ) && sleep 0']
STDOUT '' STDERR ''
<buildah-container> PUT /tmp/tmpu3oJkp TO /root/.ansible/tmp/ansible-tmp-1498856124.09-5712721270452/file.py
<buildah-container> RUN ['buildah', 'run', '--', 'buildah-container', '/bin/sh', '-c', 'chmod u+x /root/.ansible/tmp/ansible-tmp-1498856124.09-5712721270452/ /root/.ansible/t
mp/ansible-tmp-1498856124.09-5712721270452/file.py && sleep 0']
STDOUT '' STDERR ''
<buildah-container> RUN ['buildah', 'run', '--', 'buildah-container', '/bin/sh', '-c', '/tmp/ansible-test-coverage-cBoE0Y/coverage/injector.py /root/.ansible/tmp/ansible-tmp-
1498856124.09-5712721270452/file.py; rm -rf "/root/.ansible/tmp/ansible-tmp-1498856124.09-5712721270452/" > /dev/null 2>&1 && sleep 0']
STDOUT '/bin/sh: 1: /tmp/ansible-test-coverage-cBoE0Y/coverage/injector.py: not found\n' STDERR '/bin/sh: 1: /tmp/ansible-test-coverage-cBoE0Y/coverage/injector.py: not found
\n'
<buildah-container> RUN ['buildah', 'umount', '--', 'buildah-container']
RC 0 STDOUT '' STDERR ''
fatal: [buildah-container]: FAILED! => {
    "changed": false,
    "failed": true,
    "module_stderr": "/bin/sh: 1: /tmp/ansible-test-coverage-cBoE0Y/coverage/injector.py: not found\n",
    "module_stdout": "",
    "msg": "MODULE FAILURE",
    "rc": 0
}
        to retry, use: --limit @/home/tt/g/ansible/test/integration/targets/connection/test_connection.retry
...

How come that coverage is in place when I haven't used --coverage option?

Signed-off-by: Tomas Tomecek <ttomecek@redhat.com>

mattclay · 2017-07-02T19:57:49Z

@TomasTomecek You'll need to set ANSIBLE_TEST_REMOTE_INTERPRETER='' in the test script to prevent the injector from being used, since the remote host isn't the same as the local controller.

This can be done in the same way the connection_ssh test sets environment vars while reusing the existing connection test.

TomasTomecek · 2017-07-03T09:07:32Z

thanks @mattclay, that helped a lot

Unfortunately I stumbled into another one: containers/buildah#179

<buildah-container> RUN ['buildah', 'mount', '--', 'buildah-container']
MOUNTPOINT /var/lib/containers/storage/overlay/9b20ff75c09d5a38c2fa4690790b4f972e57655621cade4dddd15d03433a7e40/merged RC 0 STDERR ''
<buildah-container> RUN ['buildah', 'run', '--', 'buildah-container', '/bin/sh', '-c', '/usr/bin/python && sleep 0']
STDOUT '' STDERR ''
<buildah-container> RUN ['buildah', 'umount', '--', 'buildah-container']
RC 0 STDOUT '' STDERR ''
fatal: [buildah-container]: FAILED! => {
    "changed": false,
    "failed": true,
    "module_stderr": "",
    "module_stdout": "",
    "msg": "MODULE FAILURE",
    "rc": 0
}
        to retry, use: --limit @/home/tt/g/ansible/test/integration/targets/connection/test_connection.retry

PLAY RECAP *******************************************************************************************************************************************************************
buildah-container          : ok=3    changed=1    unreachable=0    failed=1

Command exited with status 2 after 2.08892011642 seconds.

(this will be squashed into a single commit before merge) * add docs for the conn plugin * fix issue invoking the integration tests * add a way to invoke commands inside the container as a different user Signed-off-by: Tomas Tomecek <ttomecek@redhat.com>

ansibot · 2017-07-03T09:57:32Z

The test ansible-test sanity --test shellcheck failed with the following errors:

test/integration/targets/connection_buildah/runme.sh:5:32: SC1007 Remove space after = if trying to assign a value (for empty string, use var='' ... ).
test/integration/targets/connection_buildah/runme.sh:7:32: SC1007 Remove space after = if trying to assign a value (for empty string, use var='' ... ).

click here for bot help

Signed-off-by: Tomas Tomecek <ttomecek@redhat.com>

TomasTomecek · 2017-07-11T14:46:08Z

Any update here? @bcoca

TomasTomecek · 2017-07-12T05:46:08Z

Thanks! I'm glad you squashed the commits.

it-praktyk · 2017-07-17T20:55:30Z

@TomasTomecek thank you for your job!

I'm very interesting to start using Ansible + Buildha but ... I don't know how can I start using that connection type? Should I use https://docs.ansible.com/ansible-container/ ?

TomasTomecek · 2017-07-18T07:25:47Z

@it-praktyk Here are some brief instructions for testing: https://github.com/TomasTomecek/ansible/blob/8fd3b209a81255781efbfaf614bfbade828a9565/test/integration/targets/connection_buildah/test_connection.inventory

Let's get through a list of commands how you can use it with ansible in practice, this is just from top of my head. You need to make sure that you created the buildah container before running the playbook and named the container same as the host you specified in the playbook.

$ sudo buildah from --name=buildah-container python:2

$ cat playbook.yaml
- hosts: buildah-container
...

$ cat inventory
buildah-container

$ sudo ansible-playbook -c buildah -i inventory playbook.yaml

This PR includes an integration test so you can check it out.

In the meantime, I'm working on support for buildah in ansible-container -- unfortunately it's a lot of work and it will take me weeks (or maybe even months) to finish it and make it usable.

it-praktyk · 2017-07-19T08:25:41Z

Thank you for explanation. I'll try track a progress. Please let me know if will you need someone for testing.

TomasTomecek · 2018-02-04T12:20:39Z

I have finally pushed myself into writing a blog post how this PR can be utilized: https://blog.tomecek.net/post/building-containers-with-buildah-and-ansible/

chouseknecht suggested changes Jun 29, 2017

View reviewed changes

ansibot added the needs_revision This PR fails CI tests or a maintainer has requested a review/revision of the PR. label Jun 29, 2017

nitzmahone added new_plugin This PR includes a new plugin. and removed needs_triage Needs a first human triage before being processed. labels Jun 29, 2017

ansibot added the needs_rebase https://docs.ansible.com/ansible/devel/dev_guide/developing_rebasing.html label Jun 30, 2017

TomasTomecek mentioned this pull request Jun 30, 2017

add support to copying files from container containers/buildah#177

Closed

bcoca requested changes Jun 30, 2017

View reviewed changes

add connection plugin for buildah

8b94eee

Signed-off-by: Tomas Tomecek <ttomecek@redhat.com>

TomasTomecek force-pushed the add-buildah-connection-plugin branch from 8a67ea7 to 8b94eee Compare June 30, 2017 14:18

bcoca requested changes Jun 30, 2017

View reviewed changes

ansibot removed the needs_rebase https://docs.ansible.com/ansible/devel/dev_guide/developing_rebasing.html label Jun 30, 2017

TomasTomecek added 2 commits June 30, 2017 22:21

fixup

b2df931

* create a method to invoke buildah * mount container filesystem persistently so we can access it during put and fetch * use copyfile function for copying files Signed-off-by: Tomas Tomecek <ttomecek@redhat.com>

revert tests

3a38c82

Signed-off-by: Tomas Tomecek <ttomecek@redhat.com>

fixup

5882c2b

Signed-off-by: Tomas Tomecek <ttomecek@redhat.com>

progress

987db11

(this will be squashed into a single commit before merge) * add docs for the conn plugin * fix issue invoking the integration tests * add a way to invoke commands inside the container as a different user Signed-off-by: Tomas Tomecek <ttomecek@redhat.com>

fix shellcheck warning

8fd3b20

Signed-off-by: Tomas Tomecek <ttomecek@redhat.com>

TomasTomecek mentioned this pull request Jul 7, 2017

add a way to disable PTY allocation containers/buildah#181

Closed

ansibot added stale_ci This PR has been tested by CI more than one week ago. Close and re-open this PR to get it retested. stale_review Updates were made after the last review and the last review is more than 7 days old. labels Jul 11, 2017

bcoca approved these changes Jul 12, 2017

View reviewed changes

bcoca merged commit 5b898a7 into ansible:devel Jul 12, 2017

TomasTomecek deleted the add-buildah-connection-plugin branch July 12, 2017 05:44

ansibot added feature This issue/PR relates to a feature request. and removed feature_pull_request labels Mar 5, 2018

ansible locked and limited conversation to collaborators Apr 26, 2019

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

add connection plugin for buildah #26170

add connection plugin for buildah #26170

TomasTomecek commented Jun 28, 2017

ansibot commented Jun 28, 2017

chouseknecht left a comment

chouseknecht Jun 29, 2017

TomasTomecek Jun 30, 2017

chouseknecht commented Jun 29, 2017

TomasTomecek commented Jun 30, 2017

bcoca Jun 30, 2017

bcoca Jun 30, 2017

TomasTomecek Jun 30, 2017

bcoca Jun 30, 2017

TomasTomecek Jun 30, 2017

bcoca Jun 30, 2017

bcoca Jun 30, 2017

TomasTomecek Jun 30, 2017

bcoca Jun 30, 2017

TomasTomecek Jun 30, 2017

bcoca Jun 30, 2017

bcoca Jun 30, 2017

TomasTomecek Jun 30, 2017

bcoca Jun 30, 2017

bcoca Jun 30, 2017

TomasTomecek commented Jun 30, 2017

mattclay commented Jun 30, 2017

TomasTomecek commented Jul 1, 2017

mattclay commented Jul 2, 2017

TomasTomecek commented Jul 3, 2017

ansibot commented Jul 3, 2017

TomasTomecek commented Jul 11, 2017

TomasTomecek commented Jul 12, 2017

it-praktyk commented Jul 17, 2017

TomasTomecek commented Jul 18, 2017 •

edited

Loading

it-praktyk commented Jul 19, 2017

TomasTomecek commented Feb 4, 2018

add connection plugin for buildah #26170

add connection plugin for buildah #26170

Conversation

TomasTomecek commented Jun 28, 2017

SUMMARY

ISSUE TYPE

COMPONENT NAME

ANSIBLE VERSION

ADDITIONAL INFORMATION

ansibot commented Jun 28, 2017

chouseknecht left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

chouseknecht commented Jun 29, 2017

TomasTomecek commented Jun 30, 2017

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

TomasTomecek commented Jun 30, 2017

mattclay commented Jun 30, 2017

TomasTomecek commented Jul 1, 2017

mattclay commented Jul 2, 2017

TomasTomecek commented Jul 3, 2017

ansibot commented Jul 3, 2017

TomasTomecek commented Jul 11, 2017

TomasTomecek commented Jul 12, 2017

it-praktyk commented Jul 17, 2017

TomasTomecek commented Jul 18, 2017 • edited Loading

it-praktyk commented Jul 19, 2017

TomasTomecek commented Feb 4, 2018

TomasTomecek commented Jul 18, 2017 •

edited

Loading