New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add connection plugin for buildah #26170

Merged
merged 6 commits into from Jul 12, 2017

Conversation

Projects
None yet
7 participants
@TomasTomecek
Contributor

TomasTomecek commented Jun 28, 2017

Signed-off-by: Tomas Tomecek ttomecek@redhat.com

SUMMARY

This PR introduces a new connection plugin for buildah.

The connection plugin should be utilized in Ansible Container to perform builds of container images without the need to use a dedicated daemon (read, dockerd).

ISSUE TYPE
  • Feature Pull Request
COMPONENT NAME

plugin/connection/buildah

ANSIBLE VERSION
2.3.1
ADDITIONAL INFORMATION

If you look at the code, you can see a bunch of TODOs -- should I resolve all of them? The biggest one is: copying files from the container -- buildah is expected to create new container images, hence it doesn't support copying files from the container now -- shall I request such feature?

CC @chouseknecht @gregdek @j00bar @nalind

@ansibot

This comment has been minimized.

Contributor

ansibot commented Jun 28, 2017

The test ansible-test sanity --test boilerplate failed with the following error:

Command "test/sanity/code-smell/boilerplate.sh" returned exit status 2.
>>> Standard Output
== Missing __metaclass__ = type ==
./lib/ansible/plugins/connection/buildah.py

The test ansible-test sanity --test pep8 failed with the following error:

lib/ansible/plugins/connection/buildah.py:109:5: E301 expected 1 blank line, found 0

The test ansible-test sanity --test pylint failed with the following error:

lib/ansible/plugins/connection/buildah.py:107:8: notimplemented-raised NotImplemented raised - should raise NotImplementedError

click here for bot help

@chouseknecht

One minor nit. Outside of that, looks great!

lib/ansible/plugins/connection/buildah.py Outdated
@ensure_connect
def exec_command(self, cmd, in_data=None, sudoable=False):
""" Run a command on the docker host """

This comment has been minimized.

@chouseknecht

chouseknecht Jun 29, 2017

Member

I think this comment is misleading. It's actually running a command inside a buildah container.

This comment has been minimized.

@TomasTomecek

TomasTomecek Jun 30, 2017

Contributor

I forgot to update all the comments, these come from the docker connection plugin.

@chouseknecht

This comment has been minimized.

Member

chouseknecht commented Jun 29, 2017

This is a thought, and not a blocker...

Not being able to copy a file from the container is interesting, and problematic potentially from an Ansible Container standpoint.

During a build, Ansible Container mounts /usr from conductor to /_usr on the target container. This is to enable sharing the Python runtime, package managers, and the like, without having to install them into the target container. I'm not sure you can do that with buildah.

I notice there is a buildah mount command, which allows mounting the container file system. Could it be that's Dan's intention is that you mount the container's filesystem, and perform operations against it directly, using something like chroot.

@TomasTomecek

This comment has been minimized.

Contributor

TomasTomecek commented Jun 30, 2017

Chris, that is a very good point -- how would we utilize conductor with buildah?

I'm assuming that mount operation should be sufficient for the /_usr operation. Here's a RFE for buildah: containers/buildah#177

I will try to finish the PR today.

lib/ansible/plugins/connection/buildah.py Outdated
cmd_bytes = to_bytes(cmd, errors='surrogate_or_strict')
cmd_args_list = shlex.split(cmd_bytes)
local_cmd = ['buildah', 'run', '--', self._container_id]

This comment has been minimized.

@bcoca

bcoca Jun 30, 2017

Member

assumes that buildah is in path, i would check first and return a fatal exception if this is missing informing the user the need for it to exist.

lib/ansible/plugins/connection/buildah.py Outdated
stderr = to_bytes(stderr, errors='surrogate_or_strict')
display.vvvvv("STDOUT %s STDERR %s" % (stdout, stderr))
def fetch_file(self, in_path, out_path):

This comment has been minimized.

@bcoca

bcoca Jun 30, 2017

Member

see the 'dd' workaround we use in chroot and other 'local plugins' to get around this limitation

This comment has been minimized.

@TomasTomecek

TomasTomecek Jun 30, 2017

Contributor

I did it with cat >$out_path, is it okay?

This comment has been minimized.

@bcoca

bcoca Jun 30, 2017

Member

that has many implications, starting by requiring specific shells and issues with encodings

This comment has been minimized.

@TomasTomecek

TomasTomecek Jun 30, 2017

Contributor

I don't understand how shells have anything to do with this: it's not even invoked via shell. The sample I posted above is a shortcut to:

local_cmd = ['buildah', 'run', '--', self._container_id, 'cat',
             to_bytes(in_path, errors='surrogate_or_strict')]
with open(to_bytes(out_path, errors='surrogate_or_strict'), 'wb') as out_file:
    p = subprocess.Popen(local_cmd, shell=False, stdin=subprocess.PIPE,
                         stdout=out_file, stderr=subprocess.PIPE)

which I hoped would be obvious, sorry for not being precisely specific.

I understand that encodings might get messy. I'll take a look on the other workarounds. I fear that doing it with dd would not be possible since dd would get executed in the container and we need to get the file to host (some mounting magic could do it).

This comment has been minimized.

@bcoca

bcoca Jun 30, 2017

Member

> is a redirection, which ONLY works inside shells, and some shells don't use it the same way.

lib/ansible/plugins/connection/buildah.py Outdated
def close(self):
super(Connection, self).close()
# TODO: we should probably get rid of ~/.ansible directory in the container

This comment has been minimized.

@bcoca

bcoca Jun 30, 2017

Member

no to the TODO, as this assumes exclusive access, which is not guaranteed

This comment has been minimized.

@TomasTomecek

TomasTomecek Jun 30, 2017

Contributor

okay, I will remove the TODO then

lib/ansible/plugins/connection/buildah.py Outdated
def _connect(self):
""" Create a container from specified container image, via host """
super(Connection, self)._connect()

This comment has been minimized.

@bcoca

bcoca Jun 30, 2017

Member

should this check that the instance actually exists?

This comment has been minimized.

@TomasTomecek

TomasTomecek Jun 30, 2017

Contributor

you mean the container exists?

This comment has been minimized.

@bcoca
lib/ansible/plugins/connection/buildah.py Outdated
#
# You should have received a copy of the GNU General Public License
# along with Ansible. If not, see <http://www.gnu.org/licenses/>.
from __future__ import (absolute_import, division, print_function)

This comment has been minimized.

@bcoca

bcoca Jun 30, 2017

Member

not a requirement, but it would be nice to have docs here, in 2.4 we are adding docs to connection plugins also (see ssh plugin)

This comment has been minimized.

@TomasTomecek

TomasTomecek Jun 30, 2017

Contributor

will definitely provide documentation

lib/ansible/plugins/connection/buildah.py Outdated
# TODO: save actual user and print it
def _connect(self):
""" Create a container from specified container image, via host """

This comment has been minimized.

@bcoca

bcoca Jun 30, 2017

Member

comment does not match code ... also connections should not be creating hosts, just connecting to them

add connection plugin for buildah
Signed-off-by: Tomas Tomecek <ttomecek@redhat.com>
lib/ansible/plugins/connection/buildah.py Outdated
stderr = to_bytes(stderr, errors='surrogate_or_strict')
display.vvvvv("STDOUT %s STDERR %s" % (stdout, stderr))
def fetch_file(self, in_path, out_path):

This comment has been minimized.

@bcoca

bcoca Jun 30, 2017

Member

that has many implications, starting by requiring specific shells and issues with encodings

@ansibot ansibot removed the needs_rebase label Jun 30, 2017

@TomasTomecek

This comment has been minimized.

Contributor

TomasTomecek commented Jun 30, 2017

@bcoca thanks for comments, I will try to resolve all of them on Monday.

I have one question: how should I implement integration tests? Originally I did it the same way other connection plugins were doing it -- after the rebase, I realized it's the old way so I turned it into a role by copying the playbook of generic connection test -- please review.

@mattclay

This comment has been minimized.

Member

mattclay commented Jun 30, 2017

@TomasTomecek You should be able to re-use the existing generic connection test without copying the playbook. Look at connection_lxc for an example.

You'll need to test locally with ansible-test integration connection_buildah since it won't run in CI.

TomasTomecek added some commits Jun 30, 2017

fixup
 * create a method to invoke buildah
 * mount container filesystem persistently so we can access it
   during put and fetch
 * use copyfile function for copying files

Signed-off-by: Tomas Tomecek <ttomecek@redhat.com>
revert tests
Signed-off-by: Tomas Tomecek <ttomecek@redhat.com>
@TomasTomecek

This comment has been minimized.

Contributor

TomasTomecek commented Jul 1, 2017

@mattclay thanks for the tip! Unfortunately I'm running into an issue which I have no idea how to resolve:

root@laptop $ ansible-test integration --local connection_buildah
...
TASK [remove remote file with unicode filename and content] ******************************************************************************************************************
task path: /home/tt/g/ansible/test/integration/targets/connection/test_connection.yml:21
Using module_utils file /home/tt/g/ansible/lib/ansible/module_utils/_text.py
Using module_utils file /home/tt/g/ansible/lib/ansible/module_utils/basic.py
Using module_utils file /home/tt/g/ansible/lib/ansible/module_utils/pycompat24.py
Using module_utils file /home/tt/g/ansible/lib/ansible/module_utils/six/__init__.py
Using module_utils file /home/tt/g/ansible/lib/ansible/module_utils/six/_six.py
Using module file /home/tt/g/ansible/lib/ansible/modules/files/file.py
<buildah-container> RUN ['buildah', 'mount', '--', 'buildah-container']
MOUNTPOINT /var/lib/containers/storage/overlay/9b20ff75c09d5a38c2fa4690790b4f972e57655621cade4dddd15d03433a7e40/merged RC 0 STDERR ''
<buildah-container> RUN ['buildah', 'run', '--', 'buildah-container', '/bin/sh', '-c', 'echo ~ && sleep 0']
STDOUT '' STDERR ''
<buildah-container> RUN ['buildah', 'run', '--', 'buildah-container', '/bin/sh', '-c', '( umask 77 && mkdir -p "` echo /root/.ansible/tmp/ansible-tmp-1498856124.09-5712721270
452 `" && echo ansible-tmp-1498856124.09-5712721270452="` echo /root/.ansible/tmp/ansible-tmp-1498856124.09-5712721270452 `" ) && sleep 0']
STDOUT '' STDERR ''
<buildah-container> PUT /tmp/tmpu3oJkp TO /root/.ansible/tmp/ansible-tmp-1498856124.09-5712721270452/file.py
<buildah-container> RUN ['buildah', 'run', '--', 'buildah-container', '/bin/sh', '-c', 'chmod u+x /root/.ansible/tmp/ansible-tmp-1498856124.09-5712721270452/ /root/.ansible/t
mp/ansible-tmp-1498856124.09-5712721270452/file.py && sleep 0']
STDOUT '' STDERR ''
<buildah-container> RUN ['buildah', 'run', '--', 'buildah-container', '/bin/sh', '-c', '/tmp/ansible-test-coverage-cBoE0Y/coverage/injector.py /root/.ansible/tmp/ansible-tmp-
1498856124.09-5712721270452/file.py; rm -rf "/root/.ansible/tmp/ansible-tmp-1498856124.09-5712721270452/" > /dev/null 2>&1 && sleep 0']
STDOUT '/bin/sh: 1: /tmp/ansible-test-coverage-cBoE0Y/coverage/injector.py: not found\n' STDERR '/bin/sh: 1: /tmp/ansible-test-coverage-cBoE0Y/coverage/injector.py: not found
\n'
<buildah-container> RUN ['buildah', 'umount', '--', 'buildah-container']
RC 0 STDOUT '' STDERR ''
fatal: [buildah-container]: FAILED! => {
    "changed": false,
    "failed": true,
    "module_stderr": "/bin/sh: 1: /tmp/ansible-test-coverage-cBoE0Y/coverage/injector.py: not found\n",
    "module_stdout": "",
    "msg": "MODULE FAILURE",
    "rc": 0
}
        to retry, use: --limit @/home/tt/g/ansible/test/integration/targets/connection/test_connection.retry
...

How come that coverage is in place when I haven't used --coverage option?

fixup
Signed-off-by: Tomas Tomecek <ttomecek@redhat.com>
@mattclay

This comment has been minimized.

Member

mattclay commented Jul 2, 2017

@TomasTomecek You'll need to set ANSIBLE_TEST_REMOTE_INTERPRETER='' in the test script to prevent the injector from being used, since the remote host isn't the same as the local controller.

This can be done in the same way the connection_ssh test sets environment vars while reusing the existing connection test.

@TomasTomecek

This comment has been minimized.

Contributor

TomasTomecek commented Jul 3, 2017

thanks @mattclay, that helped a lot

Unfortunately I stumbled into another one: containers/buildah#179

<buildah-container> RUN ['buildah', 'mount', '--', 'buildah-container']
MOUNTPOINT /var/lib/containers/storage/overlay/9b20ff75c09d5a38c2fa4690790b4f972e57655621cade4dddd15d03433a7e40/merged RC 0 STDERR ''
<buildah-container> RUN ['buildah', 'run', '--', 'buildah-container', '/bin/sh', '-c', '/usr/bin/python && sleep 0']
STDOUT '' STDERR ''
<buildah-container> RUN ['buildah', 'umount', '--', 'buildah-container']
RC 0 STDOUT '' STDERR ''
fatal: [buildah-container]: FAILED! => {
    "changed": false,
    "failed": true,
    "module_stderr": "",
    "module_stdout": "",
    "msg": "MODULE FAILURE",
    "rc": 0
}
        to retry, use: --limit @/home/tt/g/ansible/test/integration/targets/connection/test_connection.retry

PLAY RECAP *******************************************************************************************************************************************************************
buildah-container          : ok=3    changed=1    unreachable=0    failed=1

Command exited with status 2 after 2.08892011642 seconds.
progress
(this will be squashed into a single commit before merge)

 * add docs for the conn plugin
 * fix issue invoking the integration tests
 * add a way to invoke commands inside the container as a different user

Signed-off-by: Tomas Tomecek <ttomecek@redhat.com>
@ansibot

This comment has been minimized.

Contributor

ansibot commented Jul 3, 2017

The test ansible-test sanity --test shellcheck failed with the following errors:

test/integration/targets/connection_buildah/runme.sh:5:32: SC1007 Remove space after = if trying to assign a value (for empty string, use var='' ... ).
test/integration/targets/connection_buildah/runme.sh:7:32: SC1007 Remove space after = if trying to assign a value (for empty string, use var='' ... ).

click here for bot help

fix shellcheck warning
Signed-off-by: Tomas Tomecek <ttomecek@redhat.com>
@TomasTomecek

This comment has been minimized.

Contributor

TomasTomecek commented Jul 11, 2017

Any update here? @bcoca

@bcoca

bcoca approved these changes Jul 12, 2017

@bcoca bcoca merged commit 5b898a7 into ansible:devel Jul 12, 2017

1 check passed

Shippable Run 27278 status is SUCCESS.
Details

@TomasTomecek TomasTomecek deleted the TomasTomecek:add-buildah-connection-plugin branch Jul 12, 2017

@TomasTomecek

This comment has been minimized.

Contributor

TomasTomecek commented Jul 12, 2017

Thanks! I'm glad you squashed the commits.

@it-praktyk

This comment has been minimized.

Contributor

it-praktyk commented Jul 17, 2017

@TomasTomecek thank you for your job!

I'm very interesting to start using Ansible + Buildha but ... I don't know how can I start using that connection type? Should I use https://docs.ansible.com/ansible-container/ ?

@TomasTomecek

This comment has been minimized.

Contributor

TomasTomecek commented Jul 18, 2017

@it-praktyk Here are some brief instructions for testing: https://github.com/TomasTomecek/ansible/blob/8fd3b209a81255781efbfaf614bfbade828a9565/test/integration/targets/connection_buildah/test_connection.inventory

Let's get through a list of commands how you can use it with ansible in practice, this is just from top of my head. You need to make sure that you created the buildah container before running the playbook and named the container same as the host you specified in the playbook.

$ sudo buildah from --name=buildah-container python:2

$ cat playbook.yaml
- hosts: buildah-container
...

$ cat inventory
buildah-container

$ sudo ansible-playbook -c buildah -i inventory playbook.yaml

This PR includes an integration test so you can check it out.

In the meantime, I'm working on support for buildah in ansible-container -- unfortunately it's a lot of work and it will take me weeks (or maybe even months) to finish it and make it usable.

@it-praktyk

This comment has been minimized.

Contributor

it-praktyk commented Jul 19, 2017

Thank you for explanation. I'll try track a progress. Please let me know if will you need someone for testing.

@TomasTomecek

This comment has been minimized.

Contributor

TomasTomecek commented Feb 4, 2018

I have finally pushed myself into writing a blog post how this PR can be utilized: https://blog.tomecek.net/post/building-containers-with-buildah-and-ansible/

@ansibot ansibot added feature and removed feature_pull_request labels Mar 5, 2018

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment