add connection plugin for buildah

[TomasTomecek]

Signed-off-by: Tomas Tomecek ttomecek@redhat.com

SUMMARY

This PR introduces a new connection plugin for buildah.

The connection plugin should be utilized in Ansible Container to perform builds of container images without the need to use a dedicated daemon (read, dockerd).

ISSUE TYPE

• Feature Pull Request

COMPONENT NAME

plugin/connection/buildah

ANSIBLE VERSION

2.3.1

ADDITIONAL INFORMATION

If you look at the code, you can see a bunch of TODOs -- should I resolve all of them? The biggest one is: copying files from the container -- buildah is expected to create new container images, hence it doesn't support copying files from the container now -- shall I request such feature?

CC @chouseknecht @gregdek @j00bar @nalind

[ansibot]

The test ansible-test sanity --test boilerplate failed with the following error:

Command "test/sanity/code-smell/boilerplate.sh" returned exit status 2.
>>> Standard Output
== Missing __metaclass__ = type ==
./lib/ansible/plugins/connection/buildah.py

The test ansible-test sanity --test pep8 failed with the following error:

lib/ansible/plugins/connection/buildah.py:109:5: E301 expected 1 blank line, found 0

The test ansible-test sanity --test pylint failed with the following error:

lib/ansible/plugins/connection/buildah.py:107:8: notimplemented-raised NotImplemented raised - should raise NotImplementedError

click here for bot help

[chouseknecht]

One minor nit. Outside of that, looks great!

[chouseknecht]

I think this comment is misleading. It's actually running a command inside a buildah container.

[TomasTomecek]

I forgot to update all the comments, these come from the docker connection plugin.

[chouseknecht]

This is a thought, and not a blocker...

Not being able to copy a file from the container is interesting, and problematic potentially from an Ansible Container standpoint.

During a build, Ansible Container mounts /usr from conductor to /_usr on the target container. This is to enable sharing the Python runtime, package managers, and the like, without having to install them into the target container. I'm not sure you can do that with buildah.

I notice there is a buildah mount command, which allows mounting the container file system. Could it be that's Dan's intention is that you mount the container's filesystem, and perform operations against it directly, using something like chroot.

[TomasTomecek]

Chris, that is a very good point -- how would we utilize conductor with buildah?

I'm assuming that mount operation should be sufficient for the /_usr operation. Here's a RFE for buildah: containers/buildah#177

I will try to finish the PR today.

[bcoca]

assumes that buildah is in path, i would check first and return a fatal exception if this is missing informing the user the need for it to exist.

[bcoca]

see the 'dd' workaround we use in chroot and other 'local plugins' to get around this limitation

[TomasTomecek]

I did it with cat >$out_path, is it okay?

[bcoca]

that has many implications, starting by requiring specific shells and issues with encodings

[TomasTomecek]

I don't understand how shells have anything to do with this: it's not even invoked via shell. The sample I posted above is a shortcut to:

local_cmd = ['buildah', 'run', '--', self._container_id, 'cat',
             to_bytes(in_path, errors='surrogate_or_strict')]
with open(to_bytes(out_path, errors='surrogate_or_strict'), 'wb') as out_file:
    p = subprocess.Popen(local_cmd, shell=False, stdin=subprocess.PIPE,
                         stdout=out_file, stderr=subprocess.PIPE)

which I hoped would be obvious, sorry for not being precisely specific.

I understand that encodings might get messy. I'll take a look on the other workarounds. I fear that doing it with dd would not be possible since dd would get executed in the container and we need to get the file to host (some mounting magic could do it).

[bcoca]

> is a redirection, which ONLY works inside shells, and some shells don't use it the same way.

[bcoca]

no to the TODO, as this assumes exclusive access, which is not guaranteed

[TomasTomecek]

okay, I will remove the TODO then

[bcoca]

should this check that the instance actually exists?

[TomasTomecek]

you mean the container exists?

[bcoca]

yep

[bcoca]

not a requirement, but it would be nice to have docs here, in 2.4 we are adding docs to connection plugins also (see ssh plugin)

[TomasTomecek]

will definitely provide documentation

[bcoca]

comment does not match code ... also connections should not be creating hosts, just connecting to them

[bcoca]

that has many implications, starting by requiring specific shells and issues with encodings

[TomasTomecek]

@bcoca thanks for comments, I will try to resolve all of them on Monday.

I have one question: how should I implement integration tests? Originally I did it the same way other connection plugins were doing it -- after the rebase, I realized it's the old way so I turned it into a role by copying the playbook of generic connection test -- please review.

[mattclay]

@TomasTomecek You should be able to re-use the existing generic connection test without copying the playbook. Look at connection_lxc for an example.

You'll need to test locally with ansible-test integration connection_buildah since it won't run in CI.

[TomasTomecek]

@mattclay thanks for the tip! Unfortunately I'm running into an issue which I have no idea how to resolve:

root@laptop $ ansible-test integration --local connection_buildah
...
TASK [remove remote file with unicode filename and content] ******************************************************************************************************************
task path: /home/tt/g/ansible/test/integration/targets/connection/test_connection.yml:21
Using module_utils file /home/tt/g/ansible/lib/ansible/module_utils/_text.py
Using module_utils file /home/tt/g/ansible/lib/ansible/module_utils/basic.py
Using module_utils file /home/tt/g/ansible/lib/ansible/module_utils/pycompat24.py
Using module_utils file /home/tt/g/ansible/lib/ansible/module_utils/six/__init__.py
Using module_utils file /home/tt/g/ansible/lib/ansible/module_utils/six/_six.py
Using module file /home/tt/g/ansible/lib/ansible/modules/files/file.py
<buildah-container> RUN ['buildah', 'mount', '--', 'buildah-container']
MOUNTPOINT /var/lib/containers/storage/overlay/9b20ff75c09d5a38c2fa4690790b4f972e57655621cade4dddd15d03433a7e40/merged RC 0 STDERR ''
<buildah-container> RUN ['buildah', 'run', '--', 'buildah-container', '/bin/sh', '-c', 'echo ~ && sleep 0']
STDOUT '' STDERR ''
<buildah-container> RUN ['buildah', 'run', '--', 'buildah-container', '/bin/sh', '-c', '( umask 77 && mkdir -p "` echo /root/.ansible/tmp/ansible-tmp-1498856124.09-5712721270
452 `" && echo ansible-tmp-1498856124.09-5712721270452="` echo /root/.ansible/tmp/ansible-tmp-1498856124.09-5712721270452 `" ) && sleep 0']
STDOUT '' STDERR ''
<buildah-container> PUT /tmp/tmpu3oJkp TO /root/.ansible/tmp/ansible-tmp-1498856124.09-5712721270452/file.py
<buildah-container> RUN ['buildah', 'run', '--', 'buildah-container', '/bin/sh', '-c', 'chmod u+x /root/.ansible/tmp/ansible-tmp-1498856124.09-5712721270452/ /root/.ansible/t
mp/ansible-tmp-1498856124.09-5712721270452/file.py && sleep 0']
STDOUT '' STDERR ''
<buildah-container> RUN ['buildah', 'run', '--', 'buildah-container', '/bin/sh', '-c', '/tmp/ansible-test-coverage-cBoE0Y/coverage/injector.py /root/.ansible/tmp/ansible-tmp-
1498856124.09-5712721270452/file.py; rm -rf "/root/.ansible/tmp/ansible-tmp-1498856124.09-5712721270452/" > /dev/null 2>&1 && sleep 0']
STDOUT '/bin/sh: 1: /tmp/ansible-test-coverage-cBoE0Y/coverage/injector.py: not found\n' STDERR '/bin/sh: 1: /tmp/ansible-test-coverage-cBoE0Y/coverage/injector.py: not found
\n'
<buildah-container> RUN ['buildah', 'umount', '--', 'buildah-container']
RC 0 STDOUT '' STDERR ''
fatal: [buildah-container]: FAILED! => {
    "changed": false,
    "failed": true,
    "module_stderr": "/bin/sh: 1: /tmp/ansible-test-coverage-cBoE0Y/coverage/injector.py: not found\n",
    "module_stdout": "",
    "msg": "MODULE FAILURE",
    "rc": 0
}
        to retry, use: --limit @/home/tt/g/ansible/test/integration/targets/connection/test_connection.retry
...

How come that coverage is in place when I haven't used --coverage option?

[mattclay]

@TomasTomecek You'll need to set ANSIBLE_TEST_REMOTE_INTERPRETER='' in the test script to prevent the injector from being used, since the remote host isn't the same as the local controller.

This can be done in the same way the connection_ssh test sets environment vars while reusing the existing connection test.

[TomasTomecek]

thanks @mattclay, that helped a lot

Unfortunately I stumbled into another one: containers/buildah#179

<buildah-container> RUN ['buildah', 'mount', '--', 'buildah-container']
MOUNTPOINT /var/lib/containers/storage/overlay/9b20ff75c09d5a38c2fa4690790b4f972e57655621cade4dddd15d03433a7e40/merged RC 0 STDERR ''
<buildah-container> RUN ['buildah', 'run', '--', 'buildah-container', '/bin/sh', '-c', '/usr/bin/python && sleep 0']
STDOUT '' STDERR ''
<buildah-container> RUN ['buildah', 'umount', '--', 'buildah-container']
RC 0 STDOUT '' STDERR ''
fatal: [buildah-container]: FAILED! => {
    "changed": false,
    "failed": true,
    "module_stderr": "",
    "module_stdout": "",
    "msg": "MODULE FAILURE",
    "rc": 0
}
        to retry, use: --limit @/home/tt/g/ansible/test/integration/targets/connection/test_connection.retry

PLAY RECAP *******************************************************************************************************************************************************************
buildah-container          : ok=3    changed=1    unreachable=0    failed=1

Command exited with status 2 after 2.08892011642 seconds.

[ansibot]

The test ansible-test sanity --test shellcheck failed with the following errors:

test/integration/targets/connection_buildah/runme.sh:5:32: SC1007 Remove space after = if trying to assign a value (for empty string, use var='' ... ).
test/integration/targets/connection_buildah/runme.sh:7:32: SC1007 Remove space after = if trying to assign a value (for empty string, use var='' ... ).

click here for bot help

[TomasTomecek]

Any update here? @bcoca

[TomasTomecek]

Thanks! I'm glad you squashed the commits.

[it-praktyk]

@TomasTomecek thank you for your job!

I'm very interesting to start using Ansible + Buildha but ... I don't know how can I start using that connection type? Should I use https://docs.ansible.com/ansible-container/ ?

[TomasTomecek]

@it-praktyk Here are some brief instructions for testing: https://github.com/TomasTomecek/ansible/blob/8fd3b209a81255781efbfaf614bfbade828a9565/test/integration/targets/connection_buildah/test_connection.inventory

Let's get through a list of commands how you can use it with ansible in practice, this is just from top of my head. You need to make sure that you created the buildah container before running the playbook and named the container same as the host you specified in the playbook.

$ sudo buildah from --name=buildah-container python:2

$ cat playbook.yaml
- hosts: buildah-container
...

$ cat inventory
buildah-container

$ sudo ansible-playbook -c buildah -i inventory playbook.yaml

This PR includes an integration test so you can check it out.

In the meantime, I'm working on support for buildah in ansible-container -- unfortunately it's a lot of work and it will take me weeks (or maybe even months) to finish it and make it usable.

[it-praktyk]

Thank you for explanation. I'll try track a progress. Please let me know if will you need someone for testing.

[TomasTomecek]

I have finally pushed myself into writing a blog post how this PR can be utilized: https://blog.tomecek.net/post/building-containers-with-buildah-and-ansible/