Aborting, target uses selinux but python bindings (libselinux-python) aren't installed!

[dschier-wtd]

Update 2018-04-18: For those encounteing missing selinux when running with virtualenvs you can install the pure python https://pypi.org/project/selinux/ shim which will load the real selinux bindings from outside the virtualenv.

Molecule and Ansible details

ansible --version && molecule --version

ansible 2.7.6
  config file = None
  configured module search path = ['/home/ds/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
  ansible python module location = /home/ds/.venv-molecule-p3/lib/python3.7/site-packages/ansible
  executable location = /home/ds/.venv-molecule-p3/bin/ansible
  python version = 3.7.2 (default, Jan 16 2019, 19:49:22) [GCC 8.2.1 20181215 (Red Hat 8.2.1-6)]
molecule, version 2.19.0

Molecule installation method (one of):

• pip

Ansible installation method (one of):

• pip
• OS package

Detail any linters or test runners used:

Desired Behavior

molecule test should run locally on Fedora 29 in virtualenv

Actual Behaviour

molecule test brakes due to missing libselinux-python at Step Create Dockerfiles from image names

Creating an env with site-packages is not able to perform due to pyyaml version.
Creating an env without site packages is not able to perform due to missing selinux python bindings.
Copying the selinux packages in the venv does not help.

Output of molecule test without site packages.

    TASK [Create Dockerfiles from image names] *************************************
    task path: /home/ds/.venv-molecule-p3/lib/python3.7/site-packages/molecule/provisioner/ansible/playbooks/docker/create.yml:21
    <127.0.0.1> ESTABLISH LOCAL CONNECTION FOR USER: ds
    <127.0.0.1> EXEC /bin/sh -c 'echo ~ds && sleep 0'
    <127.0.0.1> EXEC /bin/sh -c '( umask 77 && mkdir -p "` echo /home/ds/.ansible/tmp/ansible-tmp-1549316301.9148593-163859707096071 `" && echo ansible-tmp-1549316301.9148593-163859707096071="` echo /home/ds/.ansible/tmp/ansible-tmp-1549316301.9148593-163859707096071 `" ) && sleep 0'
    Using module file /home/ds/.venv-molecule-p3/lib/python3.7/site-packages/ansible/modules/files/stat.py
    <127.0.0.1> PUT /home/ds/.ansible/tmp/ansible-local-24132gpawu2kd/tmpxnk6_h_z TO /home/ds/.ansible/tmp/ansible-tmp-1549316301.9148593-163859707096071/AnsiballZ_stat.py
    <127.0.0.1> EXEC /bin/sh -c 'chmod u+x /home/ds/.ansible/tmp/ansible-tmp-1549316301.9148593-163859707096071/ /home/ds/.ansible/tmp/ansible-tmp-1549316301.9148593-163859707096071/AnsiballZ_stat.py && sleep 0'
    <127.0.0.1> EXEC /bin/sh -c '/home/ds/.venv-molecule-p3/bin/python3 /home/ds/.ansible/tmp/ansible-tmp-1549316301.9148593-163859707096071/AnsiballZ_stat.py && sleep 0'
    Using module file /home/ds/.venv-molecule-p3/lib/python3.7/site-packages/ansible/modules/files/file.py
    <127.0.0.1> PUT /home/ds/.ansible/tmp/ansible-local-24132gpawu2kd/tmpi8lrc2o7 TO /home/ds/.ansible/tmp/ansible-tmp-1549316301.9148593-163859707096071/AnsiballZ_file.py
    <127.0.0.1> EXEC /bin/sh -c 'chmod u+x /home/ds/.ansible/tmp/ansible-tmp-1549316301.9148593-163859707096071/ /home/ds/.ansible/tmp/ansible-tmp-1549316301.9148593-163859707096071/AnsiballZ_file.py && sleep 0'
    <127.0.0.1> EXEC /bin/sh -c '/home/ds/.venv-molecule-p3/bin/python3 /home/ds/.ansible/tmp/ansible-tmp-1549316301.9148593-163859707096071/AnsiballZ_file.py && sleep 0'
    <127.0.0.1> PUT /home/ds/.ansible/tmp/ansible-local-24132gpawu2kd/tmpfd5trz4z/Dockerfile.j2 TO /home/ds/.ansible/tmp/ansible-tmp-1549316301.9148593-163859707096071/source
    <127.0.0.1> EXEC /bin/sh -c 'chmod u+x /home/ds/.ansible/tmp/ansible-tmp-1549316301.9148593-163859707096071/ /home/ds/.ansible/tmp/ansible-tmp-1549316301.9148593-163859707096071/source && sleep 0'
    Using module file /home/ds/.venv-molecule-p3/lib/python3.7/site-packages/ansible/modules/files/copy.py
    <127.0.0.1> PUT /home/ds/.ansible/tmp/ansible-local-24132gpawu2kd/tmpmmw347_6 TO /home/ds/.ansible/tmp/ansible-tmp-1549316301.9148593-163859707096071/AnsiballZ_copy.py
    <127.0.0.1> EXEC /bin/sh -c 'chmod u+x /home/ds/.ansible/tmp/ansible-tmp-1549316301.9148593-163859707096071/ /home/ds/.ansible/tmp/ansible-tmp-1549316301.9148593-163859707096071/AnsiballZ_copy.py && sleep 0'
    <127.0.0.1> EXEC /bin/sh -c '/home/ds/.venv-molecule-p3/bin/python3 /home/ds/.ansible/tmp/ansible-tmp-1549316301.9148593-163859707096071/AnsiballZ_copy.py && sleep 0'
    <127.0.0.1> EXEC /bin/sh -c 'rm -f -r /home/ds/.ansible/tmp/ansible-tmp-1549316301.9148593-163859707096071/ > /dev/null 2>&1 && sleep 0'
    failed: [localhost] (item={'name': 'instance', 'image': 'centos:7'}) => {
        "changed": false,
        "checksum": "40a922dca6c7a93024ac1ab412987783252b69c2",
        "diff": [
            {
                "after": "# Molecule managed\n\nFROM centos:7\n\nRUN if [ $(command -v apt-get) ]; then apt-get update && apt-get install -y python sudo bash ca-certificates && apt-get clean; \\\n    elif [ $(command -v dnf) ]; then dnf makecache && dnf --assumeyes install python sudo python-devel python2-dnf bash && dnf clean all; \\\n    elif [ $(command -v yum) ]; then yum makecache fast && yum install -y python sudo yum-plugin-ovl bash && sed -i 's/plugins=0/plugins=1/g' /etc/yum.conf && yum clean all; \\\n    elif [ $(command -v zypper) ]; then zypper refresh && zypper install -y python sudo bash python-xml && zypper clean -a; \\\n    elif [ $(command -v apk) ]; then apk update && apk add --no-cache python sudo bash ca-certificates; \\\n    elif [ $(command -v xbps-install) ]; then xbps-install -Syu && xbps-install -y python sudo bash ca-certificates && xbps-remove -O; fi\n",
                "after_header": "/home/ds/.ansible/tmp/ansible-local-24132gpawu2kd/tmpfd5trz4z/Dockerfile.j2",
                "before": ""
            }
        ],
        "invocation": {
            "module_args": {
                "_original_basename": "Dockerfile.j2",
                "attributes": null,
                "backup": false,
                "checksum": "40a922dca6c7a93024ac1ab412987783252b69c2",
                "content": null,
                "delimiter": null,
                "dest": "/tmp/molecule/ansible-role-test/default/Dockerfile_centos_7",
                "directory_mode": null,
                "follow": false,
                "force": true,
                "group": null,
                "local_follow": null,
                "mode": null,
                "owner": null,
                "regexp": null,
                "remote_src": null,
                "selevel": null,
                "serole": null,
                "setype": null,
                "seuser": null,
                "src": "/home/ds/.ansible/tmp/ansible-tmp-1549316301.9148593-163859707096071/source",
                "unsafe_writes": null,
                "validate": null
            }
        },
        "item": {
            "image": "centos:7",
            "name": "instance"
        },
        "msg": "Aborting, target uses selinux but python bindings (libselinux-python) aren't installed!"
    }
    
    PLAY RECAP *********************************************************************
    localhost                  : ok=0    changed=0    unreachable=0    failed=1   
    
    
ERROR: 
An error occurred during the test sequence action: 'create'. Cleaning up.

Please feel free to reach out to me here or in IRC @freenode #ansible-molecule. I will do my best to support you.

Thanks a lot in advance!

[decentral1se]

#1668 is related. /cc @ssbarnea

[dschier-wtd]

Yesterday I had a small chat in IRC #ansible-molecule

We came to the following workaround/solution:

# Create VENV
virtualenv --no-site-packages VENV
# Symlink system selinux packages + c-library
ln -s /usr/lib64/python3.7/site-packages/selinux/ VENV/lib/python3.7/site-packages/selinux
ln -s /usr/lib64/python3.7/site-packages/_selinux.cpython-37m-x86_64-linux-gnu.so VENV/lib/python3.7/site-packages/_selinux.cpython-37m-x86_64-linux-gnu.so
# Activate VENV
source ENV/bin/activate
# Install packages
pip install ansible molecule docker-py

Afterwards the VENV is usable. Maybe it can be added to the docs, that for some selinux systems it is needed to symlink the selinux libraries + the c-library.

[ssbarnea]

@daniel-wtd Do not ever try to use "docker-py" package, is 4 years old, it was replaced by "docker". I woudl go so far to even request its removal from pypi and replacing it with a placeholder that installs docker one.

[ssbarnea]

The root core issue here is different, is a design issue with molecule where by default it does hide the output when creation fails, something that is really common for new users and not only for new users.

I think we should find a way to address this and avoid the case where user needs to run again with --debug switch in order to find-out what went wrong. I would call it an unfortunate design decision, but we shoudl be able to find a way to address it.

[dschier-wtd]

@ssbarnea maybe the video from https://github.com/ansible/molecule/README.md leads to the opinion to use docker-py.

EDIT: it works woith docker-py and docker

The root cause here is not the docker package, but that the support for selinux is a little bit problematic. Using system-site-packages leads to problems and not using them forces to manual symlinking the selinux packages.

[decentral1se]

I think we should find a way to address this and avoid the case where user needs to run again with --debug switch in order to find-out what went wrong. I would call it an unfortunate design decision, but we shoudl be able to find a way to address it.

Just to link, I have a proposal out for an idea to fix this in #1666 (comment).

[ssbarnea]

This problem is not limited to Fedora, is happening with CentOS too. I think it may prove to be a good enough reason to delay release of 2.20. The magic line is:

Aborting, target uses selinux but python bindings (libselinux-python) aren't installed!

[ssbarnea]

While trying to fix it I realised that we don't need to install libselinux into the container because the problem was with template module itself, cased by ansible/ansible#34340

This is guaranteed to happen if you run tox on a redhat distro withotu sitespackages=true as libselinux bindings will be missing and there is no way to install them with pip.

[dschier-wtd]

Since Fedora has also some of the ansible packages in their repositories, it may happen, that sitepackages=true conflicts with the other python modules, already installed on the machine.

Is there a way, to import/link only selinux into the virtualenv?

[webknjaz]

@daniel-wtd what if you used tox --sitepackages only if needed?

WRT linking just selinux, I guess you could try symlinking or copying bindings to your virtualenv from system site-pakages as per https://dmsimard.com/2016/01/08/selinux-python-virtualenv-chroot-and-ansible-dont-play-nice/ AS LONG AS VENV'S PYTHON MATCHES THE SYSTEM ONE.

Other than that, the community could try putting together an OS-specific wheel dist compilation process to tackle the ability to do pip install libselinux which would solve this in a more generic way.
Though, it would need to go to selinux upstream so that it could be kept up-to-date.

[ssbarnea]

First use sitepackages does not cause conflicts because pip would remove syumlinked packages if they are not compatible and install compatible ones (newer or older).

The is still a very small risk which is more about the fact that other packages already installed on the system may become available during testing, for example pytest or flake8 plugins. In the past I had few cases where this caused different behavior during local testing, but it was easy to deal with it (they can be disabled in config).

Regarding installing libselinux inside virtualenvs, I know some workarounds but do really not want to go this route, involved doing manual file copy of the selinux module binaries from system into the virtualenv after creation. It is very specific to each distro/version and likely to break in so many cases... not to mention the additional ugliness added inside tox.ini for performing these steps.

There are other reasons why we may want to use python system packages: for avoiding to compile binary modules like openssl.

PS. Yep, I asked about selinux binary, I am not sure is doable yet with current wheel. I know few projects had partial success with some libraries but with selinux it may be too specific to each disto and version to make it possible. Even so, we kinda need a fix "now".

[webknjaz]

for avoiding to compile binary modules like openssl.

All openssl wrappers I know already ship proper wheels. So it's not a good example.

Even so, we kinda need a fix "now".

Agreed. I can live with this now, but let's at least add more comprehensive comments there explaining the situation and downsides.

[ssbarnea]

I made an update on the proposed change and also included a link to this bug, which will allow anyone that wonders about the resons behind to see the full discussion.

[gundalow]

2019-03-13 IRC Meeting. Agreed this isn't 2.20 blocking

[Lirt]

Will this be fixed and released into 2.20.2, or 2.21.0?

[ssbarnea]

@Lirt This is not really molecule bug and it cannot eveb be attributed to Ansible, so is not really a priority to address it. Still you are welcomed to apply the workaround. If you are lucky simple installing selinux froom pypi may sort it for you.

[Dmitry1987]

I don't understand why I have selinux disabled on the CentOS 7.7 distros (on Azure) but Ansible when runs from local workstation says it's not installed so a docker_container module fails.

Why does it even require any selinux packages, when that one is disabled? o.O
Is there some clean fix, or maybe will be, within Ansible future versions?

[canit00]

@Dmitry1987 take a look here for an easy straight forward workaround.

[queglay]

It looks like in these circumstances we have to revert to python 2.7, but we get warnings saying python 2.7 will be deprecated which is why I started to update everything and ran into this problem.
We have to find all instances of yum and copy in playbooks and append this:

  vars:
    ansible_python_interpreter: "{{ package_python_interpreter }}"

Where package_python_interpreter is deifined as python2.7 for any centos 7 host.
Finding these edge cases and overriding with host vars is a workaround, but not great.

I hope functions like and yum and copy will eventually work in python3 on Centos 7, before python 2.7 is deprecated.