Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Molecule seems to fail when Ansible is installed from rpm #2173

Closed
alexsaezm opened this issue Jul 15, 2019 · 13 comments
Closed

Molecule seems to fail when Ansible is installed from rpm #2173

alexsaezm opened this issue Jul 15, 2019 · 13 comments
Labels
help wanted packaging installation, packaging, built containers upstream-bug

Comments

@alexsaezm
Copy link

Issue Type

  • Bug report

Molecule and Ansible details

ansible 2.8.2
  config file = /etc/ansible/ansible.cfg
  configured module search path = ['/home/asaezmor/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
  ansible python module location = /usr/lib/python3.6/site-packages/ansible
  executable location = /usr/bin/ansible
  python version = 3.6.8 (default, Apr  3 2019, 17:26:03) [GCC 8.2.1 20180905 (Red Hat 8.2.1-3)]
molecule, version 2.20.1

Molecule installation method (one of):

  • pip

Ansible installation method (one of):

  • pip
  • rpm

Desired Behavior

The execution should work no matter how Ansible was installed.

Actual Behaviour

When installed from a rpm file (ansible-2.8.2-1.el8ae.noarch.rpm) the execution failed.
Using pip, everything works.

$ molecule --debug converge
--> Validating schema /home/asaezmor/Developer/src/github.com/alexsaezm/fedora-lab/molecule/default/molecule.yml.
Validation completed successfully.
--> Test matrix
    
└── default
    ├── dependency
    ├── create
    ├── prepare
    └── converge
    
--> Scenario: 'default'
--> Action: 'dependency'
Skipping, missing the requirements file.
--> Scenario: 'default'
--> Action: 'create'
DEBUG: ANSIBLE ENVIRONMENT
---
ANSIBLE_CONFIG: /tmp/molecule/fedora-lab/default/ansible.cfg
ANSIBLE_FILTER_PLUGINS: /home/asaezmor/.local/lib/python3.6/site-packages/molecule/provisioner/ansible/plugins/filters:/tmp/molecule/fedora-lab/default/plugins/filters:/home/asaezmor/Developer/src/github.com/alexsaezm/fedora-lab/plugins/filters
ANSIBLE_LIBRARY: /home/asaezmor/.local/lib/python3.6/site-packages/molecule/provisioner/ansible/plugins/libraries:/tmp/molecule/fedora-lab/default/library:/home/asaezmor/Developer/src/github.com/alexsaezm/fedora-lab/library
ANSIBLE_ROLES_PATH: /tmp/molecule/fedora-lab/default/roles:/home/asaezmor/Developer/src/github.com/alexsaezm

DEBUG: MOLECULE ENVIRONMENT
---
MOLECULE_DEBUG: 'True'
MOLECULE_DEPENDENCY_NAME: galaxy
MOLECULE_DRIVER_NAME: docker
MOLECULE_ENV_FILE: /home/asaezmor/Developer/src/github.com/alexsaezm/fedora-lab/.env.yml
MOLECULE_EPHEMERAL_DIRECTORY: /tmp/molecule/fedora-lab/default
MOLECULE_FILE: /home/asaezmor/Developer/src/github.com/alexsaezm/fedora-lab/molecule/default/molecule.yml
MOLECULE_INSTANCE_CONFIG: /tmp/molecule/fedora-lab/default/instance_config.yml
MOLECULE_INVENTORY_FILE: /tmp/molecule/fedora-lab/default/inventory/ansible_inventory.yml
MOLECULE_LINT_NAME: yamllint
MOLECULE_PROJECT_DIRECTORY: /home/asaezmor/Developer/src/github.com/alexsaezm/fedora-lab
MOLECULE_PROVISIONER_LINT_NAME: ansible-lint
MOLECULE_PROVISIONER_NAME: ansible
MOLECULE_SCENARIO_DIRECTORY: /home/asaezmor/Developer/src/github.com/alexsaezm/fedora-lab/molecule/default
MOLECULE_SCENARIO_NAME: default
MOLECULE_VERIFIER_LINT_NAME: flake8
MOLECULE_VERIFIER_NAME: testinfra
MOLECULE_VERIFIER_TEST_DIRECTORY: /home/asaezmor/Developer/src/github.com/alexsaezm/fedora-lab/molecule/default/tests

DEBUG: SHELL REPLAY
ANSIBLE_CONFIG=/tmp/molecule/fedora-lab/default/ansible.cfg ANSIBLE_FILTER_PLUGINS=/home/asaezmor/.local/lib/python3.6/site-packages/molecule/provisioner/ansible/plugins/filters:/tmp/molecule/fedora-lab/default/plugins/filters:/home/asaezmor/Developer/src/github.com/alexsaezm/fedora-lab/plugins/filters ANSIBLE_LIBRARY=/home/asaezmor/.local/lib/python3.6/site-packages/molecule/provisioner/ansible/plugins/libraries:/tmp/molecule/fedora-lab/default/library:/home/asaezmor/Developer/src/github.com/alexsaezm/fedora-lab/library ANSIBLE_ROLES_PATH=/tmp/molecule/fedora-lab/default/roles:/home/asaezmor/Developer/src/github.com/alexsaezm MOLECULE_DEBUG=True MOLECULE_DEPENDENCY_NAME=galaxy MOLECULE_DRIVER_NAME=docker MOLECULE_ENV_FILE=/home/asaezmor/Developer/src/github.com/alexsaezm/fedora-lab/.env.yml MOLECULE_EPHEMERAL_DIRECTORY=/tmp/molecule/fedora-lab/default MOLECULE_FILE=/home/asaezmor/Developer/src/github.com/alexsaezm/fedora-lab/molecule/default/molecule.yml MOLECULE_INSTANCE_CONFIG=/tmp/molecule/fedora-lab/default/instance_config.yml MOLECULE_INVENTORY_FILE=/tmp/molecule/fedora-lab/default/inventory/ansible_inventory.yml MOLECULE_LINT_NAME=yamllint MOLECULE_PROJECT_DIRECTORY=/home/asaezmor/Developer/src/github.com/alexsaezm/fedora-lab MOLECULE_PROVISIONER_LINT_NAME=ansible-lint MOLECULE_PROVISIONER_NAME=ansible MOLECULE_SCENARIO_DIRECTORY=/home/asaezmor/Developer/src/github.com/alexsaezm/fedora-lab/molecule/default MOLECULE_SCENARIO_NAME=default MOLECULE_VERIFIER_LINT_NAME=flake8 MOLECULE_VERIFIER_NAME=testinfra MOLECULE_VERIFIER_TEST_DIRECTORY=/home/asaezmor/Developer/src/github.com/alexsaezm/fedora-lab/molecule/default/tests

DEBUG: COMMAND
/usr/bin/ansible-playbook --skip-tags=molecule-notest,notest --diff --inventory=/tmp/molecule/fedora-lab/default/inventory /home/asaezmor/.local/lib/python3.6/site-packages/molecule/provisioner/ansible/playbooks/docker/create.yml -vvv

    ansible-playbook 2.8.2
      config file = /tmp/molecule/fedora-lab/default/ansible.cfg
      configured module search path = ['/home/asaezmor/.local/lib/python3.6/site-packages/molecule/provisioner/ansible/plugins/libraries', '/tmp/molecule/fedora-lab/default/library', '/home/asaezmor/Developer/src/github.com/alexsaezm/fedora-lab/library']
      ansible python module location = /usr/lib/python3.6/site-packages/ansible
      executable location = /usr/bin/ansible-playbook
      python version = 3.6.8 (default, Apr  3 2019, 17:26:03) [GCC 8.2.1 20180905 (Red Hat 8.2.1-3)]
    Using /tmp/molecule/fedora-lab/default/ansible.cfg as config file
    host_list declined parsing /tmp/molecule/fedora-lab/default/inventory/ansible_inventory.yml as it did not pass it's verify_file() method
    script declined parsing /tmp/molecule/fedora-lab/default/inventory/ansible_inventory.yml as it did not pass it's verify_file() method
    Parsed /tmp/molecule/fedora-lab/default/inventory/ansible_inventory.yml inventory source with yaml plugin
    
    PLAYBOOK: create.yml ***********************************************************
    1 plays in /home/asaezmor/.local/lib/python3.6/site-packages/molecule/provisioner/ansible/playbooks/docker/create.yml
 [WARNING]: Skipping plugin (/home/asaezmor/.local/lib/python3.6/site-
packages/molecule/provisioner/ansible/plugins/filters/molecule_core.py) as it
seems to be invalid: No module named 'molecule'

    
    PLAY [Create] ******************************************************************
    META: ran handlers
    
    TASK [Log into a Docker registry] **********************************************
    task path: /home/asaezmor/.local/lib/python3.6/site-packages/molecule/provisioner/ansible/playbooks/docker/create.yml:8
    fatal: [localhost]: FAILED! => {
        "msg": "An unhandled exception occurred while templating '{{ lookup('file', molecule_file) | molecule_from_yaml }}'. Error was a <class 'ansible.errors.AnsibleError'>, original message: template error while templating string: no filter named 'molecule_from_yaml'. String: {{ lookup('file', molecule_file) | molecule_from_yaml }}"
    }
    
    PLAY RECAP *********************************************************************
    localhost                  : ok=0    changed=0    unreachable=0    failed=1    skipped=0    rescued=0    ignored=0
    
ERROR: 
@alexsaezm alexsaezm added the bug label Jul 15, 2019
@ssbarnea
Copy link
Member

Have you tried using code from master branch? If you cannot replicate it with master we can close it.

@alexsaezm
Copy link
Author

Good catch. With master works. Thanks!

@alexsaezm alexsaezm reopened this Jul 15, 2019
@alexsaezm
Copy link
Author

I replicated it using the latest version. My bad, I thought it was fixed but I didn't try exactly the same thing.

So:

  1. Ansible from rpm
  2. molecule-2.22rc3
$ molecule init role -r test-molecule
--> Initializing new role test-molecule...
Initialized role in /tmp/test-molecule successfully.
$ cd test-molecule/
$ cat molecule/default/molecule.yml 
---
dependency:
  name: galaxy
driver:
  name: podman
lint:
  name: yamllint
platforms:
  - name: instance
    image: centos:7
provisioner:
  name: ansible
  lint:
    name: ansible-lint
verifier:
  name: testinfra
  lint:
    name: flake8
$ molecule test
$ molecule test                                                                                                                                                                                                    
--> Validating schema /tmp/test-molecule/molecule/default/molecule.yml.                                                                                                                                            
Validation completed successfully.                                                                                                                                                                                 
--> Test matrix                                                                                                                                                                                                    
                                                                                                                                                                                                                   
└── default                                                                                                                                                                                                        
    ├── lint                                                                                                                                                                                                       
    ├── dependency                                                                                                                                                                                                 
    ├── cleanup                                                                                                                                                                                                    
    ├── destroy                                                                                                                                                                                                    
    ├── syntax                                                                                                                                                                                                     
    ├── create                                                                                                                                                                                                     
    ├── prepare                                                                                                                                                                                                    
    ├── converge                                                                                                                                                                                                   
    ├── idempotence                                                                                                                                                                                                
    ├── side_effect                                                                                                                                                                                                
    ├── verify                                                                                                                                                                                                     
    ├── cleanup
    └── destroy

--> Scenario: 'default'
--> Action: 'lint'
--> Executing Yamllint on files found in /tmp/test-molecule/...
Lint completed successfully.
--> Executing Flake8 on files found in /tmp/test-molecule/molecule/default/tests/...
Lint completed successfully.
--> Executing Ansible Lint on /tmp/test-molecule/molecule/default/playbook.yml...
    [701] Role info should contain platforms
    /tmp/test-molecule/meta/main.yml:2
    {'meta/main.yml': {'galaxy_info': {'author': 'your name', 'description': 'your description', 'company': 'your company (optional)', 'license': 'license (GPLv2, CC-BY, etc)', 'min_ansible_version': 1.2, 'gala$y_tags': [], '__line__': 3, '__file__': '/tmp/test-molecule/meta/main.yml'}, 'dependencies': [], '__line__': 2, '__file__': '/tmp/test-molecule/meta/main.yml'}}                                                  

    [703] Should change default metadata: author
    /tmp/test-molecule/meta/main.yml:2
    {'meta/main.yml': {'galaxy_info': {'author': 'your name', 'description': 'your description', 'company': 'your company (optional)', 'license': 'license (GPLv2, CC-BY, etc)', 'min_ansible_version': 1.2, 'gala$y_tags': [], '__line__': 3, '__file__': '/tmp/test-molecule/meta/main.yml'}, 'dependencies': [], '__line__': 2, '__file__': '/tmp/test-molecule/meta/main.yml'}}                                                  

    [703] Should change default metadata: description
    /tmp/test-molecule/meta/main.yml:2
    {'meta/main.yml': {'galaxy_info': {'author': 'your name', 'description': 'your description', 'company': 'your company (optional)', 'license': 'license (GPLv2, CC-BY, etc)', 'min_ansible_version': 1.2, 'gala$y_tags': [], '__line__': 3, '__file__': '/tmp/test-molecule/meta/main.yml'}, 'dependencies': [], '__line__': 2, '__file__': '/tmp/test-molecule/meta/main.yml'}}                                                  

    [703] Should change default metadata: company
    /tmp/test-molecule/meta/main.yml:2
    {'meta/main.yml': {'galaxy_info': {'author': 'your name', 'description': 'your description', 'company': 'your company (optional)', 'license': 'license (GPLv2, CC-BY, etc)', 'min_ansible_version': 1.2, 'galax
y_tags': [], '__line__': 3, '__file__': '/tmp/test-molecule/meta/main.yml'}, 'dependencies': [], '__line__': 2, '__file__': '/tmp/test-molecule/meta/main.yml'}}

    [703] Should change default metadata: license
    /tmp/test-molecule/meta/main.yml:2
    {'meta/main.yml': {'galaxy_info': {'author': 'your name', 'description': 'your description', 'company': 'your company (optional)', 'license': 'license (GPLv2, CC-BY, etc)', 'min_ansible_version': 1.2, 'galax
y_tags': [], '__line__': 3, '__file__': '/tmp/test-molecule/meta/main.yml'}, 'dependencies': [], '__line__': 2, '__file__': '/tmp/test-molecule/meta/main.yml'}}

An error occurred during the test sequence action: 'lint'. Cleaning up.
--> Scenario: 'default'
--> Action: 'cleanup'
Skipping, cleanup playbook not configured.
--> Scenario: 'default'
--> Action: 'destroy'
 [WARNING]: Skipping plugin (/home/asaezmor/Developer/src/github.com/ansible/mo
lecule-2.22rc3/molecule/provisioner/ansible/plugins/filters/molecule_core.py)
as it seems to be invalid: No module named 'molecule'


    PLAY [Destroy] *****************************************************************

    TASK [Destroy molecule instance(s)] ********************************************
    fatal: [localhost]: FAILED! => {"msg": "An unhandled exception occurred while templating '{{ lookup('file', molecule_file) | molecule_from_yaml }}'. Error was a <class 'ansible.errors.AnsibleError'>, origina
l message: template error while templating string: no filter named 'molecule_from_yaml'. String: {{ lookup('file', molecule_file) | molecule_from_yaml }}"}

    PLAY RECAP *********************************************************************
    localhost                  : ok=0    changed=0    unreachable=0    failed=1    skipped=0    rescued=0    ignored=0

ERROR: 
  1. When I use Ansible from pip:
$ molecule test                                                                                                                                                                                                    
--> Validating schema /tmp/test-molecule/molecule/default/molecule.yml.                                                                                                                                            
Validation completed successfully.                                                                                                                                                                                 
--> Test matrix                                                                                                                                                                                                    
                                                                                                                                                                                                                   
└── default                                                                                                                                                                                                        
    ├── lint                                                                                                                                                                                                       
    ├── dependency                                                                                                                                                                                                 
    ├── cleanup                                                                                                                                                                                                    
    ├── destroy                                                                                                                                                                                                    
    ├── syntax                                                                                                                                                                                                     
    ├── create                                                                                                                                                                                                     
    ├── prepare                                                                                                                                                                                                    
    ├── converge                                                                                                                                                                                                   
    ├── idempotence                                                                                                                                                                                                
    ├── side_effect                                                                                                                                                                                                
    ├── verify                                                                                                                                                                                                     
    ├── cleanup
    └── destroy

--> Scenario: 'default'
--> Action: 'lint'
--> Executing Yamllint on files found in /tmp/test-molecule/...
Lint completed successfully.
--> Executing Flake8 on files found in /tmp/test-molecule/molecule/default/tests/...
Lint completed successfully.
--> Executing Ansible Lint on /tmp/test-molecule/molecule/default/playbook.yml...
    [701] Role info should contain platforms
    /tmp/test-molecule/meta/main.yml:2
    {'meta/main.yml': {'galaxy_info': {'author': 'your name', 'description': 'your description', 'company': 'your company (optional)', 'license': 'license (GPLv2, CC-BY, etc)', 'min_ansible_version': 1.2, 'galaxy_tags': [], '__line__': 3, '__file__': '/tmp/test-molecule/meta/main.yml'}, 'dependencies': [], '__line__': 2, '__file__': '/tmp/test-molecule/meta/main.yml'}}                                                  

    [703] Should change default metadata: author
    /tmp/test-molecule/meta/main.yml:2
    {'meta/main.yml': {'galaxy_info': {'author': 'your name', 'description': 'your description', 'company': 'your company (optional)', 'license': 'license (GPLv2, CC-BY, etc)', 'min_ansible_version': 1.2, 'galaxy_tags': [], '__line__': 3, '__file__': '/tmp/test-molecule/meta/main.yml'}, 'dependencies': [], '__line__': 2, '__file__': '/tmp/test-molecule/meta/main.yml'}}                                                  

    [703] Should change default metadata: description
    /tmp/test-molecule/meta/main.yml:2
    {'meta/main.yml': {'galaxy_info': {'author': 'your name', 'description': 'your description', 'company': 'your company (optional)', 'license': 'license (GPLv2, CC-BY, etc)', 'min_ansible_version': 1.2, 'galaxy_tags': [], '__line__': 3, '__file__': '/tmp/test-molecule/meta/main.yml'}, 'dependencies': [], '__line__': 2, '__file__': '/tmp/test-molecule/meta/main.yml'}}                                                  

    [703] Should change default metadata: company
    /tmp/test-molecule/meta/main.yml:2
    {'meta/main.yml': {'galaxy_info': {'author': 'your name', 'description': 'your description', 'company': 'your company (optional)', 'license': 'license (GPLv2, CC-BY, etc)', 'min_ansible_version': 1.2, 'galaxy_tags': [], '__line__': 3, '__file__': '/tmp/test-molecule/meta/main.yml'}, 'dependencies': [], '__line__': 2, '__file__': '/tmp/test-molecule/meta/main.yml'}}                                                  

    [703] Should change default metadata: license
    /tmp/test-molecule/meta/main.yml:2
    {'meta/main.yml': {'galaxy_info': {'author': 'your name', 'description': 'your description', 'company': 'your company (optional)', 'license': 'license (GPLv2, CC-BY, etc)', 'min_ansible_version': 1.2, 'galaxy_tags': [], '__line__': 3, '__file__': '/tmp/test-molecule/meta/main.yml'}, 'dependencies': [], '__line__': 2, '__file__': '/tmp/test-molecule/meta/main.yml'}}                                                  

An error occurred during the test sequence action: 'lint'. Cleaning up.
--> Scenario: 'default'
--> Action: 'cleanup'
Skipping, cleanup playbook not configured.
--> Scenario: 'default'
--> Action: 'destroy'

    PLAY [Destroy] *****************************************************************

    TASK [Destroy molecule instance(s)] ********************************************
    changed: [localhost] => (item=None)
    changed: [localhost]

    TASK [Wait for instance(s) deletion to complete] *******************************
    changed: [localhost] => (item=None)
    changed: [localhost]

    PLAY RECAP *********************************************************************
    localhost                  : ok=2    changed=2    unreachable=0    failed=0    skipped=0    rescued=0    ignored=0                                                                                            

--> Pruning extra files from scenario ephemeral directory

So I think something is different between the rpm and the pip installations.

@decentral1se
Copy link
Contributor

Wait but now it is just failing with linting errors?

@ssato
Copy link

ssato commented Jul 18, 2019

(Excuse me from the side.)

I think that the RPM does not work because all of filters and plugins (.py files) under molecule/provisioner/ansible/ are missing (and might not be installed) from it.

   ...
ASK [Destroy molecule instance(s)] ********************************************
    fatal: [localhost]: FAILED! => {"msg": ... no filter named 'molecule_from_yaml'. ...
                                               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
   ...

@ssbarnea
Copy link
Member

ssbarnea commented Jul 19, 2019

We faced the same issue yesterday at https://bugs.launchpad.net/tripleo/+bug/1837143 and apparently this is caused by the decision of making the scripts installed use python -s which disables access to user installed packages. I am not sure why this was done but is clearly affecting a huge number of users.

@ssbarnea ssbarnea added packaging installation, packaging, built containers upstream-bug help wanted and removed bug labels Jul 19, 2019
@ssbarnea
Copy link
Member

One thing is clear: this is not a molecule bug and is likely what I would call a packaging bug (hopefully not an intended feature). I would usually try to raise it as an Ansible bug but if you look at https://releases.ansible.com/ansible/rpm/release/ you will not find an ansible 2.8 based on python3.6 so.... you mainly used a non-official ansible distribution. Time to file it un bugzilla against RHEL.

@alexsaezm
Copy link
Author

@decentral1se no, sorry, was another test in a new role.
@ssbarnea I used the rpm file from Red Hat.

@tutunak
Copy link

tutunak commented Sep 2, 2019

I have the same problem on fedora 30 with ansible 2.8.2 installed from the official repository.

@tutunak
Copy link

tutunak commented Sep 2, 2019

@alexsaezm do you use Fedora?

@tutunak
Copy link

tutunak commented Sep 2, 2019

I've removed molecule from pip3 and pip2, remove ansible by dnf and then install ansible and molecule from pip3 and it's working fine for me now.

@decentral1se
Copy link
Contributor

Should we close this or?

@matihost
Copy link

Can it be reopened until the issue is fixed on Ansible rpm packaging side?
It is there RH Bugzilla or issue already posted anywhere?

@tutunak your workaround works for me thanks!
Snippet for anybody else:

sudo yum -y remove ansible
pip3 install ansible --user
pip3 install "molecule[lint]" --user
pip3 install "molecule[docker]" --user

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
help wanted packaging installation, packaging, built containers upstream-bug
Projects
None yet
Development

No branches or pull requests

6 participants