Filter cannot be loaded on ansible-core 2.14.11 at packages in venv. 2.14.10 is ok.

[hugotkk]

Summary

While working on OpenStack installation using kolla-ansible, I encountered an issue concerning custom filters, specifically when utilising ansible-core 2.14.11. Downgrade to 2.14.10 is fine.

Observation:

• I thought it was the kolla-ansible issue but there was no recent related update on the kolla-ansible repository.
• The error coincided with the release of ansible-core version 2.14.11, suggesting a potential version-specific problem. Reverting to version 2.14.10 resolved the issue, affirming the hypothesis.

Issue Type

Bug Report

Component Name

pip install 'ansible>=6,<8'

Ansible Version

(env) vagrant@coa-controller:~$ ansible --version
ansible [core 2.14.11]
  config file = None
  configured module search path = ['/home/vagrant/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
  ansible python module location = /home/vagrant/env/lib/python3.10/site-packages/ansible
  ansible collection location = /home/vagrant/.ansible/collections:/usr/share/ansible/collections
  executable location = /home/vagrant/env/bin/ansible
  python version = 3.10.12 (main, Jun 11 2023, 05:26:28) [GCC 11.4.0] (/home/vagrant/env/bin/python3)
  jinja version = 3.1.2
  libyaml = True

Configuration

(env) vagrant@coa-controller:~$ ansible-config dump --only-changed -t all
CONFIG_FILE() = None

OS / Environment

ubuntu 22.04

Steps to Reproduce

For context, I've been following this guide strictly: https://docs.openstack.org/kolla-ansible/latest/user/quickstart.html.

I started my work weeks ago. It was working fine until I started a fresh installation (from OS) on 9th Oct.

It stopped working at

kolla-ansible -i all-in-one deploy

Expected Results

The playbook should run to the end

PLAY RECAP ***************************************************************************************************************************************************
localhost : ok=346 changed=196 unreachable=0 failed=0 skipped=255 rescued=0 ignored=1

Actual Results

It stopped at the middle


TASK [haproxy-config : Copying over nova-cell:nova-novncproxy haproxy config] ...
fatal: [localhost]: FAILED! => {"msg": "An unhandled exception occurred while templating '{{ cell_proxy_project_services | namespace_haproxy_for_cell(cell_name) }}'. Error was a <class 'ansible.errors.AnsibleError'>, original message: template error while templating string: Could not load \"namespace_haproxy_for_cell\": 'namespace_haproxy_for_cell'. String: {{ cell_proxy_project_services | namespace_haproxy_for_cell(cell_name) }}. Could not load \"namespace_haproxy_for_cell\": 'namespace_haproxy_for_cell'"}

It complains the filter namespace_haproxy_for_cell could not be loaded

But the filter is defined at env/lib/python3.10/site-packages/kolla_ansible/filters.py

### Code of Conduct

- [X] I agree to follow the Ansible Code of Conduct

[ansibot]

Files identified in the description:

• lib/ansible/modules/pip.py
• bin/ansible

If these files are incorrect, please update the component name section of the description or use the component bot command.

[artificial-intelligence]

@hugotkk fwiw I'll try to workaround this in kolla via this patch, currently testing it:

https://review.opendev.org/c/openstack/kolla/+/897843

[artificial-intelligence]

looking at the changes merged between 2.14.10 and 2.14.11 I have a hunch this might be related to #81644 but it needs to be investigated first.

[srieger1]

for everybody encountering this issue before a fix is released in kolla-ansible or ansible-core: It is enough to downgrade ansible-core to 2.14.10 on the kolla-ansible deployment node without (re)building the kolla-toolbox container image using the Dockerfile provied in https://review.opendev.org/c/openstack/kolla/+/897843. E.g., by running:

pip install ansible-core==2.14.10

before starting kolla-ansible deploy.

Thanks to @artificial-intelligence and @hugotkk for the fix! 🎉

[nitzmahone]

I suspect I know what's going on here, but it looks like getting kolla-ansible to run far enough to verify the problem requires a ridiculous amount of setup for someone that doesn't have Openstack stuff just laying around everywhere (or taking pruning shears to big chunks of its codebase to no-op everything). Is there a container/VM image laying around somewhere that I can just fire up to mess with this?

[mnasiadka]

@nitzmahone not really a container or VM image - but https://docs.openstack.org/kolla-ansible/latest/user/quickstart.html is probably pretty straightforward if you have a VM for experimentation.

If you have some pointers or ideas what might be the cause - we're open to them - I was planning to have a look into that in coming days.

[osfrickler]

So the issue is somehow triggered by #81644, if I revert that commit on top of stable/2.14, the issue goes away. The relevant locations in kolla-ansible are this filter plugin in one role:

https://github.com/openstack/kolla-ansible/blob/master/ansible/roles/nova-cell/filter_plugins/filters.py

and the use of this filter in variable handed to import_role:

https://github.com/openstack/kolla-ansible/blob/master/ansible/roles/nova-cell/tasks/cell_proxy_loadbalancer.yml

I can reproduce the issue with a simple playbook calling two nested minimal roles like in the above case. I'll create a clean reproducer tomorrow but maybe this is already enough to make progress.

[sivel]

Please check to see if #82002 resolves your issue.

I believe this is a duplicate of #81897

[osfrickler]

Just applying the first part of the patch for lib/ansible/plugins/loader.py solves the issue both in my test case and for the kolla-ansible run.

[mnasiadka]

Same here, fixes the issue.

[bcoca]

closing this as dupe (linked above in @sivel's comment), see that ticket for final resolution