The rpm python bindings needed for scan playbooks is missing

[AlanCoding]

The import here is breaking:

https://github.com/ansible/awx-facts-playbooks/blob/c5b4693061097fea5cf852e7d0792b1a9e75248c/library/scan_packages.py#L57

These are still supported, and will need to work in AWX-EE. They didn't used to fail, but some recent changes here are causing it to fail.

TASK [Scan packages (Unix/Linux)] **********************************************
An exception occurred during task execution. To see the full traceback, use -vvv. The error was: ModuleNotFoundError: No module named 'rpm'
fatal: [registered_host]: FAILED! => {"changed": false, "module_stderr": "Traceback (most recent call last):\\n  File \\"/root/.ansible/tmp/ansible-tmp-1609915382.0675998-90-214846177902729/AnsiballZ_scan_packages.py\\", line 99, in <module>\\n    _ansiballz_main()\\n  File \\"/root/.ansible/tmp/ansible-tmp-1609915382.0675998-90-214846177902729/AnsiballZ_scan_packages.py\\", line 91, in _ansiballz_main\\n    invoke_module(zipped_mod, temp_path, ANSIBALLZ_PARAMS)\\n  File \\"/root/.ansible/tmp/ansible-tmp-1609915382.0675998-90-214846177902729/AnsiballZ_scan_packages.py\\", line 40, in invoke_module\\n    runpy.run_module(mod_name='ansible.modules.scan_packages', init_globals=None, run_name='__main__', alter_sys=True)\\n  File \\"/usr/lib64/python3.8/runpy.py\\", line 207, in run_module\\n    return _run_module_code(code, init_globals, run_name, mod_spec)\\n  File \\"/usr/lib64/python3.8/runpy.py\\", line 97, in _run_module_code\\n    _run_code(code, mod_globals, init_globals,\\n  File \\"/usr/lib64/python3.8/runpy.py\\", line 87, in _run_code\\n    exec(code, run_globals)\\n  File \\"/tmp/ansible_scan_packages_payload_pxffu5v8/ansible_scan_packages_payload.zip/ansible/modules/scan_packages.py\\", line 111, in <module>\\n  File \\"/tmp/ansible_scan_packages_payload_pxffu5v8/ansible_scan_packages_payload.zip/ansible/modules/scan_packages.py\\", line 98, in main\\n  File \\"/tmp/ansible_scan_packages_payload_pxffu5v8/ansible_scan_packages_payload.zip/ansible/modules/scan_packages.py\\", line 57, in rpm_package_list\\nModuleNotFoundError: No module named 'rpm'\\n", "module_stdout": "", "msg": "MODULE FAILURE\\nSee stdout/stderr for the exact error", "rc": 1}

I can confirm that this is present in the AWX images before execution environments. It is a global package:

bash-4.4$ pip3 show rpm
Name: rpm
Version: 4.14.3
Summary: Python bindings for rpm
Home-page: http://www.rpm.org/
Author: UNKNOWN
Author-email: rpm-maint@lists.rpm.org
License: UNKNOWN
Location: /usr/lib64/python3.6/site-packages
Requires: 

I have not yet fully determined where this comes from, but the source appears to be part of "rpm" itself. This matches with the content in the AWX image site-packages, plus some *.so files.

https://github.com/rpm-software-management/rpm/tree/master/python/rpm

[AlanCoding]

We did get some help with this previously, and to document I'm going to have to quote some people here, hope you don't mind.

<something something about ansible-core packaging, python3.6 vs 3.8>

@shanemcd asked, on what I believe is this issue:

@sivel what does that mean for the dnf module / stuff that requires selinux bindings for things in 2.11?

Then we got a reply

@shanemcd you would likely want to define a localhost ansible_connection=local in inventory
to avoid using implicit localhost, and relying on interpreter discovery

Maybe this either isn't the same issue (rpm bindings, not selinux bindings), or there's another gap where the bindings need to be installed for the desired interpreter, because my testing with recent images yields:

$ podman run --rm --tty --interactive quay.io/ansible/awx-ee:latest python3.8 -c "import rpm"
Traceback (most recent call last):
  File "<string>", line 1, in <module>
ModuleNotFoundError: No module named 'rpm'

Recap:

• In https://hub.docker.com/r/ansible/awx, we have rpm in /usr/lib64/python3.6/site-packages
• In quay.io/ansible/ansible-runner:devel we do not have rpm in either /usr/lib/python3.8/site-packages or /usr/lib/python3.6/site-packages
• We also don't have it in quay.io/ansible/awx-ee:latest for either interpreter

That is the bug.

[AlanCoding]

I tried to track down some candidates that might have provided this through dnf list installed in both images. The most likely I saw were:

  "python3-rpm-macros.noarch",

  "python-srpm-macros.noarch",
  "python3-rpm-generators.noarch",

  "python36-devel.x86_64",

[AlanCoding]

This question looks pretty illustrative of the problem:

https://stackoverflow.com/questions/15412482/installing-rpm-module-for-non-system-python

Advice here is "don't do that". Don't use a non-system python and expect to get the rpm bindings.

[ryanpetrello]

@wenottingham how much do you care about compatibility and ongoing support for https://github.com/ansible/awx-facts-playbooks

[wenottingham]

I think #25 (comment) is going down the wrong road. We don't need any of this in the EE image - the errors from the scan playbook would be on the managed hosts, and that's where the import error is coming from.

What sorts of hosts are you running the playbook against?

[AlanCoding]

This is surfaced as a part of testing Insights - because a host's insights id is discovered through fact gathering. To do this testing, a host is created in AWX API with properties:

{
        "name": "registered_host",
        "inventory": "<host inventory>",
        "variables": "{\"ansible_host\": \"127.0.0.1\", \"ansible_connection\": \"local\"}"
    }

(with much from awxkit defaults)

Then a job template is created with properties:

{
        "name": "<some name>",
        "inventory": "<inventory with that host>",
        "project": "<project for https://github.com/ansible/awx-facts-playbooks>",
        "playbook": "scan_facts.yml",
        "limit": "registered_host",
        "use_fact_cache": true
    }

The scan_facts.yml playbook here targets hosts: all. With the JT limit, it targets "registered_host", and runs locally.

In terms of test infrastructure, I don't know that we use an independent managed host these days, much less a managed host that is configured for our pytest-ansible actions to be directed towards. Speaking extremely speculatively, maybe we could replace "ansible_connection": "local" with "ansible_host": "<server url>". We target the server URL for testing the awx.awx inventory plugin (I assume is working but could be wrong), but this is not the same. We need SSH credentials, which are locally present in the test executor somehow, but would have to be moved around in some novel ways. This would take work. There's a reason we never did it before.

---

Putting those challenges aside, what you're proposing isn't just having this playbook not work on hosts set to local connection - this will make the playbook fail whenever a host in the inventory sets local connection. That doesn't sound reasonable, and I'd think you would prefer to change the host pattern or skip those hosts in scan_facts.yml.

Avoiding local connection in that playbook may be pretty trivial, but that would still leave the insights testing problem unsolved.

---

I should mention that @pabelanger proposed having both pythons installed in the EE, and then we can use ansible_python_interpreter to use the system python (3.6) which can import rpm, but have everything else work in the installed python (3.8). I have to agree this would work, but it would create a lot of user confusion with EEs, or maybe just this EE in specific.

I mention that because I find it about equally distasteful as the other options I mentioned here.

[wenottingham]

We could also solve the "get the insights id" by just using ansible/awx#8650 instead.

[AlanCoding]

That can maybe put an insights id into the hostvars, not into the ansible facts for the hosts, which is how that integration works.

[wenottingham]

My understanding is that the insights id in facts is used to map "whatever" hosts to what insights thinks they are in the playbook. If we just say that for insights remdediations you pull the inventory from insights, it sidesteps that process entirely and should just work. cc @chrismeyersfsu

[AlanCoding]

solved using /usr/libexec/platform-python but needs docs for some of the things said here