GCP Inventory sync failed #3180

movergan · 2019-02-07T11:30:41Z

ISSUE TYPE

Bug Report

COMPONENT NAME

Python 3 in latest awx_task:3.0.0 docker image

SUMMARY

GCE Inventory sync fails

ENVIRONMENT

AWX version: 3.0.0
AWX install method: docker standalone
Operating System: Debian 8
Web Browser: N/A

STEPS TO REPRODUCE

Try to sync GCE inventory

EXPECTED RESULTS

Synced fine

ACTUAL RESULTS

Fails with:

RuntimeError: ansible-inventory failed (rc=1) with stdout:
stderr:
 [WARNING]:  * Failed to parse /var/lib/awx/venv/awx/lib64/python3.6/site-
packages/awx/plugins/inventory/gce.py with script plugin: Inventory script
(/var/lib/awx/venv/awx/lib64/python3.6/site-
packages/awx/plugins/inventory/gce.py) had an execution error: Traceback (most
recent call last):   File "/var/lib/awx/venv/awx/lib64/python3.6/site-
packages/awx/plugins/inventory/gce.py", line 508, in <module>
GceInventory()   File "/var/lib/awx/venv/awx/lib64/python3.6/site-
packages/awx/plugins/inventory/gce.py", line 170, in __init__     self.driver =
self.get_gce_driver()   File "/var/lib/awx/venv/awx/lib64/python3.6/site-
packages/awx/plugins/inventory/gce.py", line 318, in get_gce_driver     gce =
get_driver(Provider.GCE)(*args, **kwargs)   File
"/var/lib/awx/venv/ansible/lib/python2.7/site-
packages/libcloud/compute/drivers/gce.py", line 1795, in __init__
super(GCENodeDriver, self).__init__(user_id, key, **kwargs)   File
"/var/lib/awx/venv/ansible/lib/python2.7/site-
packages/libcloud/common/base.py", line 975, in __init__     self.connection =
self.connectionCls(*args, **conn_kwargs)   File
"/var/lib/awx/venv/ansible/lib/python2.7/site-
packages/libcloud/compute/drivers/gce.py", line 99, in __init__
credential_file=credential_file, **kwargs)   File
"/var/lib/awx/venv/ansible/lib/python2.7/site-
packages/libcloud/common/google.py", line 767, in __init__     user_id, key,
auth_type, credential_file, scopes, **kwargs)   File
"/var/lib/awx/venv/ansible/lib/python2.7/site-
packages/libcloud/common/google.py", line 653, in __init__     self.user_id,
self.key, self.scopes, **kwargs)   File
"/var/lib/awx/venv/ansible/lib/python2.7/site-
packages/libcloud/common/google.py", line 486, in __init__     raise
GoogleAuthError('PyCrypto library required for '
libcloud.common.google.GoogleAuthError: 'PyCrypto library required for Service
Account Authentication.'
 [WARNING]: Unable to parse /var/lib/awx/venv/awx/lib64/python3.6/site-
packages/awx/plugins/inventory/gce.py as an inventory source
ERROR! No inventory was parsed, please check your configuration and options.

ADDITIONAL INFORMATION

The text was updated successfully, but these errors were encountered:

thenetengineer · 2019-02-12T18:35:30Z

I had a similar issue with a script pulling inventory from Netbox, and it appears that AWX is using a PYTHONPATH that only includes the Python 2.7 site-packages, and not the new Python 3.6 site-packages.

I solved it by changing the PYTHONPATH to also include the new Python 3.6.

skinlayers · 2019-02-21T09:16:40Z

I'm running into the same thing with GKE:

2019-02-21 08:52:29,422 INFO     awx.main.commands.inventory_import Reading Ansible inventory source: /var/lib/awx/venv/awx/lib64/python3.6/site-packages/awx/plugins/inventory/gce.py
2019-02-21 08:52:29,423 INFO     awx.main.commands.inventory_import Using VIRTUAL_ENV: /var/lib/awx/venv/ansible
2019-02-21 08:52:29,424 INFO     awx.main.commands.inventory_import Using PATH: /var/lib/awx/venv/ansible/bin:/var/lib/awx/venv/awx/bin:/var/lib/awx/venv/awx/bin:/var/lib/awx/venv/awx/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
2019-02-21 08:52:29,424 INFO     awx.main.commands.inventory_import Using PYTHONPATH: /var/lib/awx/venv/ansible/lib/python2.7/site-packages:
Traceback (most recent call last):
  File "/var/lib/awx/venv/awx/bin/awx-manage", line 11, in <module>
    load_entry_point('awx==3.0.1.0', 'console_scripts', 'awx-manage')()
  File "/var/lib/awx/venv/awx/lib64/python3.6/site-packages/awx/__init__.py", line 140, in manage
    execute_from_command_line(sys.argv)
  File "/var/lib/awx/venv/awx/lib64/python3.6/site-packages/django/core/management/__init__.py", line 364, in execute_from_command_line
    utility.execute()
  File "/var/lib/awx/venv/awx/lib64/python3.6/site-packages/django/core/management/__init__.py", line 356, in execute
    self.fetch_command(subcommand).run_from_argv(self.argv)
  File "/var/lib/awx/venv/awx/lib64/python3.6/site-packages/django/core/management/base.py", line 283, in run_from_argv
    self.execute(*args, **cmd_options)
  File "/var/lib/awx/venv/awx/lib64/python3.6/site-packages/django/core/management/base.py", line 330, in execute
    output = self.handle(*args, **options)
  File "/var/lib/awx/venv/awx/lib64/python3.6/site-packages/awx/main/management/commands/inventory_import.py", line 1049, in handle
    raise exc
  File "/var/lib/awx/venv/awx/lib64/python3.6/site-packages/awx/main/management/commands/inventory_import.py", line 949, in handle
    data = AnsibleInventoryLoader(source=source, is_custom=self.is_custom, venv_path=venv_path).load()
  File "/var/lib/awx/venv/awx/lib64/python3.6/site-packages/awx/main/management/commands/inventory_import.py", line 195, in load
    return self.command_to_json(base_args + ['--list'])
  File "/var/lib/awx/venv/awx/lib64/python3.6/site-packages/awx/main/management/commands/inventory_import.py", line 178, in command_to_json
    self.method, proc.returncode, stdout, stderr))
RuntimeError: ansible-inventory failed (rc=1) with stdout:
stderr:
 [WARNING]:  * Failed to parse /var/lib/awx/venv/awx/lib64/python3.6/site-
packages/awx/plugins/inventory/gce.py with script plugin: Inventory script
(/var/lib/awx/venv/awx/lib64/python3.6/site-
packages/awx/plugins/inventory/gce.py) had an execution error: Traceback (most
recent call last):   File "/var/lib/awx/venv/awx/lib64/python3.6/site-
packages/awx/plugins/inventory/gce.py", line 508, in <module>
GceInventory()   File "/var/lib/awx/venv/awx/lib64/python3.6/site-
packages/awx/plugins/inventory/gce.py", line 170, in __init__     self.driver =
self.get_gce_driver()   File "/var/lib/awx/venv/awx/lib64/python3.6/site-
packages/awx/plugins/inventory/gce.py", line 318, in get_gce_driver     gce =
get_driver(Provider.GCE)(*args, **kwargs)   File
"/var/lib/awx/venv/ansible/lib/python2.7/site-
packages/libcloud/compute/drivers/gce.py", line 1795, in __init__
super(GCENodeDriver, self).__init__(user_id, key, **kwargs)   File
"/var/lib/awx/venv/ansible/lib/python2.7/site-
packages/libcloud/common/base.py", line 975, in __init__     self.connection =
self.connectionCls(*args, **conn_kwargs)   File
"/var/lib/awx/venv/ansible/lib/python2.7/site-
packages/libcloud/compute/drivers/gce.py", line 99, in __init__
credential_file=credential_file, **kwargs)   File
"/var/lib/awx/venv/ansible/lib/python2.7/site-
packages/libcloud/common/google.py", line 767, in __init__     user_id, key,
auth_type, credential_file, scopes, **kwargs)   File
"/var/lib/awx/venv/ansible/lib/python2.7/site-
packages/libcloud/common/google.py", line 653, in __init__     self.user_id,
self.key, self.scopes, **kwargs)   File
"/var/lib/awx/venv/ansible/lib/python2.7/site-
packages/libcloud/common/google.py", line 486, in __init__     raise
GoogleAuthError('PyCrypto library required for '
libcloud.common.google.GoogleAuthError: 'PyCrypto library required for Service
Account Authentication.'
 [WARNING]: Unable to parse /var/lib/awx/venv/awx/lib64/python3.6/site-
packages/awx/plugins/inventory/gce.py as an inventory source
ERROR! No inventory was parsed, please check your configuration and options.

skinlayers · 2019-02-25T05:14:44Z

I was able to fix it in Kubernetes/GKE by adding pycryptodome==3.7.3 to requirements/requirements_ansible.in and requirements/requirements_ansible.txt. I attempted to follow the instructions for updating requirements in requirements/README.md, but other packages broke in the process.

kdelee · 2019-02-26T16:14:52Z

@rooftopcellist this may be of interest to you, when I saw PyCrypto involved

ryanpetrello · 2019-03-18T13:24:34Z

related: apache/libcloud#1280

ryanpetrello · 2019-03-19T20:12:21Z

FYI libcloud now no longer uses PyCrypto (apache/libcloud#1280); once a new release lands on PyPI, we'll update AWX's dependencies.

awxbot added state:needs_triage type:bug labels Feb 7, 2019

mabashian added the component:api label Feb 11, 2019

skinlayers mentioned this issue Feb 25, 2019

Add PyCryptodome to requirements_ansible.*, related #3180 #3305

Closed

AlanCoding added state:needs_devel and removed state:needs_triage labels Feb 26, 2019

kdelee added the flag:🎱 label Feb 27, 2019

ryanpetrello mentioned this issue Mar 21, 2019

pin apache-libcloud to a version that doesn't use PyCrypto #3466

Merged

ryanpetrello added state:needs_test and removed state:in_progress labels Mar 22, 2019

ryanpetrello closed this as completed Mar 22, 2019

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

GCP Inventory sync failed #3180

GCP Inventory sync failed #3180

movergan commented Feb 7, 2019

thenetengineer commented Feb 12, 2019

skinlayers commented Feb 21, 2019 •

edited

Loading

skinlayers commented Feb 25, 2019 •

edited

Loading

kdelee commented Feb 26, 2019 •

edited

Loading

ryanpetrello commented Mar 18, 2019

ryanpetrello commented Mar 19, 2019

GCP Inventory sync failed #3180

GCP Inventory sync failed #3180

Comments

movergan commented Feb 7, 2019

ISSUE TYPE

COMPONENT NAME

SUMMARY

ENVIRONMENT

STEPS TO REPRODUCE

EXPECTED RESULTS

ACTUAL RESULTS

ADDITIONAL INFORMATION

thenetengineer commented Feb 12, 2019

skinlayers commented Feb 21, 2019 • edited Loading

skinlayers commented Feb 25, 2019 • edited Loading

kdelee commented Feb 26, 2019 • edited Loading

ryanpetrello commented Mar 18, 2019

ryanpetrello commented Mar 19, 2019

skinlayers commented Feb 21, 2019 •

edited

Loading

skinlayers commented Feb 25, 2019 •

edited

Loading

kdelee commented Feb 26, 2019 •

edited

Loading