AttributeError: 'module' object has no attribute 'JSONDecodeError'

[tomdeblende]

Summary

When using amazon.aws.ec2_metadata_facts on CentOS 7, Amazon Linux 1 and Amazon Linux 2, I get this error:

An exception occurred during task execution. To see the full traceback, use -vvv. The error was: AttributeError: 'module' object has no attribute 'JSONDecodeError'

On more recent OS'es it works fine.

Issue Type

Bug Report

Component Name

amazon.aws.ec2_metadata_facts

Ansible Version

$ ansible --version
ansible [core 2.14.2]
  config file = /etc/ansible/ansible.cfg
  configured module search path = ['/home/ansible/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
  ansible python module location = /usr/local/lib/python3.9/site-packages/ansible
  ansible collection location = /home/ansible/.ansible/collections:/usr/share/ansible/collections
  executable location = /usr/local/bin/ansible
  python version = 3.9.16 (main, Dec  8 2022, 00:00:00) [GCC 11.3.1 20220421 (Red Hat 11.3.1-2)] (/usr/bin/python3)
  jinja version = 3.1.2
  libyaml = True

Collection Versions

$ ansible-galaxy collection list
Collection                    Version
----------------------------- -------
amazon.aws                    5.2.0
ansible.netcommon             4.1.0
ansible.posix                 1.5.1
ansible.utils                 2.9.0
ansible.windows               1.13.0
arista.eos                    6.0.0
awx.awx                       21.11.0
azure.azcollection            1.14.0
check_point.mgmt              4.0.0
chocolatey.chocolatey         1.4.0
cisco.aci                     2.3.0
cisco.asa                     4.0.0
cisco.dnac                    6.6.3
cisco.intersight              1.0.23
cisco.ios                     4.3.1
cisco.iosxr                   4.1.0
cisco.ise                     2.5.12
cisco.meraki                  2.15.0
cisco.mso                     2.2.1
cisco.nso                     1.0.3
cisco.nxos                    4.0.1
cisco.ucs                     1.8.0
cloud.common                  2.1.2
cloudscale_ch.cloud           2.2.4
community.aws                 5.2.0
community.azure               2.0.0
community.ciscosmb            1.0.5
community.crypto              2.10.0
community.digitalocean        1.23.0
community.dns                 2.5.0
community.docker              3.4.0
community.fortios             1.0.0
community.general             6.3.0
community.google              1.0.0
community.grafana             1.5.3
community.hashi_vault         4.1.0
community.hrobot              1.7.0
community.libvirt             1.2.0
community.mongodb             1.4.2
community.mysql               3.5.1
community.network             5.0.0
community.okd                 2.2.0
community.postgresql          2.3.2
community.proxysql            1.5.1
community.rabbitmq            1.2.3
community.routeros            2.7.0
community.sap                 1.0.0
community.sap_libs            1.4.0
community.skydive             1.0.0
community.sops                1.6.0
community.vmware              3.3.0
community.windows             1.12.0
community.zabbix              1.9.1
containers.podman             1.10.1
cyberark.conjur               1.2.0
cyberark.pas                  1.0.17
dellemc.enterprise_sonic      2.0.0
dellemc.openmanage            6.3.0
dellemc.os10                  1.1.1
dellemc.os6                   1.0.7
dellemc.os9                   1.0.4
dellemc.powerflex             1.5.0
dellemc.unity                 1.5.0
f5networks.f5_modules         1.22.0
fortinet.fortimanager         2.1.7
fortinet.fortios              2.2.2
frr.frr                       2.0.0
gluster.gluster               1.0.2
google.cloud                  1.1.2
grafana.grafana               1.1.0
hetzner.hcloud                1.9.1
hpe.nimble                    1.1.4
ibm.qradar                    2.1.0
ibm.spectrum_virtualize       1.11.0
infinidat.infinibox           1.3.12
infoblox.nios_modules         1.4.1
inspur.ispim                  1.2.0
inspur.sm                     2.3.0
junipernetworks.junos         4.1.0
kubernetes.core               2.3.2
lowlydba.sqlserver            1.3.1
mellanox.onyx                 1.0.0
netapp.aws                    21.7.0
netapp.azure                  21.10.0
netapp.cloudmanager           21.22.0
netapp.elementsw              21.7.0
netapp.ontap                  22.2.0
netapp.storagegrid            21.11.1
netapp.um_info                21.8.0
netapp_eseries.santricity     1.4.0
netbox.netbox                 3.10.0
ngine_io.cloudstack           2.3.0
ngine_io.exoscale             1.0.0
ngine_io.vultr                1.1.3
openstack.cloud               1.10.0
openvswitch.openvswitch       2.1.0
ovirt.ovirt                   2.4.1
purestorage.flasharray        1.16.2
purestorage.flashblade        1.10.0
purestorage.fusion            1.3.0
sensu.sensu_go                1.13.2
splunk.es                     2.1.0
t_systems_mms.icinga_director 1.32.0
theforeman.foreman            3.8.0
vmware.vmware_rest            2.2.0
vultr.cloud                   1.7.0
vyos.vyos                     4.0.0
wti.remote                    1.0.4

AWS SDK versions

$ pip show boto boto3 botocore
WARNING: Package(s) not found: boto
Name: boto3
Version: 1.26.67
Summary: The AWS SDK for Python
Home-page: https://github.com/boto/boto3
Author: Amazon Web Services
Author-email:
License: Apache License 2.0
Location: /usr/local/lib/python3.9/site-packages
Requires: botocore, jmespath, s3transfer
Required-by:
---
Name: botocore
Version: 1.29.67
Summary: Low-level, data-driven core of boto 3.
Home-page: https://github.com/boto/botocore
Author: Amazon Web Services
Author-email:
License: Apache License 2.0
Location: /usr/local/lib/python3.9/site-packages
Requires: jmespath, python-dateutil, urllib3
Required-by: boto3, s3transfer

Configuration

$ ansible-config dump --only-changed
CONFIG_FILE() = /etc/ansible/ansible.cfg
DEFAULT_BECOME(/etc/ansible/ansible.cfg) = True
DEFAULT_BECOME_ASK_PASS(/etc/ansible/ansible.cfg) = False
DEFAULT_BECOME_METHOD(/etc/ansible/ansible.cfg) = sudo
DEFAULT_BECOME_USER(/etc/ansible/ansible.cfg) = root
DEFAULT_FORKS(/etc/ansible/ansible.cfg) = 50
DEFAULT_HOST_LIST(/etc/ansible/ansible.cfg) = ['/etc/ansible/aws_ec2.yaml']
DEFAULT_REMOTE_USER(/etc/ansible/ansible.cfg) = ansible
HOST_KEY_CHECKING(/etc/ansible/ansible.cfg) = False
INVENTORY_ENABLED(/etc/ansible/ansible.cfg) = ['aws_ec2']

OS / Environment

Amazon Linux 2022

Steps to Reproduce

- amazon.aws.ec2_metadata_facts:

Expected Results

Being able to use the variables set by amazon.aws.ec2_metadata_facts.

Actual Results

TASK [common : amazon.aws.ec2_metadata_facts] *************************************************************************************************************************************************************************
An exception occurred during task execution. To see the full traceback, use -vvv. The error was: AttributeError: 'module' object has no attribute 'JSONDecodeError'
fatal: [ip-192-168-0-28.eu-west-1.compute.internal]: FAILED! => {"changed": false, "module_stderr": "Shared connection to ip-192-168-0-28.eu-west-1.compute.internal closed.\r\n", "module_stdout": "Traceback (most recent call last):\r\n  File \"/home/ansible/.ansible/tmp/ansible-tmp-1676026317.2103488-26967-221943897239799/AnsiballZ_ec2_metadata_facts.py\", line 107, in <module>\r\n    _ansiballz_main()\r\n  File \"/home/ansible/.ansible/tmp/ansible-tmp-1676026317.2103488-26967-221943897239799/AnsiballZ_ec2_metadata_facts.py\", line 99, in _ansiballz_main\r\n    invoke_module(zipped_mod, temp_path, ANSIBALLZ_PARAMS)\r\n  File \"/home/ansible/.ansible/tmp/ansible-tmp-1676026317.2103488-26967-221943897239799/AnsiballZ_ec2_metadata_facts.py\", line 48, in invoke_module\r\n    run_name='__main__', alter_sys=True)\r\n  File \"/usr/lib64/python2.7/runpy.py\", line 188, in run_module\r\n    fname, loader, pkg_name)\r\n  File \"/usr/lib64/python2.7/runpy.py\", line 82, in _run_module_code\r\n    mod_name, mod_fname, mod_loader, pkg_name)\r\n  File \"/usr/lib64/python2.7/runpy.py\", line 72, in _run_code\r\n    exec code in run_globals\r\n  File \"/tmp/ansible_amazon.aws.ec2_metadata_facts_payload_i3wvN6/ansible_amazon.aws.ec2_metadata_facts_payload.zip/ansible_collections/amazon/aws/plugins/modules/ec2_metadata_facts.py\", line 598, in <module>\r\n  File \"/tmp/ansible_amazon.aws.ec2_metadata_facts_payload_i3wvN6/ansible_amazon.aws.ec2_metadata_facts_payload.zip/ansible_collections/amazon/aws/plugins/modules/ec2_metadata_facts.py\", line 591, in main\r\n  File \"/tmp/ansible_amazon.aws.ec2_metadata_facts_payload_i3wvN6/ansible_amazon.aws.ec2_metadata_facts_payload.zip/ansible_collections/amazon/aws/plugins/modules/ec2_metadata_facts.py\", line 568, in run\r\n  File \"/tmp/ansible_amazon.aws.ec2_metadata_facts_payload_i3wvN6/ansible_amazon.aws.ec2_metadata_facts_payload.zip/ansible_collections/amazon/aws/plugins/modules/ec2_metadata_facts.py\", line 531, in fetch\r\nAttributeError: 'module' object has no attribute 'JSONDecodeError'\r\n", "msg": "MODULE FAILURE\nSee stdout/stderr for the exact error", "rc": 1}

Code of Conduct

• I agree to follow the Ansible Code of Conduct

[ansibullbot]

Files identified in the description:
None

If these files are inaccurate, please update the component name section of the description or use the !component bot command.

click here for bot help

[tremble]

Many thanks for taking the time to report this issue.

The issue here is that between Python 3 and Python 2 the Exception raised by the json module changed. With the Python 3 exception not being available on older Python 2 hosts.

While the bulk of this collection has dropped support for Python <3.6, I believe it's appropriate for this specific module to continue to support Python 2.7, since it doesn't depend on the botocore SDK (which drives our Python support policy).

I've added a fix in #1359, and included a specific integration test for running this against Python 2.x hosts.

[tomdeblende]

I have seen development activity for this issue and waited for the new 7.3.0 Ansible version. It does not seem to be resolved in this version? Is that correct, or am I missing something?

[tremble]

@tomdeblende it looks like the fix didn't get backported to the 5.x release of this collection (I missed that the automatic backport failed). Collections can be updated separately from Ansible. This fix will be available with Release 6.0.0 of the collection, and hopefully 5.4.0 if a manual backport is successful

[tomdeblende]

@tremble where can I check when 5.4.0 will be released? Looking at https://github.com/ansible-collections/amazon.aws/releases I see 5.3.0 is not listed while it is already available.

[tremble]

@tomdeblende There's no official date yet. The short answer is "Real Soon Now".

Unfortunately none of the usual suspects were able to meet the monthly working group meeting yesterday which is when we'd normally discuss the release. I'll poke folks and see if we can get the process started.