s3_bucket | DigitalOcean compatibility (KeyError: Rules)

[eeshugerman]

Summary

When I try to create a DigitalOcean Space with s3_bucket from v2 of this collection, the task fails.

Issue Type

Bug Report

Component Name

s3_bucket

Ansible Version

$ ansible --version
ansible [core 2.11.5] 
  config file = /home/elliott/devel/PikaNet/ansible.cfg
  configured module search path = ['/home/elliott/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
  ansible python module location = /home/elliott/.virtualenvs/ansible-4.5/lib/python3.9/site-packages/ansible
  ansible collection location = /home/elliott/.ansible/collections:/usr/share/ansible/collections
  executable location = /home/elliott/.virtualenvs/ansible-4.5/bin/ansible
  python version = 3.9.7 (default, Aug 31 2021, 13:28:12) [GCC 11.1.0]
  jinja version = 3.0.1
  libyaml = True
 

Collection Versions

$ ansible-galaxy collection list
# /home/elliott/.ansible/collections/ansible_collections
Collection Version
---------- -------
amazon.aws 2.0.0  

# /home/elliott/.virtualenvs/ansible-4.5/lib/python3.9/site-packages/ansible_collections
Collection                    Version
----------------------------- -------
amazon.aws                    1.5.0  
ansible.netcommon             2.4.0  
ansible.posix                 1.3.0  
ansible.utils                 2.4.0  
ansible.windows               1.7.2  
arista.eos                    2.2.0  
awx.awx                       19.2.2 
azure.azcollection            1.9.0  
check_point.mgmt              2.0.0  
chocolatey.chocolatey         1.1.0  
cisco.aci                     2.0.0  
cisco.asa                     2.0.3  
cisco.intersight              1.0.17 
cisco.ios                     2.4.0  
cisco.iosxr                   2.4.0  
cisco.meraki                  2.4.2  
cisco.mso                     1.2.0  
cisco.nso                     1.0.3  
cisco.nxos                    2.5.1  
cisco.ucs                     1.6.0  
cloudscale_ch.cloud           2.2.0  
community.aws                 1.5.0  
community.azure               1.0.0  
community.crypto              1.9.2  
community.digitalocean        1.9.0  
community.docker              1.9.1  
community.fortios             1.0.0  
community.general             3.6.0  
community.google              1.0.0  
community.grafana             1.2.1  
community.hashi_vault         1.3.2  
community.hrobot              1.1.1  
community.kubernetes          1.2.1  
community.kubevirt            1.0.0  
community.libvirt             1.0.2  
community.mongodb             1.3.1  
community.mysql               2.1.1  
community.network             3.0.0  
community.okd                 1.1.2  
community.postgresql          1.4.0  
community.proxysql            1.2.0  
community.rabbitmq            1.1.0  
community.routeros            1.2.0  
community.skydive             1.0.0  
community.sops                1.1.0  
community.vmware              1.13.0 
community.windows             1.6.0  
community.zabbix              1.4.0  
containers.podman             1.7.0  
cyberark.conjur               1.1.0  
cyberark.pas                  1.0.7  
dellemc.enterprise_sonic      1.1.0  
dellemc.openmanage            3.6.0  
dellemc.os10                  1.1.1  
dellemc.os6                   1.0.7  
dellemc.os9                   1.0.4  
f5networks.f5_modules         1.11.0 
fortinet.fortimanager         2.1.3  
fortinet.fortios              2.1.2  
frr.frr                       1.0.3  
gluster.gluster               1.0.1  
google.cloud                  1.0.2  
hetzner.hcloud                1.6.0  
hpe.nimble                    1.1.3  
ibm.qradar                    1.0.3  
infinidat.infinibox           1.2.4  
inspur.sm                     1.3.0  
junipernetworks.junos         2.5.0  
kubernetes.core               1.2.1  
mellanox.onyx                 1.0.0  
netapp.aws                    21.6.0 
netapp.azure                  21.8.1 
netapp.cloudmanager           21.9.0 
netapp.elementsw              21.6.1 
netapp.ontap                  21.10.0
netapp.um_info                21.7.0 
netapp_eseries.santricity     1.2.13 
netbox.netbox                 3.1.1  
ngine_io.cloudstack           2.1.0  
ngine_io.exoscale             1.0.0  
ngine_io.vultr                1.1.0  
openstack.cloud               1.5.0  
openvswitch.openvswitch       2.0.0  
ovirt.ovirt                   1.6.2  
purestorage.flasharray        1.10.0 
purestorage.flashblade        1.6.0  
sensu.sensu_go                1.12.0 
servicenow.servicenow         1.0.6  
splunk.es                     1.0.2  
t_systems_mms.icinga_director 1.21.1 
theforeman.foreman            2.2.0  
vyos.vyos                     2.5.1  
wti.remote                    1.0.1  

AWS SDK versions

This is on the remote host right? I don't have any boto stuff installed on the controller.

$ pip show boto boto3 botocore
WARNING: Package(s) not found: boto
Name: boto3
Version: 1.13.14
Summary: The AWS SDK for Python
Home-page: https://github.com/boto/boto3
Author: Amazon Web Services
Author-email: UNKNOWN
License: Apache License 2.0
Location: /usr/lib/python3/dist-packages
Requires: 
Required-by: 
---
Name: botocore
Version: 1.20.0
Summary: Low-level, data-driven core of boto 3.
Home-page: https://github.com/boto/botocore
Author: Amazon Web Services
Author-email: None
License: Apache License 2.0
Location: /usr/lib/python3/dist-packages
Requires: 
Required-by: 

Configuration

$ cat ansible.cfg
[defaults]
interpreter_python = auto
stdout_callback = yaml
vault_password_file = .vault_pass
force_color = true

OS / Environment

Debian 11

Steps to Reproduce

- name: Create a DigitialOcean space (bucket)
  amazon.aws.s3_bucket:
    name: '{{ digitalocean_spaces.backups_bucket }}'
    state: present
    aws_access_key: '{{ digitalocean_spaces.key }}'
    aws_secret_key: '{{ digitalocean_spaces.secret }}'
    s3_url: 'https://{{ digitalocean_spaces.endpoint }}'

Expected Results

The task succeeds, space/bucket is created

Actual Results

An exception occurred during task execution. To see the full traceback, use -vvv. The error was: KeyError: 'Rules'
fatal: [default]: FAILED! => changed=false 
  module_stderr: |-
    Shared connection to 127.0.0.1 closed.
  module_stdout: |-
    Traceback (most recent call last):
      File "/home/vagrant/.ansible/tmp/ansible-tmp-1631983227.5101855-200375-103025631810603/AnsiballZ_s3_bucket.py", line 100, in <module>
        _ansiballz_main()
      File "/home/vagrant/.ansible/tmp/ansible-tmp-1631983227.5101855-200375-103025631810603/AnsiballZ_s3_bucket.py", line 92, in _ansiballz_main
        invoke_module(zipped_mod, temp_path, ANSIBALLZ_PARAMS)
      File "/home/vagrant/.ansible/tmp/ansible-tmp-1631983227.5101855-200375-103025631810603/AnsiballZ_s3_bucket.py", line 40, in invoke_module
        runpy.run_module(mod_name='ansible_collections.amazon.aws.plugins.modules.s3_bucket', init_globals=dict(_module_fqn='ansible_collections.amazon.aws.plugins.modules.s3_bucket', _modlib_path=modlib_path),
      File "/usr/lib/python3.9/runpy.py", line 210, in run_module
        return _run_module_code(code, init_globals, run_name, mod_spec)
      File "/usr/lib/python3.9/runpy.py", line 97, in _run_module_code
        _run_code(code, mod_globals, init_globals,
      File "/usr/lib/python3.9/runpy.py", line 87, in _run_code
        exec(code, run_globals)
      File "/tmp/ansible_amazon.aws.s3_bucket_payload__iooj36v/ansible_amazon.aws.s3_bucket_payload.zip/ansible_collections/amazon/aws/plugins/modules/s3_bucket.py", line 977, in <module>
      File "/tmp/ansible_amazon.aws.s3_bucket_payload__iooj36v/ansible_amazon.aws.s3_bucket_payload.zip/ansible_collections/amazon/aws/plugins/modules/s3_bucket.py", line 971, in main
      File "/tmp/ansible_amazon.aws.s3_bucket_payload__iooj36v/ansible_amazon.aws.s3_bucket_payload.zip/ansible_collections/amazon/aws/plugins/modules/s3_bucket.py", line 491, in create_or_update_bucket
      File "/tmp/ansible_amazon.aws.s3_bucket_payload__iooj36v/ansible_amazon.aws.s3_bucket_payload.zip/ansible_collections/amazon/aws/plugins/modules/s3_bucket.py", line 786, in get_bucket_ownership_cntrl
    KeyError: 'Rules'
  msg: |-
    MODULE FAILURE
    See stdout/stderr for the exact error
  rc: 1

Code of Conduct

• I agree to follow the Ansible Code of Conduct

[ansibullbot]

Files identified in the description:

• [plugins/modules/s3_bucket.py](https://github.com/['ansible-collections/amazon.aws', 'ansible-collections/community.aws', 'ansible-collections/community.vmware']/blob/main/plugins/modules/s3_bucket.py)

If these files are inaccurate, please update the component name section of the description or use the !component bot command.

click here for bot help

[ansibullbot]

cc @abikouo @jillr @s-hertel @tremble @wimnat
click here for bot help

[goneri]

Hi @eeshugerman, we don't have access to a digital ocean test environment. @slapula Is see you're the last person who've worked on the digital ocean support for this module. Do you have an idea what's going on?

[tremble]

@eeshugerman We don't have any way to properly debug this issue.

I've pushed #530 which might fix the issue. It would be helpful if you could confirm this.

[eeshugerman]

Thanks! I'll give it a try this weekend.