Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

s3_bucket -- DigitialOcean Spaces compatibility #390

Closed
eeshugerman opened this issue Jun 20, 2021 · 6 comments · Fixed by #391
Closed

s3_bucket -- DigitialOcean Spaces compatibility #390

eeshugerman opened this issue Jun 20, 2021 · 6 comments · Fixed by #391
Labels
bug This issue/PR relates to a bug has_pr module module plugins plugin (any type) python3 traceback

Comments

@eeshugerman
Copy link

eeshugerman commented Jun 20, 2021

SUMMARY

I'm upgrading from Ansible v2.8 to v4.1. With v2.8, I was able to use s3_bucket with DO Spaces -- I just needed to set requester_pays: null. Now, with v4.1, it seems there's no way to prevent the module's get_bucket_encryption call, and DO Spaces doesn't implement encryption stuff, so the task fails. I tried all of the permitted values for encryption ('none', 'AES256' 'aws:kms'), as well as not setting it at all -- it made no difference.

See https://github.com/ansible-collections/amazon.aws/blob/main/plugins/modules/s3_bucket.py#L420-L426.
My naive assessment is that the get_bucket_encryption call (and its try/except block) would need to be moved into the if encryption is not None block.

ISSUE TYPE
  • Bug Report
COMPONENT NAME

s3_bucket

ANSIBLE VERSION
ansible [core 2.11.1] 
  config file = /home/elliott/devel/PikaNet/ansible.cfg
  configured module search path = ['/home/elliott/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
  ansible python module location = /home/elliott/.virtualenvs/ansible-4/lib/python3.9/site-packages/ansible
  ansible collection location = /home/elliott/.ansible/collections:/usr/share/ansible/collections
  executable location = /home/elliott/.virtualenvs/ansible-4/bin/ansible
  python version = 3.9.5 (default, May 24 2021, 12:50:35) [GCC 11.1.0]
  jinja version = 3.0.1
  libyaml = True
CONFIGURATION
ANSIBLE_FORCE_COLOR(/home/elliott/devel/PikaNet/ansible.cfg) = True
DEFAULT_STDOUT_CALLBACK(/home/elliott/devel/PikaNet/ansible.cfg) = yaml
DEFAULT_VAULT_PASSWORD_FILE(/home/elliott/devel/PikaNet/ansible.cfg) = /home/elliott/devel/PikaNet/.vault_pass
INTERPRETER_PYTHON(/home/elliott/devel/PikaNet/ansible.cfg) = auto                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            
OS / ENVIRONMENT

Ubuntu 18.04

STEPS TO REPRODUCE
amazon.aws.s3_bucket:
  name: 'my-bucket'
  state: present
  aws_access_key: '{{ digitalocean_spaces.key }}'
  aws_secret_key: '{{ digitalocean_spaces.secret }}'
  s3_url: 'https://sfo2.digitaloceanspaces.com'
EXPECTED RESULTS

The task succeeds.

ACTUAL RESULTS
The full traceback is:
Traceback (most recent call last):
  File "/tmp/ansible_amazon.aws.s3_bucket_payload_murbosko/ansible_amazon.aws.s3_bucket_payload.zip/ansible_collections/amazon/aws/plugins/modules/s3_bucket.py", line 388, in create_or_update_bucket
  File "/tmp/ansible_amazon.aws.s3_bucket_payload_murbosko/ansible_amazon.aws.s3_bucket_payload.zip/ansible_collections/amazon/aws/plugins/module_utils/cloud.py", line 154, in retry_func
    raise e
  File "/tmp/ansible_amazon.aws.s3_bucket_payload_murbosko/ansible_amazon.aws.s3_bucket_payload.zip/ansible_collections/amazon/aws/plugins/module_utils/cloud.py", line 144, in retry_func
    return f(*args, **kwargs)
  File "/tmp/ansible_amazon.aws.s3_bucket_payload_murbosko/ansible_amazon.aws.s3_bucket_payload.zip/ansible_collections/amazon/aws/plugins/modules/s3_bucket.py", line 522, in get_bucket_encryption
  File "/usr/lib/python3/dist-packages/botocore/client.py", line 316, in _api_call
    return self._make_api_call(operation_name, kwargs)
  File "/usr/lib/python3/dist-packages/botocore/client.py", line 635, in _make_api_call
    raise error_class(parsed_response, operation_name)
botocore.exceptions.ClientError: An error occurred (NotImplemented) when calling the GetBucketEncryption operation: Server does not support the functionality required to fulfill the request. Please see https://developers.digitalocean.com/documentation/spaces/#aws-s3-compatibility
fatal: [default]: FAILED! => changed=false 
  boto3_version: 1.4.2
  botocore_version: 1.16.19
  error:
    code: NotImplemented
    message: Server does not support the functionality required to fulfill the request. Please see https://developers.digitalocean.com/documentation/spaces/#aws-s3-compatibility
  invocation:
    module_args:
      aws_access_key: <REDACTED>
      aws_ca_bundle: null
      aws_config: null
      aws_secret_key: VALUE_SPECIFIED_IN_NO_LOG_PARAMETER
      ceph: false
      debug_botocore_endpoint_logs: false
      delete_public_access: false
      ec2_url: null
      encryption: null
      encryption_key_id: null
      force: false
      name: my-bucket
      policy: null
      profile: null
      public_access: null
      purge_tags: true
      region: null
      requester_pays: null
      s3_url: https://sfo2.digitaloceanspaces.com
      security_token: null
      state: present
      tags: null
      validate_certs: true
      versioning: null
  msg: 'Failed to get bucket encryption: An error occurred (NotImplemented) when calling the GetBucketEncryption operation: Server does not support the functionality required to fulfill the request. Please see https://developers.digitalocean.com/documentation/spaces/#aws-s3-compatibility'
  response_metadata:
    http_headers:
      content-length: '268'
      content-type: application/xml
      strict-transport-security: max-age=15552000; includeSubDomains; preload
      x-amz-error-code: NotImplemented
      x-amz-error-message: Server does not support the functionality required to fulfill the request. Please see https://developers.digitalocean.com/documentation/spaces/#aws-s3-compatibility
      x-do-spaces-error: unsupported_operation_s3:GetBucketEncryptionConfiguration
    http_status_code: 501
    retry_attempts: 0
@ansibullbot
Copy link

Files identified in the description:
None

If these files are inaccurate, please update the component name section of the description or use the !component bot command.

click here for bot help

@ansibullbot ansibullbot added bug This issue/PR relates to a bug needs_triage python3 traceback labels Jun 20, 2021
@eeshugerman
Copy link
Author

!component =plugins/modules/s3_bucket.py

@ansibullbot
Copy link

Files identified in the description:

If these files are inaccurate, please update the component name section of the description or use the !component bot command.

click here for bot help

@ansibullbot
Copy link

@ansibullbot ansibullbot added module module plugins plugin (any type) labels Jun 20, 2021
ansible-zuul bot pushed a commit that referenced this issue Jun 30, 2021
s3_bucket - handle "NotImplemented" for encryption

SUMMARY
fixes: #390
DigitalOcean doesn't support the encryption settings (and returns a NotImplemented error).
Additionally:

Slight logic tweak to avoid the line-too-long linting error
Avoid importing specific exceptions for consistency with other modules

ISSUE TYPE

Bugfix Pull Request

COMPONENT NAME
s3_bucket
ADDITIONAL INFORMATION

Reviewed-by: Alina Buzachis <None>
@tremble
Copy link
Contributor

tremble commented Jun 30, 2021

The fix for this issue should be available when we release version 2.0.0 of the amazon.aws collection.

@eeshugerman
Copy link
Author

thanks @tremble!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug This issue/PR relates to a bug has_pr module module plugins plugin (any type) python3 traceback
Projects
None yet
Development

Successfully merging a pull request may close this issue.

4 participants