Fix S3 unavailable region error

[dougluce]

ISSUE TYPE

• Bugfix Pull Request

COMPONENT NAME

cloud/amazon/s3.py

ANSIBLE VERSION

ansible 2.1.0
  config file = /Users/aluce/.ansible.cfg
  configured module search path = Default w/o overrides

SUMMARY

This addresses the error:

  fatal: [site]: FAILED! => {"changed": false, "failed": true, "msg": "Failed to connect to S3: Region  does not seem to be available for awsmodule boto.s3. If the region definitely exists, you may need to upgrade boto or extend with endpoints_path"}

Commit 0dd58e9 changed the logic so an exception is thrown (by connect_to_aws) before the s3 is None check is performed. This change changese the None check to a catch so the old logic can compensate.

[willthames]

What does this mean? Can we fix this in connect_to_aws?

[gregdek]

Thanks @dougluce for this PR. This module is maintained by the Ansible core team, so it can take a while for patches to be reviewed. Thanks for your patience.

[This message brought to you by your friendly Ansibull-bot.]

[bradmering]

I just upgraded to Ansible 2.1 and started getting this error as well. I can confirm that @dougluce patch solves the problem (or at least prevents premature failure). The if s3 is None: never is processed because the connect_to_aws throws an exception and exits out to the failure. Catching and attempting the alternate method in the catch seems to solve the problem. My ansible script, which ran fine under 2.0, runs right now under 2.1

[willthames]

Ah, it seems that s3 was the only module in the change referenced by @dougluce using connect_to_aws that wasn't already wrapping its connection in try/except.

From a coding point of view, this gets my 👍 - with @bradmering's testing, we should be good to go.

[filipenf]

I've cherry-picked this commit and tested locally and can confirm it fixes the bug!

[willthames]

Looks like this needs back porting to 1.9.x too (see #2421)

[criloz]

fixed the bug. thanks

[willthames]

@gregdek shipit

[gregdek]

Thanks @dougluce for this PR. Unfortunately, it is not mergeable in its current state due to merge conflicts. Please rebase your PR. When you are done, please comment with text 'ready_for_review' and we will put this PR back into review.

[This message brought to you by your friendly Ansibull-bot.]

[dougluce]

ready_for_review

[gregdek]

Thanks @dougluce for this PR. This module is maintained by the Ansible core team, so it can take a while for patches to be reviewed. Thanks for your patience.

Core team: please review according to guidelines (http://docs.ansible.com/ansible/developing_modules.html#module-checklist) and comment with 'needs_revision' or merge as appropriate.

[This message brought to you by your friendly Ansibull-bot.]

[dbarrosop]

What's the status of this? Ansible 2.1, released yesterday, contains this bug although it was reported a month and a half ago.

[jamescarr]

Indeed. This defect practically rendered the s3 module broken outright. All of our playbooks that use s3 are 100% broken.

[allenluce]

I've been using a line like this in my requirements.txt to keep things going until the fix is released:

-e git+https://github.com/dougluce/ansible@fix-s3-region-error#egg=ansible

It's fine as a stopgap but it's important to revise as soon as the proper Ansible release is available.

[willthames]

shipit

[gregdek]

Thanks again to @dougluce for this PR, and thanks @ansible for reviewing. Marking for inclusion.

[This message brought to you by your friendly Ansibull-bot.]

[mrvisser]

Is there a workaround for this in the meantime before it's released?

edit: I see Allen posted something. I'm installing from the apt repo, not sure if that excludes me from this workaround.

[jamescarr]

@mrvisser I just took the s3.py file as it is at this commit and dropped it in our library dir with a comment to remove once this fix is out. :-)

[mrvisser]

Thanks @jamescarr . Yesterday, I switched to the pip build which, while it was painful to get working, will allow us to be a bit more nimble deploying onto specific versions in the future. For anyone who may do the same while being blocked by this, here's a packer provisioner that was required from the base ec2 ubuntu ami to get ansible installed at a particular version (v2.0.2.0-1 in this case):

"provisioners": [
    {
      "type": "shell",
      "inline": [
        "sudo apt-get update",
        "sudo apt-get install software-properties-common -y",

        "sudo apt-get install build-essential python-dev python-setuptools git libffi-dev libssl-dev -y",
        "sudo easy_install pip",
        "sudo -H pip install git+git://github.com/ansible/ansible.git@v2.0.2.0-1",
        "sudo -H pip install -U distribute",

        "sudo apt-get install python-boto -y"
      ]
    }
]

[jamescarr]

@mrvisser thanks a ton for sharing that. Friday morning I too witnessed some pain at switching to a pip based install and simply gave up since I had a few other in-flight pull requests to wrap up.