SSM connection doesn't use regional S3 endpoint #1190

davidgiga1993 · 2022-06-01T07:55:38Z

Summary

When using ansible_connection: aws_ssm the url used to download data from a bucket is always *.s3.amazonaws.com regardless of the region specified in ansible_aws_ssm_region.

This causes issues since in restricted environments the EC2 instance has to use the S3 gateway endpoint which is only available using the region specific url, for example *.s3.eu-central-1.amazonaws.com

Issue Type

Bug Report

Component Name

s3

Ansible Version

$ ansible --version
ansible [core 2.12.6]
  config file = /etc/ansible/ansible.cfg
  configured module search path = ['/home/ssm-user/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
  ansible python module location = /usr/lib/python3/dist-packages/ansible
  ansible collection location = /home/ssm-user/.ansible/collections:/usr/share/ansible/collections
  executable location = /usr/bin/ansible
  python version = 3.10.4 (main, Apr  2 2022, 09:04:19) [GCC 11.2.0]
  jinja version = 3.0.3
  libyaml = True

Collection Versions

$ ansible-galaxy collection list

AWS SDK versions

$ pip show boto boto3 botocore
Name: boto3
Version: 1.24.0
Summary: The AWS SDK for Python
Home-page: https://github.com/boto/boto3
Author: Amazon Web Services
Author-email:
License: Apache License 2.0
Location: /usr/local/lib/python3.8/dist-packages
Requires: s3transfer, botocore, jmespath
Required-by:
---
Name: botocore
Version: 1.27.0
Summary: Low-level, data-driven core of boto 3.
Home-page: https://github.com/boto/botocore
Author: Amazon Web Services
Author-email:
License: Apache License 2.0
Location: /usr/local/lib/python3.8/dist-packages
Requires: python-dateutil, jmespath, urllib3
Required-by: s3transfer, boto3

Configuration

$ ansible-config dump --only-changed

OS / Environment

Ubuntu 22

Steps to Reproduce

ll:
  hosts:
    test:
      ansible_connection: aws_ssm
      ansible_aws_ssm_instance_id: "i-...."

  vars:
    ansible_aws_ssm_bucket_name: my-bucket
    ansible_aws_ssm_region: eu-central-1

Expected Results

Ansible should use the region specific s3 url for download

Actual Results

EXEC curl 'https://my-bucket.s3.amazonaws.com/...

Code of Conduct

I agree to follow the Ansible Code of Conduct

The text was updated successfully, but these errors were encountered:

ansibullbot · 2022-06-01T07:56:43Z

Files identified in the description:
None

If these files are inaccurate, please update the component name section of the description or use the !component bot command.

click here for bot help

…n var (#1176) SUMMARY Fix issue where syntax error is reported if using ssm connection and the target node is located in a different region to the s3 bucket. Fixes #1190, #637 ISSUE TYPE Bugfix Pull Request COMPONENT NAME aws_ssm connection plugin ADDITIONAL INFORMATION When using SSM for ansible connection and the target node is in a different region to the s3 bucket used, the playbook immediately errors with the following. (There are no issues when both target and s3 bucket are in the same region) fatal: [i-04444a7f03cc2bffd]: FAILED! => {"ansible_facts": {}, "changed": false, "failed_modules": {"ansible.legacy.setup": {"failed": true, "module_stderr": "", "module_stdout": " File \"/tmp/ansible/ansible-tmp-1653576081.8378458-29658-258097978113216/AnsiballZ_setup.py\", line 1\r\r\n <?xml version=\"1.0\" encoding=\"UTF-8\"?>\r\r\n ^\r\r\nSyntaxError: invalid syntax\r\r", "msg": "MODULE FAILURE\nSee stdout/stderr for the exact error", "rc": 1}}, "msg": "The following modules failed to execute: ansible.legacy.setup\n"} The tmp file has the following contents <?xml version="1.0" encoding="UTF-8"?> <Error><Code>AuthorizationQueryParametersError</Code><Message>Error parsing the X-Amz-Credential parameter; the region 'us-east-1' is wrong; expecting 'eu-west-1'</Message><Region>eu-west-1</Region><RequestId>4VTAGR4C1V9ATBJT</RequestId><HostId>OahjGsFQHlr3ihxobH/yyH7Mzxq98mwjcb6+J3Y2EifDU7FykCe8b6QJTNodIG5WSquVeJF+Zsk=</HostId></Error> Steps to reproduce: run an ansible playbook using the following: aws_ec2 inventory aws_ssm connection type specify an s3 bucket (ansible_aws_ssm_bucket_name var) that is in a different location to the target node The presigned url generated includes the region the s3 bucket is in, so this region must be used for the session obtained in the _get_url function.

…n var (#1176) SUMMARY Fix issue where syntax error is reported if using ssm connection and the target node is located in a different region to the s3 bucket. Fixes #1190, #637 ISSUE TYPE Bugfix Pull Request COMPONENT NAME aws_ssm connection plugin ADDITIONAL INFORMATION When using SSM for ansible connection and the target node is in a different region to the s3 bucket used, the playbook immediately errors with the following. (There are no issues when both target and s3 bucket are in the same region) fatal: [i-04444a7f03cc2bffd]: FAILED! => {"ansible_facts": {}, "changed": false, "failed_modules": {"ansible.legacy.setup": {"failed": true, "module_stderr": "", "module_stdout": " File \"/tmp/ansible/ansible-tmp-1653576081.8378458-29658-258097978113216/AnsiballZ_setup.py\", line 1\r\r\n <?xml version=\"1.0\" encoding=\"UTF-8\"?>\r\r\n ^\r\r\nSyntaxError: invalid syntax\r\r", "msg": "MODULE FAILURE\nSee stdout/stderr for the exact error", "rc": 1}}, "msg": "The following modules failed to execute: ansible.legacy.setup\n"} The tmp file has the following contents <?xml version="1.0" encoding="UTF-8"?> <Error><Code>AuthorizationQueryParametersError</Code><Message>Error parsing the X-Amz-Credential parameter; the region 'us-east-1' is wrong; expecting 'eu-west-1'</Message><Region>eu-west-1</Region><RequestId>4VTAGR4C1V9ATBJT</RequestId><HostId>OahjGsFQHlr3ihxobH/yyH7Mzxq98mwjcb6+J3Y2EifDU7FykCe8b6QJTNodIG5WSquVeJF+Zsk=</HostId></Error> Steps to reproduce: run an ansible playbook using the following: aws_ec2 inventory aws_ssm connection type specify an s3 bucket (ansible_aws_ssm_bucket_name var) that is in a different location to the target node The presigned url generated includes the region the s3 bucket is in, so this region must be used for the session obtained in the _get_url function. (cherry picked from commit 1be7da1)

…n var (#1176) (#1291) [PR #1176/1be7da11 backport][stable-4] ssm connection: pull bucket region info rather than taking from region var This is a backport of PR #1176 as merged into main (1be7da1). SUMMARY Fix issue where syntax error is reported if using ssm connection and the target node is located in a different region to the s3 bucket. Fixes #1190, #637 ISSUE TYPE Bugfix Pull Request COMPONENT NAME aws_ssm connection plugin ADDITIONAL INFORMATION When using SSM for ansible connection and the target node is in a different region to the s3 bucket used, the playbook immediately errors with the following. (There are no issues when both target and s3 bucket are in the same region) fatal: [i-04444a7f03cc2bffd]: FAILED! => {"ansible_facts": {}, "changed": false, "failed_modules": {"ansible.legacy.setup": {"failed": true, "module_stderr": "", "module_stdout": " File \"/tmp/ansible/ansible-tmp-1653576081.8378458-29658-258097978113216/AnsiballZ_setup.py\", line 1\r\r\n <?xml version=\"1.0\" encoding=\"UTF-8\"?>\r\r\n ^\r\r\nSyntaxError: invalid syntax\r\r", "msg": "MODULE FAILURE\nSee stdout/stderr for the exact error", "rc": 1}}, "msg": "The following modules failed to execute: ansible.legacy.setup\n"} The tmp file has the following contents <?xml version="1.0" encoding="UTF-8"?> <Error><Code>AuthorizationQueryParametersError</Code><Message>Error parsing the X-Amz-Credential parameter; the region 'us-east-1' is wrong; expecting 'eu-west-1'</Message><Region>eu-west-1</Region><RequestId>4VTAGR4C1V9ATBJT</RequestId><HostId>OahjGsFQHlr3ihxobH/yyH7Mzxq98mwjcb6+J3Y2EifDU7FykCe8b6QJTNodIG5WSquVeJF+Zsk=</HostId></Error> Steps to reproduce: run an ansible playbook using the following: aws_ec2 inventory aws_ssm connection type specify an s3 bucket (ansible_aws_ssm_bucket_name var) that is in a different location to the target node The presigned url generated includes the region the s3 bucket is in, so this region must be used for the session obtained in the _get_url function. Reviewed-by: Mark Chappell <None>

…n var (#1176) (#1290) [PR #1176/1be7da11 backport][stable-3] ssm connection: pull bucket region info rather than taking from region var This is a backport of PR #1176 as merged into main (1be7da1). SUMMARY Fix issue where syntax error is reported if using ssm connection and the target node is located in a different region to the s3 bucket. Fixes #1190, #637 ISSUE TYPE Bugfix Pull Request COMPONENT NAME aws_ssm connection plugin ADDITIONAL INFORMATION When using SSM for ansible connection and the target node is in a different region to the s3 bucket used, the playbook immediately errors with the following. (There are no issues when both target and s3 bucket are in the same region) fatal: [i-04444a7f03cc2bffd]: FAILED! => {"ansible_facts": {}, "changed": false, "failed_modules": {"ansible.legacy.setup": {"failed": true, "module_stderr": "", "module_stdout": " File \"/tmp/ansible/ansible-tmp-1653576081.8378458-29658-258097978113216/AnsiballZ_setup.py\", line 1\r\r\n <?xml version=\"1.0\" encoding=\"UTF-8\"?>\r\r\n ^\r\r\nSyntaxError: invalid syntax\r\r", "msg": "MODULE FAILURE\nSee stdout/stderr for the exact error", "rc": 1}}, "msg": "The following modules failed to execute: ansible.legacy.setup\n"} The tmp file has the following contents <?xml version="1.0" encoding="UTF-8"?> <Error><Code>AuthorizationQueryParametersError</Code><Message>Error parsing the X-Amz-Credential parameter; the region 'us-east-1' is wrong; expecting 'eu-west-1'</Message><Region>eu-west-1</Region><RequestId>4VTAGR4C1V9ATBJT</RequestId><HostId>OahjGsFQHlr3ihxobH/yyH7Mzxq98mwjcb6+J3Y2EifDU7FykCe8b6QJTNodIG5WSquVeJF+Zsk=</HostId></Error> Steps to reproduce: run an ansible playbook using the following: aws_ec2 inventory aws_ssm connection type specify an s3 bucket (ansible_aws_ssm_bucket_name var) that is in a different location to the target node The presigned url generated includes the region the s3 bucket is in, so this region must be used for the session obtained in the _get_url function. Reviewed-by: Mark Chappell <None>

bodnarbm · 2022-08-27T17:40:59Z

Can we reopen this issue?

The referenced PR in the close action above (#1176) does not seem to resolve this particular issue. That PR was targeting a fix for pulling the region information for the S3 bucket used for SSM file transfers from the bucket metadata itself, but the pre-signed URLs generated for the S3 downloads are still pointing at the global S3 endpoint, and not the region specific one.

This still creates the issue indicated in #637 (which also needs to be reopened), where newly created S3 buckets returned 307 redirects from the global endpoint to use the regional endpoint instead within the first 24 hours, and then the signature fails as the signature was intended for the global endpoint and not the regional endpoint.

…nsible-collections#1200) route53_health_check: Add health_check info on updating health check SUMMARY Fixes ansible-collections#1190 Added health check info in output when updating a Route53 health check with use_unique_names: true. ISSUE TYPE Bugfix Pull Request COMPONENT NAME route53_health_check Reviewed-by: Mike Graves <mgraves@redhat.com> Reviewed-by: GomathiselviS <None> Reviewed-by: Mandar Kulkarni <mandar242@gmail.com>

ansibullbot added bug This issue/PR relates to a bug needs_triage python3 labels Jun 1, 2022

JLukeBlakey mentioned this issue Jun 1, 2022

ssm connection: pull bucket region info rather than taking from region var #1176

Merged

tremble closed this as completed in #1176 Jun 30, 2022

patchback bot mentioned this issue Jun 30, 2022

[PR #1176/1be7da11 backport][stable-3] ssm connection: pull bucket region info rather than taking from region var #1290

Merged

patchback bot mentioned this issue Jun 30, 2022

[PR #1176/1be7da11 backport][stable-4] ssm connection: pull bucket region info rather than taking from region var #1291

Merged

tremble mentioned this issue Jul 10, 2022

Invoke-WebRequest : The remote server returned an error: (400) Bad Request #969

Closed

1 task

renovate bot mentioned this issue Aug 8, 2022

chore(deps): update dependency community.aws to v4.1.1 linxside/ansible-ee#44

Merged

1 task

bodnarbm mentioned this issue Aug 27, 2022

aws_ssm connection plugin: S3 Signed Url invalid for newly created S3 Bucket #637

Closed

1 task

tremble mentioned this issue Feb 6, 2023

ssm-connector: better debugging and exception in exec_command file_transport_command methods #1017

Closed

1 task

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

SSM connection doesn't use regional S3 endpoint #1190

SSM connection doesn't use regional S3 endpoint #1190

davidgiga1993 commented Jun 1, 2022

ansibullbot commented Jun 1, 2022

bodnarbm commented Aug 27, 2022

SSM connection doesn't use regional S3 endpoint #1190

SSM connection doesn't use regional S3 endpoint #1190

Comments

davidgiga1993 commented Jun 1, 2022

Summary

Issue Type

Component Name

Ansible Version

Collection Versions

AWS SDK versions

Configuration

OS / Environment

Steps to Reproduce

Expected Results

Actual Results

Code of Conduct

ansibullbot commented Jun 1, 2022

bodnarbm commented Aug 27, 2022