generate_presigned_url results in SignatureDoesNotMatch

[viktormill]

Describe the bug
Calling generate_presigned_url in AWS region 'eu-north-1' results in error 'SignatureDoesNotMatch. The request signature we calculated does not match the signature you provided. Check your key and signing method.'

If calling generate_presigned_url in AWS region 'eu-west-1' this works fine.

The presigned url looks like this
https://my-bucket.s3.amazonaws.com/test.txt?X-Amz-Algorithm...

Steps to reproduce
Using Boto3 version 1.17.78 and calling generate_presigned_url

s3_client = boto3.client('s3')
s3_source_signed_url = s3_client.generate_presigned_url(
  'get_object',
  Params={
  'Bucket': 'my-bucket,
  'Key': 'my-key'
  },
  ExpiresIn=3600
)

Expected behavior
generate_presigned_url to include region in the presigned url e.g. https://my-bucket.s3.eu-north-1.amazonaws.com/test.txt?X-Amz-Algorithm...

Debug logs
'SignatureDoesNotMatch. The request signature we calculated does not match the signature you provided. Check your key and signing method.'

[tim-finnigan]

Hi @viktormill, thanks for reaching out. By default, the region is not included in the presigned URL as noted here: #1982 (comment). But some users have provided workarounds in that issue by setting endpoint_url like this: #1982 (comment).

In regard to the error your seeing, many people have suggested that setting the Config like this will solve the problem:
#1644 (comment)

Can you let us know if that worked? If you’re still seeing an issue, please send us the full debug logs (with sensitive information redacted) by adding boto3.set_stream_logger('') to your code.

[viktormill]

@tim-finnigan I think this is a DNS bug on AWS on bucket creation,
If I immediately create a bucket on S3 in e.g. eu-north-1, this bug happen.
To solve it, both of your links to solutions works:
Virtual-based:

boto3.client(
  's3',
  endpoint_url=f'https://s3.{region}.amazonaws.com',
  config=boto3.session.Config(s3={'addressing_style': 'virtual'})
)

Or
Path-based:

boto3.client('s3', config=boto3.session.Config(s3={'addressing_style': 'path'}, signature_version='s3v4'))

Though If I wait for a few hours up to one day, the above fixes aren't needed.
Also this bug seems to never happen on eu-west-1.

I also read that aws are planning to deprecate s3 path based urls: https://aws.amazon.com/blogs/aws/amazon-s3-path-deprecation-plan-the-rest-of-the-story/

So I assume that virtual based would be recommended?

[tim-finnigan]

Hi @viktormill, thanks for your response. I was able to reproduce the SignatureDoesNotMatch error for objects in both eu-north-1 and eu-west-1 buckets. The easiest solution was just to provide the region_name and endpoint_url as noted in this comment: #1149 (comment)

s3_client = session.client(
    's3', region_name='eu-central-1', endpoint_url='https://s3.eu-central-1.amazonaws.com'
)

Thanks again for raising visibility to this. I will discuss with the team and see if we can better document this scenario. Please let us know if you have any more questions or feedback.

[github-actions]

⚠️COMMENT VISIBILITY WARNING⚠️

Comments on closed issues are hard for our team to see.
If you need more assistance, please either tag a team member or open a new issue that references this one.
If you wish to keep having a conversation with other community members under this issue feel free to do so.

[Ling-Jun]

This is still happening in 2024. I have created this following script:

• this works: s3_client = session.client('s3', region_name='ca-central-1', endpoint_url='https://s3.ca-central-1.amazonaws.com', config=boto3.session.Config(signature_version='s3v4'))
• this raises the same error as above: s3_client = session.client('s3', region_name='ca-central-1', config=boto3.session.Config(signature_version='s3v4')), the error does disappear after seeming a day