Alpakka fails to generate correct S3 Chinese endpoints.

[Elderry]

Versions used

Alpakka version: 0.17

Expected Behavior

After configured correctly, alpakka can upload files to Chinese S3 endpoints.

Actual Behavior

Fail to upload.

Relevant logs

java.util.concurrent.CompletionException: akka.stream.StreamTcpException: Connection failed.

Cause Description

The bug is caused by the same reason of Issue 175, but the PR claimed to fix that only hard code for the us-east-1 case. For region cn-north-1 and cn-northwest-1. The correct endpoint is s3.<region>.amazonaws.com.cn, according to the
AWS reference, instead of s3-<region>.amazonaws.com generated by alpakka.

[2m]

Thanks for the ticket! Would you be open to creating a PR that fixes it?

I also noticed from the AWS reference docs that us-east-1 does not a special treatment anymore. The following format works for us-east-1 and other regions (except for cn- regions) as well:

s3.<region>.amazonaws.com

[ennru]

This is now fixed with #2336

[avajeeh]

The fix #2336 is to standardize the URI for S3 for most of the AWS region but in China, the URI is having '.cn' appended to the host url. Can it be confirmed that this fix will work for cn-northwest-1 and cn-north-1 regions?

[seglo]

Hi @avajeeh. If that's the case then I assume that it won't work for Chinese regions. You can confirm by writing a test in s3/src/test/scala/akka/stream/alpakka/s3/impl/auth/CanonicalRequestSpec.scala. Would you be open to creating a PR to add the exceptions for these regions?

[avajeeh]

What if instead, we make the aws domain name configurable which will default to 'amazonaws.com'?

[seglo]

There's already a setting for that to support S3 compatible implementations (not hosted by Amazon). Have you tried working around the issue with alpakka.s3.endpoint-url?

https://github.com/akka/alpakka/blob/master/s3/src/main/resources/reference.conf#L87..L91

[avajeeh]

Thanks, this helped me as I can see that it goes ahead. I tried with the following as my parameter,
alpakka.s3.endpoint-url=https://{bucket}.s3.cn-northwest-1.amazonaws.com.cn:443

It resulted in another failure though which is as follows
Caused by: akka.stream.alpakka.s3.S3Exception: The authorization header is malformed; the region 'us-east-1' is wrong; expecting 'cn-northwest-1' (Status code: 400 Bad Request, Code: AuthorizationHeaderMalformed, RequestId: 7B24EDB71801CD16, Resource: )

I update the default-region from us-east-1 to cn-northwest-1 because thats the only place it was referring to us-east-1 region. This resulted in another error as follows,

Exception in thread "main" akka.stream.alpakka.s3.S3Exception: The AWS Access Key Id you provided does not exist in our records. (Status code: 403 Forbidden, Code: InvalidAccessKeyId, RequestId: 4A6AF21929DE2106, Resource: )

I cross checked and my credentials are correct so now I am completly lost on how to proceed furthur. Can you please help?

[seglo]

Hi @avajeeh. Would you be in a position to experiment with modifying the S3 connector yourself to support the .cn TLD? I suspect it would be a relatively easy addition, but I don't have the means to verify that it works against an actual Chinese AWS region. You should be able to implement it following the same pattern as #2336.

[avajeeh]

I can give it a try. Is there any other way we can collaborate if I need any feedback with regards to implementation?

[seglo]

That's great! You can either create a new issue and we can collaborate there or a PR. We also have a gitter channel, but it's for all open source akka development (https://gitter.im/akka/dev).