repeated bucket name in the URL on AWS and issue with :path_style

[valerioleo]

I am trying to set fog for aws for my rails application, together with CarrierWave.
I created a CarrierWave initializer with al the settings needed.
Files are uploaded without problem in my S3 bucket but there are still two things to fix:

1. the url saved in my model is different from the one saved in the S3 bucket.
   For example:

The real link of the image (retrieved from the bucket control panel) is:
https://s3.eu-central-1.amazonaws.com/my_bucket_name/my_bucket_name/my_bucket_name/uploads/restaurant/gallery/1/image.jpeg

While in my model the image is saved as:
https://my_APP_name.s3.amazonaws.com/uploads/restaurant/gallery/1/image.jpg
Which is the non styled url (but I explicitly declared :path_style => true)

I tried to change :path_style to false but in that case amazon retrieves me this error:

hostname "my_bucket_name.my_bucket_name.s3.eu-central-1.amazonaws.com" does not match the server certificate (OpenSSL::SSL::SSLError).
I sincerely have no clue about how to fix it.

2. As you can see, I don't know for which reason but the images are uploaded in several subdirectories (/my_bucket_name/my_bucket_name/my_bucket_name/restaurant/...) and this is driving me crazy.

I attach here the configuration file which I believe is fine. Thanks everybody for the support.

CarrierWave.configure do |config|
    config.fog_provider = 'fog/aws' 
    config.fog_credentials = {
        :provider => 'AWS',
        :aws_access_key_id => 'KEY',
        :aws_secret_access_key => 'SECRET_KEY',
        :region => 'eu-west-1',
        :path_style => true
    }
    config.fog_directory = 'my_bucket_name'    
    config.fog_public = true

end

PS: "my_bucket_name" and "my_app_name" are fantasy names just to help explaining the issue and they are not the real names. Anyway, real names are composed exclusively of lowercase letter.

[geemus]

The error with path-style false may have to do with the bucket name including underscores, as I think this might not match with the wildcard SSL on s3. I'm uncertain about the repeated domain names though. Someone within the carrierwave project may better be able to help there as I don't think it has to do with fog itself, so much as the integration.

[valerioleo]

The error with path-style false may have to do with the bucket name including underscores

Hi Geemus! Thanks for the reply.
In this issue post I changed both bucket and app name just to make easier to explain the problem.
The original bucket name (it's called "restaurantplates") has neither underscores nor periods, it is just lowercase letters. And the same is for the app name.

[geemus]

Ah, thanks for that clarification. I knew that sometimes particular names conflicted with the wildcard cert S3 uses, but it doesn't sound like that is the case here after all. Were you able to get in touch with anybody from carrierwave to offer advice? I haven't worked with it in such a long time I worry I may be missing something obvious. Thanks!

[karwank]

Any progress with this?

[RomanKisilenko]

Same issue here, is there any fix known?

[karwank]

It turned out that I had wrong region in configuration. Changing ragion fixed the issue.

[geemus]

Yeah, when this issue occurs it usually relates to a failed attempt to follow redirects when the region is set incorrectly for the connection. Unfortunately, they are not true redirects, just a series of inconsistent error messages that we have done our best to turn into working redirects (sometimes it just works, but we have had a number of cases where it ends up doing things like this, unfortunately). Setting the region to match the file is a good way to avoid this issue altogether, otherwise I would welcome help improving the redirect code (but know that I've spent a lot of time struggling with it and not had much luck improving the situation due to various inconsistencies and the difficulty to test all the possibilities).

[scsmith]

I'm not sure if this is 100% the OP's issue but from what I can see S3 has changed how it's region redirects occur.

Previously if you used a path_style request it would go from s3.amazonaws.com -> eu-west-1.s3.amazonaws.com and make the request again.

Instead now it seems to redirect from s3.amazonaws.com -> bucketname.s3.amazonaws.com.

Because of this Fog can't recognise the region and will continue to sign with the wrong region (us-east-1) by default. It seems like this differs from the logic in the redirect code, as stated, or at least there needs to be a new detection method for signing with the wrong region in addition to what is already present.

[geemus]

@scsmith that could very well be. The behavior I saw in the past was that different requests redirected in different ways (which made it really hard to debug or workaround), perhaps that is no longer the case though.

[vutran0111]

@scsmith Thanks bro 😃

[lustan3216]

I'm have the same issue.
And i found out its might be AWS redirection problem.
How i solved is change region, i move bucket from Tokyo to Singapore.
It jsut working.

[geemus]

@lustan3216 yeah, there have definitely been some weird edge cases relating to redirection. Glad you found a fix and thanks for taking time to share details.

[github-actions]

This issue has been marked inactive and will be closed if no further activity occurs.