cannot seem to get fog to connect to s3 at all...

[rwboyer]

I ran into this problem trying to use another gem that uses fog and wanted to attempt to debug if it was that other gem or me or fog but here's what I get...

Excon::Errors::SocketError: hostname "typkit.rwboyer.com.s3.amazonaws.com" does not match the server certificate (OpenSSL::SSL::SSLError)

Using the latests greatest gems on rubygems for Fog and for excon as of today (and the previous versions as well)...

I get the same exact errors using even this really simple code:

fog = Fog::Storage.new(
    :provider => "AWS",
    :aws_access_key_id => ENV['AMAZON_ACCESS_KEY_ID'], 
    :aws_secret_access_key => ENV['AMAZON_SECRET_ACCESS_KEY']
)

directory = fog.directories.get("typkit.rwboyer.com")

the environment is fine considering it's used for all my production s3 stuff just doesn't seem to work here. What am I doing wrong or is it some sort of bug???

Thanks

RB

[geemus]

I know that some folks have had issues with this (s3 certificates aren't setup to handle buckets with dots, I think). You can force the right stuff by passing :path_style => true in the initializer, I believe. Hopefully that will help anyway, do let me know and we can continue troubleshooting as needed.

[rwboyer]

Will give that a shot right now...

Was also thinking of setting the Excon peer verify thing to false - looking through the code it looks at first glance like :connection_options seems to go strait through. Is this correct?

[rwboyer]

geemus - thanks that does solve the issue for now but would like to know if :connection_options => {} is passed thru to excon as well...

Now I just have to figure out if that other gem has options or hack at it so it does... ;-)

RB

[rwboyer]

One more question - any way to set :path_style in the environment???? so I don't have to hack up the 42 gem dependency chain from what I am trying to use all the way through to rails asset_sync??

Thanks...

RB

[geemus]

@rwboyer connection options should be passed through, if I remember correctly. Disabling ssl as you mention is another option but has it's own downsides. I think you should be able to just setup :path_style in the credentials that are used (along with key/secret) and it should be used wherever. But I haven't used asset_sync specifically.

[rwboyer]

@geemus = thanks a bunch but I don't really "get" what you mean by pass it through via credentials...

like how exactly? I am sitting like 8 gems above fog and here's what i've got...

sync.fog_provider = 'AWS' # Your storage provider
sync.fog_directory = 'typkit.rwboyer.com' # Your bucket name
sync.fog_region = 'us-east-1'
sync.aws_access_key_id          = ENV['AMAZON_ACCESS_KEY_ID'] 
sync.aws_secret_access_key      = ENV['AMAZON_SECRET_ACCESS_KEY']

etc, etc. so not sure where you want me to just chuck in :path_style => true

may just be having a dumb-ass attack after a long week...

in any case = any chance fog picks it up out of an environment setting????

RB

[geemus]

Fair enough. I think you can cheat a bit by using fog_credentials (which are intended mostly for just reading credentials from a file). Basically around where you are doing that stuff, I think you should be able to add:
Fog.credentials = { :path_style => true } and it should work as you might expect. Do let me know if that doesn't cut it, but I think from reviewing the code just now it should be legit.

[rwboyer]

@geemus

That would be friggin awesome - will try it as soon as I am off this current job today or tomorrow... didn't push that code to github so it's on a machine not with me atm...

Thanks a lot

[rwboyer]

@geemus

While we're at it and I am on a tear for niggling issues I don't feel like fixing myself which usually means some really ugly "temporary" hacking that never gets un-temporary-ized here's another Fog related thing... actually two.

First off - related to the discussion/work-around discussed above. What would be the harm in using path_style true as a default? I don't really see any from the outside looking in. Am I the ONLY one using S3 to serve pages and a gem that depends on rails asset sync? I would assume anyone serving pages from S3 with a real domain name would be using bucket names with dots so it would seem appropriate to just reverse the default to me. Or am I the odd man out and that would somehow cause a bunch of issues I am not seeing? I kind of like the whole ruby/rails/gems thing of using the most sensible defaults vs having to set shitloads of parameters... otherwise it's a whole lot like IBM VM systems programming in the 80's - sorry flashback.

Second - In an effort to circumvent actual debugging of other people's components due to laziness/busy-ness and too much work lately on de-lousing a bunch of other people's wordpress/php muck I went to plan B and used yet another gem that does NOT use rails asset sync (since i am not a big rails fan anyway) but does use Fog directly and happens to pass thru :path_style if so desired. Unfortunately it had another esoteric issue that seems to be Fog/Excon related. Specifically it dies with an Excon error:

/Users/rwboyer/.rvm/gems/ruby-2.0.0-p247/gems/excon-0.27.6/lib/excon/socket.rb:138:in `getaddrinfo': getaddrinfo: nodename nor servname provided, or not known (SocketError) (Excon::Errors::SocketError)

After google-researching this for all of 2 minutes all I could find of relevance was someone using Carrierwave to deal with massive amounts of ICON files a long time ago that seems never to have been figured out/resolved - it just sort of went away after a few weeks or got silently resolved.

In anycase I opened and issue with that gem as well here

fredjean/middleman-s3_sync#28

but was wondering if you had any insight into somehow the stuff Excon needs just disappearing when using Fog?

Funny since all this was in an effort was to ditch my own crappy S3 sync using AWS api based code to something in actual use by other folks. Just for reference I am testing on a static site that has maybe a couple thousand pages and another couple thousand JPEG image assets.

Thanks

RB

[guigs]

I had the same issue. Adding Fog.credentials = { path_style: true } in application.rb solves it! Thanks!

[rwboyer]

@guigs @geemus - probably a great idea to add to the README since Fog + S3 seems to be a natural fit and what seems like a ton of other gems pickup Fog to do JUST this.

[christhekeele]

I got this working with Fog.credentials = { path_style: true } in my initializers.

[mariovisic]

Is there any harm in using path style by default? This issue has only recently started to affect me using the asset_sync gem. Previously it would work fine. Are there any cases where the path style will not work?

[guigs]

It used to work for me too. I could't find what have changed that cause the certificate error (without using path_style).

[geemus]

Subdomain style is noticeably more performant. So it seemed better to have that as the default. A lot of others do use S3 in the way you describe (but you can still do this without dots in the bucket name and just direct to them with DNS). Also, if you use http (instead of https) it also isn't an issue. As such, we have not made this default (so far anyway). Does that make sense?

[fredjean]

@geemus @rwboyer I actually switched fredjean/middleman-s3_sync to set path_style to true by default a few weeks ago. It turned out to be a lucky move on my part. Of course, people using middleman-s3_sync can choose to turn it off...

As a reminder, you can configure S3 to act as a web server. In this case, the name of the bucket must match the CNAME or ALIAS record (supported by Route53, DNSimple and others, not standard DNS) exactly, dots and all... It does not however limit how to access the bucket. Both path style and bucket-name-as-subdomain style works when accessing the S3 resources via API.

[mariovisic]

It's only more performant if a region isn't listed, right? The performance loss comes from having to do an extra lookup/redirect for the region, is the result then cached and then reused later on? I'm asking because I might set the path style as default for my app and i'm wondering if the performance penalty will be paid for each request.

@geemus were there any changes that have been introduced that could have caused the issue? Is HTTPS now used by default over HTTP for example?

Should HTTPS be disabled when using subdomain styles as the certificates never match up anyways?

FYI: This is the check in the excon gem that is causing the error: https://github.com/geemus/excon/blob/master/lib/excon/ssl_socket.rb#L82

[geemus]

@fredjean true, I have seen the s3 as website thing and even done it. I think even then you end up with https verification issues since the certs wouldn't match.

@mariovisic I haven't re-benchmarked it more recently. But when I did the performance impact was for every request (it just is even worse with region mismatches). I'm not aware of defaults changing that would cause the issue, but it is possible. I'd be reticent to simply turn of ssl verification as users might well expect it to work. failing fast seems like it might be better in that regard. I suppose you could also explicitly turn off ssl verification directly. I dunno, it is tricky.

[fredjean]

@geemus Yep.. I'm the one that reminds you periodically about the S3 bucket as a web site use case ;)

Thanks for the work on fog and excon. It's really nice technology to build things on top of.

[mariovisic]

@geemus Maybe the solution to this issue could be to warn/error with a useful message explaining to set path style or use http over https.

[geemus]

@mariovisic that seems more feasible. I think trying to change the default might have too many other/larger ramifications.

[plribeiro3000]

@geemus i think this issue may be solved by @carlosgaldino commit, Can you confirm that?

[geemus]

I am not certain, but given the lack of activity, I'm willing to wager it resolved in some way. Closing, but will re-open if people comment and make it clear there is more to be done.

[shwoodard]

I'm having this issue, but I don't have dots in my bucket name. It is like, foo-bar.

[geemus]

@shwoodard could you open a new issue with the relevant details to your case? Thanks!

[shwoodard]

The issue was that I am using carrierwave and it does not support the
latest fog; I did not lock it down in my Gemfile.

On Mon Jan 05 2015 at 1:27:54 PM Wesley Beary notifications@github.com
wrote:

@shwoodard https://github.com/shwoodard could you open a new issue with
the relevant details to your case? Thanks!

—
Reply to this email directly or view it on GitHub
#2357 (comment).

[geemus]

Ah, perhaps we need to nudge carrierwave to update that then.

[shwoodard]

Agreed.

On Tue Jan 06 2015 at 10:50:06 AM Wesley Beary notifications@github.com
wrote:

Ah, perhaps we need to nudge carrierwave to update that then.

—
Reply to this email directly or view it on GitHub
#2357 (comment).