Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

S3: Signature Does Not Match #583

Closed
mmarschall opened this issue Nov 2, 2011 · 12 comments

Comments

@mmarschall
Copy link

@mmarschall mmarschall commented Nov 2, 2011

Hi,

I'm running Fog against an S3 compatible service by German provider Hosteurope. While I can successfully connect to their service using various S3 clients, it fails using Fog :-(

Here is the exception:

irb(main):001:0> storage = Fog::Storage.new({:provider => 'AWS', :aws_access_key_id => "UCNVGEXEVTP5AKFMQ93R", :aws_secret_access_key => "...", :endpoint => 'https://cs.hosteurope.de'})
=> #<Fog::AWS::Storage::Real:0x10f37aec8 @hmac=#<Fog::HMAC:0x10f373088 @key="...", @digest=#<OpenSSL::Digest::Digest: da39a3ee5e6b4b0d3255bfef95601890afd80709>, @signer=#Proc:0x000000010a7b6eb0@/Library/Ruby/Gems/1.8/gems/fog-0.7.2/lib/fog/core/hmac.rb:22>, @aws_secret_access_key="...", @port=443, @scheme="https", @connection=#<Fog::Connection:0x10f35afb0 @excon=#<Excon::Connection:0x10f35aec0 @socket_key="cs.hosteurope.de:443", @connection={:port=>"443", :path=>"", :headers=>{}, :query=>nil, :scheme=>"https", :host=>"cs.hosteurope.de", :mock=>nil}>, @Persistent=nil>, @host="cs.hosteurope.de", @path="", @endpoint="https://cs.hosteurope.de", @aws_access_key_id="UCNVGEXEVTP5AKFMQ93R">
irb(main):002:0> storage.get_bucket("helpster-cms-backup")
Excon::Errors::Forbidden: Expected(200) <=> Actual(403 Forbidden)
request => {:expects=>200, :port=>"443", :path=>"/", :headers=>{"Authorization"=>"AWS UCNVGEXEVTP5AKFMQ93R:xDF89OO+i4dWEaLivctwZM4cO4M=", "Date"=>"Wed, 26 Oct 2011 13:43:03 +0000", "Content-Length"=>0, "Host"=>"helpster-cms-backup.cs.hosteurope.de:443"}, :idempotent=>true, :method=>"GET", :query=>{}, :host=>"helpster-cms-backup.cs.hosteurope.de", :scheme=>"https", :mock=>nil}
response => #<Excon::Response:0x10f34fc78 @Body="\nSignatureDoesNotMatch\n\n\n", @headers={"Content-Type"=>"application/xml", "Date"=>"Wed, 26 Oct 2011 13:43:04 GMT", "Content-Length"=>"110", "Server"=>"RestServer/1.0", "Cache-Control"=>"no-cache", "Connection"=>"close"}, @status=403>
from /Library/Ruby/Gems/1.8/gems/excon-0.6.3/lib/excon/connection.rb:179:in request' from /Library/Ruby/Gems/1.8/gems/fog-0.7.2/lib/fog/core/connection.rb:20:inrequest'
from /Library/Ruby/Gems/1.8/gems/fog-0.7.2/lib/fog/storage/aws.rb:323:in request' from /Library/Ruby/Gems/1.8/gems/fog-0.7.2/lib/fog/storage/requests/aws/get_bucket.rb:55:inget_bucket'
from (irb):2

Any ideas?

@geemus

This comment has been minimized.

Copy link
Member

@geemus geemus commented Nov 2, 2011

I wish I could be more helpful, but I'm really not sure. It is
possible that you are passing incorrect credentials (so double check
that first). Beyond that, I tried to find some further documentation
on their website but it is in German (which I don't speak) and all I
could really find was stuff that seemed to say S3 compatible but
didn't have extra details. The downside to the "compatible" approach
is that there are many different versions of S3 that work slightly
differently and have different features. fog is aimed at the more
recent versions, whereas I suspect some other tools are aimed at older
versions. So I'm not really sure how best to proceed with this...

On Wed, Nov 2, 2011 at 08:59, Matthias Marschall
reply@reply.github.com
wrote:

Hi,

I'm running Fog against an S3 compatible service by German provider Hosteurope. While I can successfully connect to their service using various S3 clients, it fails using Fog :-(

Here is the exception:

irb(main):001:0> storage = Fog::Storage.new({:provider => 'AWS', :aws_access_key_id => "UCNVGEXEVTP5AKFMQ93R", :aws_secret_access_key => "...", :endpoint => 'https://cs.hosteurope.de'})
=> #<Fog::AWS::Storage::Real:0x10f37aec8 @hmac=#<Fog::HMAC:0x10f373088 @key="...", @digest=#<OpenSSL::Digest::Digest: da39a3ee5e6b4b0d3255bfef95601890afd80709>, @signer=#Proc:0x000000010a7b6eb0@/Library/Ruby/Gems/1.8/gems/fog-0.7.2/lib/fog/core/hmac.rb:22>, @aws_secret_access_key="...", @port=443, @scheme="https", @connection=#<Fog::Connection:0x10f35afb0 @excon=#<Excon::Connection:0x10f35aec0 @socket_key="cs.hosteurope.de:443", @connection={:port=>"443", :path=>"", :headers=>{}, :query=>nil, :scheme=>"https", :host=>"cs.hosteurope.de", :mock=>nil}>, @Persistent=nil>, @host="cs.hosteurope.de", @path="", @endpoint="https://cs.hosteurope.de", @aws_access_key_id="UCNVGEXEVTP5AKFMQ93R">
irb(main):002:0> storage.get_bucket("helpster-cms-backup")
Excon::Errors::Forbidden: Expected(200) <=> Actual(403 Forbidden)
 request => {:expects=>200, :port=>"443", :path=>"/", :headers=>{"Authorization"=>"AWS UCNVGEXEVTP5AKFMQ93R:xDF89OO+i4dWEaLivctwZM4cO4M=", "Date"=>"Wed, 26 Oct 2011 13:43:03 +0000", "Content-Length"=>0, "Host"=>"helpster-cms-backup.cs.hosteurope.de:443"}, :idempotent=>true, :method=>"GET", :query=>{}, :host=>"helpster-cms-backup.cs.hosteurope.de", :scheme=>"https", :mock=>nil}
 response => #<Excon::Response:0x10f34fc78 @Body="\nSignatureDoesNotMatch\n\n\n", @headers={"Content-Type"=>"application/xml", "Date"=>"Wed, 26 Oct 2011 13:43:04 GMT", "Content-Length"=>"110", "Server"=>"RestServer/1.0", "Cache-Control"=>"no-cache", "Connection"=>"close"}, @status=403>
       from /Library/Ruby/Gems/1.8/gems/excon-0.6.3/lib/excon/connection.rb:179:in request'        from /Library/Ruby/Gems/1.8/gems/fog-0.7.2/lib/fog/core/connection.rb:20:inrequest'
       from /Library/Ruby/Gems/1.8/gems/fog-0.7.2/lib/fog/storage/aws.rb:323:in request'        from /Library/Ruby/Gems/1.8/gems/fog-0.7.2/lib/fog/storage/requests/aws/get_bucket.rb:55:inget_bucket'
       from (irb):2

Any ideas?

Reply to this email directly or view it on GitHub:
https://github.com/geemus/fog/issues/583

@mmarschall

This comment has been minimized.

Copy link
Author

@mmarschall mmarschall commented Nov 2, 2011

Thanks for the quick response. I'm sure that the credentials are right because I can use them with other clients.

I'm in contact with their support. Is there any information which would be helpful? I could try to get it from them...

Am 02.11.2011 um 15:10 schrieb Wesley Beary:

I wish I could be more helpful, but I'm really not sure. It is
possible that you are passing incorrect credentials (so double check
that first). Beyond that, I tried to find some further documentation
on their website but it is in German (which I don't speak) and all I
could really find was stuff that seemed to say S3 compatible but
didn't have extra details. The downside to the "compatible" approach
is that there are many different versions of S3 that work slightly
differently and have different features. fog is aimed at the more
recent versions, whereas I suspect some other tools are aimed at older
versions. So I'm not really sure how best to proceed with this...

On Wed, Nov 2, 2011 at 08:59, Matthias Marschall
reply@reply.github.com
wrote:

Hi,

I'm running Fog against an S3 compatible service by German provider Hosteurope. While I can successfully connect to their service using various S3 clients, it fails using Fog :-(

Here is the exception:

irb(main):001:0> storage = Fog::Storage.new({:provider => 'AWS', :aws_access_key_id => "UCNVGEXEVTP5AKFMQ93R", :aws_secret_access_key => "...", :endpoint => 'https://cs.hosteurope.de'})
=> #<Fog::AWS::Storage::Real:0x10f37aec8 @hmac=#<Fog::HMAC:0x10f373088 @key="...", @digest=#<OpenSSL::Digest::Digest: da39a3ee5e6b4b0d3255bfef95601890afd80709>, @signer=#Proc:0x000000010a7b6eb0@/Library/Ruby/Gems/1.8/gems/fog-0.7.2/lib/fog/core/hmac.rb:22>, @aws_secret_access_key="...", @port=443, @scheme="https", @connection=#<Fog::Connection:0x10f35afb0 @excon=#<Excon::Connection:0x10f35aec0 @socket_key="cs.hosteurope.de:443", @connection={:port=>"443", :path=>"", :headers=>{}, :query=>nil, :scheme=>"https", :host=>"cs.hosteurope.de", :mock=>nil}>, @Persistent=nil>, @host="cs.hosteurope.de", @path="", @endpoint="https://cs.hosteurope.de", @aws_access_key_id="UCNVGEXEVTP5AKFMQ93R">
irb(main):002:0> storage.get_bucket("helpster-cms-backup")
Excon::Errors::Forbidden: Expected(200) <=> Actual(403 Forbidden)
request => {:expects=>200, :port=>"443", :path=>"/", :headers=>{"Authorization"=>"AWS UCNVGEXEVTP5AKFMQ93R:xDF89OO+i4dWEaLivctwZM4cO4M=", "Date"=>"Wed, 26 Oct 2011 13:43:03 +0000", "Content-Length"=>0, "Host"=>"helpster-cms-backup.cs.hosteurope.de:443"}, :idempotent=>true, :method=>"GET", :query=>{}, :host=>"helpster-cms-backup.cs.hosteurope.de", :scheme=>"https", :mock=>nil}
response => #<Excon::Response:0x10f34fc78 @Body="\nSignatureDoesNotMatch\n\n\n", @headers={"Content-Type"=>"application/xml", "Date"=>"Wed, 26 Oct 2011 13:43:04 GMT", "Content-Length"=>"110", "Server"=>"RestServer/1.0", "Cache-Control"=>"no-cache", "Connection"=>"close"}, @status=403>
from /Library/Ruby/Gems/1.8/gems/excon-0.6.3/lib/excon/connection.rb:179:in request' from /Library/Ruby/Gems/1.8/gems/fog-0.7.2/lib/fog/core/connection.rb:20:inrequest'
from /Library/Ruby/Gems/1.8/gems/fog-0.7.2/lib/fog/storage/aws.rb:323:in request' from /Library/Ruby/Gems/1.8/gems/fog-0.7.2/lib/fog/storage/requests/aws/get_bucket.rb:55:inget_bucket'
from (irb):2

Any ideas?

Reply to this email directly or view it on GitHub:
https://github.com/geemus/fog/issues/583

Reply to this email directly or view it on GitHub:
https://github.com/geemus/fog/issues/583#issuecomment-2603759

@geemus

This comment has been minimized.

Copy link
Member

@geemus geemus commented Nov 2, 2011

Nothing is jumping out at me, but I just realized I'm also looking at a different version. Looks like you are currently using fog 0.7.2, you may have better luck with 1.0.0. I'd update that first and then if the problem persists we can keep digging. Sorry I didn't see that right away, somehow got into problem fixing mode too quickly when I should have been surveying the big picture. Thanks!

@mmarschall

This comment has been minimized.

Copy link
Author

@mmarschall mmarschall commented Nov 2, 2011

ok, upgraded to fog 1.0.0 - same issue:

irb(main):004:0> storage = Fog::Storage.new({:provider => 'AWS', :aws_access_key_id => "UCNVGEXEVTP5AKFMQ93R", :aws_secret_access_key => "...", :endpoint => 'https://cs.hosteurope.de'})
=> #<Fog::Storage::AWS::Real:0x10f16efd0 @aws_secret_access_key="...", @hmac=#<Fog::HMAC:0x10eed4068 @key="...", @digest=#<OpenSSL::Digest::Digest: da39a3ee5e6b4b0d3255bfef95601890afd80709>, @signer=#<Proc:0x000000010f179980@/Library/Ruby/Gems/1.8/gems/fog-1.0.0/lib/fog/core/hmac.rb:22>>, @port=443, @scheme="https", @aws_access_key_id="UCNVGEXEVTP5AKFMQ93R", @host="cs.hosteurope.de", @connection=#<Fog::Connection:0x10eed20d8 @persistent=nil, @excon=#<Excon::Connection:0x10eed19a8 @connection={:read_timeout=>60, :mock=>nil, :host=>"cs.hosteurope.de", :port=>"443", :path=>"", :write_timeout=>60, :query=>nil, :scheme=>"https", :headers=>{}, :connect_timeout=>60}, @socket_key="cs.hosteurope.de:443">>, @path="", @connection_options={}, @endpoint="https://cs.hosteurope.de">
irb(main):005:0> storage.directories
Excon::Errors::Forbidden: Expected(200) <=> Actual(403 Forbidden)
  request => {:host=>"cs.hosteurope.de", :mock=>nil, :read_timeout=>60, :path=>"/", :port=>"443", :write_timeout=>60, :query=>nil, :url=>"cs.hosteurope.de", :expects=>200, :scheme=>"https", :method=>"GET", :headers=>{"Authorization"=>"AWS UCNVGEXEVTP5AKFMQ93R:vZla3aTDxXPm9V4JMFzltXUrD3o=", "Date"=>"Wed, 02 Nov 2011 14:51:55 +0000", "Content-Length"=>0, "Host"=>"cs.hosteurope.de:443"}, :connect_timeout=>60, :idempotent=>true}
  response => #<Excon::Response:0x10eea8f30 @status=403, @headers={"Content-Type"=>"application/xml", "Date"=>"Wed, 02 Nov 2011 14:51:59 GMT", "Content-Length"=>"110", "Server"=>"RestServer/1.0", "Cache-Control"=>"no-cache", "Connection"=>"close"}, @body="<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>SignatureDoesNotMatch</Code>\n<Message></Message>\n</Error>\n">
    from /Library/Ruby/Gems/1.8/gems/excon-0.7.6/lib/excon/connection.rb:194:in `request'
    from /Library/Ruby/Gems/1.8/gems/fog-1.0.0/lib/fog/core/connection.rb:20:in `request'
    from /Library/Ruby/Gems/1.8/gems/fog-1.0.0/lib/fog/aws/storage.rb:366:in `request'
    from /Library/Ruby/Gems/1.8/gems/fog-1.0.0/lib/fog/aws/requests/storage/get_service.rb:32:in `get_service'
    from /Library/Ruby/Gems/1.8/gems/fog-1.0.0/lib/fog/aws/models/storage/directories.rb:13:in `all'
    from /Library/Ruby/Gems/1.8/gems/fog-1.0.0/lib/fog/core/collection.rb:131:in `lazy_load'
    from /Library/Ruby/Gems/1.8/gems/fog-1.0.0/lib/fog/core/collection.rb:11:in `empty?'
    from /Library/Ruby/Gems/1.8/gems/fog-1.0.0/lib/fog/core/collection.rb:75:in `inspect'
    from /Library/Ruby/Gems/1.8/gems/formatador-0.2.1/lib/formatador.rb:92:in `indent'
    from /Library/Ruby/Gems/1.8/gems/fog-1.0.0/lib/fog/core/collection.rb:68:in `inspect'
    from /System/Library/Frameworks/Ruby.framework/Versions/1.8/usr/lib/ruby/1.8/irb.rb:310:in `output_value'
    from /System/Library/Frameworks/Ruby.framework/Versions/1.8/usr/lib/ruby/1.8/irb.rb:159:in `eval_input'
    from /System/Library/Frameworks/Ruby.framework/Versions/1.8/usr/lib/ruby/1.8/irb.rb:271:in `signal_status'
    from /System/Library/Frameworks/Ruby.framework/Versions/1.8/usr/lib/ruby/1.8/irb.rb:155:in `eval_input'
    from /System/Library/Frameworks/Ruby.framework/Versions/1.8/usr/lib/ruby/1.8/irb.rb:154:in `eval_input'
    from /System/Library/Frameworks/Ruby.framework/Versions/1.8/usr/lib/ruby/1.8/irb.rb:71:in `start'
    from /System/Library/Frameworks/Ruby.framework/Versions/1.8/usr/lib/ruby/1.8/irb.rb:70:in `catch'
    from /System/Library/Frameworks/Ruby.framework/Versions/1.8/usr/lib/ruby/1.8/irb.rb:70:in `start'
    from /usr/bin/irb:13

One pointer I got from Hosteurope is that s3anywhere for Android had the same issue in 1.7 but had it fixed in 1.8. Very thin, I know... :-(

@mmarschall

This comment has been minimized.

Copy link
Author

@mmarschall mmarschall commented Nov 2, 2011

The only thing s3anywhere says about the change from 1.7 to 1.8 is this:

"HostEurope storage support added."

Maybe worth asking them whether they want to share what exactly they did to make it work?

@geemus

This comment has been minimized.

Copy link
Member

@geemus geemus commented Nov 2, 2011

I'll ask around and see if I can find anything to go on.

@mmarschall

This comment has been minimized.

Copy link
Author

@mmarschall mmarschall commented Nov 2, 2011

Great. Please let me know if I can help...

Am 02.11.2011 um 16:14 schrieb Wesley Beary:

I'll ask around and see if I can find anything to go on.

Reply to this email directly or view it on GitHub:
#583 (comment)

@cvedel

This comment has been minimized.

Copy link

@cvedel cvedel commented Nov 3, 2011

Hello,

I believe the problem is in the specification of the endpoint. You specify:

:endpoint => 'https://cs.hosteurope.de'

This result in the value of path being an empty string instead of /. The string signed by fog is not valid because it has no resource and you get the SignatureDoesNotMatch error.
If instead, you use:

:endpoint => 'https://cs.hosteurope.de/'

it will work better.

Hope this helps,

Christophe

@mmarschall mmarschall closed this Nov 3, 2011
@mmarschall

This comment has been minimized.

Copy link
Author

@mmarschall mmarschall commented Nov 3, 2011

Wow, Christophe, great! That worked! Thanks a lot!

@geemus

This comment has been minimized.

Copy link
Member

@geemus geemus commented Nov 3, 2011

Ah, fog should probably add the slash if it is missing then (perhaps). Glad to know this is fixed in any event.

@mmarschall

This comment has been minimized.

Copy link
Author

@mmarschall mmarschall commented Nov 3, 2011

That might be an option...

@geemus

This comment has been minimized.

Copy link
Member

@geemus geemus commented Nov 4, 2011

I believe the following commit should prevent others from having this issue:
1baa2e8

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
3 participants
You can’t perform that action at this time.