Skip to content

AWS::S3::Errors::SignatureDoesNotMatch #817

dbachet opened this Issue Apr 7, 2012 · 27 comments

10 participants

dbachet commented Apr 7, 2012

I'm migrating my app from Rails 3.0.9 to 3.2.3 / ruby 1.9.3. I was using paperclip(2.3.11) with aws-s3 gem to store my pictures.

Now I want to use the last version of paperclip(3.0.1) with aws-sdk gem.

When trying to upload a file, I get the error:

The request signature we calculated does not match the signature you provided. Check your key and signing method.

It's creating the original style image on the S3 bucket, but not the other styles.

Here is also what I get from console:

    [paperclip] Saving attachments.
    [paperclip] saving /presentation_pictures/43/original/Image_1.png
    [AWS S3 200 0.813272 0 retries] put_object(:acl=>:public_read,:bucket_name=>"*******_dev",:content_type=>"image/png",:data=>#<Paperclip::UploadedFileAdapter:0x2e144b4 @target=#<ActionDispatch::Http::UploadedFile:0x2dbb1fc @original_filename="Image 1.png", @content_type="image/png", @headers="Content-Disposition: form-data; name=\"presentation_picture[picture]\"; filename=\"Image 1.png\"\r\nContent-Type: image/png\r\n", @tempfile=#<File:/var/folders/tb/tbmv4LE9EwGNPr1QqU2S0E+++TI/-Tmp-/RackMultipart20120407-33502-1gajfe9>>, @tempfile=#<File:/var/folders/tb/tbmv4LE9EwGNPr1QqU2S0E+++TI/-Tmp-/Image 1.png20120407-33502-an4dcy>>,:key=>"presentation_pictures/43/original/Image_1.png")

A downgrade to paperclip 3.0.0 solved the problem. That made me think that it can be a paperclip issue ? :s

So I write here :)

I've set my aws.yml file as:

    development: &development
      bucket: bucket_name_for_dev
      access_key_id: *****
      secret_access_key: *******
      <<: *development
      bucket: bucket_name_for_staging

      <<: *development
      bucket: bucket_name_for_prod

my model contains :

    has_attached_file :picture, :styles => { :medium => "300x300>", :thumb => "50x50>" }, :default_url => 'picture_missing.png',
         :storage => :s3,
         :bucket => "bucket_name_for_dev",
         :s3_credentials => Rails.root.join("config/aws.yml"),
         :path => "/presentation_pictures/:id/:style/:filename",
         :url  => ":s3_eu_url"

    attr_accessible :picture
    attr_accessor :picture_url

I'll stick to 3.0.0, then it's not a prior problem to me. But if I can help you, don't hesitate to ask me information if you think it's a paperclip issue.

Today, I'm not able to launch tests as... I don't know how :/ But I could try!

Issue happened with:

Ruby 1.9.3p125
Rails 3.2.3
ImageMagick 6.6.9-4
paperclip 3.0.1
aws-sdk 1.3.9
nokogiri 1.4.7

Thx a lot for your job!


I had this same issue this afternoon after upgrading. Downgrading to 3.0.0 worked for me.

thoughtbot, inc. member
sikachu commented Apr 16, 2012

Hi guys,

Did Paperclip 3.0.2 solve this problem for you? I think this is related to the binary file open.


I was on master 6 days ago when it was failing (had to be on master because I fell victim to the Rack::Test::UploadedFile problem). Do you think a commit in the last few days has fixed it? I can give it a shot today/tomorrow.

thoughtbot, inc. member
sikachu commented Apr 16, 2012

Do you mind giving me the ref you're pointing to? (in your Gemfile.lock)


Good call. I reverted from a2a4c7a

dbachet commented Apr 16, 2012


By my side, the 3.0.2 version didn't fix the problem. :(

thoughtbot, inc. member
sikachu commented Apr 17, 2012

Hey guys,

I've been trying to replicate your problems on my side for a while, using the newly generated app with Paperclip from current master. I can't really replicate the problem, and not sure what caused the issue.

So, looking from the configuration, are you guys both using the EU bucket? Did you guys set the host value in your configuration?

I really would like to resolve this problem for you guys. Do you mind creating a app that replicate the problem? Also, the new version of aws-sdk was released as well, so you might want to try it to see if it fixes your problem.

Thank you for your patience.


My configuration is simply this. My buckets are all in the US Standard region (US-EAST-1, I believe). I am on aws-sdk 1.3.9.

{:storage => :s3,
                                    :bucket => "mybucketname",
                                    :s3_credentials => ...,
                                    :path => ":class/:attachment/:hash/:style.:extension",
                                    :hash_secret => ...}

What host value should I try configuring?

thoughtbot, inc. member

The same issue started popping up for me today. I get validation errors if I fall back to 3.0.0. Anyone have a solution for this?


The following fails for me unless I remove the line marked with OFFENDING LINE in the comment:

VALID_CONTENT_TYPES = %w(image/jpg image/jpeg image/png image/gif)
has_attached_file :logo,
:styles => {:medium => '300x300>'}, # OFFENDING LINE
:storage => :s3,
:s3_credentials => Rails.root.join("config","s3.yml"),
:path => "/company_logos/:id/:filename"
validates_attachment_content_type :logo, :content_type => VALID_CONTENT_TYPES, :message => "should be \"jpg, jpeg, png or gif\""

I hope this helps find the error.

granoff commented Apr 20, 2012

Fwiw, I am experience this problem as well. I tried v3.0.0 but it ddn't work locally (my Mac). When I went to 3.0.2, the problem went away locally, but when I pushed to Heroku, the error came back. I cannot remove my :styles directive as it is rather complex and important to how we use S3. I am at a loss as to what to try next. Hopefully, some solution will become apparent! :-)

granoff commented Apr 20, 2012

In a final act of desperation (it was late :-) ) I heeded a warning in the readme that using the master branch as the gem source, rather than a released version, might resolve issues.... so I tried it, and it worked!

That doesn't leave me with a really warm feeling because for a production system I'd rather not rely on a bleeding edge repo, but that's the nature of this ecosystem, isn't it? :-)

The issue noted in the above link (a fork of paperclip) addresses something which may or may not be part of this repo... I have not yet culled the most recent commits for it.

narbo commented Apr 21, 2012

Hello, I think I encountered the same issue (as far as I understand) and managed to get everything to work perfectly fixing the following line (NicoArbogast/paperclip@66441fc). I found the explanation/answer in here igor-alexandrov/paperclip-aws#10.

thoughtbot, inc. member
jyurek commented Apr 23, 2012

@granoff, we try to release regularly (bi-weekly, actually), so if master works for you now it should be in a released version soon. @garyharan, does master solve the problem for you as well?

@NicoArbogast, that's great that that solves the problem. Do you happen to have a test for that change, though? I'd have to assume not since we had such a hard time even reproducing the problem.

narbo commented Apr 24, 2012

Hi, I am sorry I don't have any test for that change. It was an issue I encountered only on my prod env. I tried the patch following the discussion here igor-alexandrov/paperclip-aws#10 and it worked. I'm sorry I did not have time to dig deeper into this issue.

thoughtbot, inc. member
sikachu commented Apr 24, 2012

Oh, I think I can see this problem on *nix system. Let me check if we have covered it.

So, the reason is that using file command in Linux would return this:

> `file -b --mime-type paperclip.xls`
 => "application/\n" 

We need to strip that \n.


The only place I had the issue was on my staging server and right now I have users testing it so I won't be able to check until tomorrow.

thoughtbot, inc. member
thoughtbot, inc. member
sikachu commented Apr 24, 2012

I've add a test to prevent regression in 3d814fb. 3.0.3 will go out soon.

@sikachu sikachu closed this Apr 24, 2012
jpadvo commented Jun 2, 2012

The patch @NicoArbogast recommended works for me, but it doesn't look like it is included in 3.03 or 3.04, I had to use his repo or download the gem and apply the fix myself.

@sikachu sikachu reopened this Jun 2, 2012
thoughtbot, inc. member
sikachu commented Jun 2, 2012

Reopen this. I'm going to have a look at this one.

@sikachu sikachu added a commit that referenced this issue Jun 8, 2012
@sikachu sikachu Make sure no newline from type_from_file_command
This relates to #817
thoughtbot, inc. member
sikachu commented Jun 8, 2012

@jpadvo I've tried to reproduce this and still unable to. I've also added a test case to make sure that we have stripped newline character from file content type.

Can I have more information from you?

  • What's the OS are you using?
  • What version of file command (file --version)
  • Can you test this command against a file in your directory, and post me the result?
    ruby -e "puts \`file -b --mime test.png\`.inspect"

Thank you.

jpadvo commented Jun 8, 2012

Sure, thanks for looking into it!

I'm using OS X Lion.

Output of the first command:

$ file --version
magic file from /usr/share/file/magic

...and the second, on two different formats of image file:

$ ruby -e "puts \`file -b --mime uZXKQ.jpeg\`.inspect"
"image/jpeg; charset=binary\n"

$ ruby -e "puts \`file -b --mime dZQMk.png\`.inspect"
"image/png; charset=binary\n"
nzifnab commented Aug 28, 2012

Hrm... did this ever get fixed? I've upgraded one of our apps to Rails 3.2 and have been running into this error with S3 on both heroku, and locally on my Mac. I followed the stacktrace to this line in paperclip:

Where I logged the value of write_options and see that it is:

write_options = {:content_type=>"image/jpeg\r", :acl=>:public_read}

Changing the linked line from paperclip's gem to:

write_options = {
  content_type => file.content_type.to_s.chomp,
  :acl => acl

Fixes the problem for me. Looking above at this issue it seems like people had already pinpointed the source of the problem (content_type was getting a newline character), is there a specific reason it was never fixed in paperclip's code?

nzifnab commented Aug 28, 2012

After further exploration I've narrowed the root of the problem to this:

Where that @target variable is:

#<ActionDispatch::Http::UploadedFile:0x0000010341a218 @original_filename="myimg.png", @content_type="image/png\r", @headers=...<SNIP>
@nzifnab nzifnab added a commit to nzifnab/paperclip that referenced this issue Aug 28, 2012
@nzifnab nzifnab Rack gets a newline at the end of the content_type in certain instanc…
…es: 'image/jpeg\r' which causes S3 to be unable to accept the file. Fixes #817
nzifnab commented Aug 30, 2012

Hmm. We have two versions of file upload: One of them sends the file to another one of our apps via Net::HTTP::Multipart (something like that), and then that core app handles the upload with paperclip, and that is the one that was getting the carriage return appended to content_type. This was fixed by pointing the paperclip gem to my fork (with the fix above). We upgraded another of our apps which also does uploads, but does them right in the app (not sending them to the core service like the first app). I forgot to upgrade the gem to point to my fork but it appears to be working just fine. They both send to S3, and they both use the same S3 credentials and such.

Maybe it's the Net::HTTP version of the file that's getting a newline added at some time during the process. Curious.

@mike-burns mike-burns added a commit that closed this issue Sep 7, 2012
@nzifnab nzifnab Rack gets a newline at the end of the content_type in certain instanc…
…es: 'image/jpeg\r' which causes S3 to be unable to accept the file. Fixes #817
@mike-burns mike-burns closed this in eb92834 Sep 7, 2012
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Something went wrong with that request. Please try again.