thoughtbot / paperclip

Easy file attachment management for ActiveRecord — Read more

Filtered By:

Sort By: Priority | Votes | Last Updated

Loading…

32.

6 votes

feature
x

s3
x

patch
x

Patch to allow per style S3 permissions
2 comments Created 8 months ago by qrush

Reported by Andrew

I have forked and patched paperclip on github at http://github.com/andrewtimberlake/paperclip/commit/5b2f2e6fb6abf9625cbf50cd59f86267cfc0a83d

This change allows you to set a hash for s3_permissions like
:s3_permissions => {:original => 'private', :thumb => 'public-read'} This is useful for a situation where a cropped thumbnail needs to be quick to download but the original file needs to be protected (and it's download might be managed through the web app itself instead of directly off S3)
You can also set a :default key in your permissions hash which will apply to all styles except those specifically stated.

The original functionality is unchanged as s3_permissions can still accept a string which will apply to all styles.

Includes tests

Comments

dosire Fri Sep 04 05:39:22 -0700 2009 | link

This is very useful, please implement soon.

Comment
This is very useful, please implement soon.
or cancel

heimidal Fri Oct 23 09:57:05 -0700 2009 | link

This would be a really great feature to have included in Paperclip.

Comment
This would be a really great feature to have included in Paperclip.
or cancel

Please log in to comment.
37.

3 votes

bug
x

s3
x

patch
x

to_file method for s3 Storage
5 comments Created 8 months ago by qrush
Reported by Giovanni Cangiani

Hi,

the to_file method for s3 Storage does not return a File object but an RightAws::S3::Key instead. This is consistent with the comment in the code (in the format most representative of the current storage), but not with what the reprocess! method in Attachment expects (an object that responds to "read" method).

I am willing to submit a patch for this but there are two ways and I'd like to know which one you prefer.
1. actually write the s3_bucket.key.data into a Tempfile and return the file
2. implement read/write methods for RightAws::S3::Key
What do you think ?

Last question. Is there any foundamental reason for chosing RightAWS instead of the usual AWS::S3 ?

Thanks,

giovanni

Comments
qrush Fri Jun 05 13:43:58 -0700 2009 | link

Comment from Jon Yurek

Well, considering the method is to_file, it seems like making a Tempfile would be the right way, but in the interest of performing as few transactions over the network as possible, having the Key object respond properly to #read is likely better.

As for RightAWS... I don't have a specific reason anymore, although I did when I wrote it and I can't recall what it was and I don't have enough reason to change it now that it's written. :) Why, is there something the matter with RightAWS?

Comment
**Comment from Jon Yurek** Well, considering the method is to_file, it seems like making a Tempfile would be the right way, but in the interest of performing as few transactions over the network as possible, having the Key object respond properly to #read is likely better. As for RightAWS... I don't have a specific reason anymore, although I did when I wrote it and I can't recall what it was and I don't have enough reason to change it now that it's written. :) Why, is there something the matter with RightAWS?
or cancel
qrush Fri Jun 05 13:45:58 -0700 2009 | link
Comments from James McKinney

the Key object has a #get method, which works just like #read on File. All you would have to do is create an alias (the Key object has no #read method atm).

RightAws has some advantages over AWS::S3. For example, RightAws can connect to S3 Europe. AWS::S3 can't. RightAws makes it very easy to iterate over keys and "folders" on S3. RightAws will reconnect to S3 if S3 gives a 500 Error; AWS::S3 just fails out. RightAws in general just has a lot more methods than AWS::S3 (delete_folder, copying/moving between buckets, including US to EU, ...).

RightAws also implements the EC2, SQS, etc. APIs, so for teams who want to save on gems, RightAws is better than AWS::S3.

=======

See my comment in ticket #33: http://thoughtbot.lighthouseapp.com/projects/8794-paperclip/tickets/33#ticket-33-8

The example I give in that comment shows that, between the two options given by the reporter, the correct solution must be to write the data to a Tempfile, not just to implement #read and #write on RightAws::S3::Key. The example again is:

@user = User.new @user.avatar = User.find(1).avatar.to_file @user.save

A third alternative would require rewriting either Attachment#assign or its dependent methods to handle the assignment of a RightAws::S3::Key, which seems much more involved that just patching the #to_file method.

=======

Actually, speaking of that third alternative, it seems like if we can make it possible for RightAws::S3::Key to #include IOStream, and for it to fulfill the tests in #valid_assignment?, then we'll be on our way to a proper solution.
Comment
**Comments from James McKinney** the Key object has a #get method, which works just like #read on File. All you would have to do is create an alias (the Key object has no #read method atm). RightAws has some advantages over AWS::S3. For example, RightAws can connect to S3 Europe. AWS::S3 can't. RightAws makes it very easy to iterate over keys and "folders" on S3. RightAws will reconnect to S3 if S3 gives a 500 Error; AWS::S3 just fails out. RightAws in general just has a lot more methods than AWS::S3 (delete_folder, copying/moving between buckets, including US to EU, ...). RightAws also implements the EC2, SQS, etc. APIs, so for teams who want to save on gems, RightAws is better than AWS::S3. ======= See my comment in ticket #33: http://thoughtbot.lighthouseapp.com/projects/8794-paperclip/tickets/33#ticket-33-8 The example I give in that comment shows that, between the two options given by the reporter, the correct solution must be to write the data to a Tempfile, not just to implement #read and #write on RightAws::S3::Key. The example again is: <pre> @user = User.new @user.avatar = User.find(1).avatar.to_file @user.save </pre> A third alternative would require rewriting either Attachment#assign or its dependent methods to handle the assignment of a RightAws::S3::Key, which seems much more involved that just patching the #to_file method. ======= Actually, speaking of that third alternative, it seems like if we can make it possible for RightAws::S3::Key to #include IOStream, and for it to fulfill the tests in #valid_assignment?, then we'll be on our way to a proper solution.
or cancel
qrush Fri Jun 05 13:46:19 -0700 2009 | link

Comment from Jon Yurek

I think including the IOStream methods would probably work best. Getting the Key as a File would necessitate downloading it right then, which isn't always what you want (but that does make "to_file" a bit of a misnomer).

Comment
**Comment from Jon Yurek** I think including the IOStream methods would probably work best. Getting the Key as a File would necessitate downloading it right then, which isn't always what you want (but that does make "to_file" a bit of a misnomer).
or cancel

qrush Fri Jun 05 13:46:49 -0700 2009 | link

Comment from Jerry Cheung

For our use, we only call to_file when we actually wanted the file downloaded for processing. I've attached the Tempfile fix described in the first comment in case someone needs it before the final optimized fix is available.

https://thoughtbot.lighthouseapp.com/attachments/90378/s3_storage_to_file.patch

Comment
**Comment from Jerry Cheung** For our use, we only call to_file when we actually wanted the file downloaded for processing. I've attached the Tempfile fix described in the first comment in case someone needs it before the final optimized fix is available. https://thoughtbot.lighthouseapp.com/attachments/90378/s3_storage_to_file.patch
or cancel

qrush Fri Jun 05 13:49:07 -0700 2009 | link

Comment from timcowlishaw

FYI - i've forked the paperclip gitrepo and applied this patch here, if it's useful for anyone.

http://github.com/timcowlishaw/paperclip/tree/master

Comment
**Comment from timcowlishaw** FYI - i've forked the paperclip gitrepo and applied this patch here, if it's useful for anyone. http://github.com/timcowlishaw/paperclip/tree/master
or cancel

Please log in to comment.

13.

4 votes

s3
x

bug
x

SystemStackError with S3

8 comments Created 8 months ago by qrush

Reported by sam:

Hi,

The latest version of of paperclip (as of 2009-05-21) has a major issue with S3 storage. Attempts to do anything related to paths or URLs (so basically everything) result in a SystemStackError: stack level too deep.

Here's a sample of output:

SystemStackError: stack level too deep
    from /Users/sam/workspace/FloresFrescas.com/vendor/plugins/paperclip/lib/paperclip/storage.rb:149:in `s3_path_url'
    from /Users/sam/workspace/FloresFrescas.com/vendor/plugins/paperclip/lib/paperclip/interpolations.rb:31:in `send'
    from /Users/sam/workspace/FloresFrescas.com/vendor/plugins/paperclip/lib/paperclip/interpolations.rb:31:in `interpolate'
    from /Users/sam/workspace/FloresFrescas.com/vendor/plugins/paperclip/lib/paperclip/interpolations.rb:30:in `gsub'
    from /Users/sam/workspace/FloresFrescas.com/vendor/plugins/paperclip/lib/paperclip/interpolations.rb:30:in `interpolate'
    from /Library/Ruby/Gems/1.8/gems/activerecord-2.2.2/lib/active_record/attribute_methods.rb:211:in `inject'
    from /Users/sam/workspace/FloresFrescas.com/vendor/plugins/paperclip/lib/paperclip/interpolations.rb:29:in `each'
    from /Users/sam/workspace/FloresFrescas.com/vendor/plugins/paperclip/lib/paperclip/interpolations.rb:29:in `inject'
    from /Users/sam/workspace/FloresFrescas.com/vendor/plugins/paperclip/lib/paperclip/interpolations.rb:29:in `interpolate'
    from /Users/sam/workspace/FloresFrescas.com/vendor/plugins/paperclip/lib/paperclip/attachment.rb:391:in `interpolate'
    from /Users/sam/workspace/FloresFrescas.com/vendor/plugins/paperclip/lib/paperclip/attachment.rb:103:in `url'
    from /Users/sam/workspace/FloresFrescas.com/vendor/plugins/paperclip/lib/paperclip/interpolations.rb:46:in `url'
    from /Users/sam/workspace/FloresFrescas.com/vendor/plugins/paperclip/lib/paperclip/interpolations.rb:31:in `send'
    from /Users/sam/workspace/FloresFrescas.com/vendor/plugins/paperclip/lib/paperclip/interpolations.rb:31:in `interpolate'
    from /Users/sam/workspace/FloresFrescas.com/vendor/plugins/paperclip/lib/paperclip/interpolations.rb:30:in `gsub'
    from /Users/sam/workspace/FloresFrescas.com/vendor/plugins/paperclip/lib/paperclip/interpolations.rb:30:in `interpolate'
... 6074 levels...
    from /Users/sam/workspace/FloresFrescas.com/vendor/plugins/paperclip/lib/paperclip/interpolations.rb:29:in `inject'
    from /Users/sam/workspace/FloresFrescas.com/vendor/plugins/paperclip/lib/paperclip/interpolations.rb:29:in `interpolate'
    from /Users/sam/workspace/FloresFrescas.com/vendor/plugins/paperclip/lib/paperclip/attachment.rb:391:in `interpolate'
    from /Users/sam/workspace/FloresFrescas.com/vendor/plugins/paperclip/lib/paperclip/attachment.rb:112:in `path'
    from /Users/sam/workspace/FloresFrescas.com/vendor/plugins/paperclip/lib/paperclip/storage.rb:149:in `s3_path_url'
    from /Users/sam/workspace/FloresFrescas.com/vendor/plugins/paperclip/lib/paperclip/interpolations.rb:31:in `send'
    from /Users/sam/workspace/FloresFrescas.com/vendor/plugins/paperclip/lib/paperclip/interpolations.rb:31:in `interpolate'
    from /Users/sam/workspace/FloresFrescas.com/vendor/plugins/paperclip/lib/paperclip/interpolations.rb:30:in `gsub'
    from /Users/sam/workspace/FloresFrescas.com/vendor/plugins/paperclip/lib/paperclip/interpolations.rb:30:in `interpolate'
    from /Library/Ruby/Gems/1.8/gems/activerecord-2.2.2/lib/active_record/attribute_methods.rb:211:in `inject'
    from /Users/sam/workspace/FloresFrescas.com/vendor/plugins/paperclip/lib/paperclip/interpolations.rb:29:in `each'
    from /Users/sam/workspace/FloresFrescas.com/vendor/plugins/paperclip/lib/paperclip/interpolations.rb:29:in `inject'
    from /Users/sam/workspace/FloresFrescas.com/vendor/plugins/paperclip/lib/paperclip/interpolations.rb:29:in `interpolate'
    from /Users/sam/workspace/FloresFrescas.com/vendor/plugins/paperclip/lib/paperclip/attachment.rb:391:in `interpolate'
    from /Users/sam/workspace/FloresFrescas.com/vendor/plugins/paperclip/lib/paperclip/attachment.rb:103:in `url'

I'm guessing this has something to do with the new default style code. When I reverted to a version of the plugin from March (commit a1b39cc65) I had no problems. I've not tried any other revisions.

Cheers,
sam

Okay, I managed to stumble upon a fix. Simply by adding the :path to the has_attached_file it operates as it should do. I didn't realise this was a requirement, but after testing older versions and getting problems with saving to S3 I found adding :path was the only way to get anything to work.

Cheers, sam

Comments

nirvdrum Tue Aug 11 10:54:26 -0700 2009 | link

Confirmed.

scsmith Thu Aug 27 04:11:06 -0700 2009 | link

This is still a problem. Just for clarification does specifying :path option cause the :url option to be ignored thus preventing cloud front and domain style urls?

ghazel Tue Sep 08 00:14:34 -0700 2009 | link

Same problem, on paperclip 2.3.1

liamks Wed Oct 14 19:28:00 -0700 2009 | link

Confirmed with 2.3.1.1

dustmoo Wed Nov 04 07:25:43 -0800 2009 | link

Specifying the :path option does prevent paperclip from entering the never-ending loop.

i.e: :path => ":attachment/:id/:style/:filename"

digx Mon Nov 09 01:31:46 -0800 2009 | link

confirmed here when using s3

rebelcolony Wed Nov 25 01:07:06 -0800 2009 | link

sorry, how would i add path to:

has_attached_file :image, :storage => :s3, :s3_credentials => "#{RAILS_ROOT}/config/s3.yml", :styles => { :thumb => "190x124>", :large => "400x300>" }

rebelcolony Wed Nov 25 01:22:37 -0800 2009 | link

ok just added the :path so
has_attached_file :image, :storage => :s3, :s3_credentials => "#{RAILS_ROOT}/config/s3.yml", :path => ":attachment/:id/:style.:extension", :styles => { :thumb => "190x124>", :large => "400x300>" }
and seems to work.

Please log in to comment.

15.

3 votes

s3
x

feature
x

Using Rails' Built-in Asset Hosts for S3-stored Attachments
0 comments Created 8 months ago by qrush

Reported by Marshall Sontag

The Problem:

I recently decided to move all of my assets, including paperclip attachments, to S3/Cloudfront. To take advantage of Rails' built-in support for multiple asset hosts, I created 4 CNAME subdomains (assets0.domain.com, assets1.domain.com, etc) that all forwarded to Cloudfront. However, because Paperclip was set to s3 storage, it would only return an S3-based URL, or it would allow me to specify only a single CNAME host. I could not use Rails' built-in asset_host settings to serve my assets.

The Solution:

Rails' built-in image_tag helper (more specifically, the image_path helper) will use the asset_host setting in your environment (if set) to calculate what the URL should be. In order to do this, it needs to be handed a relative URL (i.e. "photos/1/image.jpg"). Paperclip is not currently equipped to return a relative URL when using S3 Storage, so it needs this ability. Toward this end, I created a 4th option for the S3 :url setting (which I have called :asset_host) that returns a relative path so that Rails can do the proper URL calculation.

The Patch:

Fresh commit, complete with leading slash and 2 different asset host settings. One with bucket for S3, one without bucket for Cloudfront:

http://github.com/marshmallowcreme/paperclip/tree/master

Comments

Please log in to comment.
9.

-1 votes

s3
x

bug
x

Copying s3-based attachments
1 comment Created 8 months ago by qrush
Reported by Dylan Markow

Using S3, it doesn't seem that copying attachments from one record to another works. The comments in attachment.rb state that "In addition to form uploads, you can also assign another Paperclip attachment: new_user.avatar = old_user.avatar", but when I try this, the second record's attachment remains empty and nothing gets uploaded to amazon's servers.
```
a = Post.find(1)
b = Post.find(2)
a.attachment_file_name
# => "Attachment.pdf"
b.attachment_file_name
# => nil
b.attachment = a.attachment
b.save
b.attachment_file_name
# => nil
```
Comments
briandunn Fri Jul 17 08:19:33 -0700 2009 | link
came across this too. here is a failing test:

context "photo" do setup {add_photo_to_person; @person.save} should "be reasignable" do assert @person.photo.file? other_dude = people(:turk) other_dude.photo = @person.photo other_dude.save! assert other_dude.photo.file? end

end
other_dude.photo.file? fails when paperclip is configured for s3.
Comment
came across this too. here is a failing test: context "photo" do setup {add_photo_to_person; @person.save} should "be reasignable" do assert @person.photo.file? other_dude = people(:turk) other_dude.photo = @person.photo other_dude.save! assert other_dude.photo.file? end end other_dude.photo.file? fails when paperclip is configured for s3.
or cancel
Please log in to comment.
10.

1 vote

feature
x

s3
x

Support additional mime types
7 comments Created 8 months ago by qrush

Reported by thibaut Assus

In file lib/paperclip/upfile.rb Module Upfile method content_type I was wondering if it was possible to add our proper mime types (as mp3) It would be wonderful !

Comments

qrush Fri Jun 05 08:38:09 -0700 2009 | link

Comments from S. Brent Faulkner:

Not directly intended to fix this but a patch I submitted for resolution to #144 adds support for Rack::Mime (if loaded)... this commit adds many extra mime types...

http://github.com/sbfaulkner/paperclip/commit/dd773d20d3fc555f48440c5aa6130af565df448b

Comment
Comments from S. Brent Faulkner: Not directly intended to fix this but a patch I submitted for resolution to #144 adds support for Rack::Mime (if loaded)... this commit adds many extra mime types... http://github.com/sbfaulkner/paperclip/commit/dd773d20d3fc555f48440c5aa6130af565df448b
or cancel

qrush Fri Jun 05 08:38:54 -0700 2009 | link

Comments from Sam Stokes:

Relatedly, I needed Paperclip::Upfile to know about the application/pdf MIME type, so this one-line commit 89dcdd50... adds that. It also demonstrates that it's really easy to add new MIME types.

My understanding is that Paperclip::Upfile's ability to guess the content type doesn't normally get used, because if you upload something from a web form, then the browser supplies the content type and Paperclip just uses that. So this mostly comes up during testing (e.g. using webrat to fake out the browser).

Also (sorry for the double post but this might be useful to some people), here's a quick and dirty monkey-patch to stick in a suitable place (like features/support/env.rb) if you need to quickly add a new content-type.

Comment
Comments from Sam Stokes: Relatedly, I needed Paperclip::Upfile to know about the application/pdf MIME type, so this one-line commit [89dcdd50...](http://github.com/samstokes/paperclip/commit/89dcdd50db4d01610404c3ff03e56d71278d0716) adds that. It also demonstrates that it's really easy to add new MIME types. My understanding is that Paperclip::Upfile's ability to guess the content type doesn't normally get used, because if you upload something from a web form, then the browser supplies the content type and Paperclip just uses that. So this mostly comes up during testing (e.g. using webrat to fake out the browser). Also (sorry for the double post but this might be useful to some people), here's a [quick and dirty monkey-patch](http://gist.github.com/118590) to stick in a suitable place (like `features/support/env.rb`) if you need to quickly add a new content-type.
or cancel

qrush Fri Jun 05 08:41:23 -0700 2009 | link

Comments from Paul Cortens:

This line of code in storage.rb sets the content-type on S3 to the content-type of the originally uploaded file. If the file has been converted to another image format, this is not accurate.

key.put(nil, @s3_permissions, {'Content-type' => instance_read(:content_type)})

I am guessing that we could use a method similar to content_type in upfile.rb to detect the correct content type from the format specified in the style hash. Or we could have an option to explicitly set the content-type for each style in the style hash.

What do you think?

Comment
Comments from Paul Cortens: This line of code in storage.rb sets the content-type on S3 to the content-type of the originally uploaded file. If the file has been converted to another image format, this is not accurate. `key.put(nil, @s3_permissions, {'Content-type' => instance_read(:content_type)})` I am guessing that we could use a method similar to content_type in upfile.rb to detect the correct content type from the format specified in the style hash. Or we could have an option to explicitly set the content-type for each style in the style hash. What do you think?
or cancel

korobkov Tue Sep 22 04:30:18 -0700 2009 | link

I made a fork (http://github.com/korobkov/paperclip), where I I've added support for MIME-type resolution by magic, rather than extension, using http://github.com/minad/mimemagic gem.
It was written on pure ruby, so it will also work on non-posix-compliant systems without /usr/bin/file utility.

Comment
I made a fork (http://github.com/korobkov/paperclip), where I I've added support for MIME-type resolution by magic, rather than extension, using http://github.com/minad/mimemagic gem. It was written on pure ruby, so it will also work on non-posix-compliant systems without `/usr/bin/file` utility.
or cancel

masterkain Sun Nov 29 01:32:49 -0800 2009 | link

still no love..

Comment
still no love..
or cancel

korobkov Mon Nov 30 04:27:45 -0800 2009 | link

I've switched MIME-type resolution from http://github.com/minad/mimemagic to http://github.com/Narnach/file_wrapper, which wraps /usr/bin/file...

Comment
I've switched MIME-type resolution from http://github.com/minad/mimemagic to http://github.com/Narnach/file_wrapper, which wraps /usr/bin/file...
or cancel

monde Sun Dec 27 13:26:07 -0800 2009 | link

I implemented correct content_type detection with file --mime-type command:
http://github.com/monde/paperclip/commit/2ca52df1aa45a8a9cb3d086a6be77168b7f5be52

The file command is able to determine a wide range of content types beyond what is possible with a simple check of a file name's extention.

Comment
I implemented correct content_type detection with file --mime-type command: http://github.com/monde/paperclip/commit/2ca52df1aa45a8a9cb3d086a6be77168b7f5be52 The file command is able to determine a wide range of content types beyond what is possible with a simple check of a file name's extention.
or cancel

Please log in to comment.