Skip to content

Paperclip loops hundreds of times and times out #866

Closed
johnroa opened this Issue May 10, 2012 · 20 comments
@johnroa
johnroa commented May 10, 2012

SE thread: http://stackoverflow.com/questions/10142159/paperclip-s3-paperclip-saving-attachments-times-out

Rails 3.1.1
Heroku

When I access one particular model of my app in everything from updating attributes to creating to deleting, the logs show:

[paperclip] Saving attachments
[paperclip] Saving attachments
[paperclip] Saving attachments
[paperclip] Saving attachments
[paperclip] Saving attachments
[paperclip] Saving attachments

about 200 times, and then the app times out. It only happens on one particular model.

The paperclip config in the model is:

# File upload
has_attached_file :questimage, 
    :styles => {
        :original => {
            :geometry => "2048",
            :quality => 70
        },
        :full   =>  "933",
        :thumb => "300x200#",
        :small => "85x85#"
    },
    :convert_options => { :all => '-auto-orient' },
    :storage => :s3,
:s3_credentials => YAML.load_file('config/s3.yml'),
:path => ":attachment/:id_:style.:extension"

One in about five attempts everything works fine. The other four, the app times out and dies.

Here is the strange parts:

a) this only happens in production. In staging, which has the exact same environment config file, it works perfectly. It also works perfectly in development (all of which use s3).

b) this happens even when a file is not being manipulated, such as changing an attribute on the record from true to false.

The log do not reveal anything except the hundreds of "[paperclip] Saving attachments." before timing out.

Completely stuck on this. Guessing I've found an odd bug but am having a hard time tracing it.

Thank you!

@jyurek
thoughtbot, inc. member
jyurek commented May 11, 2012

Wow, even when there's no manipulation of the image it still does that? Can you set Paperclip.options[:log] = true and Paperclip.options[:log_command] = true in an initializer? This might give us more visibility.

Also, if you comment out the has_attached_file call, does the problem still happen?

@mike-burns
thoughtbot, inc. member

Hi @johnroa , which version of Ruby is this? Does it still happen?

Thanks,
-Mike

@yuvalkarmi

@johnroa here's my best guess: you have an after_update filter in your model that calls the paperclip reprocessing (maybe you're generating a thumbnail after letting the user crop the photo?).

I think you local machine behaves differently than heroku because you have different versions of paperclip installed on your local machine and on the server. My guess is that you didn't specify the exact version to use in your Gemfile.

Try updating paperclip on your local machine to the latest version and I bet my chips that you'll see the same problem.

So why is this problem caused with the newer version (which happens to run on heroku)?
The older version of paperclip doesn't update any attributes on the model after reprocessing, and so the processing occurs once and all is good. The newer version (which heroku installs) updates the PAPERCLIPATTACHMENT_updated_at column after reprocessing (replace PAPERCLIPATTACHMENT with the name of your attachment). That results in the reprocessing invoking the after_update filter again, and again, and again until heroku shuts it down.

The way I would solve it (if I'm correct about your setup) would be to nix the after_update filter and just call the paperclip reprocessing manually only when the user submits a crop request.

That way updating the model doesn't cause this infinite loop.

Does that help?

@YavorIvanov

We have almost the same problem occurring on our production server (non heroku related). The thing is we use after_post_process and after_save Still debugging it but thanks for your explanation. It helps us a lot to locate the problem.

Update: going back from 3.2.0 to 3.1.4 fixed the problem. It seems the problem also occurs on after_save

@johnroa
johnroa commented Sep 27, 2012

Hi everyone. Very sorry I didn't respond to this earlier. I wasn't getting the notifications.

Anyway, @ykarmi was close to what the final issue/solution was. I had an "audit" function running when certain kinds of records were saved, which touched every row in the database. Even though the paperclip image was not being affected, paperclip was still processing. On my development server, this was a non-issue because I was only working with a few hundred rows. On Heroku, I was working with a live data set of thousands of rows, and Heroku has a reasonably short timeout.

The fix was to put my audit function in a background process using DelayedJob.

Hope this helps!

@inetufo
inetufo commented Sep 30, 2012

@YavorIvanov I have the same issue, going back from 3.2.0 to 3.1.4 solves the problem...

@freegenie

Having the same issue here. So the latest verison if paperclip does not allow a reprocess to be wrapped in a transaction (inside the after_save cycle)?

@janx janx referenced this issue in janx/rails_admin_jcrop Nov 12, 2012
Closed

Stack Level Too Deep #5

@corwinstephen

Seems a little rude to interfere with the ability to use an after_update callback, since that's how so many Paperclip users handle cropping. Are there any plans to accomodate this in the future?

@tilsammans

@ykarmi thanks for the explanation! This was exactly the issue with me.

Doing the custom cropping in an after_update callback is everywhere as a "best practice" unfortunately, so this is bound to trip up more people. Doing the cropping in a specific controller action is probably better practice though, so doing that solved my problem.

@djcp
djcp commented Dec 7, 2012

Glad it's fixed, it sounds like the problem was (mostly) implementation related.

@djcp djcp closed this Dec 7, 2012
@jgarber
jgarber commented May 21, 2013

This is going to trip up everyone who ever followed RailsCast #182. I ran into this while upgrading paperclip.

@jgarber
jgarber commented May 21, 2013

This hack seems to work:

  def reprocess_photo
    photo.assign(photo)
    photo.save
  end
@krismeister

@jgarber , I was troubleshooting this all night. Thank you so much for the fix.

@YavorIvanov

Seems like it is still tripping people up. Maybe someone who knows RB should tell him to place a notice or something. I think this is the most common place to start after documentation so I will email him this thread.

@rohineematale

@jgarber .. Thank you very much.. 👍

@pettero
pettero commented Jul 4, 2013

@jgarber You got a beer with your name on it in Stockholm

@jgarber
jgarber commented Jul 6, 2013

@pettero Careful, I might take you up on it someday! 😄

@corwinstephen

@jgarber you've got a beer coming at you from me as well homie.

@osifoanosike

@jgarber you the best man! ur comment still very relevant!

@melsatar

@jgarber, Great work, you saved us. Brilliant hack

@nanosplit nanosplit referenced this issue in awijeet/Image_cropping_in_rails4 May 23, 2016
@nanosplit nanosplit the magic crop on update 960277e
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Something went wrong with that request. Please try again.