Errno::ENOENT when "storage :fog" is set #688

Closed
rafakuch opened this Issue Apr 9, 2012 · 8 comments

Comments

Projects
None yet
5 participants

rafakuch commented Apr 9, 2012

This issue happens only when I set "storage :fog" in my Uploader class. When "storage :file" is set, everything goes fine.

I have a form with a "two-step submission" where the first step handles the image upload and the second one handles image cropping. I'm using image_cache as a hidden_field in my form in order to achieve that.

I also set config.delete_tmp_file_after_storage = false in my fog.rb initializer to see what's going on with the temp files:

In the first submission, CW caches the files in the following directories:

.../public/uploads/tmp/20120409-1646-23999-2221/medium_IMG.JPG
.../public/uploads/tmp/20120409-1646-23999-9408/IMG.JPG

But, in the second submission, it tries to get the "medium_IMG.JPG" file from the wrong directory (please see full trace below). Actually, when storage :file is set, CW caches another two files (because of the second step form submission) and everything works. But with storage :fog, CW doesn't cache the files again.

Errno::ENOENT in AvatarsController#update

No such file or directory - .../public/uploads/tmp/20120409-1646-23999-9408/medium_IMG.JPG

carrierwave (0.6.1) lib/carrierwave/sanitized_file.rb:157:in `initialize'
carrierwave (0.6.1) lib/carrierwave/sanitized_file.rb:157:in `open'
carrierwave (0.6.1) lib/carrierwave/sanitized_file.rb:157:in `read'
carrierwave (0.6.1) lib/carrierwave/storage/fog.rb:251:in `store'
carrierwave (0.6.1) lib/carrierwave/storage/fog.rb:84:in `store!'
carrierwave (0.6.1) lib/carrierwave/uploader/store.rb:59:in `block in store!'
carrierwave (0.6.1) lib/carrierwave/uploader/callbacks.rb:17:in `with_callbacks'
carrierwave (0.6.1) lib/carrierwave/uploader/store.rb:58:in `store!'
carrierwave (0.6.1) lib/carrierwave/uploader/versions.rb:237:in `block in store_versions!'
carrierwave (0.6.1) lib/carrierwave/uploader/versions.rb:237:in `each'
carrierwave (0.6.1) lib/carrierwave/uploader/versions.rb:237:in `store_versions!'
carrierwave (0.6.1) lib/carrierwave/uploader/callbacks.rb:18:in `block in with_callbacks'
carrierwave (0.6.1) lib/carrierwave/uploader/callbacks.rb:18:in `each'
carrierwave (0.6.1) lib/carrierwave/uploader/callbacks.rb:18:in `with_callbacks'
carrierwave (0.6.1) lib/carrierwave/uploader/store.rb:58:in `store!'
carrierwave (0.6.1) lib/carrierwave/mount.rb:345:in `store!'
carrierwave (0.6.1) lib/carrierwave/mount.rb:217:in `store_image!'
activesupport (3.2.1) lib/active_support/callbacks.rb:427:in `_run__767968874__save__101160981__callbacks'
activesupport (3.2.1) lib/active_support/callbacks.rb:405:in `__run_callback'
activesupport (3.2.1) lib/active_support/callbacks.rb:385:in `_run_save_callbacks'
activesupport (3.2.1) lib/active_support/callbacks.rb:81:in `run_callbacks'
activerecord (3.2.1) lib/active_record/callbacks.rb:264:in `create_or_update'
activerecord (3.2.1) lib/active_record/persistence.rb:84:in `save'
activerecord (3.2.1) lib/active_record/validations.rb:50:in `save'
activerecord (3.2.1) lib/active_record/attribute_methods/dirty.rb:22:in `save'
activerecord (3.2.1) lib/active_record/transactions.rb:241:in `block (2 levels) in save'
activerecord (3.2.1) lib/active_record/transactions.rb:295:in `block in with_transaction_returning_status'
activerecord (3.2.1) lib/active_record/connection_adapters/abstract/database_statements.rb:190:in `transaction'
activerecord (3.2.1) lib/active_record/transactions.rb:208:in `transaction'
activerecord (3.2.1) lib/active_record/transactions.rb:293:in `with_transaction_returning_status'
activerecord (3.2.1) lib/active_record/transactions.rb:241:in `block in save'
activerecord (3.2.1) lib/active_record/transactions.rb:252:in `rollback_active_record_state!'
activerecord (3.2.1) lib/active_record/transactions.rb:240:in `save'
activerecord (3.2.1) lib/active_record/persistence.rb:213:in `block in update_attributes'
activerecord (3.2.1) lib/active_record/transactions.rb:295:in `block in with_transaction_returning_status'
activerecord (3.2.1) lib/active_record/connection_adapters/abstract/database_statements.rb:190:in `transaction'
activerecord (3.2.1) lib/active_record/transactions.rb:208:in `transaction'
activerecord (3.2.1) lib/active_record/transactions.rb:293:in `with_transaction_returning_status'
activerecord (3.2.1) lib/active_record/persistence.rb:211:in `update_attributes'
app/controllers/avatars_controller.rb:31:in `update'
actionpack (3.2.1) lib/action_controller/metal/implicit_render.rb:4:in `send_action'
actionpack (3.2.1) lib/abstract_controller/base.rb:167:in `process_action'
actionpack (3.2.1) lib/action_controller/metal/rendering.rb:10:in `process_action'
actionpack (3.2.1) lib/abstract_controller/callbacks.rb:18:in `block in process_action'
activesupport (3.2.1) lib/active_support/callbacks.rb:436:in `_run__278112054__process_action__317663717__callbacks'
activesupport (3.2.1) lib/active_support/callbacks.rb:405:in `__run_callback'
activesupport (3.2.1) lib/active_support/callbacks.rb:385:in `_run_process_action_callbacks'
activesupport (3.2.1) lib/active_support/callbacks.rb:81:in `run_callbacks'
actionpack (3.2.1) lib/abstract_controller/callbacks.rb:17:in `process_action'
actionpack (3.2.1) lib/action_controller/metal/rescue.rb:29:in `process_action'
actionpack (3.2.1) lib/action_controller/metal/instrumentation.rb:30:in `block in process_action'
activesupport (3.2.1) lib/active_support/notifications.rb:123:in `block in instrument'
activesupport (3.2.1) lib/active_support/notifications/instrumenter.rb:20:in `instrument'
activesupport (3.2.1) lib/active_support/notifications.rb:123:in `instrument'
actionpack (3.2.1) lib/action_controller/metal/instrumentation.rb:29:in `process_action'
actionpack (3.2.1) lib/action_controller/metal/params_wrapper.rb:205:in `process_action'
activerecord (3.2.1) lib/active_record/railties/controller_runtime.rb:18:in `process_action'
actionpack (3.2.1) lib/abstract_controller/base.rb:121:in `process'
actionpack (3.2.1) lib/abstract_controller/rendering.rb:45:in `process'
actionpack (3.2.1) lib/action_controller/metal.rb:203:in `dispatch'
actionpack (3.2.1) lib/action_controller/metal/rack_delegation.rb:14:in `dispatch'
actionpack (3.2.1) lib/action_controller/metal.rb:246:in `block in action'
actionpack (3.2.1) lib/action_dispatch/routing/route_set.rb:66:in `call'
actionpack (3.2.1) lib/action_dispatch/routing/route_set.rb:66:in `dispatch'
actionpack (3.2.1) lib/action_dispatch/routing/route_set.rb:30:in `call'
journey (1.0.1) lib/journey/router.rb:68:in `block in call'
journey (1.0.1) lib/journey/router.rb:56:in `each'
journey (1.0.1) lib/journey/router.rb:56:in `call'
actionpack (3.2.1) lib/action_dispatch/routing/route_set.rb:589:in `call'
warden (1.1.0) lib/warden/manager.rb:35:in `block in call'
warden (1.1.0) lib/warden/manager.rb:34:in `catch'
warden (1.1.0) lib/warden/manager.rb:34:in `call'
actionpack (3.2.1) lib/action_dispatch/middleware/best_standards_support.rb:17:in `call'
rack (1.4.1) lib/rack/etag.rb:23:in `call'
rack (1.4.1) lib/rack/conditionalget.rb:35:in `call'
actionpack (3.2.1) lib/action_dispatch/middleware/head.rb:14:in `call'
actionpack (3.2.1) lib/action_dispatch/middleware/params_parser.rb:21:in `call'
actionpack (3.2.1) lib/action_dispatch/middleware/flash.rb:242:in `call'
rack (1.4.1) lib/rack/session/abstract/id.rb:205:in `context'
rack (1.4.1) lib/rack/session/abstract/id.rb:200:in `call'
actionpack (3.2.1) lib/action_dispatch/middleware/cookies.rb:338:in `call'
activerecord (3.2.1) lib/active_record/query_cache.rb:64:in `call'
activerecord (3.2.1) lib/active_record/connection_adapters/abstract/connection_pool.rb:443:in `call'
actionpack (3.2.1) lib/action_dispatch/middleware/callbacks.rb:28:in `block in call'
activesupport (3.2.1) lib/active_support/callbacks.rb:405:in `_run__846527525__call__101160981__callbacks'
activesupport (3.2.1) lib/active_support/callbacks.rb:405:in `__run_callback'
activesupport (3.2.1) lib/active_support/callbacks.rb:385:in `_run_call_callbacks'
activesupport (3.2.1) lib/active_support/callbacks.rb:81:in `run_callbacks'
actionpack (3.2.1) lib/action_dispatch/middleware/callbacks.rb:27:in `call'
actionpack (3.2.1) lib/action_dispatch/middleware/reloader.rb:65:in `call'
actionpack (3.2.1) lib/action_dispatch/middleware/remote_ip.rb:31:in `call'
actionpack (3.2.1) lib/action_dispatch/middleware/debug_exceptions.rb:16:in `call'
actionpack (3.2.1) lib/action_dispatch/middleware/show_exceptions.rb:56:in `call'
railties (3.2.1) lib/rails/rack/logger.rb:26:in `call_app'
railties (3.2.1) lib/rails/rack/logger.rb:16:in `call'
actionpack (3.2.1) lib/action_dispatch/middleware/request_id.rb:22:in `call'
rack (1.4.1) lib/rack/methodoverride.rb:21:in `call'
rack (1.4.1) lib/rack/runtime.rb:17:in `call'
activesupport (3.2.1) lib/active_support/cache/strategy/local_cache.rb:72:in `call'
rack (1.4.1) lib/rack/lock.rb:15:in `call'
actionpack (3.2.1) lib/action_dispatch/middleware/static.rb:53:in `call'
railties (3.2.1) lib/rails/engine.rb:479:in `call'
railties (3.2.1) lib/rails/application.rb:220:in `call'
rack (1.4.1) lib/rack/content_length.rb:14:in `call'
railties (3.2.1) lib/rails/rack/log_tailer.rb:14:in `call'
rack (1.4.1) lib/rack/handler/webrick.rb:59:in `service'
/home/rafael/.rvm/rubies/ruby-1.9.2-p318/lib/ruby/1.9.1/webrick/httpserver.rb:111:in `service'
/home/rafael/.rvm/rubies/ruby-1.9.2-p318/lib/ruby/1.9.1/webrick/httpserver.rb:70:in `run'
/home/rafael/.rvm/rubies/ruby-1.9.2-p318/lib/ruby/1.9.1/webrick/server.rb:183:in `block in start_thread'

Parameters:

{"utf8"=>"✓",
 "_method"=>"put",
 "authenticity_token"=>"GOWxcQqMOvkPyyZNU8xF/Fx2TMIHXU6TgGCvjPpJ4eg=",
 "avatar"=>{"image_cache"=>"20120409-1646-23999-9408/IMG.JPG",
 "crop_x"=>"0",
 "crop_y"=>"0",
 "crop_w"=>"470",
 "crop_h"=>"470"},
 "commit"=>"Update",
 "locale"=>"en",
 "user_id"=>"2"}

app/uploaders/image_uploader.rb

class ImageUploader < CarrierWave::Uploader::Base

  include CarrierWave::RMagick

  storage :fog
  #storage :file

  def store_dir
    "uploads/#{model.class.to_s.underscore}/#{mounted_as}/#{model.id}"
  end

  process :scale => [470, nil]

  def scale(width, height)
    resize_to_limit(width, height) if model.valid?
  end

  version :medium do
    process :crop
    resize_to_fill(150,150)
  end

  def crop
    if model.crop_x.present?
      resize_to_limit(470, nil)
      manipulate! do |img|
        x = model.crop_x.to_i
        y = model.crop_y.to_i
        w = model.crop_w.to_i
        h = model.crop_h.to_i
        img.crop!(x, y, w, h)
      end
    end
  end

  def extension_white_list
     %w(jpg jpeg gif png)
  end

end

/app/models/avatar.rb

class Avatar < ActiveRecord::Base

  mount_uploader :image, ImageUploader

  belongs_to  :avatarable, :polymorphic => true

  attr_accessor :crop_x, :crop_y, :crop_w, :crop_h

  after_save :crop_image

  def crop_image
    image.recreate_versions! if crop_x.present?
  end

end

I tried to investigate further but I'm not too experienced with Rails/Ruby...

Owner

bensie commented Apr 13, 2012

Can you please try this on master to see if it was fixed by #686?

I've got this exact same issue. I'm trying out master right now to see if it works.

Yep, that worked.

Owner

bensie commented Apr 13, 2012

🤘 Gem release coming shortly.

@bensie bensie closed this Apr 13, 2012

I digged in the code and figured out a solution. The code above is a simplified version of the one I'm working on, which generates versions conditionally when the second submission occurs using recreate_versions!. I guess this is related to #571. So, I applied two modifications regarding:

  1. store! in Fog class tries to store files that are not cached (that's why I was getting the Errno::ENOENT error)
  2. cache_stored_file! may raise "TypeError Exception: can't convert File into String" if storage is set to Fog

As I'm not much experienced, I don't feel comfortable to evaluate if this solution is reasonable (probably is not an elegant one), but it seems to work here. I will let the issue opened if someone would like to comment something...

module CarrierWave
  module Storage
    class Fog < Abstract
      def store!(file)
        f = CarrierWave::Storage::Fog::File.new(uploader, self, uploader.store_path)
        f.store(file) if file.size > 0
        f
      end
    end
  end

  module Uploader
    module Cache
      def cache_stored_file!
        opened_file = file.read.is_a?(String) ? file : File.open(file.read)
        sanitized = SanitizedFile.new :tempfile => StringIO.new(opened_file.read),
                                      :filename => File.basename(path),
                                      :content_type => file.content_type
        cache! sanitized
      end
    end
  end
end

@bensie you were pretty fast handling the issue! lol #686 didn't worked for me, maybe because of my conditional versions... But thanks anyway!

I had a similar issue with model validation errors, fog storage, and the cache files. With the recent gem updates, things seem to be working ok even after a validation error. However, some process methods weren't being called/reprocessed. Had to do the following to reprocess things after a validation error and when the file is already cached:

after :retrieve_from_cache, :do_something_to_model

I am running into the same problem. See my latest issue submission #843. What is :do_something_to_model?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment