Spree occasionally stops serving Amazon S3 images

[freshnewegg]

I set up Amazon s3 to store images and it works successfully as detailed on http://guides.spreecommerce.org/developer/s3_storage.html. However, occasionally, spree stops serving the s3 url's and starts serving local ones.

Context

I'm trying to serve product information (specifically the image url) via the spree api. I'm making call toward the api/v1/products endpoint.

Expected Behavior

I was expecting the server to always serve the s3 image url even if the network has problems. It shouldn't revert to local storage.

Actual Behavior

After the server runs for a while, it randomly starts serving local images.

Possible Fix

Restarting the server makes it serve s3 images again but this is not a realistic solution

Steps to Reproduce

Not sure what causes it...

Your Environment

• Version used: 3.1.0.rc1
• Gemfile and Gemfile.lock as text in a Gist: https://gist.github.com/freshnewegg/6f9d2dd49897504f7085f12da31fbb36
• Any relevant stack traces ("Full trace" preferred):

[priyank-gupta]

@freshnewegg Whats your s3 configuration ? and where you did it ?

[BlakeLucchesi]

I can confirm I see the same behavior but only when serving my app locally (using $ heroku local).

I have the related ENV variables specified in my .env file which the server picks up on start.

app/initializers/spree.rb

Spree.config do |config|
  # Example:
  # Uncomment to stop tracking inventory levels in the application
  # config.track_inventory_levels = false
  attachment_config = {

    s3_credentials: {
      access_key_id:     ENV['AWS_ACCESS_KEY_ID'],
      secret_access_key: ENV['AWS_SECRET_ACCESS_KEY'],
      bucket:            ENV['S3_BUCKET_NAME']
    },

    storage:        :s3,
    s3_headers:     { "Cache-Control" => "max-age=31557600" },
    s3_protocol:    "https",
    bucket:         ENV['S3_BUCKET_NAME'],
    url:            ":s3_domain_url",

    styles: {
        mini:     "48x48>",
        small:    "100x100>",
        product:  "240x240>",
        large:    "600x600>"
    },

    path:           "/:class/:id/:style/:basename.:extension",
    default_url:    "/:class/:id/:style/:basename.:extension",
    default_style:  "product"
  }

  attachment_config.each do |key, value|
    Spree::Image.attachment_definitions[:attachment][key.to_sym] = value
  end

  config[:company] = true
  config[:admin_products_per_page] = 250
  config[:orders_per_page] = 100
  config[:always_include_confirm_step] = false
end

Spree.user_class = "Spree::LegacyUser"

Spree::Auth::Config[:registration_step] = false;

[freshnewegg]

@priyank-gupta

in config/initializers/spree.rb

attachment_config = {

  s3_credentials: {
    access_key_id:     ENV['AWS_ACCESS_KEY_ID'],
    secret_access_key: ENV['AWS_SECRET_ACCESS_KEY'],
    bucket:            ENV['S3_BUCKET_NAME']
  },

  storage:        :s3,
  s3_headers:     { "Cache-Control" => "max-age=31557600" },
  s3_protocol:    "https",
  bucket:         ENV['S3_BUCKET_NAME'],
  url:            ":s3_domain_url",

  styles: {
      mini:     "48x48>",
      small:    "100x100>",
      product:  "240x240>",
      large:    "600x600>"
  },

  path:           "/:class/:id/:style/:basename.:extension",
  default_url:    "/:class/:id/:style/:basename.:extension",
  default_style:  "product"
}

attachment_config.each do |key, value|
  Spree::Image.attachment_definitions[:attachment][key.to_sym] = value
end

Update: I was using aws v1 before, after updating to aws v2, I haven't seen any problems yet but that may be pure coincidence as it was random before.

Similar to Blake, the issue only occurs locally.

[AlecKriebel]

@freshnewegg Experiencing this issue too. You were able to upgrade to aws v2? Spree_core requires paperclip, and the version Spree_core requires doesn't work with aws v2 I thought, right?

[freshnewegg]

There's a branch of paperclip that is fixed with aws-v2

:git=> 'https://github.com/thoughtbot/paperclip', :ref => '523bd46c768226893f23889079a7aa9c73b57d68'

I'm using that until they do a main release

[damianlegawiec]

Spree 3.3 comes with Paperclip 5.x and AWS SDK > 2 support so we can assume this is fixed :)

[ysads]

I think this issue isn't fixed yet...

Here, I'm using:
spree 3.3.2
aws-sdk 3.0.1
paperclip 5.1.0

And Spree still stops serving S3 images. It seems that everytime the server crashes - even for the dumbest errors -, the image serving breaks and start to locally find files.

Does anyone know a possible explanation for that behaviour?

[kinduff]

I ended up decorating Spree's image model:

# app/models/spree/image_decorator.rb
Spree::Image.class_eval do
  attachment_config = {
    s3_credentials: {
      access_key_id:     ENV["AWS_ACCESS_KEY_ID"],
      secret_access_key: ENV["AWS_SECRET_ACCESS_KEY"],
    },

    storage:      :s3,
    s3_headers: { 'Expires' => 1.year.from_now.httpdate,
                     'Cache-Control' => 'max-age=31557600' },
    s3_protocol:  'https',
    s3_region:    ENV["AWS_REGION"],
    bucket:       ENV["S3_BUCKET_NAME"],
    url:          ':s3_domain_url',

   path: proc { |attachment| ":attachment/:class/:id/:style/:basename.:extension" },
   default_style:  'product'
  }

  attachment_config.each do |key, value|
    Spree::Image.attachment_definitions[:attachment][key.to_sym] = value
  end
end

Above example is working for me using hosted at Amazon's EC2 + S3

• spree (3.4.1)
• rails (5.1.4)
• ruby (2.4)
• aws-sdk (3.0.1)
• paperclip (5.1.0)

Had to move the bucket key outside the s3_credentials because it was raising a weird error about undefined method each or something. Same error if I put this in the Spree's initializer file.

[willwoodlief]

I spent a couple hours tracking this bug before I realized the paperclip settings were being forgotten. The solution of decorating the image model, above, worked for me! Before I fixed it, I was able to trigger this error reliably by throwing an exception in the variant_decorator.rb inside the method available?, then refreshing the same product page again, which had several variants and one image, after commenting out the exception. Writing this in case people want to replicate the bug to test. I am positive there is a better way to replicate it lol

[ysads]

I came to the same scenario as @willwoodlief. It seems that, when an exception is fired, the code within initializers folder gets overwritten by Spree's default configuration. I realized that because I also have a method in image class which becomes "not_found" when an exception gets fired. I'm still very confused about this behaviour... Does anyone know what can possibly cause such strange scenario?

[bbonislawski]

I'll look into that. I've heard about some problems with that and will test that. Reopning issue for now. Thank you for mentioning problems

[waleolakareem]

My first time using spree for a project and I had this problem too. I came to found out that it was quite easy to set up S3 with spree since the new version of spree uses active storage. Please just follow this link
https://edgeguides.rubyonrails.org/active_storage_overview.html and for your secret key set up follow this link
https://www.engineyard.com/blog/rails-encrypted-credentials-on-rails-5.2

It is quite easy now. Do not over think it, took me 3 days trying to use paperclip set up and all that. Save yourself the valuable time.