Track Active Storage variants in the database

[georgeclaghorn]

Currently, when a variant is requested, we check whether it exists in the service. If it doesn’t, we generate it on-demand and store it for reuse.

The existence check can add unacceptable latency to variant serving. We’ve seen existence checks to third-party services take upwards of 500ms in normal circumstances. Further, checking whether an object exists at a particular key before uploading it triggers eventual consistency in S3:

Amazon S3 provides read-after-write consistency for PUTS of new objects in your S3 bucket in all Regions with one caveat. The caveat is that if you make a HEAD or GET request to the key name (to find if the object exists) before creating the object, Amazon S3 provides eventual consistency for read-after-write.

This means that when a variant is requested for the first time, generated, and stored in S3, subsequent redirects to the variant’s service URL can fail indefinitely thereafter. The result is sporadically broken images.

This PR addresses the above concerns by doing away with the existence check. When a variant is requested for the first time, we generate it, store it, and track it in the application database. Now we can know whether we’ve already generated a variant without making a remote call to the storage service.

[jeremy]

May be desirable to extract this method so apps can override to connect to whatever db role. Probably premature.

[jeremiemv]

Thanks @georgeclaghorn
This will be a great improvement :)

[simonfranzen]

Hey. This seems like a very nice feature. I am struggling a bit. Is is merged?
I upgraded from rails 5 to rails 6 recently. I have created the active_storage_variant_records table and enabled it in the config, but it's not storing any entries in the db.

ActiveStorage::VariantWithRecord is also not loaded.

undefined method `variant_records' for #ActiveStorage::Blob:0x00007fbcffe12670

[sedubois]

@simonfranzen this ships in 6.1

[buhrmi]

Can't wait for this to be released. Will there be a way to include variants when querying the database to prevent N+1 queries?

[vizcay]

Hi @georgeclaghorn, this is great work and pretty much needed when the S3 server goes down as happened a few times to me and others (#39327).

I've quickly tried to monkey patch ActiveStorage on Rails 6.0 to track already created variants and worked but ended up hitting the problem of the N+1 Query as I wasn't able to preload al the variations when loading all the images.

Before investing more time there, there is no way of using this on Rails 6.0 without bundling all rails from master, right?

[Cremz]

This is indeed a great improvement, however the N+1 issue is very frustrating. I rather hit the server and check for the images instead of doing the preloaded query, then 3 x n+1 calls for each variant. (even worse when the model has multiple attached images). Any progress on a fix for this?

[rept]

@georgeclaghorn great addition. Am I right to assume that for existing variants (so already created before this new feature) the check will do done as follows:

• check in DB if variant exists
• if not exists, check in service
• if found there: insert variant record as well
• if not found there: create variant as usual

Another question: is the following intended?

After saving the image I do this to create a resized version:

device_capture.source_image.attach(io: image, filename: 'file.jpg', content_type: "image/jpg")
device_capture.source_image.analyze
device_capture.save
device_capture.source_image.variant(resize: "256x144^").processed

This version gets created and uploaded fine (see log below). However when I look in the table active_storage_variant_records there are no records created for this? I have to mention, I do this right after saving the initial image.

INFO -- : GCS Storage (166.2ms) Uploaded file to key: variants/bqybptq4psz72e8cirbktuou0fu9/88b54084c3b692042cf443b2ff6a221419d09ab4b6dfc1df7bef5f8b85ef0411

[ghiculescu]

For those concerned about n+1s, please give #40842 a try and let me know if it improves things for you.

[Laykou]

How does it work with existing variants in db? After deploying this to production will existing variants still work as addressed above? #37901 (comment)

• check in DB if variant exists
• if not exists, check in service
• if found there: insert variant record as well
• if not found there: create variant as usual

[louim]

@Laykou, yes, existing variants will create a new record in the database the first time they are accessed and will skip the service exist call after that.

[hong-godaddy]

Are we able to backport this to 5.2?

[louim]

Hey @hong-godaddy, features are not backported to earlier versions of Rails.

[KSH-code]

S3 has supported Strong consistency model now from Dec. 2020.

Is it possible to change default as false?

[tbhockey]

First of all, thank you for this feature 🙏

I'm coming from Rails 5 and have upgraded to 6.1 and have enabled this with config.active_storage.track_variants = true and while it's working beautifully, but we are getting quite a few broken images. I deleted all VariantRecord objects to allow them to rebuild, but strangely this fixed some images and now others are broken. Any idea how to properly resolve this?

[matthee]

Hi @tbhockey! I've experienced a similar problem now. Here is how i solved it:

tl;dr run ActiveStorage::VariantRecord.destroy_all

First, I tried to delete my variants like so:

ActiveStorage::Attachment.where(record_type: "ActiveStorage::VariantRecord").destroy_all

This looks fine at first glance. However, ActiveStorage actually creates ActiveStorage::VariantRecord records for each variant in the database. When you then browse to a page, where the variant is requested, ActiveStorage looks for the corresponding ActiveStorage::VariantRecord record in the database. If it is found, rails figures that the variant must also exist in the storage service (which it does not), leading to a broken image tag in the browser.

In order to fix this problem the ActiveStorage::VariantRecord records must also be removed from the database.

Running the following command in the rails console finally fixed my problem and rails happily regenerates all my variants on the fly:

ActiveStorage::VariantRecord.destroy_all

[tbhockey]

Thanks for the reply @matthee . I actually tried this as well, but for me, what happened was after doing that, different images were broken and others were fixed 😆

We ended up just having to go through and manually delete the Variants for any of the broken images. Since then, everything has been fine. But absolutely no clue what was causing that.