Use ImageProcessing gem for ActiveStorage variants

[janko]

ImageProcessing gem is a wrapper around MiniMagick and ruby-vips, and implements an interface for common image resizing and processing. This is the canonical image processing gem recommended in Shrine, and that's where it developed from. The initial implementation was extracted from Refile, which also implements on-the-fly transformations.

Some features that ImageProcessing gem adds on top of MiniMagick:

• resizing macros
  • #resize_to_limit
  • #resize_to_fit
  • #resize_to_fill
  • #resize_and_pad
• automatic orientation
• automatic thumbnail sharpening
• avoids the complex and inefficient MiniMagick::Image class
• will use magick CLI instead of convert on ImageMagick 7 in the near future

However, maybe the biggest feature of the ImageProcessing gem is that it has an alternative implementation that uses libvips. Libvips is an alternative to ImageMagick that can process images very rapidly (we've seen up to 10x faster processing compared to ImageMagick).

What's great is that the ImageProcessing gem provides the same interface for both implementations. The macros are named the same, and the libvips implementation also does auto orientation and thumbnail sharpening; only the operations/options specific to ImageMagick/libvips differ. The integration provided by this PR should work for both implementations.

The PR should maintain almost 100% backwards compatibility; there are two incompatibilities:

• users would now have to include the image_processing gem into their Gemfiles instead of mini_magick
• :combine_options was removed because ImageProcessing::MiniMagick builds a single convert command, so it's not needed anymore

I know that Rails 5.2 is on feature freeze before the release, but I think if we agree we want ImageProcessing to be used for Active Storage, it's better to get it in now, due to the two mentioned backwards incompatibilities.

In short, Active Storage relying on ImageProcessing means it will have much better MiniMagick defaults, it will get libvips support for free (which is a big deal in terms of performance), it will get convenient resizing macros that work on both implementations, and it means the Active Storage project will have minimal maintenance in the image processing department.

[rails-bot]

Thanks for the pull request, and welcome! The Rails team is excited to review your changes, and you should hear from @kamipo (or someone else) soon.

If any changes to this PR are deemed necessary, please add them as extra commits. This ensures that the reviewer can see what has changed since they last reviewed the code. Due to the way GitHub handles out-of-date commits, this should also make it reasonably obvious what issues have or haven't been addressed. Large or tricky changes may require several passes of review and changes.

This repository is being automatically checked for code quality issues using Code Climate. You can see results for this analysis in the PR status below. Newly introduced issues should be fixed before a Pull Request is considered ready to review.

Please see the contribution instructions for more information.

[y-yagi]

r? @georgeclaghorn

[rafaelfranca]

The shipt for Rails 5.2 has really sailed, no new features or changes that are not going to fix regression bugs are going to be introduced a this point.

It looks like we can implement this in a way that those two incompatibilities can be managed.

The gemfile change is not a problem. We can support the two backends in Rails 6.0, deprecating the old one.

We can also deprecate the combine_options option when the backend is image_processing.

In rails 6.1 we remove the support to mini_magick and we get all the benefits of image_processing.

Am I missing something why this can't wait to 6.0?

[janko]

You’re probably right, it should be doable later in the way you described. When I created the PR I thought Rails 5.2 was still in the beta phase, and thought it makes sense to squeeze it in.

Let’s then take the time to properly discuss it after 5.2 is released.

[janko]

I added backwards compatibility with the MiniMagick implementation and :combine_options. Let me know if there is anything else missing.

[georgeclaghorn]

Nice work, @janko-m! Thanks for your patience.

[georgeclaghorn]

The name of this configuration option is more generic than the feature it controls. Let‘s name it something like image_processor or variant_processor.

[janko]

Good point, I chose variant_processor – f01e249

[georgeclaghorn]

transform generates the variant, so it can close the variant as well, and take care to do so even if uploading fails:

def process
  download_blob_to_tempfile do |input|
    transform image do |output|
      upload output
    end
  end
end

# ...

def transform(image)
  format = "png" unless WEB_IMAGE_CONTENT_TYPES.include?(blob.content_type)
  result = variation.transform(image, format: format)

  begin
    yield result
  ensure
    result.close!
  end
end

[janko]

Agreed, updated in 0d811fd.

[dixpac]

Since you are maintainer of image_processing gem, is there a way to add resize in the gem.
If I understood well, inside image_processing gem resize_to_fit is resize from IM...just proposing this so there is no need to change all resize calls to resize_to_fit inside existing apps 😀

[janko]

ImageProessing gem still supports all ImageMagick operations, the changes in tests and documentation are just to show/test the usage of macros. Any operation sent that is not a macro (#resize_to_fit, #resize_to_limit etc.) is interpreted to be an ImageMagick operation (via #method_missing).

So, with this change you still use :resize and any other ImageMagick operations, but you can also additionally use :resize_to_fit, :resize_to_limit and others.

[dixpac]

Ahh, that makes sense 👍 ....not sure tho that we need change in documentations/guides then, for me
resize "100x100" feels nicer than resize_to_fit [100, 100], but this is just mine preference 😀

[janko]

Yeah, in this particular case resize "100x100" is probably nicer. The difference shows in resize_to_limit [100, 100] which is resize "100x100>" (I find the > a bit cryptic), or resize_to_fill and resize_and_pad which are macros of multiple ImageMagick operations.

While we're here, we should probably recommend using resize "100x100>" (or #resize_to_limit), as when you physically enlarge an image that is smaller than specified dimensions, it will generally look worse than if you just display the unchanged image enlarged. That of course won't happen with thumbnails as small as 100x100, but it could with larger thumbnails.

What do you think?

[janko]

Actually, then people would have to remember to add the width/height HTML attributes to the <img> tags. I'll just revert to using resize, and additionally show an example of using a resize_* macro.

[janko]

Updated in 7fc8b6d.

[janko]

I also removed the comment that the image will be downloaded into memory, because that's not true anymore, even before this pull request. In both cases #download_blob_to_tempfile is used, which streams the file into disk.

[janko]

@dixpac Actually, I just remembered, one advantage of #resize_to_fit over #resize is that it also sharpens the thumbnail after resizing. You can see an example showing the difference in my blog post:

In short, when an image is resized, the algorithm will make the thumbnail appear a bit blurry compared to original (regardless of compression settings). That's why some image programs like Adobe Photoshop will automatically sharpen the image after resizing, and you can do the same with ImageMagick/libvips.

So, I think #resize_to_fit would be a better default.

[dixpac]

@janko-m that is a great point, I didn't know this.Thanks.
I thought that resize_to_fit is same as resize. So image_processing gem is automatically sharping the image. Indeed then resize_to_fit feels as better default then :)

Just curious, how much overhead sharpening adds over regular resize :)?

[janko]

Ok, I made some benchmarks. As expected, the sharpening overhead depends on the thumbnail size – the smaller the thumbnail is, the less time it takes to sharpen it. With ImageMagick I've observed the following for a 1600x900 source image:

• 800x800 – 1.87x slower
• 500x500 – 1.40x slower
• 300x300 – 1.18x slower
• 100x100 – 1.05x slower

For large source images the difference is naturally much smaller, as it takes more time to resize it and equal time to sharpen the resized image. But 1600x900 is probably a better average measure.

On libvips it doesn't go above 1.20x slower, on average it's only about 1.10x slower.

[georgeclaghorn]

Thanks, @janko-m! Here are a few more comments. This also needs a changelog entry.

[georgeclaghorn]

Add "the" before image_processing.

[georgeclaghorn]

Correct "procecssor" to "processor."

[janko]

@georgeclaghorn Done!

[georgeclaghorn]

This test is failing in CI:

Error:
ActiveStorage::VariantTest#test_works_for_vips_processor:
NoMethodError: undefined method `source' for #<String:0x00000000043b5728>

Can you take a look?

[janko]

Fixed in f2e2cef. Requiring ruby-vips will raise a LoadError if libvips is not installed, and the #processor method was accidentally swallowing it.