Consider alternatives to Gregwar/Image for image manipulation

[fredrikekelund]

I'm having a good time working with Grav for a new site! The biggest pain point I've encountered so far (that wasn't fixed in the 1.1 release) is the image processing pipeline. The API that Grav provides with the help of Gregwar/Image is neat, and I realize it's a pretty big selling point for Grav as well. But I've encountered two issues with Gregwar/Image that are kind of blocking for my use case.

Firstly, there's color profile handling. Although Gregwar/Image has an extensible back-end, there's only an implementation for GD. And GD doesn't deal with image metadata, meaning it won't embed any other color profiles than the default sRGB when writing files. Images that were initially rendered with eg. an Adobe RGB color profile loose luster because of this, and while this may seem like just a nuisance, it really is a deal breaker for some people.

Secondly, GD doesn't perform all that well. GraphicsMagick for example should be a better candidate if you want to optimize for performance. Even though the performance of Gregwar/Image isn't the fault of Grav, it affects the perception of Grav's overall performance. I have an image heavy page on the site I'm building (about 70 images á 2500px in width), and when generating that page from scratch, PHP times out with the default execution timeout setting of 30s.

And as an additional note, these issues don't seem to be addressed any time soon, since development on Gregwar/Image seems pretty stale. There isn't much activity in the issue tracker, or on the commit front (the latest commits up until just recently were from November/December last year). The color profile issue has been brought up in Gregwar/Image#101, but no solution has been suggested yet.

I understand that swapping out Gregwar/Image is probably a pretty major decision, but maybe an API compatibility layer could ease such a transition.

I'm not sure what the best alternative is, but avalanche123/Imagine seems like a more mature and capable option. It supports color profiles and has different back-end implementations for GD, ImageMagick and GraphicsMagick.

[rhukster]

I'm actually quite aware of the lack of updates in Gregwar/Image library. Unfortunatly when Grav was in it's early stages, it was the only game in town. In the past couple of years I have seen some better options pop up, and I've been watching them.

Ideally I would like to change to something more actively developed, and one that supports both gd and imagick, but the problem is that it would need to have a compatibility layer to ensure existing sites continue to work as expected. I've not had time to evaluate and check each library and ensure that we can replicate each function we already support.

Some i'm aware of:

• http://glide.thephpleague.com/
• https://github.com/avalanche123/Imagine
• http://image.intervention.io/
• http://stefangabos.ro/php-libraries/zebra-image/
• http://imanee.io/

It would definitely be open to someone helping out and implementing one of these with a compatibility layer for the existing calls.

[mahagr]

I like the approach of Glide as it generates the files on fly instead of during the page generation. It also allows ACL on images, which is great.

[fredrikekelund]

@mahagr I agree that image generation at authoring time rather than at request time is a good premise. But that can only be done with the admin plugin, right? And it wouldn't really be dependent on what image manipulation library is chosen, would it? I mean, PHP can't idly observe a directory, but if a page is edited through the admin, the page could be rendered after being saved. This would of course be neat, but maybe it's a separate issue?

[mahagr]

Admin plugin isn't needed to do that. Yes, it doesn't matter which library is chosen, though Glide does come with more logic built in, which means its more plug and play than others. On the other hand, having that logic also restricts things...

[fredrikekelund]

Maybe we mean different things by "on the fly" then, do you mean that the images would be generated when they are requested by the browser?

[mahagr]

Yes, images would be generated when requested by the browser is what Glide does.

[fredrikekelund]

Well, that might make page generation faster, but wouldn't it be a much bigger change compared to just replacing Gregwar/Image with some other library to perform the image manipulation?

Maintaining the current system, but changing the library and exposing a few new system config params eg. for determining whether GD or Imagick should be used sounds like a compelling option, IMO.

[rhukster]

I don't think we want to radically change things at this point. We'll save any major architectural changes for Grav 2.0 :) One thing to consider is that HTTP server type solutions need to be smart about how they cache images so they don't do the same work more than once.

Anyway for this PR, there are multiple parts:

1. replace the existing Gregwar/Image library with another
2. ensure all existing methods that are documented have a wrapper function to perform the same function with the new library. Also expose any new and potentially useful methods.
3. implement a caching layer so that the the exact same image+operations is cached to /images/ folder once, and not processed if exists.
4. tests should be built to help with development and ensure compatibility between existing method calls.

The exact folder structure of that cache in images/ folder is not important, so the cache mechanism is flexible. Could be another doctrine cache which we are already using (with file-based storage only), or anything else.

[OleVik]

A thing to note with several of the aforementioned, and other, libraries is this:

1. Some lack a built-in method-stack handling, in the form of: GraphicsMagick > ImageMagick > GD
2. Some poorly implement proper destruct and detach sequences: This is the most notable problem when used with source images (>2MB in size). They tend to crash due to memory limits - not actually because of a lack of memory, but memory not being free'd up.
3. They virtually all perform the same operations, barring some lacking extensive features, varying mostly in quality of code.

The GraphicsMagick > ImageMagick > GD stack seems to be most advantageous in terms of handling large or many images, as well as speed and required processing power. I recently started looking into an alternative to my Node-habits for image manipulation, and properly generating responsive images in a publishing/development workflow is a must in 2016.

[rhukster]

Some good points @OleVik!

I think #1 is not really an issue though. Although many of these libs don't provide auto fallback to supported library, it wouldn't be hard to have this check ourselves. We can easily check for a function or method provided by the library, and fallback and instantiate the specific image handling object that is supported.

[fredrikekelund]

I've done some investigation, and I wanted to post a quick update. Of the proposed libraries, Intervention seems to be the best candidate. It has a ready made caching solution and supports Imagick. However, the caching solution has the problem that all filters need to be applied inside of a single callback. As far as I understand, the current ImageMedium logic would need to be rearchitected so that filter calls from the Twig template are aggregated and then called in batch when eg. the ImageMedium#url method is called. Rearchitecting the ImageMedium logic would have the added benefit of ensuring that the ImageMedium#derivatives method could be called in any order together with the other filter methods in the template, as we could reorder the calls as necessary when finally generating the image. This probably isn't accomplished in an hour, but it doesn't seem like an unreasonable undertaking for someone who's fairly familiar with Grav's codebase, and it won't break API compatibility.

Notwithstanding the rearchitecting described above, I keep coming back to the idea of forking Gregwar/Image to write an Imagick adapter instead of using another library. The module is totally capable of using other adapters, they even have an Imagick one lying around, they just haven't finished it (there's a half ready PR for it as well). If they won't accept a PR from such a fork, maybe Grav could depend on the forked version until further notice.

[flaviocopes]

The last PR merged on Gregwar/Image is mine, to fix an orientation issue I discovered by using it in Grav, about 10 months ago.
Merged, but no release since then, which means it's not yet available to the general public :)

So I don't hold my breath on that imagick adapter to be available soon, hope I'm mistaken, but :)

If we can find a (nearly) drop-in alternative which provides better performance, and with more active maintainers (we don't want to maintain our own fork I suppose), would be great of course.

[rhukster]

I've added a Hacktoberfest tag to see if we can get some help with this as I'm a bit stretched with other things at the moment.

[lepascal]

Are there any news regarding this? Any solution to handle a large amount of images? I am developing a portfolio site that contains 500+ images on the front page – which are lazy loaded to save bandwidth. However, all are generated on page load and the server is always timing out.

[Japhys]

I was just made aware of this issue. Posted in GRAV community with a similar issue (and a small clip to illustrate). Is this still being considered?

[rhukster]

Imagick was never fully developed for Gregwar/Image. At this point, we're quite tightly integrated with this package which only supports GD. I think the next major version of Grav 2.0 will use something different.

[Japhys]

@rhukster Thanks for the reply Andy. I guess I'll have to be patient, no reason to change for something else. Very happy with Grav overall.