Interoperability with WP Retina 2x plugin

[bradt]

Continuing the conversation from http://wordpress.org/support/topic/retina-2x-amazon-s3?replies=7.

Jordy, maybe you can help save me some time with answers to the following:

1. When uploading a new media item, what is the best way to get the file paths of the generated @2x images?
2. When removing a media item, what would be the best way to get a list of the @2x files for that media item? Are you storing this info in the post meta?

Thanks,
Brad

[jordymeow]

Hey Bradt,

The 2 questions probably have the same answer. I don't store anything in the database (I don't want to mess-up the DB with those files), so I create/update the @2x files whenever the original media is uploaded or replaced, and delete the @2x files when the original media is removed.

From an attachment_id, here is how you can get the retina files:

$meta = wp_get_attachment_metadata( $attachment_id );
$sizes = $meta['sizes'];
$originalfile = $meta['file'];
$pathinfo = pathinfo( $originalfile );
$uploads = wp_upload_dir();
$basepath = trailingslashit( $uploads['basedir'] ) . $pathinfo['dirname'];
foreach ( $sizes as $name => $attr ) {
  $pathinfo = pathinfo( $attr['file'] );
  $retina_file = $pathinfo['filename'] . '@2x.' . $pathinfo['extension'];
  // file_exists( $retina_file ) ? ...

}

I hope it's clear enough, let me know if you need more details :)

[bradt]

Awesome, thanks for the sample code.

Hmm, I'm curious, why you don't just add these extra image sizes using WP's add_image_size function?

[jordymeow]

There are so many reasons! To avoid conflict and unexpected behavior with other plugins, to avoid having those files available directly to the users (when inserting images into the posts for example), one of the main reason is that the files need to be named in a certain way ("@2x"), and there would be too many functions to overload/hack, and there are probably many other reasons... The retina files should not be considered as new sizes in the system because they are an alternative.

[bradt]

Gotcha, thanks for satisfying my curiosity.

I'm wondering, at what point do you create the @2x images? My hook to upload them to S3 will obviously have to come after this. Any suggestions on what hook I should use?

Also, what would you say would be the best way to detect if your plugin is active? Maybe class_exists or function_exists on one of your classes or functions?

[jordymeow]

You can check if the function "wr2x_init" exits. It will never be removed or renamed.

I create them on the fly, at the same time as the normal images are created. The user can also do it "later". Don't bother, whenever you upload or re/upload a certain image, just check if there is the @2x equivalent and if there is upload it as well. I really believe that if you do this everything will work smoothly :)

[bradt]

Hmm, ideally there would be a hook (do_action) in your plugin after the @2x images are created. Something like wr2x_images_generated that would pass the file paths of the generated images and maybe the attachment id as parameters. Then I could just add_hook in my plugin and handle uploading them to S3 when the hook is called. I wouldn't have to check whether or not your plugin is active and most importantly, I would know that your @2x images were just generated and it would work when the user generates the images "later".

[jordymeow]

I could do that but right now, when are you uploading the media files? When they are created and modified? What if your plugin is installed after, will it detect the retina files as well?

[bradt]

Right now, I hook into wp_update_attachment_metadata, which executes when media is uploaded, when the image is modified via the image editor, or if a plugin like Regenerate Thumbnails is run. So yes, when they are created or modified.

However, my plugin only pushes images to S3 that are uploaded/modified after it was installed. There is no feature to upload existing media items. So, if you install it on a site that already has a bunch of uploaded images, those images won't be served from S3. Only newly uploaded/modified images will be served from S3. I plan to add a feature to upload existing images in the future. For that feature, I could just check if the @2x images exist as you had previously suggested. I don't think I'd even bother checking if your plugin is active or not. If there's a @2x image, just upload it anyways.

But to handle the case where your @2x images are generated "later", it seems like I need a hook to know that I should upload those to S3.

[realdannys]

Don't let this discussion die guys!

[bradt]

I appreciate the enthusiasm, but Jordy has said he is busy and will get to it when he has time. You might want to consider sending him a donation for his work on this free plugin. I'm not saying your request will get done faster if you make a donation, but I know when I personally receive a donation for my work on free plugins, it adds a bit of motivation.

[jordymeow]

Thanks a lot Brad! The real issue is now that I fried my MacBook Pro
Retina... spilled coffee on it... I dared asking Apple for help, at least
for the repair or something since I am developing something that indirectly
help them as well. But they wouldn't do anything for me. Just asking me for
half the price of the computer...

By the way, I think you only send the message to me and yourself :)

On Sat, Feb 16, 2013 at 3:09 AM, Brad Touesnard notifications@github.comwrote:

I appreciate the enthusiasm, but Jordy has said he is busyhttp://wordpress.org/support/topic/retina-2x-amazon-s3?replies=16#post-3829168and will get to it when he has time. You might want to consider sending him
a donation for his work on this free plugin. I'm not saying your request
will get done faster if you make a donation, but I know when I personally
receive a donation for my work on free plugins, it adds a bit of motivation.

—
Reply to this email directly or view it on GitHubhttps://github.com//issues/4#issuecomment-13619813.

[jordymeow]

Hey Bradt,

I uploaded a new version of my plugin in the trunk. It's not released yet under a new version since I would like to do more testing (and maybe a few more features) but it's a very good release.

I added two action as well and I added them in my FAQ. There is what I wrote:

The retina files have to be sent to the CDN, then the plugin should work fine (using the client-method). The plugin in charge of sending the files to the CDN is not the WP Retina 2x plugin, and cannot. The developer of the other plugin has to implement support for the Retina files. It should be very easy! In order to help those developers, I created two WordPress actions (when a retina file is added or removed) and they both send two arguments: the attachment id and the full path to the retina file.

• wr2x_retina_file_added
• wr2x_retina_file_removed

Are you okay with this? You can implement it already :)

[bradt]

Very nice! I'll try it out soon and report back. :)

[realdannys]

How did you get on with this Brad?

[bradt]

Busy busy. Haven't gotten to it yet.

[bradt]

I'm open to pull requests if you want to do it yourself.

[realdannys]

I wouldn't have a clue where to start Brad.

[jordymeow]

In the meantime, you can use WP Retina 2x with the new method called "HTML Rewriter". This way, you can stop using a CDN and try Cloudflare or Google PageSpeed Service instead. It's fast, it works perfectly, and it's even easier to use (no extra plugin or operations). I would love to know why people are still going for CDN though, Cloudflare/GPSS seem like amazing solutions to me.

[realdannys]

Prob not the place - but Cloudflare/GPS are replacements for Amazon Cloudfront rather than S3. The files still need to be hosted somewhere for Cloudflare/GPS/Cloundfront to provide them globally on their network - thats where S3 comes in, instead of having them on your server. S3 is simple storage which is infinitely expandable which is great for streaming files or high traffic etc and not putting strain on server and it links well with EC2 for elastic cloud server implementations where you don't really have a core server as such.

Id still be using Cloudflare with this setup too.

[jordymeow]

Actually from my point of view GPS is a S3 service as well. The only
difference is that you can use your WordPress normally, play with your
files there, then GPS will cache the files on its own servers. Sounds much
easier and cleaner to me :)

On Sat, Mar 16, 2013 at 8:14 PM, realdannys notifications@github.comwrote:

Prob not the place - but Cloudflare/GPS are replacements for Amazon
Cloudfront rather than S3. The files still need to be hosted somewhere for
Cloudflare/GPS/Cloundfront to provide them globally on their network -
thats where S3 comes in, instead of having them on your server. S3 is
simple storage which is infinitely expandable which is great for streaming
files or high traffic etc and not putting strain on server and it links
well with EC2 for elastic cloud server implementations where you don't
really have a core server as such.

Id still be using Cloudflare with this setup too.

—
Reply to this email directly or view it on GitHubhttps://github.com//issues/4#issuecomment-15003127
.

[realdannys]

No GPS is definitely a pure CDN - I doubt id use it over Cloudflare or Cloudfront either - AWS has multiple advantages to many services, its not for everyone but its proper cloud hosting with flexible server instances - in the GPS approach you'd need a dedicated server which you can't expand or reduce as traffic demands amoungst many other things you can do with AWS.

[bradt]

Thanks Jordy, I hadn't heard of GPSS. I'll have to give these a try.

[realdannys]

Anything new Brad :)

[bradt]

I won't be getting to this until after launch: http://wcmia13.deliciousbrains.com

[realdannys]

Hi Brad,

Any update on this plugin? I hope you can find a couple of days to spare to support the retina plugin - and maybe an option to delete original files once moved to S3?

Dan

On 10 Apr 2013, at 02:29, Brad Touesnard notifications@github.com wrote:

I won't be getting to this until after launch: http://wcmia13.deliciousbrains.com

—
Reply to this email directly or view it on GitHub.

[realdannys]

Hi Guys/Jordy,

This still doesn't work properly for me. It works correctly on Brad's side in that it now uploads the retina images to S3 when they are created. However Jordy's plugin doesn't re-write the URL if they are on S3, it appears to be looking for them in the website directory instead of at the re-written URL. Maybe there should be a way to point to a sub domain/CDN if you have the images Jordy?

[jordymeow]

Sorry, I don't have much time looking into this issue these days + I am not using Amazon S3. However, if you find a way to resolve this issue I will definitely enhance the plugin right away.

[realdannys]

No problem Jordy. Could you maybe point me in the right direction for where the plugin pulls or checks the resulting URL. From what I can workout its trying to base the url on "home url"/file/ etc etc.

If I could hardcode the "home URL" to be the new sub domain/URL then at least it would be checking the correct link for the same files. But I can find lots of references to base URL and not sure which one to start editing, my initial attempts have been poking in the dark with no results at all and I'm guessing that each method has its own way of doing it, id be happy just getting the retina.js method to start with.

[jordymeow]

Unfortunately, I am not the developer of the retina.js script, so if there is a solution you should check on their github (https://github.com/imulus/retinajs). I recommend you to use the HTML Rewriter method; in that case, all the code is in the wp-retina-2x.php, function wr2x_html_rewrite.

[realdannys]

Hi guys,

Just though I should let you know I finally found a solution to this.

Now the change in Brad's plugin has been made, the retina images are perfectly uploaded to S3 when the images are made in the image library, so thats great.

I had no luck changing the HTML rewriter method for some reason, but I did find this fork of the retina.js on github, which tackles the problems with CORS requests in S3 and fixes them perfectly - https://github.com/chrishunt/retinajs/blob/master/src/retina.js

So for anyone else looking all they need to do is update the retina.js file in Jordy's plugin with that version and it'll work with any CDN or subdomain they have their images hosted on.

[travelmassive]

Hi all I am going to Lazarus this issue because it's worthy of some additional comments.

I have purchased both WP Offload S3 and WP Retina 2x Pro.

As a customer, it's a bit disappointing that even after so many years these two great plugin fail to integrate :(

I'm certain we're only a few lines of code away from this working. What can be done to resolve this?

I see that @jordymeow offered up a function 'wr2x_retina_file_added' which is still in his code base. What we need is some advice or sample code from @bradt for how to hook this in to WP Offload S3 so that the retina images get moved over to S3 after they are generated.

I also tried the (since removed from WP Offload S3) code suggested in this gist but it doesn't work -> https://gist.github.com/polevaultweb/d5e2af9150e0495d078b

I am certain I'm not the only person looking for a solution here. Thanks!

[jordymeow]

I am not sure I should ask this question, but why aren't you using a conventional CDN? It works, it integrates with all kinds of plugins, it doesn't require uploads, can refresh, reset cache when images are updated/replaced, and also cache all your static content (not only images). I use S3 for non-WordPress related projects, and I like it, but for WordPress projects, I would definitely not use it.

For your issue, it would work with Retina and the PictureFill method, you would need to check the http check but that's all. Of course, S3 would need to upload the retina photos as well.

[travelmassive]

Thanks for quick reply - re: CDN. Why do I use S3 + Cloudfront? Well... I manage a multi-platform site (Drupal for our community platform, Wordpress for our blog, Discourse for our community forums) and so there is an overhead of complexity that I face as a developer and admin - multiple codebases, multiple servers, multiple databases, multiple backups, development environments, SSO, keeping everything up to date, paying licensing and server bills, the list goes on. Plus supporting our 38,000+ members. Therefore sticking to S3 + Cloudfront helps keep things sane from an ops POV.

Thinking about the solution here... perhaps a middleware plugin that can act on your wr2x_retina_file_added method to upload the file(s) to S3 so we are sure that there will be a @2x version available. Are the retina image urls kept in the database so they can be modified, or does your plugin scan the file system directly to determine what retina images can be used?

[jordymeow]

Exactly, you could do that. Just upload the file manually (or by modifying the S3 plugin) through the wr2x_retina_file_added filter. It's there for that reason basically. Then my plugin scans the system OR performs a http request (check the retina settings) to find the retina file, and include it in the srcset.

[ianmjones]

I just had a play with the two plugins, trying to get things working with a slightly altered version of our Tweaks plugin example of adding extra files for upload using our as3cf_attachment_file_paths filter, it didn't go so well.

The @2x images could be uploaded to S3 OK, but not with the bulk uploader for some reason. If you use "Remove Files From Server" then things go wonky as those extra files are not removed from local.

The biggest problem is that the 'srcset' always loses the @2x files even though they have been uploaded to S3 due to them not being part of the registered image sizes. Maybe WP Retina 2x's CDN option can fix that, I don't know, but it skips changing the URLs that WP Offload S3 swaps in for registered image sizes (WP Retina 2x has a much later priority on the relevant filter). I tried all the methods for swapping in the 2x files, no dice.

If WP Retina 2x augmented the registered image sizes and hid its new ones from user selection then I guess WP Offload S3 would just use them without issue, but as it stands there are too many gotchas for us to be able to support it.

The function I link to above used to use an apply_file_suffix function that no longer exists, it's been replaced in that Tweaks example with a simple concatenation of pathinfo parts. That's why @polevaultweb's gist no longer works too, it uses the old apply_file_suffix function.

If anyone were to come up with a cast-iron set of filters/functions that could be added to the Tweaks plugin to restore compatibility between the two plugins, we'd definitely review the PR.

[travelmassive]

Hi all

Thanks for your comments and feedback which set me in the right direction.

After some research, I came up with this solution (inline code not rendering correctly, so using quote [IJ - fixed]):

// upload retina images to s3
add_action( 'wr2x_retina_file_added', 'tm_wr2x_retina_file_added', 10, 2 );
function tm_wr2x_retina_file_added( $attachment_id, $retina_filepath ) {

    if (function_exists('as3cf_init') or (function_exists( 'as3cf_pro_init'))) {
        global $as3cf;

        $post_id = null;
        $metadata = null;
        $file_path = $retina_filepath;
        $force_new_s3_client = false;
        $remove_local_files = false;

        $s3_data = $as3cf->upload_attachment_to_s3($attachment_id, $metadata, $file_path, $force_new_s3_client, $remove_local_files);

        return ! is_wp_error( $s3_data );
    }

    return false;
}

Add the above to your theme's function.php file.

Hope this helps others. I'm using this in production and it's working for me - but it there might be side-effects so as always this suggestion is without warranty.

[jordymeow]

@travelmassive Thanks a lot for this code. That seems very easy :) I am interested in sharing this code on my FAQ, is that ok? I have also another plugin for synchronizing Lightroom with WordPress and a code in "those lines" might be nice to have, but in my case, I would need to replace already uploaded images.

[travelmassive]

@jordymeow - yes, of course feel free to share - glad I could be of some assistance.