Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

WebP Support for PW Core Engines #141

Merged
merged 32 commits into from May 23, 2019
Merged

Conversation

@horst-n
Copy link
Contributor

@horst-n horst-n commented Apr 25, 2019

@ryancramerdesign
Hi Ryan,
here is the webP support for PW core image engines.
Please refer to this forums post for some explanation:
https://processwire.com/talk/topic/14236-webp-support/page/2/?tab=comments#comment-184583

If you like, we can discuss it here or you can contact me via PM or email (info@nogajski.de).

horst-n added 12 commits Apr 24, 2019
added option "webpAdd" and "webpQuality" that allows to create a webp file as sidecar file when creating a JPEG, GIF or PNG file.
with the property $image->hasWebp we can detect, for example in a template file, if an image variation has a dependant webp file, so we can render conditional markup with srcset or picture elements.
And urlWebp and srcWebp for returning the URL.
and remove debug code
@TomS-
Copy link

@TomS- TomS- commented Apr 25, 2019

Would urlWebp work as well as srcWebp as src is an alias of url?

@horst-n
Copy link
Contributor Author

@horst-n horst-n commented Apr 25, 2019

Yes, see:

* @property-read string $urlWebp The url property of an optional WebP-dependency file (since 3.0.132).
* @property-read string $srcWebp Convenient alias for the 'urlWebp' property (since 3.0.132).

*
*/
public function setWebpQuality($n) {
$n = (int) $n;
Copy link
Contributor

@matjazpotocnik matjazpotocnik Apr 26, 2019

Could this be simplified? $this->webpQuality = min(max((int) $n, 1), 100);

Copy link
Contributor

@teppokoivula teppokoivula Apr 27, 2019

That would make the code less readable, IMHO. One-liners are fun and save lines of code (if that's a priority), but they can also take a while to sink in – the benefit of simple if-else-structure is that it's instantly understandable :)

... though that being said, I would recommend one change to this method as well:

-		if($n > 100) $n = 100;
+		else if($n > 100) $n = 100;

Micro-optimization, but if first rule is true then the second doesn't need to be processed, and this would also make the intention more obvious (again in my opinion).

Copy link
Contributor

@teppokoivula teppokoivula Apr 27, 2019

Actually here's another minor suggestion:

-		$this->webpQuality = (int) $n;
+		$this->webpQuality = $n;

(int) is pointless there, since we've already converted the value to an integer a few lines above. Again very small thing, but this extraneous type conversion can make one wonder if there's actually something strange going on here.

horst-n added 2 commits Apr 26, 2019
beginning with webp, as we also use two new options beginning with webp. Maybe this is better to avoid confusion.
webpAdd, webpQuality, webpUrl, webpSrc
@horst-n
Copy link
Contributor Author

@horst-n horst-n commented Apr 26, 2019

@ryancramerdesign
Hi Ryan, I have updated one thing in the IMagick module, but primarily want to point you to a .htaccess-only solution I have played with today, that does not need any changes on existing markup to switch a complete site to webp support. Here you go:
https://processwire.com/talk/topic/14236-webp-support/page/2/?tab=comments#comment-184669

@horst-n
Copy link
Contributor Author

@horst-n horst-n commented Apr 27, 2019

Here is example and sample output of the new manually invoked getDebugInfo method:
https://biriba.de/pw_pop3/pw_pageimage_getdebuginfo/

*
* @param int $n
*
* @return $this
Copy link
Contributor

@teppokoivula teppokoivula Apr 27, 2019

Since we're in "code review mode" here, minor note about this: return statement should always state a valid return type, and $this is not one of those. @return self should work, although @return ImageSizerEngine would be recommended (works best with IDEs).

Since specific class name can be slightly confusing if and when this class is extended, I might still go with self :)

*
* @param bool $value
*
* @return $this
Copy link
Contributor

@teppokoivula teppokoivula Apr 27, 2019

Earlier return type comment applies here as well 👌

horst-n added 3 commits Apr 29, 2019
where the regular variation exists and should not be recreated, but where a webp copy, that do not exist, is requested too.
if a webp copy exists or not. Using $this->hasWebp() within the resize method doesn't work. Changed to test for file_exists of the temporary webp copy.
@horst-n
Copy link
Contributor Author

@horst-n horst-n commented May 2, 2019

Hi @ryancramerdesign ,
with my commit from today, it seems I have covered all cases correct now. If you have any questions you know the channels to contact me. :)

horst-n added 3 commits May 3, 2019
I figured out that this is useful to have for (automated) testing and debugging too. It is much easier to get a value for conditional coding via an object:
```
$dbgInfo = $image->getDebugInfo($options, 'object');
if($dbgInfo->engines->selectedEngine == ....
```
to return the calculated URL, even if there is no webp copy available!
@ryancramerdesign
Copy link
Member

@ryancramerdesign ryancramerdesign commented May 22, 2019

@horst-n Thanks for all of your great work with this! I'm now working through the PR and so far all looks good. I do have some minor questions, mostly related to learning about this format and update.

First, I just wanted to make sure I understood correctly how this works. I'm not so familiar with webp, so these are probably dumb questions but I'll ask anyway. As far as I can tell, webp is treated here as an add-on to the existing file formats (jpeg, png, gif) rather than a new file format itself. And so one can't have a Pageimage that that is represented by a webp file, for instance? Instead, there is an extra webp URL (or path) for any GIF/JPG/PNG, available as an option for when/if the browser supports it. So this is a similar strategy like we use for HiDPI, in some respects. Does it sound like I understand it correctly so far?

2-3 years down the road when all browsers support webp, will this strategy still be the right way to go, or would you recommend something different at that point?

Also, down the road, what if new formats similar-to, but distinct-from webp come along, and we want to support those? I'm just trying to think of how the strategy best scales towards supporting new formats.

Using the .htaccess route, is it even necessary to have URL/filename methods in the PageImage class? Like the Pageimage::webpUrl() — I'm struggling a little with this because it's so format specific, which is fine for today, but I'm not sure about longer term scalability. I really like that your .htaccess solution abstracts this all away so nicely. As I look at the updates in Pageimage, I also wonder about moving webp-specific, hidpi-specific and svg-specific code into separate classes that hook into Pageimage, so that Pageimage doesn't need to know about all these things. I like what you've got so don't change anything right now — this is just future thinking.

I see reference to a webpOnly option in a few parts, but it's not clear to me where I would set that or how/when I would use that? I don't see that the code additions are setting it anywhere, but they are checking for it. I'm confused on that point just because as I understand so far, the webp support is an extra on top of the existing formats, rather than additional format on its own. So what is the webpOnly option for? :)

In your .htaccess updates for webp, it makes sense to me except for one line:

RewriteCond %{DOCUMENT_ROOT}/$1$2$3/$4.webp -f

I'm not clear about what the 4 $variables are referring to, because I'm used to seeing these referring to some prior capturing parenthesis. But in this case, I'm not clear where the $1, $2, $3 and $4 were captured? It may be that I'm just not familiar with this .htaccess feature yet.

@horst-n
Copy link
Contributor Author

@horst-n horst-n commented May 23, 2019

Hi Ryan,
today I'm a bit in a hurry, but want to answer some of the questions.


First, I just wanted to make sure I understood correctly how this works. I'm not so familiar with webp, so these are probably dumb questions but I'll ask anyway. As far as I can tell, webp is treated here as an add-on to the existing file formats (jpeg, png, gif) rather than a new file format itself. And so one can't have a Pageimage that that is represented by a webp file, for instance? Instead, there is an extra webp URL (or path) for any GIF/JPG/PNG, available as an option for when/if the browser supports it. So this is a similar strategy like we use for HiDPI, in some respects. Does it sound like I understand it correctly so far?

Yes, this is right. More information about the different role was discussed in forum posts:
https://processwire.com/talk/topic/14236-webp-support/?do=findComment&comment=185136
and
https://processwire.com/talk/topic/14236-webp-support/?do=findComment&comment=185140


2-3 years down the road when all browsers support webp, will this strategy still be the right way to go, or would you recommend something different at that point?

As I do not see any advantages from webp over jpeg and png AS MASTERIMAGES, but the same or more difficulties of wrong handling (like with jpeg), I would not recommend to change that with our current pageimage & sizers. But ... (see next question)


Also, down the road, what if new formats similar-to, but distinct-from webp come along, and we want to support those? I'm just trying to think of how the strategy best scales towards supporting new formats.

If this comes true, we need to rewrite a bigger part of the current logic / implementation of pageimage and the sizers. Then we have to support creation of one, two, or more fileformat outputs in one call to the size function, what currently isn't possible. Also to define a single outputformat, different from the source isn't possible atm. There are more points I came across as I started trying to support a "webOnly" flag, but due to my hurry, I will list those later.


Using the .htaccess route, is it even necessary to have URL/filename methods in the PageImage class? Like the Pageimage::webpUrl() — I'm struggling a little with this because it's so format specific, which is fine for today, but I'm not sure about longer term scalability. I really like that your .htaccess solution abstracts this all away so nicely. As I look at the updates in Pageimage, I also wonder about moving webp-specific, hidpi-specific and svg-specific code into separate classes that hook into Pageimage, so that Pageimage doesn't need to know about all these things. I like what you've got so don't change anything right now — this is just future thinking.

I personally also would take the .htaccess route, but there are many users that already ponted out that they want to have a fine control of where and how webp is supported or not. They want to use picture and srcset elements to create markup with webp fallbacks. Besides that, giving the users the choice over different solutions, it maybe that there are websites out, that cannot be changed to webp support by the .htaccess solution, for what ever reason.


I see reference to a webpOnly option in a few parts, but it's not clear to me where I would set that or how/when I would use that? I don't see that the code additions are setting it anywhere, but they are checking for it. I'm confused on that point just because as I understand so far, the webp support is an extra on top of the existing formats, rather than additional format on its own. So what is the webpOnly option for? :)

(Also see above.) - It was a try to support output of webp only, that isn't functional atm. There must be done greater rewrites in pageimage and the sizers, that I'm currently have no time for. And I think that it would be better if we can do it in collaboration, and it should be done with other (future) input and output formats in mind. So the rewrite will be even bigger. :)


In your .htaccess updates for webp, it makes sense to me except for one line:

RewriteCond %{DOCUMENT_ROOT}/$1$2$3/$4.webp -f

I'm not clear about what the 4 $variables are referring to, because I'm used to seeing these referring to some prior capturing parenthesis. But in this case, I'm not clear where the $1, $2, $3 and $4 were captured? It may be that I'm just not familiar with this .htaccess feature yet.

A) There is a newer strategy already, that uses the webpUrl image only, and detects if it is available. If not, it checks for a jpeg and then for a png and redirects to it.
If you primarily want to send webp, all webp supporting browsers will display it direct. Only for not supporting browsers a redirect gets issued.

    # Redirect regular PW WEBP images to JPEG or PNG where the Browser doesn't support WEBP
    # or where the WEBP file is missing:

    ## JPEG
    RewriteCond %{HTTP_ACCEPT} !image/webp   [OR]
    RewriteCond %{REQUEST_FILENAME} !-f
    RewriteCond %{DOCUMENT_ROOT}/$1$2$3/$4.jpg -f
    RewriteRule ^(.*?)(site/assets/files/)([0-9]+)/(.*)\.webp(.*)$  /$1$2$3/$4.jpg [R=307,L]

    ## PNG
    RewriteCond %{HTTP_ACCEPT} !image/webp   [OR]
    RewriteCond %{REQUEST_FILENAME} !-f
    RewriteCond %{DOCUMENT_ROOT}/$1$2$3/$4.png -f
    RewriteRule ^(.*?)(site/assets/files/)([0-9]+)/(.*)\.webp(.*)$  /$1$2$3/$4.png [R=307,L]

B) The 4 $variables are holding the parts from the RewriteRule. These are:
(.?) = are we in a subdirectory ? $1
(site/assets/files/) = is it an image under PW control ? $2
([0-9]+) = page id $3
(.
).webp = image filename without dot and extension $4
This seems to be common apache .htaccess feature, that it parses the RewriteRule first, and fills the $variables in the above RewriteCond with the found values.

https://httpd.apache.org/docs/current/mod/mod_rewrite.html#rewritecond

RewriteRule backreferences: These are backreferences of the form $N (0 <= N <= 9). $1 to $9 provide access to the grouped parts (in parentheses) of the pattern, from the RewriteRule which is subject to the current set of RewriteCond conditions. $0 provides access to the whole string matched by that pattern.

@ryancramerdesign
Copy link
Member

@ryancramerdesign ryancramerdesign commented May 23, 2019

Thanks @horst-n all makes sense now.

I'd forgotten that capturing parenthesis from the RewriteRule can be used in a prior RewriteCond, but now with coffee in hand, it's very clear how it works.

With this updated .htaccess, it looks like it would only use webp for cases where you'd called the $pageimage->webpUrl()? Whereas your previous iteration would have replaced jpg/png requests that could be fulfilled by a webp file, with the actual webp file (making it a more global drop-in webp upgrade). I like this approach. But it seems like the right strategy would depend on the context?

For people looking to upgrade to webp images without updating their site code, wouldn't the previous .htaccess strategy have been a better way to go? I did read in the thread about how people mentioned saving an image off of a website, and the drawback of having a jpg extension file with webp content in it, but I really don't think that matters for most? For me at least, I don't design websites for that purpose (people scraping the images off of it). Many might even consider this behavior a benefit as it reduces the chances of someone repurposing images they scraped from your site.

Basically, I don't think the issue described above (scraping) is one that should affect the approach. I also think it's better to avoid redirects when possible, because they double-up those requests and might potentially have SEO implications. But all in all, it seems like the best .htaccess strategy just depends on the case. Would you agree, or do you think the new version is definitely preferable?

@LostKobrakai
Copy link
Contributor

@LostKobrakai LostKobrakai commented May 23, 2019

For me at least, I don't design websites for that purpose (people scraping the images off of it). Many might even consider this behavior a benefit as it reduces the chances of someone repurposing images they scraped from your site.

I would not be that quick. From the user saving your images' point of view it's just bad UX, true, but there are also things like google images, which dynamically generates thumbnails based on the images found on sites. HTTP based thumbnailing solutions (+ CDN) generally become more and more popular (https://github.com/thephpleague/glide and lot of others / other languages) and there might be tools out there using those e.g. for thumbnails of received webmentions (newer improved alternative to pingbacks) or similar. To a degree those probably don't trust the extension, but also consult the image content to determine the type, but the enduser is certainly not the only receiver of a file with an incorrect extension.

What I would support is using images without extension and serve the file with correct content-type by browser support.

@TomS-
Copy link

@TomS- TomS- commented May 23, 2019

@LostKobrakai I must admit I'm a very big fan of glide and image as a service. However, I should imagine to implement this into ProcessWire would be a very big task. But it would be a great way of modernising ProcessWire in the regards of image processing. It also solved problems previously talked about on when to choose ImageMagick and how to handle that.

@LostKobrakai
Copy link
Contributor

@LostKobrakai LostKobrakai commented May 23, 2019

I'm not suggesting we should use glide in ProcessWire, but rather that tools like glide might also be clients of the data we serve, not just a user sitting in front of a webbrowser.

@horst-n
Copy link
Contributor Author

@horst-n horst-n commented May 23, 2019

Hi @ryancramerdesign ,

With this updated .htaccess, it looks like it would only use webp for cases where you'd called the $pageimage->webpUrl()? Whereas your previous iteration would have replaced jpg/png requests that could be fulfilled by a webp file, with the actual webp file (making it a more global drop-in webp upgrade). I like this approach. But it seems like the right strategy would depend on the context?

Yes, it definetly depends on the context. And you are right, the last .htaccess version is my preference for new sites, not the one for upgrading existing ones without touching the markup code.


For people looking to upgrade to webp images without updating their site code, wouldn't the previous .htaccess strategy have been a better way to go? I did read in the thread about how people mentioned saving an image off of a website, and the drawback of having a jpg extension file with webp content in it, but I really don't think that matters for most? For me at least, I don't design websites for that purpose (people scraping the images off of it). Many might even consider this behavior a benefit as it reduces the chances of someone repurposing images they scraped from your site.

The DropIn version looks for jpegs and rewrite (or redirect) to webp format. The difference between rewrite or redirect only is to add a [R=307] to the rewrite rule for redirecting, or leave it and only rewrite.

Additionally with that approach, to rewrite all site/assets/files jpegs it seems to be useful to skip images viewing in the admin, that contain a query string beginning with ?nc

        ## With an added GET var or GET value from the PW Page Editor, we do send the original JPEG or PNG
        RewriteCond expr "! %{QUERY_STRING} -strmatch 'nc=*'"

Basically, I don't think the issue described above (scraping) is one that should affect the approach. I also think it's better to avoid redirects when possible, because they double-up those requests and might potentially have SEO implications. But all in all, it seems like the best .htaccess strategy just depends on the case. Would you agree, or do you think the new version is definitely preferable?

Thats exactly how I see it, personally, and how I will handle it, depending on the contexts. But, for the official PW distribution and for NEW SITES, I would suggest that version with webpUrls and 307 redirect, as it should be assumed that the webp files are all created and available. So only old browsers have to redirect, but google and other SE direct see webp files and do not force redirects. Also all delivered files then have correct filetypes.

IMO this should be in the .htaccess file, as a commented section by default, and / or this and the other solutions should be readable and explained somewhere prominent in the docs. So the users can select their preferred solution, depending on the contexts. :)

@ryancramerdesign ryancramerdesign merged commit 6e580f7 into processwire:dev May 23, 2019
@ryancramerdesign
Copy link
Member

@ryancramerdesign ryancramerdesign commented May 23, 2019

@horst-n Thanks, all sounds good. I've gone ahead and merged your PR, and then I added another commit to it with several changes. The main thing is that I wanted to have some implementation of "extra" files for Pagefile/Pageimage objects, that could apply to any future webp-like format, or any extra files that might accompany Pagefile/Pageimage objects and should managed alongside them. While webp is the only "extra" format at present, I feel better just to have it wrapped around something that can be more easily be extended when/if needed.

From the API side, it's not much different except that rather than calling $image->webpUrl and $image->webpFilename, etc., you can call $image->webp->url and $image->webp->filename, etc. Though technically it still works if you call the $image->webpUrl style versions as well. But I put it in a $webp (PagefileExtra) object because it enabled me to implement a lot more related methods in their own class rather than putting more in Pageimage. Plus, all the code that works with it is managing "extras" (of any kind) rather than all being webp-specific code, which will help if we add any more formats like this. And it works with either Pagefile or Pageimage objects.

The other thing I changed is that I wanted it to create a webp version automatically if I called $image->webp->url() (or $image->webpUrl), so it does that now too. That's because if I'm calling a url() method that's usually because I plan to output it in an <img> tag, so didn't want to get a blank string back, and instead wanted it to go ahead and create the webp image. This only works on size variations, not originals, since webp images require a variation to be created. Of course, the existing webpAdd option is also good as well.

Another thing I changed is the ImageMagick webp detection, because I wasn't so sure I liked output buffering phpinfo() to detect that, when it looks like IMagick has a method that is already designed for this (Imagick::queryFormats). Though let me know if you found this isn't a good way to go.

Lastly, I wasn't able to get any webp support working in my dev environment (neither GD or IMagick), so I don't really have a way of testing it locally. Though did upload a version of the core with webp to my web server, and it appeared to work with the GD implementation... actually it worked shockingly well, in my case taking a 64kb PNG at 1200px resized down to 500px resulting in a 41k png and a 7k webp, with no visible difference (wow). Thanks again for implementing all of this in PW! This is a really nice thing to have in PW.

@adrianbj
Copy link
Contributor

@adrianbj adrianbj commented May 23, 2019

In case this is helpful for others debugging problems with WebP support, Tracy's "Versions List" now includes more details about GD and Imagick versions, and what formats they support.

@horst-n - let me know if you think I should add any other info to these sections.

Server Details

Software Version
ProcessWire 3.0.131
PHP 7.3.3
Webserver Apache/2.4.38 (Unix)
MySQL 8.0.15
Server Settings
Parameter Value
allow_url_fopen 1
max_execution_time 120 (changeable)
max_input_nesting_level 64
max_input_time 60
max_input_vars 1000
memory_limit 128M
post_max_size 8M
upload_max_filesize 20M
xdebug
xdebug.max_nesting_level
mod_rewrite 1
mod_security *confirmed off
EXIF Support 1
FreeType 1
GD Settings
Parameter Value
Version bundled (2.1.0 compatible)
GIF 1
JPG 1
PNG 1
WebP 1
iMagick Settings
Parameter Value
Version 7.0.8
GIF 1
JPG 1
PNG 1
SVG 1
PDF 1
WebP 1
Module Details
Module ClassName Version
AdminActionsUnorderedListToPages 0.1.5
AdminOnSteroids 2.0.16
AdminRestrictBranch 1.0.9
AutoTemplateStubs 0.1.1
BatchChildEditor 1.8.18
BreadcrumbDropdowns 0.1.19
CookieManagementBanner 0.4.6
EmailNewUser 1.1.10
FieldtypeColor 1.0.9
FieldtypeDecimal 1.0.1
FieldtypeMarkup 1.0.9
FieldtypeMultiplier 0.1.3
FieldtypePageIDs 0.11.0
FieldtypePhone 3.1.0
FieldtypeRockGrid 0.0.20
FieldtypeRuntimeOnly 0.1.2
FieldtypeStreetAddress 1.1.1
FieldtypeTable 0.1.9
FieldtypeVerifiedURL 0.0.1
FormBuilder 0.3.4
InputfieldColor 1.0.9
InputfieldDecimal 1.0.0
InputfieldFormBuilderFile 0.0.2
InputfieldMultiplier 0.1.1
InputfieldPhone 3.1.0
InputfieldRockGrid 0.0.18
InputfieldSelectize 1.1.0
InputfieldSelectizeAjax 1.1.0
InputfieldStreetAddress 1.1.1
InputfieldTable 0.1.9
JquerySelectize 1.0.4
MarkupLoadRSS 2.0.0
MarkupSEO 2.0.1
MarkupSitemap 0.4.1
ModuleReleaseNotes 0.10.8
ModuleSettingsImportExport 0.2.9
PageEditSoftLock 1.0.1
PageProtector 2.0.5
PageRenameOptions 1.0.5
PageSnapshot 2.0.1
PasswordForceChange 1.0.3
ProcessAdminActions 0.7.10
ProcessChangelog 1.5.7
ProcessChangelogHooks 1.2.2
ProcessChildrenCsvExport 1.8.18
ProcessCustomUploadNames 1.2.8
ProcessEmailToPage 1.3.1
ProcessFormBuilder 0.3.4
ProcessGetVideoThumbs 1.1.4
ProcessHannaCode 0.2.0
ProcessLoginHistory 1.4.3
ProcessLoginHistoryHooks 1.2.1
ProcessMigrator 0.7.7
ProcessPageEditSoftLock 1.0.1
ProcessPageFieldSelectCreator 0.5.7
ProcessPageListerPro 1.1.1
ProcessProfilerPro 0.0.1
ProcessRockFinder 0.0.3
ProcessSelectorTest 1.1.3
ProcessTableCsvExport 2.0.9
ProcessTerminal 1.0.0
ProcessTracyAdminer 1.0.6
ProcessVersionControl 2.1.3
ProcessWireUpgrade 0.0.7
ProcessWireUpgradeCheck 0.0.7
ProfilerPro 0.0.1
RestrictMultiLanguageBranch 0.1.2
RestrictTabView 1.1.3
RockFinder 1.1.2
RockMigrations 0.0.1
RockMigrationsDemoModule 0.0.1
RockModuleCreator 0.0.1
TableCsvImportExport 2.0.9
TemplateFieldWidths 0.1.9
TestModule 0.0.1
TextformatterHannaCode 0.2.0
TextformatterTagParser 2.2.0
TfaEmail 0.0.1
TfaTotp 0.0.1
TracyDebugger 4.19.13
VersionControl 2.1.6
WireMailSmtp 0.3.0

@horst-n
Copy link
Contributor Author

@horst-n horst-n commented May 24, 2019

@ryancramerdesign This is very good news. :)

@horst-n Thanks, all sounds good. I've gone ahead and merged your PR, and then I added another commit to it with several changes. The main thing is that I wanted to have some implementation of "extra" files for Pagefile/Pageimage objects, that could apply to any future webp-like format, or any extra files that might accompany Pagefile/Pageimage objects and should managed alongside them. While webp is the only "extra" format at present, I feel better just to have it wrapped around something that can be more easily be extended when/if needed.

This is awesome! You already implemented this.

From the API side, it's not much different except that rather than calling $image->webpUrl and $image->webpFilename, etc., you can call $image->webp->url and $image->webp->filename, etc. Though technically it still works if you call the $image->webpUrl style versions as well. But I put it in a $webp (PagefileExtra) object because it enabled me to implement a lot more related methods in their own class rather than putting more in Pageimage. Plus, all the code that works with it is managing "extras" (of any kind) rather than all being webp-specific code, which will help if we add any more formats like this. And it works with either Pagefile or Pageimage objects.

Cool.

The other thing I changed is that I wanted it to create a webp version automatically if I called $image->webp->url() (or $image->webpUrl), so it does that now too. That's because if I'm calling a url() method that's usually because I plan to output it in an <img> tag, so didn't want to get a blank string back, and instead wanted it to go ahead and create the webp image. This only works on size variations, not originals, since webp images require a variation to be created. Of course, the existing webpAdd option is also good as well.

And this is SuperCool! Now everyone can add webp support to existing sites by simply changing the markup from $image->width(200)->url to $image->width(200)->webp->url.
WOW!

Another thing I changed is the ImageMagick webp detection, because I wasn't so sure I liked output buffering phpinfo() to detect that, when it looks like IMagick has a method that is already designed for this (Imagick::queryFormats). Though let me know if you found this isn't a good way to go.

Wasn't aware of Imagick::queryFormats.

Lastly, I wasn't able to get any webp support working in my dev environment (neither GD or IMagick), so I don't really have a way of testing it locally. Though did upload a version of the core with webp to my web server, and it appeared to work with the GD implementation... actually it worked shockingly well, in my case taking a 64kb PNG at 1200px resized down to 500px resulting in a 41k png and a 7k webp, with no visible difference (wow). Thanks again for implementing all of this in PW! This is a really nice thing to have in PW.

Maybe this can help to update your local environment:
https://processwire.com/talk/topic/14236-webp-support/?do=findComment&comment=185046

@horst-n
Copy link
Contributor Author

@horst-n horst-n commented May 24, 2019

@ryancramerdesign only one minor thing I want to ask you to change it, is the HTML output of the verboseDebugInfo.

$return = "<pre>" . $this->wire('sanitizer')->entities($content) . "</pre>";

Can we remove the sanitizer and let $output as is? With entity encoding the thumb isn't displayed as image but as source string. I think that sanitizing is not a must here, as we only retrieve image and system informations. Or do I oversee something?

And can we change <pre> to <pre style="overflow: auto;">?

@ryancramerdesign
Copy link
Member

@ryancramerdesign ryancramerdesign commented May 24, 2019

@horst-n For that <pre> tag I was worried the inline styles might prevent people from styling it the way they wanted, but I hadn't had a chance to test it yet, so didn't realize I'd screwed it up (sorry) — I will remove that sanitizer and add that overflow style.

I was able to get ImageMagick webp support working locally (Noboru's post got me going). Seems to be working well now, but I noticed the IMagick ImageSizerEngine is failing on me for several cases and I tracked it down to the validSourceImageFormats() which only has JPG, JPG, PNG24. I went ahead and added PNG to it, and now all is working well. Just wondering why PNG, PNG8 and GIF are excluded? Is it alright if I add them back in, or does IMagick have an issue with these formats?

@ryancramerdesign
Copy link
Member

@ryancramerdesign ryancramerdesign commented May 24, 2019

@horst-n In the latest commit I went ahead and implemented that webpOnly option that you'd started, so that it's now supported in GD and IMagick engines. This was so that I could add support for the $pageimage->webp->url() to also work on the original (non-resized) images. That way we can create webp files without having to create the corresponding png/jpg.

I also found a couple of cases where the webp file was larger than the jpg file, though very rare. But since the whole point is to have a smaller file, I updated the webp->url() to return the original pageimage URL if for some reason it is smaller than the webp version. This can be disabled by calling webp->url(false) instead. It also falls back to the pageimage URL if for some reason it can't create a webp version, like if it's not supported on the server.

@horst-n
Copy link
Contributor Author

@horst-n horst-n commented May 25, 2019

@ryancramerdesign This is really awesome! So now there seems to be nothing left for the moment in regard of webp support.

@ryancramerdesign
Copy link
Member

@ryancramerdesign ryancramerdesign commented May 26, 2019

@horst-n Thanks! One question remaining about an update I made to ImageMagick and just wanted to make sure this was okay:

I noticed the IMagick ImageSizerEngine is failing on me for several cases and I tracked it down to the validSourceImageFormats() method, which only has JPG, JPG, PNG24. I went ahead and added PNG to it, and now all is working well. Just wondering why PNG, PNG8, GIF, GIF87 are excluded? Is it alright that I added them back in, or does IMagick have an issue with these formats?

@teppokoivula
Copy link
Contributor

@teppokoivula teppokoivula commented May 26, 2019

I did read in the thread about how people mentioned saving an image off of a website, and the drawback of having a jpg extension file with webp content in it, but I really don't think that matters for most? For me at least, I don't design websites for that purpose (people scraping the images off of it). Many might even consider this behavior a benefit as it reduces the chances of someone repurposing images they scraped from your site.

I'm a bit late to this party, but in my experience saving images from the site is actually a common requirement. A lot of the sites I've built have featured image galleries, press sections, etc. with images intended to be downloaded.

I'm actually not at all sure if Google would know how to handle WebP data served with JPG or PNG extension, but that's definitely another factor to consider. These days SEO isn't just about optimising textual content – it's also about optimising images so that they turn up in image search, which in turn results in traffic to the site.

@horst-n
Copy link
Contributor Author

@horst-n horst-n commented Jun 2, 2019

@horst-n Thanks! One question remaining about an update I made to ImageMagick and just wanted to make sure this was okay:

I noticed the IMagick ImageSizerEngine is failing on me for several cases and I tracked it down to the validSourceImageFormats() method, which only has JPG, JPG, PNG24. I went ahead and added PNG to it, and now all is working well. Just wondering why PNG, PNG8, GIF, GIF87 are excluded? Is it alright that I added them back in, or does IMagick have an issue with these formats?

Hi @ryancramerdesign , it is ok to add them back in. There are no issues with the formats I'm aware of.

@horst-n
Copy link
Contributor Author

@horst-n horst-n commented Jun 7, 2019

@ryancramerdesign Please you must revert the Imagick Suport for PNG8 !!! Best ASAP,
see: https://processwire.com/talk/topic/21645-pw-30132-%E2%80%93%C2%A0core-updates/?do=findComment&comment=186879 and following.

@AndZyk
Copy link

@AndZyk AndZyk commented Jul 1, 2019

@ryancramerdesign Maybe you haven't seen this issue or you plan to resolve it later, but I just wanted it to mention again, because in my opinion this is a real issue, which should be fixed fast. ;-)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Linked issues

Successfully merging this pull request may close these issues.

None yet

8 participants