Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Identifying and Optimising the LCP Image #117

Closed
dainemawer opened this issue Jan 20, 2022 · 28 comments
Closed

Identifying and Optimising the LCP Image #117

dainemawer opened this issue Jan 20, 2022 · 28 comments
Labels
Needs Discussion Anything that needs a discussion/agreement [Plugin] Image Prioritizer Issues for the Image Prioritizer plugin (dependent on Optimization Detective)

Comments

@dainemawer
Copy link
Contributor

This task is something that can be tricky to determine, but if we get it right, I think it could have huge positive effects on WordPress performance. Improving the loading performance of the Largest Contentful Paint image is one of the hallmarks of keeping LCP scores in the green. However, doing so has proven relatively tricky.

In order to reduce LCP, we need to consider a few things:

  1. The size (dimensions of the image)
  2. The resource size (mb) - is the image compressed etc
  3. How its served and where its served from (server vs CDN)
  4. How soon the browser renders the image

There are a number of things out of our control here, especially from a server, caching and infrastructure perspective. But we can look at preloading that image if we can identify that it is in fact the LCP on the page. This spoken about here on web.dev in detail: https://web.dev/optimize-lcp/#preload-important-resources especially for responsive images.

@sgomes
Copy link
Contributor

sgomes commented Jan 21, 2022

I'm afraid this goes beyond tricky. It's simply not possible to determine with certainty what the LCP will be (and whether it will be an image). Beyond the factors you mentioned, device screen sizes are also a huge factor, as is anything else that could be conditional to a device type, user, location, etc.

And this is assuming the image even appears in markup; a background image can easily become the largest element, and would be completely unknown to WordPress if defined in a theme CSS file, for example.

Since there is no generic way we can accurately determine what the LCP will be for a particular page view, my suggestion is to not try to do this as a platform, and instead leave it to the user to manually optimise if necessary.

@westonruter
Copy link
Member

The identification could be done client-side. I believe this is the approach taken with Jetpack Boost for generating the critical CSS. The frontend could be loaded up into an iframe with some PerformanceObserver code added. Once the iframe has finished loading, the PerformanceObserver could send the LCP element information to the parent page. Assuming that the LCP element (e.g. image) remains consistent across page loads, the URL for the LCP image (or font) could be stored so that a preload link could be automatically added. This process could happen in the editor after saving.

This would be similar to CLS mitigation that I've suggested in GoogleChromeLabs/layout-shift-terminator#5 and overall gathering if metrics in WordPress/gutenberg#33578 (comment).

A caveat about images: responsive images are only preloadable on Chromium-based browsers at the moment, since only they support imagesizes and imagesrcset (ref).

@sgomes
Copy link
Contributor

sgomes commented Jan 21, 2022

The identification could be done client-side.

Yes, definitely, the browser can determine that with 100% accuracy. However, that gives you the LCP for that particular client, which will not necessarily be the same across other devices, and might not even be the same for that client later (e.g. if the user rotates the device).

Common examples are different screen sizes and optional elements like cookie bars (which are only shown in a few countries), but WordPress is fully programmable and everything can be made conditional in any way you can think of.

Beyond this, the infrastructure to determine which element will be the largest for a given render is very non-trivial in the context of WordPress. It is certainly possible to implement an option to open up the site in a new window and determine what the LCP for that visit was, but I don't really see that working as anything more than a suggestion to the site owner, given all of the above. I do think that something that needs to be manually initiated like you suggest would be significantly better than automatically making the decision on behalf of the site owner, though!

@westonruter
Copy link
Member

How I see it could work is the page could be loaded in an iframe with 3 different sizes: mobile, tablet, desktop. If the same resource is common across all three, then it should definitely be preloaded. The iframe needn't be displayed to the user either. It can be done silently. After saving a post, there could be a secondary spinner with a message saying something like “Optimizing performance”. I think it can be handled automatically without requiring users to manually decide what the LCP element is.

@sgomes
Copy link
Contributor

sgomes commented Jan 21, 2022

If the same resource is common across all three, then it should definitely be preloaded.

I don't agree with that. Preload is a pretty heavy-handed approach, and it's easy for it to result in a de-optimisation instead.

Even if you account for different screen sizes and avoid preloading something if different image sizes are being used for different window sizes (which would greatly limit the usefulness of this feature, since multiple image sizes are a common best practice), perhaps you'll still be downloading the wrong image format if you assumed their browser supports or does not support a particular image type. Or maybe the image is coming from a CDN with multiple origins, and the user's browser picked a different one.

There are many more unknowables in the devices you're serving, and all kinds of variations beyond just screen size — and even there, limiting things to phone, tablet, and desktop is problematic as well. Attempting to automatically determine which resources to forcibly download will invariably lead to situations where the wrong thing is picked and the user's browser will be none the wiser about it, dutifully downloading a file at a high priority, and not making use of it in the end 😕

@westonruter
Copy link
Member

westonruter commented Jan 21, 2022

Humm. Well I don't agree, except for what you noted about different image sizes, which is what I mentioned above:

A caveat about images: responsive images are only preloadable on Chromium-based browsers at the moment, since only they support imagesizes and imagesrcset (ref).

Responsive image preloading (once supported by non-Chromium browsers) would only load the actual image needed for the device viewport.

The main difficulty with detecting the LCP image would be the browser cache. Once an image is cached, then something else could end up being the LCP. In that case, using PerformanceObserver probably wouldn't be feasible and it would be necessary to get a list of the loaded images (e.g. via performance.getEntriesByType('resource').map((r) => r.name)) and intersect them with elements that are in the viewport.

@dainemawer
Copy link
Contributor Author

I think, to a degree @sgomes - @westonruter suggestion can be scoped. For instance, LCP is only measured against items above the fold. So if the PerformanceObserver was running, then we could ensure that it was only checking for elements above-the-fold.

You're right in saying that it could potentially be any element, but again, that could be scoped: it's either going to be an image, video or text (only block-level text, which is easy enough to determine with JS). The text you cant preload, which then becomes about fonts and First Contentful Paint - and the video we could probably use the Facade method to improve.

At 10up, we've played around with trying to solve this manually and to be honest with you, I don't think WordPress is dynamic enough from a manual perspective, say writing a <link rel="preload" /> tag into the head to solve this problem either.

@westonruter suggestion of determining the common denominator across device sizes is a huge step in a direction that we couldn't even come close to without writing an extra query on every page to determine content, parse images and extract image data for preloading and even then, it's one dimensional.

@westonruter I'm not too sure how caching would affect all of this, but one other area of concern is content from Gutenberg, the user could easily change up content quite quickly, so we would need to be checking this on every post save / update. That being said, im wondering, if theres an opportunity here to bolster up how we do that on the post_save hook? I feel like a lot of performance checks / updates would need to be done through this hook (long term) considering the dynamic nature of Gutenberg

@sgomes
Copy link
Contributor

sgomes commented Jan 24, 2022

Well I don't agree, except for what you noted about different image sizes

I'm happy to go into more detail for any of those points! I don't really know what you disagreed with specifically, so I'll reformulate my comment in the hope that all of it becomes clearer. But please let me know if there's something specific you disagree with!

The gist of it is: if you preload the wrong thing, you end up making things worse. Hopefully we agree there?

I then listed a few ways we can end up accidentally preloading the wrong thing, and while the wrong image size was one of them, there are other ways we could get things wrong. It was meant to be one of several examples, not the focus of my post.

Here's another example: not every "mobile" viewport size is the same, so the LCP element might not be either. I recently dealt with a situation where text was the LCP element on a Moto G4, but on larger phones it was an image instead. That is, there's not enough granularity in a "phone", "tablet" and "desktop" split. We could try to increase granularity, but unless we account for every possible screen size, there will always be some sizes that we get wrong.

And ultimately, and much more importantly, since WordPress is fully customisable, everything could be made conditional in ways we don't know about and can't predict. For example, if a page has an image ad as its LCP and we preload that image, there's a strong likelihood that a different one will be loaded instead for an actual user, and we end up wasting bandwidth. This applies to not just ads, but anything else where random content could end up as the LCP.

And the same problem goes for things like geographically-dependant cookie bars, or dismissable newsletter subscription dialogs that only get shown the first time the user visits, both of which are real examples I've come across, and produce different LCP elements at the same screen size depending on conditions an automated approach couldn't account for without human intervention.

Responsive image preloading (once supported by non-Chromium browsers) would only load the actual image needed for the device viewport.

I wasn't very clear here, sorry about that. The full <picture> feature set is indeed supported in Chromium, including not just the preload equivalents for srcset and sizes, but also type and of course media. I don't dispute that.

The issue is that it's not supported elsewhere, and the unsupported behaviour is unsuitable to our purposes. Those browsers would download the wrong image, unless it happens to coincide with the src for the screen size in question. Taking the example in the article you link to:

<link rel="preload" as="image" href="wolf.jpg" imagesrcset="wolf_400px.jpg 400w, wolf_800px.jpg 800w, wolf_1600px.jpg 1600w" imagesizes="50vw">
[...]
<img src="wolf.jpg" srcset="wolf_400px.jpg 400w, wolf_800px.jpg 800w, wolf_1600px.jpg 1600w" sizes="50vw" alt="A rad wolf">

A desktop browser without support for the imagesrcset and imagesizes attributes would look at the preload, ignore these attributes, and fetch wolf.jpg because it's specified in href.

It would then look at the <img> tag and load something like wolf_800px.jpg or wolf_1600px.jpg instead because it supports srcset there.

( Edit: Don't just take my word for it; here's a modified fiddle that also uses imagesrcset, and ends up loading both the preload href image and one of the srcset images in Safari. )

That means we fetched wolf.jpg at high priority for nothing, which is very likely a performance regression.

Avoiding that would mean parsing and making a decision based on user agent strings, which is a bad practice that I would be strongly against, in this context.


So, to summarise, here are three facts that we can hopefully agree on:

  • Preloading the wrong resource can easily cause a performance regression.
  • We can't guarantee that our preload will download the correct resource in all situations, because there could be server-side or client-side conditions that an automated approach couldn't account for.
  • Preloading images with imagesrcset and imagesizes would often cause the wrong image to be preloaded in browsers that don't support these attributes on <link rel=preload>.

If we agree on all three of these, then it follows that, at the moment, implementing this feature should probably only be done without support for images (or at least without support for images where srcset or sizes was used), and even then it's not guaranteed that we'll get things right, making things worse when we don't.

This to me looks like a high-risk, low-reward scenario at the moment, which can only be reassessed once all WordPress-supported browsers support imagesrcset and imagesizes. And even then we can still easily get things wrong because of unknown server-side or client-side conditionals 😕

@sgomes
Copy link
Contributor

sgomes commented Jan 24, 2022

The main difficulty with detecting the LCP image would be the browser cache. Once an image is cached, then something else could end up being the LCP.

I don't think this is true. Caches will affect the LCP score, but they won't usually affect the LCP element. The LCP element is the largest qualifying element that is visible within the viewport before the user interacts with the page.

Caches won't usually affect an element's size, nor whether it renders inside or outside of the viewport. And if they do, you're probably dealing with much more serious issues than a poor LCP, like a massive FOUC of some sort.

@sgomes
Copy link
Contributor

sgomes commented Jan 24, 2022

I think, to a degree @sgomes - @westonruter suggestion can be scoped. For instance, LCP is only measured against items above the fold. So if the PerformanceObserver was running, then we could ensure that it was only checking for elements above-the-fold.

@dainemawer : My concern isn't so much the measurement part; that would be tricky to implement, but it would be feasible with the suggestions in this thread. And as for above-the-fold, as you point out, that's assumed to be the case when talking about LCP; below-the-fold elements that require user interaction to become visible (scrolling, in this case) can never become the LCP element.

My main concern is correctness. Since getting things wrong will likely result in making the situation worse than leaving things as-is, we need a pretty high degree of confidence that we're getting it right. I'm trying to demonstrate that it's impossible for us to have that degree of confidence, based on an automated solution, given the variability in screen sizes ("mobile" is in no way a single size), and the fact that there are many situations where users or devices can get different content, which could therefore mean different LCP elements.

@westonruter
Copy link
Member

Here's another example: not every "mobile" viewport size is the same, so the LCP element might not be either.

Agreed. But this is why I'm suggesting to look at the same page in the most common viewports for mobile, tablet, and desktop. If there is a common element that is the LCP across all three, then there would be a high level of confidence that it can be preloaded to improve the LCP metric.

  • Preloading images with imagesrcset and imagesizes would often cause the wrong image to be preloaded in browsers that don't support these attributes on <link rel=preload>.

The lack of cross-browser support for imagesizes and imagesrcset is indeed an issue, but I don't think it should necessarily block experimentation. Presumably Firefox and Safari will implement support for them, so WordPress could be ahead of the curve. It could spur Firefox and Safari to implement them as well, as I recall WordPress was an early adopter of responsive images which spurred browser vendors to implement support.

This to me looks like a high-risk, low-reward scenario at the moment, which can only be reassessed once all WordPress-supported browsers support imagesrcset and imagesizes. And even then we can still easily get things wrong because of unknown server-side or client-side conditionals 😕

It's true that a site can do anything. But what's more important is what 80%+ or 90%+ of sites do. If the vast majority of sites behave in a way that the detected LCP element can be preloaded reliably, then it would benefit more of the web then not doing so. In cases where it doesn't, the feature could just be opted-out of.

@sgomes
Copy link
Contributor

sgomes commented Jan 25, 2022

Agreed. But this is why I'm suggesting to look at the same page in the most common viewports for mobile, tablet, and desktop. If there is a common element that is the LCP across all three, then there would be a high level of confidence that it can be preloaded to improve the LCP metric.

Fair enough, I agree with that. I imagine it will somewhat limit the usefulness of this feature, but let's assume that it still provides enough value to justify its implementation, and so that we can continue the discussion.

The lack of cross-browser support for imagesizes and imagesrcset is indeed an issue, but I don't think it should necessarily block experimentation. Presumably Firefox and Safari will implement support for them, so WordPress could be ahead of the curve. It could spur Firefox and Safari to implement them as well, as I recall WordPress was an early adopter of responsive images which spurred browser vendors to implement support.

Responsive images were somewhat of a different situation, because it was an asymptotic improvement. Behaviour in browsers without support was no worse than before, and behaviour in browsers with support became better. This is the case for many "hint"-based APIs, where we offer new options to the browser if it knows what to do with them.

The situation here is different, in that we're discussing commanding browsers to do something (browsers can still ignore preloads, but in practice they rarely do), but only some browsers fully understand all the nuances of the command. The problem is that the browsers that don't fully understand the command will in all likelihood get worse performance than the status quo.

I won't argue that doing that could certainly push browser vendors to adopt this feature faster. But it seems dangerous and disrespectful of users to knowingly implement something that will make the situation worse for large numbers of them (WebKit is the only engine option in iOS/iPadOS, for example) without there having been any vendor signals that the feature will actually ever be implemented.

And I do acknowledge that you proposed this be done experimentally, but what form would that experimentation take? I'd be reluctant to enable it by default in Core because of the aforementioned reasons, and if it were an experimental option in Core that the site owner would have to enable, how many of them would? And how many of them would be aware of the tradeoff they're making, in that they're effectively making things slower for a potentially large chunk of their users? And is that enough to push browser adoption of the feature?

It's true that a site can do anything. But what's more important is what 80%+ or 90%+ of sites do. If the vast majority of sites behave in a way that the detected LCP element can be preloaded reliably, then it would benefit more of the web then not doing so. In cases where it doesn't, the feature could just be opted-out of.

Yes, I definitely agree that this is a numbers game, and that improving things for the majority of sites with the option to revert things in others would likely still be a good outcome.

However, this leads me to a different point that I haven't addressed yet, and is an assumption that's being made here: we're assuming that preloading the image will definitely make things better. This is not always the case.

Ultimately, bandwidth is limited, so loading priorities are a zero-sum game where moving something earlier means that something else (or everything else) gets moved later. Browsers use complex heuristics to determine what should be fetched first, and they adjust things continuously during the loading process as they discover new resources that must be fetched or complete important milestones such as layout, and preloads cut through most of that to force the browser to do something else instead.

A well-placed preload can certainly improve things, by e.g. starting an image download before the browser eventually discovers the background image two or three levels deep, in a CSS file somewhere. That's fine, but even there a preload is a bit of a blunt tool; the better solution is not to implicitly tie the HTML and CSS together with a preload in HTML and a reference to the same file in CSS (which can easily go wrong when someone decides to change the CSS to use a different image, for example), but rather to eschew the image reference in CSS altogether and instead reference the image in HTML. This allows the browser to discover the image early and use its usual heuristics to determine when to load it.

So let's set CSS-declared images aside for a second and look at images that are declared in the HTML instead, based on Chrome's priority rules. As per the reference, images that are declared in the HTML and will end up above the fold start at a Low priority, and get boosted to High priority once layout is complete and the browser realises they'll be above the fold. This means that's the window of improvement for an HTML image preload: boosting the image priority to High before layout completes; after that, the priority would be High anyway. Fair enough.

Will that produce an improvement in all situations? Not really. For example, if the image is cached, it won't be displayed any sooner, because it can only be displayed when layout is complete anyway.

But non-improvements are okay; the real problem is if we end up making things worse. So can preloading the correct LCP image make things worse? Yes, as it turns out!

Since classic, non-deferred, non-async scripts (the kind that are overwhelmingly most common in WordPress because of the enqueue mechanism) are loaded at a High priority, and since an image preload will be loaded at a High priority as well, we now run the risk of delaying first render if the script in question is in the head or sufficiently high up in the body, because we're fetching the image instead/as well. These scripts are blocking, so if they load later, the page will also render later. This will not make LCP any worse if we correctly identified the LCP image, but it will affect First Paint and FCP, which are important metrics in terms of user perception of performance. It's better for at least some of the page to render earlier (assuming that there are no layout shifts), than it is for it to fully render in one go, but later — Jake Archibald has an apt analogy for this, that he usually applies to frameworks instead.

I'm almost certain that these exact rules won't apply to WebKit and Gecko as well, but that's part of the problem; we have no guarantees that preloading the LCP image will improve LCP, nor that it won't make other metrics worse. For all I know, one of these engines could be giving preloads such a high priority that they can jump ahead of other critical, render-blocking resources like CSS as well.


Ultimately, I'm not against the idea of trying to determine the LCP element and surfacing it to the site owner. But I really do think that a preload is far too opinionated and far too forceful to be used in an automated approach like this, especially when we know that it will decrease performance in some browsers. I would certainly be more inclined to consider the importance attribute instead, because that effectively produces the same result and at least that one is ignored by browsers that don't support it, therefore not causing a high-priority downloading of the wrong thing. It's still very early for that, though, as it's still in origin trial and there are no signals from other vendors.

But even then, a useful rule in performance is that an improvement is not real until it's measured. Until it's measured, it exists in a quantum superposition of not making a difference, making things better, and making things worse 🙂

If you agree with me that it's possible for overall performance to get worse even if we use a fully supported browser, and even if we detect the correct image (and I would be interested to understand your reasoning if you disagree), then the discussion shifts back to the numbers game and which option we think is more likely to happen. I don't have any data on that, and I imagine it would be difficult to gather.

We ultimately don't know how many page visits will improve and how many will get worse based on this automated approach, and I'm personally not comfortable taking the risk of a net negative effect across all users, nor a net negative effect across all users of a particular (supported) browser in favour of a net positive for another.

@sgomes
Copy link
Contributor

sgomes commented Jan 25, 2022

I should note that Chrome DevRel's own Priority Hints explainer reads:

With the fix in Chrome 95 and the enhancement for priority hints, we hope that developers will start using preload for its intended purpose - to preload resources not detected by the parser (fonts, imports, background LCP images).

As such, the current proposal goes against Chrome's stated principles for how preloads should be used, unless we limit the discussion to CSS-declared background LCP images, or other LCP-affecting resources that get discovered late because they're not declared in the document, like fonts. Or unless we switch from preload to a different approach, like importance priority hints.

@eclarke1 eclarke1 added the Needs Discussion Anything that needs a discussion/agreement label Jan 25, 2022
@sgomes
Copy link
Contributor

sgomes commented Jan 26, 2022

After some further research into this, I'd be happy with supporting this proposal if it moved to Priority Hints instead of preloads.

Priority Hints solve some of my main concerns:

  • Browsers without full support maintain the status quo, instead of potentially suffering de-optimisations.
  • Prioritised images maintain all their existing resource selection logic very easily (preloads would make this difficult, because we'd e.g. have to unfold <picture> elements making use of media into multiple preloads and multiplex that with sizes and srcsets), and even allow for hypothetical future selection logic as well, since we're talking about a single, extra attribute that gets added to the image tag directly.
  • Prioritised images maintain source ordering at the same level of priority, which means they won't jump ahead of blocking scripts in the head, for example (a major concern in WordPress, because those are the default with the enqueue system). This avoids potentially de-optimising First Paint and FCP.
  • More generally, Priority Hints can feed into a broader set of heuristics and still leave room for the browser to handle prioritisation as it sees fit, unlike preloads, which effectively force the browser's hand.
  • They neatly avoid the issue of preloading a resource that doesn't show up in the document because, well, we can't add the importance attribute to something that isn't there!

That just leaves one risk I can think of, which is the risk of identifying the wrong LCP image through an automated process. That risk is greatly mitigated with @westonruter's suggestion of only selecting an image for optimisation if it shows up as the LCP element at three different screen sizes (phone, tablet, and desktop).

Priority Hints are still very new (currently in origin trial in Chromium, and in Chromium only), but the above means that we could start experimenting with them in the performance plugin today, and eventually promote the module to WordPress core if the experiment works well, and once at least one browser engine ships them unflagged, I'd suggest.

@westonruter @dainemawer : Does the above sound like a good plan?

@dainemawer
Copy link
Contributor Author

@sgomes with regards to running with Priority Hints is, will we see a decent enough improvement in LCP? The idea with preload is to ensure that the LCP image is part of the critical assets needed for page rendering - maybe we should run some tests to determine if there is in fact a significant difference in load time?

I guess my only other question is - what if LCP is an h1 and font related? I have some thoughts on the matter, but would be interested in hearing your side first

@sgomes
Copy link
Contributor

sgomes commented Jan 27, 2022

ensure that the LCP image is part of the critical assets needed for page rendering

@dainemawer I completely disagree with this approach at a conceptual level, as I mention in two of my posts.

To summarise, I strongly believe that making the LCP image part of the critical path is a bad idea, because that implies delaying first paint, which means a worse First Paint score, a very likely worse FCP score, and a worse user experience overall (because users are waiting longer for something to render). We shouldn't be trading a better score in one metric for a worse score in another two, when they're all important metrics.

As I see it, we want to make sure that the image loads as soon as possible, without delaying first render. importance="high" is our best bet for that, in my opinion, for the reasons I outlined in my previous post, and given the detailed information in the Priority Hints explainer that I linked to.

maybe we should run some tests to determine if there is in fact a significant difference in load time?

Testing performance changes is always the right call, as I mention in one of my above posts. It's not real until it's measured 🙂

For completeness, we should measure the preload option as well, and in particular what happens to first paint and FCP scores in both options.

And if we're doing that as an exercise in helping to inform which approach to take, we'll need to run these tests across a broad corpus of sites and pages, as well as screen sizes, to ensure that we get a representative sample of the extremely vast WordPress community and audience.

We might need to reach out to the Measurement group to see if they have any thoughts on how to do this.

I guess my only other question is - what if LCP is an h1 and font related? I have some thoughts on the matter, but would be interested in hearing your side first

Given how long the discussion has grown on images, my suggestion would be to start that conversation on a separate issue, or at the very least wait until we've reached a consensus on images here. I'd of course be happy to share my thoughts on that as well  🙂

In general, I apologise for the length of my posts and that of the resources I link to, but we do have a lot of ground we have to cover if we want to make an informed decision. These are complex topics that involve a lot of moving parts, and being aware of all of them will hopefully give us a better chance of not repeating past mistakes like (in some contexts) setting loading="lazy" on all images, including above-the-fold hero images.

@adamsilverstein
Copy link
Member

@sgomes Thank you for the detailed descriptions and feedback on this ticket, which I finally got caught up on.

I agree with most of your points about the potential tradeoff of prioritizing even the correct image and agree Priority Hints seems like a hopeful/safer path forward. On caveat: in terms of blocking scripts, I would argue most of those scripts could be deferred, we can tackle that issue elsewhere.

I think it is still worth exploring preload and measuring performance with some popular themes. When we look at WordPress data from httparchive, LCP is typically the leading offender/problem, and it is possible the approach makes things worse in only a small number of cases.

We ultimately don't know how many page visits will improve and how many will get worse based on this automated approach,

Good point. We might be able to get a signal on this from the httparchive data looking at how many sites have blocking js in head or reference css files there; I'm also looking into how many sites call document.write and plan to open some discussions on the forum there to work on that. I can also ask about

Restating this at the risk of repeating myself: one important thing we are trying to leverage here is that, as a CMS, we have a bit more information than the browser has when it begins loading the page. As you point out, sans priority hints, it actually takes a while before a browser could know what images are above the fold or likely to be the LCP. As a CMS we can often guess which image will be the LCP element. The goal here is to help the browser better prioritize that image.

In the priority hints explainer, it states:

Use the preload resource hint to download necessary resources earlier, particularly for resources that are not easily discovered early by the browser.

I would assume that does not apply to the typical LCP image, it would be discovered as soon as the html was loaded. However compared to using preload, maybe that means all header scripts would also have started loading (this is something we can test)?

@sgomes
Copy link
Contributor

sgomes commented Jan 27, 2022

Thank you, @adamsilverstein!

I agree with most of your points about the potential tradeoff of prioritizing even the correct image and agree Priority Hints seems like a hopeful/safer path forward. On caveat: in terms of blocking scripts, I would argue most of those scripts could be deferred, we can tackle that issue elsewhere.

Yes, I definitely agree with that as well! I think that blocking scripts are one of the main issues with WordPress right now, and hopefully something we can tackle as part of the performance group efforts. I'm trying to find the time to do some brainstorming around what a possible solution could look like, from a developer point of view.

However, there are some significant roadblocks there, which I suspect will lead to a need for new APIs and a slow migration process. Which is absolutely fine, but I just don't see that playing out in a short timeframe. Hopefully I'm wrong 🙂

With that in mind, I think it's probably best to handle this proposal separately, and hopefully come up with a good solution that works well whether we have blocking header scripts or not.

I think it is still worth exploring preload and measuring performance with some popular themes. When we look at WordPress data from httparchive, LCP is typically the leading offender/problem, and it is possible the approach makes things worse in only a small number of cases.

Yes, I think we can definitely take some measurements and compare both approaches. Nothing beats real data 🙂

At this stage, from a purely theoretical point of view, it seems that we can get the same benefits with the Priority Hints approach, while avoiding the biggest downsides, but it would be good to verify that.

Restating this at the risk of repeating myself: one important thing we are trying to leverage here is that, as a CMS, we have a bit more information than the browser has when it begins loading the page. As you point out, sans priority hints, it actually takes a while before a browser could know what images are above the fold or likely to be the LCP. As a CMS we can often guess which image will be the LCP element. The goal here is to help the browser better prioritise that image.

Absolutely, that's a great way of putting it and I completely agree! I guess the question is exactly how we want to convey this extra information to the browser. With a preload, you're giving it a direct command, whereas with Priority Hints you're giving it a hint, and asking it to prioritise accordingly.

Taking direct control is more dangerous to do cross-browser, because some browsers will not understand the full command (lack of support for imagesrcset and imagesizes), and they'll handle priorities differently than Chromium. Both of which can lead to de-optimisations. With Priority Hints, de-optimisations are much less likely, and you're more likely to end up with either an improvement or no change at all.

I would assume that does not apply to the typical LCP image, it would be discovered as soon as the html was loaded.

Right, the typical LCP image would be in the HTML source, so the issue of late discovery wouldn't apply. The exception would be CSS-defined background images, which I don't expect to be too common as the LCP element in general — based on purely anecdotal evidence, that is, which isn't worth much in all fairness.

However compared to using preload, maybe that means all header scripts would also have started loading (this is something we can test)?

If I'm reading the explainer correctly, preloaded images would take priority over header scripts (and maybe even CSS) in Chrome < 95. To quote:

Prior to Chrome 95, requests issued via <link rel=preload> always start before
other requests seen by the preload scanner, even if the other requests have a higher priority.

With Chrome 95, that changes, and it switches to depending on source order:

The placement of the preload hint will [now] affect when the resource is preloaded.

As far as I can tell, this would mean that declaring the preloads after the header scripts would make sure that the image download doesn't jump ahead of them, but only in Chrome >= 95. I don't know what the behaviour is in other browsers; they could be following either set of rules, or a completely different one.

I'm sure this is something we can test if we determine that preload is otherwise the better option.

However, even if we could guarantee correct ordering everywhere somehow, that would still leave us with the issue of handling browsers that aren't aware of imagesrcset, which would sadly in all likelihood still download the wrong image (the href one), whether that happens before or after the scripts.

Again, everything points to it being much safer to use Priority Hints; according to the explainer, prioritised images will still be set to High priority in Chromium, which is the same priority they'd get after the layout pass determines they're above the fold, and the same priority they'd have if they were preloaded. To me, that looks exactly like what we want: a high priority download that never delays the critical path.

@westonruter
Copy link
Member

One big challenge with using Priority Hints: injecting the importance attribute into the right image. For a preload it's relatively easy as it's just adding a new link anywhere in the head. But to inject an importance attribute to do so in a way that would catch all cases would require output buffering to locate the <img> tags.

@sgomes
Copy link
Contributor

sgomes commented Feb 2, 2022

Yes, I agree that it's technically more challenging. On the flip side, it does guarantee that the image actually exists, and avoids situations where, for whatever reason, the image being preloaded is no longer part of the document.

If the buffering approach proves too challenging, we could limit the scope to images in the post content, rather than building something generic enough to optimise theme images, or any images introduced by plugins.

@adamsilverstein
Copy link
Member

adamsilverstein commented Feb 4, 2022

One big challenge with using Priority Hints: injecting the importance attribute into the right image.

Wouldn't this be the same image we recently excluded from lazy loading? we could add the importance attribute the same way we add the loading=lazy attribute (a filter on the_content), the non lazy image would get the importance="high" attribute

@westonruter
Copy link
Member

It could be, yes. But I think there should be more verification done to ensure the non-lazy image should also be marked as important.

@westonruter
Copy link
Member

westonruter commented Oct 20, 2023

Circling back on this a year and a half later.

WordPress 6.3 included support for Priority Hints, which ended up naming the attribute fetchpriority instead of importance. WordPress assigns fetchpriority=high to the image which PHP-based heuristics determine is the most likely to be the LCP element. Nevertheless, as @felixarntz found in Analyzing the Core Web Vitals performance impact of WordPress 6.3 in the field:

The accuracy with which fetchpriority="high" is applied to the LCP image is only around 50% across all scenarios. While this is okay for the newly added support of the attribute, it is clearly something to follow up on. Getting the heuristics for applying fetchpriority right is even more challenging than not lazy-loading the LCP image especially since the LCP image may differ between different viewports, but it’s safe to say there should be more that WordPress core can do in that area. At least, it is relieving to see that the negative LCP impact of adding fetchpriority="high" to the wrong image is fairly low, compared to the negative LCP impact of lazy-loading the LCP image. See the source for fetchpriority accuracy against the LCP image and the source for LCP passing rate changes for sites that no longer lazy-load LCP image but use fetchpriority incorrectly.

As I shared above, I believe that client-side LCP element detection has the best chance at improving accuracy. (And using metrics from actual site visitors, not from simulated page loads with iframes. But I'll open other issue(s) for this specifically.) Update: See #869 for the overview issue.

One issue I've discovered is the commonality of responsive layouts having completely different images being displayed and thus there also being completely separate LCP images. I did an HTTP Archive query (GoogleChromeLabs/wpp-research#73) to see, for all pages in which both desktop and mobile have an image as the LCP element, in which fetchpriority=high was added correctly to either LCP image, the percentage of the time in which fetchpriority=high is not added to both of the images. I found that ~37% of the resulting pages fail to assign fetchpriority=high on both LCP images. This would be expected for themes with responsive design since the same markup is being served to both desktop and mobile. It also introduces a complication for using Priority Hints to optimize the LCP image since it would often be wrong for either desktop or mobile. However, preload links can solve this problem by preloading both the desktop and mobile images conditionally based on the supplied media query. (The lack of browser support in Safari can be accounted for by merely omitting the href attribute and only keeping imagesrcset and imagesizes.)

Another wrinkle for such cases when different images are shown for desktop and mobile is that an image without loading=lazy will be downloaded automatically even when it is display:none. So here it can actually be beneficial to add loading=lazy to the viewport-specific LCP image elements, as long as they get corresponding preload links.

I did an analysis to see what impact preloading has versus Priority Hints. I wrote a plugin that allows you to toggle fetchpriority=high, loading=lazy, and a preload link for the LCP image. I then used the benchmark-web-vitals command to do 100 requests to each desired permutation of those variables and obtain the median LCP for each. Here are the results:

image

Demo site URLs for 7 columns
  1. https://vary-lcp-optimizations.instawp.xyz/bison/?vary_lcp_image_optimizations%5Bimg_lazy_loading%5D=true&vary_lcp_image_optimizations%5Bimg_fetchpriority%5D=false
  2. https://vary-lcp-optimizations.instawp.xyz/bison/?vary_lcp_image_optimizations%5Bimg_lazy_loading%5D=true
  3. https://vary-lcp-optimizations.instawp.xyz/bison/?vary_lcp_image_optimizations%5Bimg_fetchpriority%5D=false
  4. https://vary-lcp-optimizations.instawp.xyz/bison/
  5. https://vary-lcp-optimizations.instawp.xyz/bison/?vary_lcp_image_optimizations%5Bpreload%5D=true
  6. https://vary-lcp-optimizations.instawp.xyz/bison/?vary_lcp_image_optimizations%5Bimg_fetchpriority%5D=false&vary_lcp_image_optimizations%5Bpreload%5D=true
  7. https://vary-lcp-optimizations.instawp.xyz/bison/?vary_lcp_image_optimizations%5Bimg_fetchpriority%5D=false&vary_lcp_image_optimizations%5Bimg_lazy_loading%5D=true&vary_lcp_image_optimizations%5Bpreload%5D=true
  • 🟥 The first two columns in red are expected to perform the very worst because there is no preloading and both are lazy-loaded; adding fetchpriority=high to a lazy-loaded image does not improve performance, and WordPress avoids this as much as possible.
  • 🟨 The third column in yellow is the case when WordPress doesn't add fetchpriority=high to the LCP image.
  • 🟩 The fourth column is the current best case in WordPress core when fetchpriority=high is added to the LCP image correctly.
  • ⬜ The fifth column is the same as the fourth, except with preload being added.
  • 🟦 The sixth and seventh columns omit fetchpriority=high from the LCP image but include a preload link, with the latter also adding loading=lazy.

These show that a preload link can be a viable alternative to Priority Hints when responsive layout results in a different LCP image depending on the viewport. Preloading the LCP image results in better LCP over just using Priority Hints, though granted this is just for this demo site which does not have blocking head scripts.

Note that the demo site includes five small images in the header because Chrome Resource Priorities and Scheduling
automatically loads the first 5 images larger than 10,000px² with high priority, so making the LCP image the 6th image ensures that we can see the effect of preloading.

All this being said, if the same element is the LCP image for both desktop and mobile, then the current approach to rely on Priority Hints is fine. Nevertheless, adding preload seems to improve LCP yet further at least on this demo site.

@sgomes
Copy link
Contributor

sgomes commented Oct 23, 2023

Thank you for this analysis, @westonruter!

I think something else that we need to consider is the cost of getting things wrong. If we're talking about the heuristics only getting things right 50% of the time, that means we're downloading an unnecessary image (at that stage) 50% of the time, potentially making LCP higher than if we were to do nothing.

That's not great as is, but using preload to fetch that image at a higher priority would make things even worse.

As such, I think it's premature to be talking about preloading when our heuristics get things wrong so often; I agree that our focus should be on improving our guesses.

But even if our heuristics were to improve, I still want to express my opposition to using preload on guessed LCP images altogether; remember that browsers that don't support imagesrcset in <link rel="preload"> (such as all WebKit browsers) will ignore that attribute and download the wrong image at a high priority. As I see it, that's reason enough to scrap that approach, as it's not fair to actively make things worse on a major engine in order to potentially improve scores slightly on the other two.

This sort of thing is exactly what fetchpriority was built for, and the results it brings are a better tradeoff between correct guess and wrong guess scenarios than preload would be, while gracefully degrading to a no-op in browsers without support.

@westonruter
Copy link
Member

@sgomes, thanks for the reply.

Thank you for this analysis, @westonruter!

I think something else that we need to consider is the cost of getting things wrong. If we're talking about the heuristics only getting things right 50% of the time, that means we're downloading an unnecessary image (at that stage) 50% of the time, potentially making LCP higher than if we were to do nothing.

That's not great as is, but using preload to fetch that image at a higher priority would make things even worse.

As such, I think it's premature to be talking about preloading when our heuristics get things wrong so often; I agree that our focus should be on improving our guesses.

Agreed. I did not intend to say that we add link preloads powered by the current PHP-based heuristics. We need client-side detection to obtain real user metrics to ensure the right elements are prioritized for the client viewport. I have written up a document for Image Loading Optimization via Client-side Detection. I'll be creating issues out of this.

But even if our heuristics were to improve, I still want to express my opposition to using preload on guessed LCP images altogether; remember that browsers that don't support imagesrcset in <link rel="preload"> (such as all WebKit browsers) will ignore that attribute and download the wrong image at a high priority. As I see it, that's reason enough to scrap that approach, as it's not fair to actively make things worse on a major engine in order to potentially improve scores slightly on the other two.

There's an easy way to work around this. Simply omit the href attribute while retaining imagesrcset. That being said, 75% of browsers currently support responsive preloads.

This sort of thing is exactly what fetchpriority was built for, and the results it brings are a better tradeoff between correct guess and wrong guess scenarios than preload would be, while gracefully degrading to a no-op in browsers without support.

Yes, I agree. In my last paragraph I say: "All this being said, if the same element is the LCP image for both desktop and mobile, then the current approach to rely on Priority Hints is fine. Nevertheless, adding preload seems to improve LCP yet further at least on this demo site." So if we can greatly improve the accuracy of detecting the LCP element across all viewports, we can start to be more aggressive about optimizing. But if the accuracy is not as high as we like, and there is only one LCP element across viewports, then we can rely on Priority Hints alone.

@sgomes
Copy link
Contributor

sgomes commented Oct 24, 2023

Thank you for clearing things up, @westonruter! Looks like I misunderstood you on a few points, sorry about that.

Agreed. I did not intend to say that we add link preloads powered by the current PHP-based heuristics. We need client-side detection to obtain real user metrics to ensure the right elements are prioritized for the client viewport. I have written up a document for Image Loading Optimization via Client-side Detection. I'll be creating issues out of this.

Thank you, and I agree that's a great step forward if we can pull it off! 👍

There's an easy way to work around this. Simply omit the href attribute while retaining imagesrcset.

That's a good approach! The spec seems to confirm this possibility, so the expectation is that all implementations going forward will handle a missing href correctly 👍

That being said, 75% of browsers currently support responsive preloads.

Right, we just need to ensure that for any feature we decide to use, we're not actively making things worse on unsupported browsers. Unsupported leading to a no-op is absolutely fine (that's textbook progressive enhancement), but unsupported leading to worse performance than before is something we should avoid. And your suggestion does indeed avoid it 👍

So if we can greatly improve the accuracy of detecting the LCP element across all viewports, we can start to be more aggressive about optimizing. But if the accuracy is not as high as we like, and there is only one LCP element across viewports, then we can rely on Priority Hints alone.

Yes, I generally agree, although I think we should probably run some more extensive testing before ramping up the aggressiveness. We need to account for a wide variety of scenarios in the performance numbers.

Ideally, we'd have a corpus of real sites we could test against. A possible automated approach would be to find existing fetchpriority attributes and use Chrome's local overrides feature to locally rewrite the HTML (to e.g. include preload as well), so that we could test multiple variations on real sites without access to the servers.

@westonruter
Copy link
Member

westonruter commented Dec 13, 2023

There's an easy way to work around this. Simply omit the href attribute while retaining imagesrcset. That being said, 75% of browsers currently support responsive preloads.

And Safari 17.2 now adds support for preloading responsive images!

image

In fact, this is now supported by >95% of user browsers. I don't think it's something we need to worry about anymore. So this code in the Image Loading Optimization module (#869) can be removed:

// Prevent preloading src for browsers that don't support imagesrcset on the link element.
if ( isset( $img_attributes['src'], $img_attributes['srcset'] ) ) {
unset( $img_attributes['src'] );
}


Bonus: Safari 17.2 also supports fetchpriority!

@westonruter
Copy link
Member

Now that the Image Prioritizer plugin has been merged into trunk, I'm going to close this as completed.

@westonruter westonruter added the [Plugin] Image Prioritizer Issues for the Image Prioritizer plugin (dependent on Optimization Detective) label Jun 5, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Needs Discussion Anything that needs a discussion/agreement [Plugin] Image Prioritizer Issues for the Image Prioritizer plugin (dependent on Optimization Detective)
Projects
None yet
Development

No branches or pull requests

5 participants