Split out image-specific aspects of Optimization Detective to dependent plugin

[westonruter]

Feature Description

As discussed at depth in #1046, Optimization Detective is to be an infrastructure/framework/API dependency plugin which isn't promoted directly to end-users. Instead, a dependent plugin like "Image Loading Optimization" (or perhaps "Image Prioritizer") would be developed which extends Optimization Detective with the image-related aspects.

To do this, we will need to ensure that Optimization Detective exposes an API that allows dependent plugins to hook into the detection and optimization process. For detection, I'm assuming this looks like a PHP filter that allows additional tags to be targeted for visiting by the tag processor. And then for optimization, this could look like an action that fires at each such tag which is visited.

[joemcgill]

I like this idea, but wonder what the scope of a new Image Loading Optimization plugin would include and whether using client side data from the Optimization Detective Framework would be an optional feature of this type of plugin or if it would be a base requirement. For example, I could see a plugin that includes some of all of these options:

• Serve modern image formats (e.g., WebP, AVIF, etc.)
• Improve responsive image handling (e.g., auto-sizes, calculation improvements, srcset generation enhancements, etc.)
• Improve resource loading: (fetch-priority, lazy-loading, etc)
• CDN integration
• Use client side optimization enhancements

Alternatively, I could see some confusion if we had a standalone plugin for Image Loading Optimization, which was only optimizations built using the Optimizations Detective API but also had individual feature plugins for things like auto-sizes, sizes calculation ehancements, webp, etc. that were not based on the new API.

[felixarntz]

@joemcgill The scope of the "Image Loading Optimization" (which now may be renamed to "Image Prioritizer") would be the same as what Optimization Detective currently includes as a proof of concept: It's focused on applying fetchpriority="high" and loading="lazy" correctly. Any other image related features should be in separate plugins, such as for instead modern image formats being part of the "Modern Image Formats" plugin (formerly "WebP Uploads").

In other words, the image loading optimization plugin would have "Optimization Detective" as a dependency (i.e. required), since without it all it could do is enhance the limited server-side heuristics that core already comes with.

[felixarntz]

@westonruter I think a good approach to refine the API to be flexible enough for their potential use-cases is to think through the additional ideas that the client-side detection could be used for. I believe you had mentioned several such ideas somewhere, so maybe we can aggregate them here and then work through them to define an API specification that would cover all of them (and hopefully similar other use-cases).

[joemcgill]

Thanks for clarifying @felixarntz. I also think the proposed new name is much more clear and limits the scope to just when an image is requested and not other optimizations like file format or responsive image strategies.

[westonruter]

As for the use cases. Image Prioritizer scope I think includes:

• ✅ Add fetchpriority=high
• Lazy-load images which are not visible in any breakpoint #1077
• Optimization Detective: Implement lazy-loading of background images #1035
• Add fetchpriority=low for above-fold images which aren't visible (e.g. hidden in mega menu or appears later in carousel).
• Improve sizes attribute based on client-side measurements (when not adding auto).
• Add missing width and height attributes to img (and video) elements. See Visual tool to scan a page and report missing width/height elements #10.

Outside Image Prioritizer:

• Integrate Embed Optimizer with Optimization Detective to skip lazy-loading LCP embeds #1054
• Auto Sizes should extend Optimization Detective to add sizes=auto when also lazy-loading images #1099
• Preconnect to domains of resources for LCP elements known to be in embeds when in initial viewport #1058
• Reserve space to reduce layout shift (perhaps scoped specifically to embeds).
• Add content-visibility to improve rendering performance.
• Prioritizing the loading of fonts used in LCP elements with text.
• Prioritize loading poster image of video LCP elements #1183

The extensibility should allow:

1. Customizing which elements are in scope for detection and optimization (currently only img tags and elements with inline background images are measured).
2. Extending the URL Metrics schema with additional metrics.
3. Injecting additional JavaScript code into the detection process to amend the submitted URL Metric. (PHP API can allow passing a list of JS module URLs which then get dynamically imported in the detect() function. Each JS module would follow common interface.)

[felixarntz]

@westonruter I agree with your list, except for:

Improve sizes attribute based on client-side measurements (when not adding auto).

That IMO isn't related to prioritizing images and therefore should be in its own plugin that could depend on Optimization Detective. In fact, @joemcgill is exploring such optimization already.