Add initial detection logic for Image Loading Optimization #876

westonruter · 2023-10-26T00:30:46Z

Summary

Add logic to detect the LCP element and images and elements with background images which are in the initial viewport.

This is currently a sub-PR of #875.

Relevant technical choices

The logic is contained in a detect.js script which gets imported as a module from an inline module script. This module script is currently printed to the page with every page load, but once storage (#871) is implemented, it will be conditionally added based on whether no existing metrics are available.

The logic in detect.js only runs if it has been invoked within 5 seconds of the page being generated. This ensures that if the page output is cached, any future conditional serving of the detection script will not end up resulting in many clients running the logic unnecessarily when page caching is involved. The 5-second window can be filtered by perflab_image_loading_detection_time_window.

The detection logic involves gathering up a list of the optimizable elements, namely images and elements with background images (for now). For each of these elements, breadcrumbs are obtained which is an array of objects containing the tag name and element index. For example, the custom logo's breadcrumbs are:

[
    {
        "tagName": "HTML",
        "index": 0
    },
    {
        "tagName": "BODY",
        "index": 1
    },
    {
        "tagName": "DIV",
        "index": 2
    },
    {
        "tagName": "HEADER",
        "index": 0
    },
    {
        "tagName": "DIV",
        "index": 0
    },
    {
        "tagName": "DIV",
        "index": 0
    },
    {
        "tagName": "DIV",
        "index": 0
    },
    {
        "tagName": "A",
        "index": 0
    },
    {
        "tagName": "IMG",
        "index": 0
    }
]

These breadcrumbs are captured as soon as the module runs so that any subsequent dynamic insertions of new elements by JavaScript won't invalidate the breadcrumbs to look up the elements on the server-rendered page at the end of output buffering.

With the list of optimizable elements in hand, an IntersectionObserver is used to keep track of when they become visible (if ever). If the initial page load is not at the top of the page then the detection logic aborts since we don't want to capture metrics for an anchor link. Similarly, the IntersectionObserver disconnects as soon as the user starts scrolling. Elements contained inside of the admin bar are ignored.

In addition to observing for visible optimizable elements, the onLCP function from web-vitals.js library is used to obtain the LCP element metric. Currently onLCP is invoked with reportAllChanges as true as otherwise it will only fire when the user clicks or presses a key. So at the very least for development purposes, reportAllChanges is enabled. When the window's load event fires, then the latest LCP candidate metric reported is considered LCP.

Finally, page metrics are gathered to send to the server, including:

The viewport width and height
Metrics for each optimizable element, including:
- Whether it is LCP
- Whether it was an LCP candidate (i.e. second most important)
- Breadcrumbs
- Intersection ratio
- Intersection rectangle
- Bounding client rectangle

For example:

{
    "viewport": {
        "width": 3201,
        "height": 1644
    },
    "elements": [
        {
            "isLCP": false,
            "isLCPCandidate": false,
            "breadcrumbs": [
                {
                    "tagName": "HTML",
                    "index": 0
                },
                {
                    "tagName": "BODY",
                    "index": 1
                },
                {
                    "tagName": "DIV",
                    "index": 2
                },
                {
                    "tagName": "HEADER",
                    "index": 0
                },
                {
                    "tagName": "DIV",
                    "index": 0
                },
                {
                    "tagName": "DIV",
                    "index": 0
                },
                {
                    "tagName": "DIV",
                    "index": 0
                },
                {
                    "tagName": "A",
                    "index": 0
                },
                {
                    "tagName": "IMG",
                    "index": 0
                }
            ],
            "intersectionRatio": 1,
            "intersectionRect": {
                "x": 1000.40625,
                "y": 79.984375,
                "width": 120.015625,
                "height": 120.015625,
                "top": 79.984375,
                "right": 1120.421875,
                "bottom": 200,
                "left": 1000.40625
            },
            "boundingClientRect": {
                "x": 1000.40625,
                "y": 79.984375,
                "width": 120.015625,
                "height": 120.015625,
                "top": 79.984375,
                "right": 1120.421875,
                "bottom": 200,
                "left": 1000.40625
            }
        },
        {
            "isLCP": false,
            "isLCPCandidate": false,
            "breadcrumbs": [
                {
                    "tagName": "HTML",
                    "index": 0
                },
                {
                    "tagName": "BODY",
                    "index": 1
                },
                {
                    "tagName": "DIV",
                    "index": 2
                },
                {
                    "tagName": "MAIN",
                    "index": 1
                },
                {
                    "tagName": "DIV",
                    "index": 1
                },
                {
                    "tagName": "FIGURE",
                    "index": 0
                },
                {
                    "tagName": "IMG",
                    "index": 0
                }
            ],
            "intersectionRatio": 1,
            "intersectionRect": {
                "x": 1275.40625,
                "y": 466.328125,
                "width": 0.015625,
                "height": 0.015625,
                "top": 466.328125,
                "right": 1275.421875,
                "bottom": 466.34375,
                "left": 1275.40625
            },
            "boundingClientRect": {
                "x": 1275.40625,
                "y": 466.328125,
                "width": 0.015625,
                "height": 0.015625,
                "top": 466.328125,
                "right": 1275.421875,
                "bottom": 466.34375,
                "left": 1275.40625
            }
        },
        {
            "isLCP": true,
            "isLCPCandidate": true,
            "breadcrumbs": [
                {
                    "tagName": "HTML",
                    "index": 0
                },
                {
                    "tagName": "BODY",
                    "index": 1
                },
                {
                    "tagName": "DIV",
                    "index": 2
                },
                {
                    "tagName": "MAIN",
                    "index": 1
                },
                {
                    "tagName": "DIV",
                    "index": 1
                },
                {
                    "tagName": "FIGURE",
                    "index": 1
                },
                {
                    "tagName": "IMG",
                    "index": 0
                }
            ],
            "intersectionRatio": 1,
            "intersectionRect": {
                "x": 1275.40625,
                "y": 490.328125,
                "width": 300.015625,
                "height": 196.015625,
                "top": 490.328125,
                "right": 1575.421875,
                "bottom": 686.34375,
                "left": 1275.40625
            },
            "boundingClientRect": {
                "x": 1275.40625,
                "y": 490.328125,
                "width": 300.015625,
                "height": 196.015625,
                "top": 490.328125,
                "right": 1575.421875,
                "bottom": 686.34375,
                "left": 1275.40625
            }
        }
    ]
}

With these metrics in hand, the server can to store them (#871) and make use of them to optimize the loading of those elements on the page (#872).

Checklist

PR has either [Focus] or Infrastructure label.
PR has a [Type] label.
PR has a milestone or the no milestone label.

…erver

…nd all JS files

.gitignore

swissspidy · 2023-11-02T17:29:22Z

modules/images/image-loading-optimization/detect.js

+		} );
+	}
+
+	// TODO: Use a local copy of web-vitals.


Let me know if you'd like help with that

~~This todo isn't to be done in this PR. I'm working on this in #878.~~

Oops. I thought you commented on a different todo :)

I was presuming that there would be a JS build process introduced to the plugin as part of #864 and so we could incorporate the use of a local copy of web-vitals as part of that. But actually it looks like it's not going to involve JS. If you think a JS build process could be incorporated as part of the PR, I'd appreciate your help. But otherwise I think this can be tackled in a subsequent PR.

@westonruter Just looking through these changes now, I think it would be good to open a separate issue/PR for introducing a build process for JS. This can be against the feature branch because no other module needs it so far, and I don't anticipate any new module being introduced in the near future where it would be needed. So we can do that as part of this one. Let's just make sure the approach could be used by other modules too.

westonruter · 2023-11-02T18:27:28Z

modules/images/image-loading-optimization/detect.js

+	// TODO: Send data to server.
+	log( pageMetrics );


This is to be done in another PR. See #878.

joemcgill

Couple of questions, but generally this looks like a good start. No reason to block this for the feature branch.

joemcgill · 2023-11-06T20:00:53Z

.eslintrc.js

@@ -8,7 +8,15 @@ const config = {
 	rules: {
 		...( wpConfig?.rules || {} ),
 		'jsdoc/valid-types': 'off',
+		'no-console': 'off',


Not sure if it's a good idea to turn this off for the whole plugin. Perhaps we can just disable it where a console.log is expected instead?

Done in 912abef

joemcgill · 2023-11-06T20:09:21Z

modules/images/image-loading-optimization/detect.js

+	const pageMetrics = {
+		viewport: {
+			width: win.innerWidth,
+			height: win.innerHeight,
+		},
+		elements: [],
+	};


Curious how you're thinking of using various viewport dimensions in the data when applying optimizations?

If it turns out that that there are different LCP images for different viewports, then fetchpriority would not be added to the img elements themselves. Instead the viewport widths captured here would then be used to construct the media attribute of preload links so that only the relevant image URL is prioritized for the given viewport. In this way, the same markup can be served to both desktop and mobile, while both still get optimized.

felixarntz · 2023-11-08T21:36:41Z

modules/images/image-loading-optimization/hooks.php

+	 * Allow this amount of milliseconds between when the page was first generated (and perhaps cached) and when the
+	 * detect function on the page is allowed to perform its detection logic and submit the request to store the results.
+	 * This avoids situations in which there is missing detection metrics in which case a site with page caching which
+	 * also has a lot of traffic could result in a cache stampede.


Add skeleton for detection script

4dcef16

westonruter added [Focus] Images Issues related to the Images focus area [Type] Feature A new feature within an existing module labels Oct 26, 2023

westonruter added this to the PL Plugin 2.8.0 milestone Oct 26, 2023

westonruter added 20 commits October 26, 2023 08:58

Update comment to reflect TTLB not TTFB

7ca7e67

Import module rather than inlining it

721517a

WIP

e2212a8

Improve logging and remove obsolete code

37a603c

WIP2

31f5ffe

WIP3

d700943

WIP4

e9aee7e

Capture image breadcrumbs early; improve waiting for intersection obs…

685c25e

…erver

Add getBreadcrumbedElements() function

6d90d75

Add missing lint-js to lint-staged and update lint-js/format-js to fi…

00be1c7

…nd all JS files

Remove yet unused yieldToMain

c9518a1

Prevent detection if page is not scrolled to top

8984176

Stop observing images as soon as scroll happens or detection is stopped

62be41b

Extend observation to elements with background images

eb1d93f

Leverage Map

ad2e46c

Assemble page metrics to send; remove unneeded attribution build

9a50aac

Remove unused getElementBreadcrumbsMap

a5eaf3a

Remove obsolete cleanup code

0fcbc69

Move logic into getElementIndex helper function

c1e6d35

Use 3rd person singular for jsdoc

3871670

westonruter commented Nov 1, 2023

View reviewed changes

.gitignore Outdated Show resolved Hide resolved

westonruter marked this pull request as ready for review November 1, 2023 23:11

westonruter requested a review from felixarntz November 1, 2023 23:11

This was referenced Nov 1, 2023

Optimization Detective: Server-applied optimizations informed by client-side detection #869

Open

Add storage for URL metrics to optimize image loading #878

Merged

westonruter requested a review from swissspidy November 2, 2023 16:42

swissspidy reviewed Nov 2, 2023

View reviewed changes

westonruter commented Nov 2, 2023

View reviewed changes

Remove accounting for /wp-* symlinks in local dev

73dbfd1

westonruter force-pushed the add/image-loading-detection branch from b73f8e8 to 73dbfd1 Compare November 2, 2023 18:48

Base automatically changed from add/image-loading-optimization-module to feature/image-loading-optimization November 2, 2023 20:55

swissspidy approved these changes Nov 6, 2023

View reviewed changes

joemcgill approved these changes Nov 6, 2023

View reviewed changes

westonruter added 2 commits November 6, 2023 17:18

Undo turning off no-console eslint rule

912abef

Ignore webp-uploads/fallback.js from linting for now

01115e5

westonruter removed this from the PL Plugin 2.8.0 milestone Nov 7, 2023

westonruter merged commit a3e8c10 into feature/image-loading-optimization Nov 7, 2023
8 checks passed

westonruter deleted the add/image-loading-detection branch November 7, 2023 02:17

felixarntz reviewed Nov 8, 2023

View reviewed changes

westonruter added no milestone PRs that do not have a defined milestone for release [Plugin] Optimization Detective Issues for the Optimization Detective plugin labels Nov 17, 2023

This was referenced Nov 29, 2023

Optimize the loading of images using stored URL metrics #884

Merged

Add Web Worker Offloading module #556

Merged

Add JS build step to utilize local copy of web-vitals.js #893

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add initial detection logic for Image Loading Optimization #876

Add initial detection logic for Image Loading Optimization #876

westonruter commented Oct 26, 2023 •

edited

swissspidy Nov 2, 2023

westonruter Nov 2, 2023 •

edited

westonruter Nov 2, 2023

westonruter Nov 2, 2023

felixarntz Nov 8, 2023

westonruter Nov 2, 2023

joemcgill left a comment

joemcgill Nov 6, 2023

westonruter Nov 7, 2023

joemcgill Nov 6, 2023

westonruter Nov 6, 2023

felixarntz Nov 8, 2023

Add initial detection logic for Image Loading Optimization #876

Add initial detection logic for Image Loading Optimization #876

Conversation

westonruter commented Oct 26, 2023 • edited

Summary

Relevant technical choices

Checklist

Choose a reason for hiding this comment

westonruter Nov 2, 2023 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

joemcgill left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

westonruter commented Oct 26, 2023 •

edited

westonruter Nov 2, 2023 •

edited