New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Lazyload images and iframes #3752

Open
wants to merge 9 commits into
base: master
from

Conversation

6 participants
@bengreenstein

bengreenstein commented Jun 8, 2018

@domenic

This is a draft of spec changes to support a lazyload attribute in iframe and img elements.


/embedded-content.html ( diff )
/iframe-embed-object.html ( diff )
/images.html ( diff )
/index.html ( diff )
/indices.html ( diff )
/infrastructure.html ( diff )
/urls-and-fetching.html ( diff )

@domenic

This comment has been minimized.

Member

domenic commented Jun 12, 2018

I did another minor pass; PTAL.

@domenic

Some issues with the metadata fetcher

source Outdated
following steps:

<ol>
<li><p>&#x231B; Let <var>request</var> be the result of <span data-x="create a potential-CORS

This comment has been minimized.

@domenic

domenic Jun 27, 2018

Member

No need for the &#x231B; parts here and elsewhere.

This comment has been minimized.

@bengreenstein
source Outdated
</span> to request as much data from the beginning of the image file as the user agent
determines is reasonable to retrieve metatdata.</p></li>

<li><p>&#x231B; <span data-x="concept-fetch">Fetch</span> <var>request</var>. Let this instance

This comment has been minimized.

@domenic

domenic Jun 27, 2018

Member

The first sentence here is good, but the others are not. This is not the image request, so it should not be associated as such. Instead, I'd say something like "Fetch request, and use the metadata contained in response as appropriate for the image."

This comment has been minimized.

@bengreenstein
source Outdated
<li><p>&#x231B; Set <var>request</var>'s <span
data-x-href="https://fetch.spec.whatwg.org/#concept-request-add-range-header">range header
</span> to request as much data from the beginning of the image file as the user agent
determines is reasonable to retrieve metatdata.</p></li>

This comment has been minimized.

@domenic

domenic Jun 27, 2018

Member

You'll want to explicitly refer to and use https://fetch.spec.whatwg.org/#concept-request-add-range-header here

This comment has been minimized.

@bengreenstein

bengreenstein Jun 28, 2018

I do. What am I missing?

This comment has been minimized.

@domenic

domenic Jun 28, 2018

Member

The issue is that you're not actually using the operation defined there. You're linking the text "range header" to it, but that's not how you call functions when writing software.

You need to actually invoke it, by using the right phrase ("add a range header") and passing it the right arguments (in this case, I believe, request, 0, and a user-agent-chosen value).

@domenic

Metadata fetching looks good to me editorially now. Eager to hear others thoughts on whether it provides the right foundation for interoperability.

Things were fixed

@annevk

You also need to update the element indexes to include this attribute for img and iframe.

Should this influence https://fetch.spec.whatwg.org/#concept-request-priority somehow?

source Outdated
@@ -25985,14 +26032,6 @@ interface <dfn>HTMLImageElement</dfn> : <span>HTMLElement</span> {

</dd>

<dt><var>image</var> . <code subdfn data-x="dom-img-decoding">decoding</code></dt>

This comment has been minimized.

@annevk

annevk Jul 25, 2018

Member

Shouldn't this be kept and something equivalent be added for lazyLoad?

This comment has been minimized.

@domenic

domenic Jul 25, 2018

Member

No. We don't generally have domintro blocks for IDL attributes that just reflect. Maybe we should removing decoding in a separate PR though.

This comment has been minimized.

@annevk

annevk Jul 26, 2018

Member

Yeah, let's do that.

source Outdated
policy</span> to the current state of the element's <code
data-x="attr-img-referrerpolicy">referrerpolicy</code> attribute.</p></li>

<li><p>Then <span data-x="concept-request-add-range-header">add a range header</span> to

This comment has been minimized.

@annevk

annevk Jul 25, 2018

Member

@jakearchibald could you please review this?

source Outdated
elements</span>. If the element's <code data-x="attr-iframe-lazyload">lazyload</code> attribute
is in the <span data-x="attr-lazyload-on-state">On</span> or <span
data-x="attr-lazyload-auto-state">Auto</span> states, and the element's CSS layout box does not
intersect the <span>viewport</span>, then running these steps may be deferred until such

This comment has been minimized.

@annevk

annevk Jul 25, 2018

Member

I don't think this works for about:blank and such, does it? What does this map to in say Blink's C++?

This comment has been minimized.

@scott-little

scott-little Dec 14, 2018

Since the wording already says "running these steps may be deferred" instead of "must", I don't think we really need to call out the trivial cases here, right?

@jakearchibald jakearchibald self-requested a review Jul 25, 2018

@jakearchibald

Also: img.decode() waits for the image fetch to complete. With lazyload, this may take a long time, or might never happen. Is that the intent? Feels like decode() is a strong signal that the developer wants the image to fetch.

source Outdated
user agent determines is reasonable to retrieve metatdata.</p></li>

<li><p><span data-x="concept-fetch">Fetch</span> <var>request</var>, and use the metadata
contained in the response as appropriate for the image.</p></li>

This comment has been minimized.

@jakearchibald

jakearchibald Jul 25, 2018

Collaborator

I'm not sure how much hand-waving is ok here, but this seems pretty hand-wavey.

We plan to use the metadata to lay out the image, right? Is that worth documenting as a possible consequence?

What happens if the response is a 404 or 500? Feels like we could call the image a failure at that point. We should probably do that for all not-ok responses, unless it looks like a consequence of making the range request (416).

What happens if the response is a 200? We'd download the whole image, would we download it again? Perhaps we'd add it to the image memory cache in that case? Or maybe terminate the fetch after fetching the metadata bytes?

What happens if the response is a 206, but we fail to parse the start? Again, we could call the image a failure, and not bother fetching any more.

What happens if the response is a 206, but the range starts at a point other than 0. It's pretty important to reject the response in this case, as reading it as if it were the start of the resource could leak data.

What happens if the response is a 206, starts at 0, but returns fewer bytes than requested?

This comment has been minimized.

@jakearchibald

jakearchibald Jul 25, 2018

Collaborator

What happens if, during the metadata fetch, the main fetch begins? What happens if the metadata fetch arrives later than the main fetch?

From earlier discussion around this feature, I think the intention is to throw away anything learned from this metadata once the main fetch happens. Eg, if the partial fetch tells us the size is 100x100, but the main fetch tells us it's 500x500, the image will be laid out at 500x500. This means we aren't merging the results of two responses, and that's pretty important for security & interoperability. It should probably be spelled out here.

This comment has been minimized.

@jakearchibald

jakearchibald Jul 25, 2018

Collaborator

In terms of validating the partial response, here's what I'm playing with in background fetch https://wicg.github.io/background-fetch/#validate-partial-response-algorithm.

This comment has been minimized.

@jakearchibald

jakearchibald Jul 25, 2018

Collaborator

I just remembered that we still load image data from a 404 etc, so ignore that part.

This comment has been minimized.

@bengreenstein

bengreenstein Dec 14, 2018

If the response is a 200 and we get the full image, or if the full image is contained in the 206, we use it and don't fetch it again. I'm not sure how much we want to codify this logic into the spec.

We should reject a range that doesn't start at 0. We best effort decode one that does, even if it is shorter than the requested range as this a common case with tiny images. If we can see that the image is larger than the amount returned and the amount returned is less than the range requested, we still try to decode best effort. Again I'm not sure how much of this we want to include in the spec.

If the main fetch begins during the metadata fetch, we cancel the metadata fetch.

The main fetch should override the metadata fetch, but of course the network stack may use data cached from the metadata fetch to reduce the size of the main fetch.

source Outdated
<p>If the <code>img</code> element's <code data-x="attr-img-lazyload">lazyload</code> attribute is
not in the <span data-x="attr-lazyload-off-state">Off</span> state, the user agent may immediately
fetch metadata from the image header. To fetch image metadata, the user agent must run the
following steps:</p>

This comment has been minimized.

@jakearchibald

jakearchibald Jul 25, 2018

Collaborator

Should we be explicit that this mustn't happen if the primary fetch for the resource has begun?

This comment has been minimized.

@jakearchibald

jakearchibald Jul 25, 2018

Collaborator

It probably shouldn't happen if the image is in the list of available images. It's a waste otherwise.

Is there any benefit to checking if the image is in the HTTP cache at this point?

source Outdated
<tr>
<td><dfn><code data-x="attr-lazyload-auto">auto</code></dfn>
<td><dfn data-x="attr-lazyload-auto-state">Auto</dfn>
<td>Indicates that the user agent may determine the fetching strategy (the default).

This comment has been minimized.

@jakearchibald

jakearchibald Jul 25, 2018

Collaborator

I think there are cases where we need to define 'auto'. Expectations are pretty strong when it comes to new Image(). Perhaps "off" is the default unless the image was created by the regular document parser (not innerHTML and createContextualFragment).

source Outdated

<p>A <dfn>lazy loading attribute</dfn> is an <span>enumerated attribute</span>. The following
table lists the keywords and states for the attribute &mdash; the keywords in the left column map
to the states in the cell in the second column on the same row as the keyword.</p>

This comment has been minimized.

@jakearchibald

jakearchibald Jul 25, 2018

Collaborator

What happens if this value is changed after element creation? Eg, if I switch from "on" to "off". Feels like the image should immediately load. Worth spelling out?

@Malvoz Malvoz referenced this pull request Aug 8, 2018

Open

Demo has issues #79

aarongable pushed a commit to chromium/chromium that referenced this pull request Aug 16, 2018

LazyLoad: Implement support for "lazyload" attribute on frames.
This CL implements support for the "lazyload" attribute on frames,
according to whatwg/html#3752, and as part of
the LazyLoad feature. The accepted values are:

"on", which causes the browser to lazily load a frame even if it's
same-origin or nested inside another lazyloaded frame,

"off", which causes the browser to avoid lazily loading this frame or
any of it's children (unless those children are marked with
lazyload="on"),

"auto", which activates the default behavior, and is therefore not
explicitly handled in code.

Bug: 873358
Change-Id: I2fde65adb15216260291b08e39888a2363f44d4a
Reviewed-on: https://chromium-review.googlesource.com/1176293
Commit-Queue: Scott Little <sclittle@chromium.org>
Reviewed-by: Kinuko Yasuda <kinuko@chromium.org>
Cr-Commit-Position: refs/heads/master@{#583841}

aarongable pushed a commit to chromium/chromium that referenced this pull request Sep 19, 2018

rajendrant Commit Bot
LazyLoad: Implement support for "lazyload" attribute on images
This CL implements support for the "lazyload" attribute on images,
according to whatwg/html#3752, and as part of
the LazyLoad feature. The accepted values are:

"off", which causes the browser to avoid lazily loading the <img> element

"on" and "auto", activate the default behavior of lazily load the <img> element

When the attribute is changed to "off", the deferred image loads immediately.

Bug: 875080
Change-Id: I839926a9827d019f23aafc40f8315476fe1b3048
Reviewed-on: https://chromium-review.googlesource.com/1197782
Reviewed-by: Hiroshige Hayashizaki <hiroshige@chromium.org>
Reviewed-by: Takashi Toyoshima <toyoshim@chromium.org>
Commit-Queue: rajendrant <rajendrant@chromium.org>
Cr-Commit-Position: refs/heads/master@{#592599}

@Malvoz Malvoz referenced this pull request Sep 25, 2018

Open

'importance' + 'lazyload' attributes #699

0 of 2 tasks complete
@sideshowbarker

I’d suggest that since the change to the CONTRIBUTING.md and README.md files arenn’t directly related to the substance of the rest of this patch (lazyload feature), it’d be preferable for the CONTRIBUTING.md and README.md changes to be in separate patches from this patch. It’d be fine if they are just a separate commits as part of PR #3752 but not sure if that would happen, given that the policy here for PRs is that they result in a single (squashed) commit that gets merged to master

@sideshowbarker

The changes to dev/style.css for adding the syntax-highlighter styles have already been merged into master, so they should be dropped from this patch.

@sideshowbarker

OK, I’m now realizing that this reason some unrelated changes are showing up in the diff here are that this patch includes some merges? So it seems like it needs to be rebased against master to clear those away?

@bengreenstein bengreenstein force-pushed the bengreenstein:lazyloading branch from d2af159 to da6a6f8 Oct 2, 2018

@annevk

This comment has been minimized.

Member

annevk commented Dec 11, 2018

@bengreenstein could you rebase this against master and then force push the resulting changeset to get rid of the merge commits? As it stands it looks like this is proposing a number of changes against dev/styles.css for instance, which doesn't seem correct.

Ben Greenstein and others added some commits May 31, 2018

Various nits and cleanups:
- Remove "lazyload hint" concept and refer instead to states of the attribute directly
- IDL attribute casing should be lazyLoad, not lazyload
- Be a bit more precise in the iframe processing model
- Remove domintro for reflecting attributes (both this one and the one added for img.decoding, erroneously)
@sideshowbarker

This comment has been minimized.

Member

sideshowbarker commented Dec 14, 2018

This branch is now in a state that it can’t be rebased against master without multiple merge conflicts.

@bengreenstein bengreenstein force-pushed the bengreenstein:lazyloading branch from b2cdf21 to c093e8c Dec 14, 2018

@bengreenstein

This comment has been minimized.

bengreenstein commented Dec 14, 2018

@sideshowbarker and @annevk Sorry about the state of the branch. Hopefully now it is all cleaned up.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment