Skip to content

HTTPS clone URL

Subversion checkout URL

You can clone with HTTPS or Subversion.

Download ZIP

Loading…

[on-hold] Explain how <img> fetching should work when it's a child of <picture> #5

Closed
yoavweiss opened this Issue · 33 comments

8 participants

@yoavweiss
Owner

The <img> tag fallback seem to have some inherent issues that result in fetching of the srcattribute value, even though it is not displayed.
I've looked around the WebCore code, and the issue seems to be inherent to <img>'s fetching mechanisms. Basically, fetching starts at the moment that srcis added. This happens before the node is added to DOM, so at that phase, the node is not aware that it will be inside <picture>and should avoid fetching.
I could not find other examples for an <img> fallback that actually works. <video> fetches internal <img>, and I couldn't find any mentions in the <video> spec of an <img> fallback.
If anyone knows of such a fallback use of <img> that works, please comment.
Otherwise, we may need to re-think the fallback <img> mechanism.

@bjankord

Can you look at how the noscript element handles nested img elements and its fetching mechanisms. The image src of an img element inside a noscript element is never fetched unless JavaScript is disabled. It seems we would want something similar for the fallback img src inside of the picture element if picture is natively supported by the browser.

@yoavweiss
Owner
@yoavweiss
Owner

It looks like that <noscript> (and some other elements) define all the content inside them as raw text, and therefore avoid loading of <img> resources. I don't think we can do that, since it means that the <source> elements will not be parsed as part of the DOM.
I think we will be better off if we'd switch to an attribute based fallback mechanism (e.g. a do-no-load attribute on the <img> tag)

@nwtn
Collaborator

Out of curiosity, what are the other elements that define the content as raw text? If one of them is semantically suitable, we could enclose the fallback <img> in it. E.g.

<picture ...>
    <source ...>
    <source ...>
    <fallbackelement><img ...></fallbackelement>
</picture>
@bjankord

Very interesting findings @yoavweiss. Like @nwtn, I'm curious what other elements define the content inside of them as raw text.

@yoavweiss
Owner

Other tags that are interpreted as raw text are <noframes>, <style>, <xmp>, <iframe> & <noembed>.
None of them fits as a semantic fallback, nor can they function as such, because we want current browsers to treat the <img> regularly.
OTOH, we can add such a "raw text" semantic tag (e.g. <picturefallback>), and that would work. However, I feel that a donotload attribute is simpler, avoids adding another tag and will serve the same purpose.
I was able to get a donotload attribute to work and prevent loading of its <img> element. If such a fallback mechanism is something we want in the spec, I'll push my changes to the WebCore repo.

@Wilto
Owner

You guys know how gung-ho I am about avoiding redundant requests, but I don’t think we should break with the video spec on this one. I think we’ll be better off filing a bug with the Chromium team on this one—there’s no reason video should suffer from this issue as well.

@yoavweiss
Owner

The thing is that defining <img> as a fallback is deviating from the <video> spec. The <video> spec does not mention <img> as a special case, and in general has an active DOM, so <img>s are loaded and <script>s are run.
The <img> fallback for video was never an official technique.
I've discussed this issue with @darobin at TPAC. The cleanest approach according to him is defining the internal <picture> DOM as an inert DOM, so that it triggers no resource loading. The problem with that approach is that it is rather experimental. It was tried at the <template> element, and as far as I understood had some issues.
In any case, if bugs need to be filed, it is against the <video> spec, not against Chromium (or all other browsers) that simply follow the spec.

@nwtn
Collaborator

An idea just popped into my head. It may not be a good idea, but it's an idea.

What if the fallback did use the same structure as <video>, namely including the fallback content in an <embed>? I haven't checked, but I recall that browsers will load images from <embed> as well as <img>.

@marcoscaceres

@nwtn, maybe gist that up? Then we can feed it into the Chromium build?

@nwtn
Collaborator

https://gist.github.com/4062299

Tried it in Chrome/Mac and @yoavweiss's Chromium build for Mac, and got expected results. Two problems: you can't have an alt attr on <embed>, and at least in Chrome I couldn't right-click the image for a normal context-menu.

Will try other browsers later, or you guys can feel free to give it a whirl.

@nwtn
Collaborator

updated gist with options for different embed/object/img combinations. object/embed seems promising.

@yoavweiss
Owner

@nwtn - I'm not sure I follow you.
In my view, the fallback image mechanism needs to answer 2 criteria:
1. Image should be fetched and displayed in today's browsers
2. Image fetching should be easily avoided without changing the loading patterns of existing elements or significantly changing the HTML parser states/algorithms.

<object><embed></object> does not load in today's browsers, so it fails the #1 requirement. I can look into avoiding fetching for other options in the gist (<embed> or <object>), but I'm not sure it'd be easier than avoiding fetching for <img> (so they may fail the #2 requirement, just like <img>)

@nwtn
Collaborator

@yoavweiss I agree on the requirements. My point is that, at least in my limited tests, <object><embed></object> did load the fallback image in today's browsers. I haven't done super robust tests yet, but I did look in current versions of Chrome and Firefox for Mac, as well as IE8. In what browsers did you find it wasn't loading?

For req 2, when I loaded that gist in your Chromium build, using a <object><embed></object> fallback did not fetch the image, at least from what I could tell looking at the network and resources panes.

@yoavweiss
Owner

@nwtn You're right!
Actually no need for <object><embed></object> which is kinda weird syntactically, but either one of them is fine as an <img> replacement fallback that does not get fetched when it's inside <picture>.
The only thing bothering me is that I have no clue why this happens. It's certainly not something I aimed for, but it seems to work for <video> as well.
So I'm voting for <object type="image/" data=""></object> as the official fallback syntax.

@nwtn
Collaborator

Cool. I think the reason I was thinking <object><embed></object> is for better support from old IE, but to be honest I'm pretty fuzzy on that and haven't had time to figure out the details.

@marcoscaceres marcoscaceres closed this issue from a commit
scottjehl This refactors the proposed markup (slightly) and picturefill JS to b…
…roaden browser support. The included update to README.md explains the changes I'm proposing to the markup pattern (there no changes to elements themselves) to support browsers that formerly did not work, like iOS4, IE browsers versions 6-10, Android browsers 1.x and up. As a result, there aren't any known "unsupported" browsers, in that every browser tested so far at least receives a fallback image - for example, IE6-8 will only see picture source elements that do not use media queries, since it doesn't support them natively (media types are ok though). Fixes #5 and Fixes #8.
404c39d
@nwtn nwtn reopened this
@marcoscaceres

@nwtn can you clarify why you reopened this?

@nwtn
Collaborator

the commit that closed it wasn't actually related to this issue, and the issue is still relevant. although using <object> or <embed> is working hack to avoid extra resource downloads, we still need to address the actual problem of how <img> fetching should work when it's a child of <picture>

@marcoscaceres

@nwtn thanks for the clarification and very good points! I will update the title of the bug to match the problem better.

@marcoscaceres

changed title to "Explain how fetching should work when it's a child of "

@yoavweiss
Owner

A couple of options to resolve this fallback issue came up on various discussions on mail and IRC
between @darobin, @nwtn and myself:
1. Rendering the DOM inside picture inert, dynamically adding (at element creation time or shortly after) internal <source> elements to the element's shadow DOM.
2. Adding a fallback attribute to the img element that will signal the browser that this element's resources should start download only after this element as added to the DOM, if and only if its parent element is not in some sort of a black-list, that will contain picture and video at first. This is better than the previously proposed donotload attribute since it is extensible to serve as a fallback to future parent elements.

The fallback attribute could be combined with other attributes that are desired to control image loading. A non-exhaustive list should probably contain:

  • Defered loading
  • Lazy loading if image reaches viewport
  • Do not preload / low priority

@nwtn proposed the syntax priority="defered|lazyload|low|fallback" which IMO works.

Personally, I favor the attribute option, since it will be simpler to specify and implement. OTOH, it is not something we can define inside the picture spec, so it'd make the fallback dependant on an external spec, which is out of our control, and may not be in scope for anyone to work on.

Thoughts?

@nwtn
Collaborator

relevant discussion re priority, lazyload, deferal, etc: https://www.w3.org/Bugs/Public/show_bug.cgi?id=17842

@yoavweiss
Owner

Actually, reading through that thread, a single "low priority" attribute value that is guaranteed to not load images before DOMContentLoaded can cover all the use cases, perhaps besides the "in viewport" use case. (which is not our concern here)
For the "fallback" use case, if an image is "low priority", the browser can avoid immediately fetching it, wait until it is added to the DOM, and trigger fetching only after we know its parent is not picture,video,etc.

@yoavweiss
Owner

After further discussion in the above mentioned bug, the proposed defer attribute is likely to guaranty only that the element won't be loaded before it is added to the document. For our purpose as a fallback, that's good enough.
The question is, do we think that obliging devs to add a defer attribute is a huge burden? Personally, I don't think that is the case.
If we do decide not to rely of defer as a fallback, we need to further investigate the "inert DOM" route, but it'd take time...

@marcoscaceres

Having devs add defer seems ok to me. It would make it consistant with img.

@Velmont

@defer ftw! I think this is a good solution, in case you didn't already catch that ;-)

@yoavweiss
Owner

I think we can say that this issue is on hold, waiting for conclusion on the <img defer>/delay front. Changing the title.

@baamenabar
Collaborator

I know this is on hold, and maybe someone already thought of this and discarded already.

During the weekend there was a proposed svg fallback technique using . It seems to have has it's issues with iPhones, but the <image> element could be used as a fallback in this case. As @yoavweiss comments ResponsiveImagesCG/meta#21 (comment) the element only gets triggered at the tokenization stage.

<picture>
<source media="(min-width: 320px)" src="img/tiny.jpg">
    <source media="(min-width: 600px)" src="img/small.jpg">
    <source media="(min-width: 800px)" src="img/original.jpg">

    <!-- fallback if picture is not supported --> 
    <image src="img/original.jpg">
</picture>

How could I test if this is getting preloaded?

@baamenabar baamenabar referenced this issue in ResponsiveImagesCG/meta
Closed

Add a tool or reference to test browser preloader. #21

@yoavweiss
Owner

Unfortunately, the problem with the img fallback are much deeper than preloading.
The problem is that even if the preloader is not kicked off, the <img src> will get loaded during HTMLImageElement's creation. That is because the element is created before it is inserted into the DOM, and isn't aware of its parents.
Modifying that behavior means modifying the way img works, which will break tons of scripts out there. I have no solutions to this other than flagging 'this image is a fallback' to the parser.
OTOH, the lazyload attribute might fit the bill as a solution here, for supporting browsers.

@yoavweiss
Owner

In fact the lazyload attribute resolves this completely.
Both preloader and parser can avoid starting fetching of the resource when the lazyload attribute is there, devs clearly have no expectations of this resource to start loading before it enters the DOM, and after it enters the DOM, the browser can verify that the element is not a direct child of <picture>.

@Velmont

Indeed. It is the get-out-of-jail-free card. :) Opt-in for a new loading technique.

@yoavweiss
Owner

Resolved by #53. Closing .

@yoavweiss yoavweiss closed this
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Something went wrong with that request. Please try again.