[on-hold] Explain how <img> fetching should work when it's a child of <picture> #5

Closed
yoavweiss opened this Issue Nov 1, 2012 · 33 comments

Comments

Projects
None yet
8 participants
Owner

yoavweiss commented Nov 1, 2012

The <img> tag fallback seem to have some inherent issues that result in fetching of the srcattribute value, even though it is not displayed.
I've looked around the WebCore code, and the issue seems to be inherent to <img>'s fetching mechanisms. Basically, fetching starts at the moment that srcis added. This happens before the node is added to DOM, so at that phase, the node is not aware that it will be inside <picture>and should avoid fetching.
I could not find other examples for an <img> fallback that actually works. <video> fetches internal <img>, and I couldn't find any mentions in the <video> spec of an <img> fallback.
If anyone knows of such a fallback use of <img> that works, please comment.
Otherwise, we may need to re-think the fallback <img> mechanism.

bjankord commented Nov 6, 2012

Can you look at how the noscript element handles nested img elements and its fetching mechanisms. The image src of an img element inside a noscript element is never fetched unless JavaScript is disabled. It seems we would want something similar for the fallback img src inside of the picture element if picture is natively supported by the browser.

Owner

yoavweiss commented Nov 6, 2012

Good thinking! I'll look at how it's done for noscript.
On Nov 6, 2012 5:16 PM, "Brett Jankord" notifications@github.com wrote:

Can you look at how the noscript element handles nested img elements and
its fetching mechanisms. The image src of an img element inside a noscript
element is never fetched unless JavaScript is disabled. It seems we would
want something similar for the fallback img src inside of the picture
element if picture is natively supported by the browser.


Reply to this email directly or view it on GitHubhttps://github.com/ResponsiveImagesCG/picture-element/issues/5#issuecomment-10116479.

Owner

yoavweiss commented Nov 9, 2012

It looks like that <noscript> (and some other elements) define all the content inside them as raw text, and therefore avoid loading of <img> resources. I don't think we can do that, since it means that the <source> elements will not be parsed as part of the DOM.
I think we will be better off if we'd switch to an attribute based fallback mechanism (e.g. a do-no-load attribute on the <img> tag)

Member

nwtn commented Nov 9, 2012

Out of curiosity, what are the other elements that define the content as raw text? If one of them is semantically suitable, we could enclose the fallback <img> in it. E.g.

<picture ...>
    <source ...>
    <source ...>
    <fallbackelement><img ...></fallbackelement>
</picture>

bjankord commented Nov 9, 2012

Very interesting findings @yoavweiss. Like @nwtn, I'm curious what other elements define the content inside of them as raw text.

Owner

yoavweiss commented Nov 9, 2012

Other tags that are interpreted as raw text are <noframes>, <style>, <xmp>, <iframe> & <noembed>.
None of them fits as a semantic fallback, nor can they function as such, because we want current browsers to treat the <img> regularly.
OTOH, we can add such a "raw text" semantic tag (e.g. <picturefallback>), and that would work. However, I feel that a donotload attribute is simpler, avoids adding another tag and will serve the same purpose.
I was able to get a donotload attribute to work and prevent loading of its <img> element. If such a fallback mechanism is something we want in the spec, I'll push my changes to the WebCore repo.

Owner

Wilto commented Nov 10, 2012

You guys know how gung-ho I am about avoiding redundant requests, but I don’t think we should break with the video spec on this one. I think we’ll be better off filing a bug with the Chromium team on this one—there’s no reason video should suffer from this issue as well.

Owner

yoavweiss commented Nov 10, 2012

The thing is that defining <img> as a fallback is deviating from the <video> spec. The <video> spec does not mention <img> as a special case, and in general has an active DOM, so <img>s are loaded and <script>s are run.
The <img> fallback for video was never an official technique.
I've discussed this issue with @darobin at TPAC. The cleanest approach according to him is defining the internal <picture> DOM as an inert DOM, so that it triggers no resource loading. The problem with that approach is that it is rather experimental. It was tried at the <template> element, and as far as I understood had some issues.
In any case, if bugs need to be filed, it is against the <video> spec, not against Chromium (or all other browsers) that simply follow the spec.

Member

nwtn commented Nov 12, 2012

An idea just popped into my head. It may not be a good idea, but it's an idea.

What if the fallback did use the same structure as <video>, namely including the fallback content in an <embed>? I haven't checked, but I recall that browsers will load images from <embed> as well as <img>.

Owner

marcoscaceres commented Nov 12, 2012

@nwtn, maybe gist that up? Then we can feed it into the Chromium build?

Member

nwtn commented Nov 12, 2012

https://gist.github.com/4062299

Tried it in Chrome/Mac and @yoavweiss's Chromium build for Mac, and got expected results. Two problems: you can't have an alt attr on <embed>, and at least in Chrome I couldn't right-click the image for a normal context-menu.

Will try other browsers later, or you guys can feel free to give it a whirl.

Member

nwtn commented Nov 12, 2012

updated gist with options for different embed/object/img combinations. object/embed seems promising.

Owner

yoavweiss commented Nov 14, 2012

@nwtn - I'm not sure I follow you.
In my view, the fallback image mechanism needs to answer 2 criteria:

  1. Image should be fetched and displayed in today's browsers
  2. Image fetching should be easily avoided without changing the loading patterns of existing elements or significantly changing the HTML parser states/algorithms.

<object><embed></object> does not load in today's browsers, so it fails the #1 requirement. I can look into avoiding fetching for other options in the gist (<embed> or <object>), but I'm not sure it'd be easier than avoiding fetching for <img> (so they may fail the #2 requirement, just like <img>)

Member

nwtn commented Nov 14, 2012

@yoavweiss I agree on the requirements. My point is that, at least in my limited tests, <object><embed></object> did load the fallback image in today's browsers. I haven't done super robust tests yet, but I did look in current versions of Chrome and Firefox for Mac, as well as IE8. In what browsers did you find it wasn't loading?

For req 2, when I loaded that gist in your Chromium build, using a <object><embed></object> fallback did not fetch the image, at least from what I could tell looking at the network and resources panes.

Owner

yoavweiss commented Nov 14, 2012

@nwtn You're right!
Actually no need for <object><embed></object> which is kinda weird syntactically, but either one of them is fine as an <img> replacement fallback that does not get fetched when it's inside <picture>.
The only thing bothering me is that I have no clue why this happens. It's certainly not something I aimed for, but it seems to work for <video> as well.
So I'm voting for <object type="image/" data=""></object> as the official fallback syntax.

Member

nwtn commented Nov 14, 2012

Cool. I think the reason I was thinking <object><embed></object> is for better support from old IE, but to be honest I'm pretty fuzzy on that and haven't had time to figure out the details.

@nwtn nwtn reopened this Apr 2, 2013

Owner

marcoscaceres commented Apr 2, 2013

@nwtn can you clarify why you reopened this?

Member

nwtn commented Apr 2, 2013

the commit that closed it wasn't actually related to this issue, and the issue is still relevant. although using <object> or <embed> is working hack to avoid extra resource downloads, we still need to address the actual problem of how <img> fetching should work when it's a child of <picture>

Owner

marcoscaceres commented Apr 2, 2013

@nwtn thanks for the clarification and very good points! I will update the title of the bug to match the problem better.

Owner

marcoscaceres commented Apr 2, 2013

changed title to "Explain how fetching should work when it's a child of "

Owner

yoavweiss commented Apr 2, 2013

A couple of options to resolve this fallback issue came up on various discussions on mail and IRC
between @darobin, @nwtn and myself:

  1. Rendering the DOM inside picture inert, dynamically adding (at element creation time or shortly after) internal <source> elements to the element's shadow DOM.
  2. Adding a fallback attribute to the img element that will signal the browser that this element's resources should start download only after this element as added to the DOM, if and only if its parent element is not in some sort of a black-list, that will contain picture and video at first. This is better than the previously proposed donotload attribute since it is extensible to serve as a fallback to future parent elements.

The fallback attribute could be combined with other attributes that are desired to control image loading. A non-exhaustive list should probably contain:

  • Defered loading
  • Lazy loading if image reaches viewport
  • Do not preload / low priority

@nwtn proposed the syntax priority="defered|lazyload|low|fallback" which IMO works.

Personally, I favor the attribute option, since it will be simpler to specify and implement. OTOH, it is not something we can define inside the picture spec, so it'd make the fallback dependant on an external spec, which is out of our control, and may not be in scope for anyone to work on.

Thoughts?

Member

nwtn commented Apr 3, 2013

relevant discussion re priority, lazyload, deferal, etc: https://www.w3.org/Bugs/Public/show_bug.cgi?id=17842

Owner

yoavweiss commented Apr 3, 2013

Actually, reading through that thread, a single "low priority" attribute value that is guaranteed to not load images before DOMContentLoaded can cover all the use cases, perhaps besides the "in viewport" use case. (which is not our concern here)
For the "fallback" use case, if an image is "low priority", the browser can avoid immediately fetching it, wait until it is added to the DOM, and trigger fetching only after we know its parent is not picture,video,etc.

Owner

yoavweiss commented Apr 7, 2013

After further discussion in the above mentioned bug, the proposed defer attribute is likely to guaranty only that the element won't be loaded before it is added to the document. For our purpose as a fallback, that's good enough.
The question is, do we think that obliging devs to add a defer attribute is a huge burden? Personally, I don't think that is the case.
If we do decide not to rely of defer as a fallback, we need to further investigate the "inert DOM" route, but it'd take time...

Owner

marcoscaceres commented Apr 7, 2013

Having devs add defer seems ok to me. It would make it consistant with img.

odinho commented Apr 7, 2013

@defer ftw! I think this is a good solution, in case you didn't already catch that ;-)

Owner

yoavweiss commented May 1, 2013

I think we can say that this issue is on hold, waiting for conclusion on the <img defer>/delay front. Changing the title.

Member

baamenabar commented Aug 20, 2013

I know this is on hold, and maybe someone already thought of this and discarded already.

During the weekend there was a proposed svg fallback technique using . It seems to have has it's issues with iPhones, but the <image> element could be used as a fallback in this case. As @yoavweiss comments ResponsiveImagesCG/meta#21 (comment) the element only gets triggered at the tokenization stage.

<picture>
<source media="(min-width: 320px)" src="img/tiny.jpg">
    <source media="(min-width: 600px)" src="img/small.jpg">
    <source media="(min-width: 800px)" src="img/original.jpg">

    <!-- fallback if picture is not supported --> 
    <image src="img/original.jpg">
</picture>

How could I test if this is getting preloaded?

@baamenabar baamenabar referenced this issue in ResponsiveImagesCG/meta Aug 20, 2013

Closed

Add a tool or reference to test browser preloader. #21

Owner

yoavweiss commented Aug 21, 2013

Unfortunately, the problem with the img fallback are much deeper than preloading.
The problem is that even if the preloader is not kicked off, the <img src> will get loaded during HTMLImageElement's creation. That is because the element is created before it is inserted into the DOM, and isn't aware of its parents.
Modifying that behavior means modifying the way img works, which will break tons of scripts out there. I have no solutions to this other than flagging 'this image is a fallback' to the parser.
OTOH, the lazyload attribute might fit the bill as a solution here, for supporting browsers.

Owner

yoavweiss commented Aug 22, 2013

In fact the lazyload attribute resolves this completely.
Both preloader and parser can avoid starting fetching of the resource when the lazyload attribute is there, devs clearly have no expectations of this resource to start loading before it enters the DOM, and after it enters the DOM, the browser can verify that the element is not a direct child of <picture>.

odinho commented Aug 22, 2013

Indeed. It is the get-out-of-jail-free card. :) Opt-in for a new loading technique.

Owner

yoavweiss commented Aug 26, 2013

Resolved by #53. Closing .

@yoavweiss yoavweiss closed this Aug 26, 2013

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment