Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use IntersectionObserver #263

Open
triblondon opened this issue May 1, 2016 · 26 comments

Comments

Projects
None yet
@triblondon
Copy link

commented May 1, 2016

This is a fantastic tool and really positive for the performance of the web. I'd love to see this using IntersectionObserver to compute the intersection of the viewport and the lazyload elements, which will avoid the need to use a scroll handler in browsers that offer support for IntersectionObserver.

I'm currently working on adding an IntersectionObserver polyfill to the polyfill service, which will provide IO support on all the browsers you support for lazySizes, so a nice-to-have would be a version of lazysizes that relies on IO with no onscroll fallback, on the basis that it can be polyfilled where needed.

@aFarkas

This comment has been minimized.

Copy link
Owner

commented May 1, 2016

I also thought about that and I also saw the polyfill already. My plan was to at least play with the API to have some know how and maybe give some input in the design process of this young API. But I have currently plenty of work and it seems that this won't change until end of August.

In general, I'm an extreme fan of new web platform features as also of performance features, but I have a simple rule: I only add a feature and relay on polyfills as soon as two different browser engines are supporting them. (It doesn't have to be a stable browser version though). (And yes: There are more, but also vague rules).

This is why I'm now preparing the switch over to requestIdleCallback. Which is already supported in Chromium and soon will be supported by Firefox. My tests are very promising although I hoped rIC would give me not only an idle time, but also a time, where the layout is guaranteed to not be dirty.

With other yes, but not now ;-).

Big question here: Is any other browser interested to implement this API soon?


But let's talk about the thing here:

I must say, that I have nearly no knowledge about the full capabilities of IntersectionObserver, but I noticed some parts.

Here are some parts about the polyfill:

  1. scroll/resize and mutationobserver
    The polyfill is only using the events above. And for scroll (a non bubbling event), it is using event propagation instead of event capturing. The later would also fire if a scrollable child in the container is scrolled, which can bring an element also into viewport. Is this how IntersectionObserver should work? I thought any change, that might bring an element into view should be used. If this is so, you can find a full list of events in lazysizes.
  2. throttle function
    This might be a very subjective view, but it helped me a lot. getBoundingClientRect can be an extreme fast method that can be called on hundreds of elements (if you have more you might want to use efficient script yielding) on a high frequency without causing jank. On the other hand it can be very slow and causes jank if it is called even only on one element, if you call it at the wrong time.

After I refactored my throttle method to: Do a normal throttle using setTimeout, then do a rAF and then call a timeout with 0. I had dramatic performance improvements.

A cleaned up version could look like this:

var throttle = function(fn){
    var running;
    var lastTime = 0;
    var run = function(){
        running = false;
        lastTime = Date.now();
        fn();
    };
    var afterRAF = function(){
        setTimeout(run);
    };
    var getRAF = function(){
        rAF(afterRAF);
    };

    return function(){
        if(running){
            return;
        }
        var delay = 125 - (Date.now() - lastTime);

        running =  true;

        if(delay < 9){
            delay = 9;
        }
        setTimeout(getRAF, delay);
    };
};

This has two interesting effects by scheduling 3 jobs instead of just one, the job takes longer if the browser is currently busy (mostly only in the loading phase), which results in slower frequency. The second thing, that seems to heavily improve things is the fact that I often get a position in the frame life cycle right after frame commit (i.e.: rAF + timeout of 0. Basically this is also the position, where rIC is scheduling tasks.) (The chance, that layout is dirty is much lower.)

If you are interested the upcoming throttle function can be seen here.

  1. Memory leaking observed elements
    I don't know how the native IntersectionObserver works, but I hope there is only a weak reference to the observed elements. However the polyfill holds a strong reference, which means if the element is removed from the document and the page author has no reference to it, the polyfill will still have one. This is a common problem of lazyloaders. For me this is currently the number one point why I can't use the current state of the polyfill.

About the IntersectionObserver spec and its use case:

lazysizes is different in the way, that I don't use one static expand (margin). It uses a dynamic one. If there are currently images in view loaded only other images in view are loaded. Images that are near the view are not downloaded. If network is idling the view is expanded to preload images, that are not in view.

This way you get two important things:

  1. images that are not in view are not in "competition" with in view images. They don't consume bandwidth and slow down in view image loading.
  2. On the other hand you can (And I think a good developer should) heavily preload not in view images, so that the user doesn't need to wait for images to load as soon as he scrolls through the page.

I have difficulties to think how I can implement this model with IntersectionObserver because of two things.

  1. It is not possible to change options on the fly.
  2. Assuming I would use two observers: It is not possible to pause and resume an observer.

The only thing I can think of, is to use one observer with the large expand option and than if the callback is called to sort these records for in view and near view images and then decide how to do things.


Use case:
I don't doubt, that this API improves developer convenience, battery lifetime and also performance for developers, which don't use the right tools to implement in view detection. (No or non efficient throttled event handlers for example.)

But I really doubt, that it would noticeable improve jank with good current implementations. Especially, I don't think, that it would dramatically improve performance compared to the upcoming version of lazysizes.

The way how scroll events do work now has dramatically changed in the last years. FF is just making the switch and now all browser have async scroll events. The actual visual scroll is now always decoupled from the scroll event itself.

This doesn't mean, that it is bad (I think it can be very good for battery lifetime), but I don't think it will produce wonders.

@triblondon

This comment has been minimized.

Copy link
Author

commented May 1, 2016

Wow, this is incredibly useful feedback for IO. I'll cross-post to the IO polyfill issue. I think to summarise the issues you raise which the polyfill should address:

  • References to observed elements held strongly may create a memory leak
  • Throttle should ideally use requestIdleCallback-like scheduling behaviour
  • Scroll events within scrollable elements that are descendents of the main document may not be being captured but have the potential to bring observed elements into the viewport.

The use case of prioritising image downloading based on distance from the visible viewport is interesting, and I believe it's not within scope of IO to help you with that, but seems not too hard to build on top. The data you would get from a callback would include the intersection ratio, and allow you to easily determine whether the object was in view or just within the margin, and prioritise accordingly.

One use case you may not have covered here is where lazySizes is included in an iframe (eg an ad) which is not scrollable, but which is scrolled into view as part of a parent document which does not use lazySizes. Do you currently have any way to determine whether the content is in the viewport? I'm aware that advertisers are currently using some pretty horrible hacks to instrument this kind of thing - mostly for analytics, but ideally we'd like them to use this kind of detection to improve performance and bandwidth usage.

@aFarkas

This comment has been minimized.

Copy link
Owner

commented May 1, 2016

The data you would get from a callback would include the intersection ratio, and allow you to easily determine whether the object was in view or just within the margin, and prioritise accordingly.

Yeah, just started to play with the API instead of looking into the docs and it will make fun to work on top of it. Can't wait for the second browser ;-).

.. Do you currently have any way to determine whether the content is in the viewport? ...

This is tough. No. But nearly any advertiser script looks like a collection of hacks written by a junior php guy, who was forced to write javascript.

@triblondon

This comment has been minimized.

Copy link
Author

commented May 1, 2016

I realise that the quality of ad code is notoriously poor, but surely that's a reason for us to try and help those developers improve it. Probably writing ad code is how many newly qualified web developers get started, and advertising is a primary use case for the web.

Anyway I thought it was worth bringing up that use case because it is the best example of how IO can add capabilities to lazySizes beyond perf improvements which obviously are always going to be arguable.

@yoavweiss

This comment has been minimized.

Copy link

commented May 2, 2016

Can't wait for the second browser ;-).

Seems like Mozilla are showing some intent to implement once previous work this would rely on is completed.

@ojanvafai

This comment has been minimized.

Copy link

commented Aug 3, 2016

@aFarkas your plan to have a larger margin and keep track of the images in that window seems reasonable to me. A thing you could do is to have a second IntersectionObserver with no margin and you only observe elements in that one that the first observer said were in the larger margin. That way you can efficiently tell if things are in the larger window and the smaller window.

Regarding code efficiency: The big performance win with using the native IntersectionObserver here would be avoiding both polling, MutationObservers and listening to all these events. That will come with a significant performance cost. IntersectionObserver performance will scale with the number of elements you're observing, but I expect that on the vast majority of pages it will be considerably more performant.

Whether you use the polyfill or not, I recommend that you use the native IntersectionObserver when available.

Regarding adoption: https://blog.mozilla.org/futurereleases/2016/07/20/reducing-adobe-flash-usage-in-firefox/ implies Firefox is aiming to ship this year. We've heard positive sentiment from other browser vendors, but not specific date targets.

1/2400 page loads in Chrome is using it now. I expect that number to grow quickly. https://www.chromestatus.com/metrics/feature/timeline/popularity/1368

@aFarkas

This comment has been minimized.

Copy link
Owner

commented Aug 4, 2016

I already have a began to write a version based on IntersectionObserver. In most cases it will make sense to use it, if you are already using a IntersectionObserver polyfill or if you target only browser, which do support IntersectionObserver.

Some notes:

The big performance win with using the native IntersectionObserver here would be avoiding both polling, MutationObservers and listening to all these events.

IntersectionObserver won't help to ditch a MutationObserver because you will need to add new elements to the IntersectionObserver.

I expect that on the vast majority of pages it will be considerably more performant.

Lazysizes without IntersectionObserver already works with 60fps on the vast majority of pages. This can't be beaten by an even more performant implementation.

@ojanvafai

This comment has been minimized.

Copy link

commented Aug 4, 2016

I already have a began to write a version based on IntersectionObserver. In most cases it will make sense to use it, if you are already using a IntersectionObserver polyfill or if you target only browser, which do support IntersectionObserver.

\o/

IntersectionObserver won't help to ditch a MutationObserver because you will need to add new elements to the IntersectionObserver.

Oh, I didn't realize it was intended to automatically observe things. You might want to consider a future iteration of this library where folks could opt to use custom elements and avoid the need for MutationObservers by using custom element callbacks, e.g. , .

I have a (very rough) first draft of something similar where you make a whole subtree lazy here https://github.com/ojanvafai/lazyelements/blob/master/lazy-elements.js. I'm definitely glad to see more lazy loading libraries emerging. This probably isn't the right forum to discuss this, but let me know if there are things you think browsers could expose to make any of this better or lighter-weight.

Lazysizes without IntersectionObserver already works with 60fps on the vast majority of pages. This can't be beaten by an even more performant implementation.

I didn't mean this as a criticism of your library. 60fps is a good minimum bar for libraries, but it's not the best judgement of library performance, especially given that most pages include many libraries that all need stay withing the frame budget when combined together.

If it's 60fps, but using 90% of the frame budget, then that means the page author has no headroom to write their own code. I'm not making any claims that this code is particularly slow, just that it could be faster and that the less of the frame budget you use, the more likely it will be that the page will actually hit 60fps with all their own code and other libraries mixed in.

@abarre

This comment has been minimized.

Copy link

commented Nov 17, 2016

Firefox ships the support of intersection observer in the version 52.
source : https://developer.mozilla.org/en-US/Firefox/Releases/52.

@aFarkas

This comment has been minimized.

Copy link
Owner

commented Nov 17, 2016

@abarre
I'm aware of this. I just added an intersectionobserver version, that you can use to test you app with. Note you will need IntersectionObserver and MutationObserver polyfill for browsers, that don't support these.

Don't expect a performance improvement though. The assumption by @ojanvafai that lazysizes would use "90% of the frame budget" is simply not true.

As long as lazysizes doesn't find any images to transform. It takes about 0.5-2ms to search for those elements throtteled by a combination of requestIdleCallback and setTimeout. (Maximum frequency 1 "search" per 125ms, but only if rIC is called.).

Which basically means, lazySizes sometimes uses 10% of the frame budget.
Unfortunately work of the IntersectionObserver is hidden from the Chrome timeline, so it is hard to compare it. I assume, that battery consumption is improved.

@abarre

This comment has been minimized.

Copy link

commented Nov 17, 2016

Thank you for this excellent feedback. I will see if it can replace our current lazyloader.

@ojanvafai

This comment has been minimized.

Copy link

commented Nov 24, 2016

@aFarkas I think you're misinterpreting my feedback as saying that your library is bad and that people shouldn't use it. I wasn't meaning to imply that at all. I'm really happy to see lazy loading libraries become more popular. I wish more of the web was built with lazy loading.

I wasn't saying that lazysizes would use 90% of the framebudget by itself. I was commenting that measuring performance using FPS is not the best method for generic libraries like this. Modern web pages regularly include multiple libraries like this that each use part of the budget and you quickly get to 90%. So, even if you're hitting 60fps, it's still worth improving performance. I see death by a dozen cuts on a lot of pages (mostly due to ad networks to be fair). :)

Your measurement of milliseconds is more meaningful. Were you testing on a mobile device? Unfortunately, the budget available to JS is somewhere closer to 6ms because the browser itself uses a big chunk of your 16ms per frame (this is highly dependent on the content though).

Regarding showing the work of IntersectionObserver, we should fix it to expose something in the timeline. Filed https://bugs.chromium.org/p/chromium/issues/detail?id=668332 to track that.

@scott-thrillist

This comment has been minimized.

Copy link

commented Jun 13, 2017

I'm seeing this issue still marked as open but notice the intersection observer js file in my lazysizes npm package. Is it being used in this plugin or not?

Edit- nevermind. I see what you're doing and just added this in my webpack.config for testing:

resolve.alias = {
    'lazysizes': path.resolve( __dirname, 'node_modules/lazysizes/src/lazysizes-intersection.js')
}
@thasmo

This comment has been minimized.

Copy link

commented Jul 19, 2017

@aFarkas Just FYI. Blink browsers (Chrome, Android, Opera), Edge 15+ have support for intersection-observer and Firefox 55 (release date on 8th August) will enable it too.

@t-kelly

This comment has been minimized.

Copy link

commented Oct 2, 2017

Any updates on this? Looks like you have some competition now https://github.com/ApoorvSaxena/lozad.js.

@aFarkas

This comment has been minimized.

Copy link
Owner

commented Oct 2, 2017

@t-kelly
I already have an IntersectionObserver version of lazysizes included in this repo. You can use it with the polyfill. But be aware that the polyfill produces memory leaks and is by fare slower than the normal lazysizes version in such unimportant browsers like Safari for iOS.

It is also easy to get me to create a totally new IntersectionObserver version of lazysizes and promote it as the main script, if you can simply produce a simple test case that demonstrates that lazysizes can produce jank under some circumstances, while IntersectionObserver based versions doesn't.

Otherwise I will simply wait, until all latest browsers support this technology.

@thasmo

This comment has been minimized.

Copy link

commented Jan 30, 2018

@aFarkas What do you think about a progressively enhanced version which uses the current behavior by default but uses IntersectionObserver if available? Of course this would still need some kind of test-case to show potential advantages/benefits.

@caillou

This comment has been minimized.

Copy link

commented Mar 27, 2018

@aFarkas Given the following situation on a page:

  • Code that relies on intersection observer to trigger animations and track visibility of DOM nodes on scroll.
  • Images that rely on lazysizes and some of its plugins (bgset, progressive, object-fit, and respimg).
  • Need to support IE11.

Which of the following scenarios would you go for:

  1. Use the regular lazysizes alongside Intersection Observer Polyfill.
  2. Use lazysizes-intersection alongside Intersection Observer Polyfill.
@aFarkas

This comment has been minimized.

Copy link
Owner

commented Mar 27, 2018

@caillou
Tough questions. I would still take 1. because the plugins are currently mainly tested with the normal version of lazysizes and lazysizes-intersection doesn't give you much more performance.

@willstocks-tech

This comment has been minimized.

Copy link

commented Feb 20, 2019

Hey @aFarkas - hope you’re well!

Is the IntersectionObserver version still being developed or is this abandoned now?

I know IntersectionObserver is really picking up ground now, but I notice the file variant hasn’t been updated in a while. I assume it’s a lot of work to convert?

@aFarkas

This comment has been minimized.

Copy link
Owner

commented Feb 21, 2019

@willstocks-tech

Yes the IntersectionObserver version is still being developed. But I indeed concentrate more on the main file.

You have to understand that IntersectionObserver is really made for people who want to either write a simple and fast lazyloader with less than 50/100 lines of code or who simply do not understand how to get element layout information in a performant way. But as soon as you want to have more control and know what you are doing it is much better to use the low level APIs.

For example I added the feature to change the expand options afterwards. In the IntersectionObserver version I would need to change the rootMargin option unfortunately that's not possible rootMargin becomes a readOnly property. Similar there is no direct control included for visibility: hidden or opacity: 0 elements.

At the end: I currently think about writing a non-backwards compatible version of lazySizes. Only reason is about writing more modern code (so for current lazySizes users it shouldn't be that interesting its more for me as a maintainer a thing). This new version might not have IntersectionObserver version at all. While I had some perf problems in some very hard circumstances at the start of this project I have never seen a perf problem that can be solved with IntersectionObserver. I also bet that if you would create such a website IntersectionObserver would have actually more problems than my technique.

@willstocks-tech

This comment has been minimized.

Copy link

commented Feb 21, 2019

Thanks @aFarkas - that all makes complete sense! TL;DR = IntersectionObserver is good for “quick and simple use cases” but if you want fine-grain control, or complex configurability and low-level access, it’s not the right tool?

I’m still new to JS, so am still learning the ropes. I know IntersectionObserver is a “go-to” at the moment for a fair few things, especially with the new “v2 spec” - which includes support for transform, opacity, filter, etc. (Worth a read: https://developers.google.com/web/updates/2019/02/intersectionobserver-v2)

I wouldn’t know where to begin to even validate whether the performance trade off is worth the development effort (is there actually any performance benefit? I know IntersectionObserver is a bit of a buzzword at the moment, but I’m not sure why as it seems to replicate a lot of existing functionality - as you said, you’re already doing what it can do and more, potentially in a more performant way...), or whether it’s more supportable! It is all super interesting though!!!!

Side note: love lazysizes - you do an amazing job, thank you 😊

@serulux

This comment has been minimized.

Copy link

commented Jun 13, 2019

Hello.

Google's documentation says that this library uses IntersectionObserver API (which is the method recommended by them for lazy loading). But I see there is a separate version of the library for that. Is still correct that if I want to use IntersectionObserver API I need to use that separate version (lazysizes-intersection.js) instead of the main one? Would that separate version include the polyfills as well?

Thanks a lot.

@aFarkas

This comment has been minimized.

Copy link
Owner

commented Jun 13, 2019

Yeah you can use intersection observer version if you want. But you won't see any noticeable perf improvements and of course have to add a polyfill (the official one has some obvious memory leak issues). Please understand that some tech people like to exaggerate especially some perf things to push new technologies.

Additionally it drives me crazy that developers - people who should be able to understand some tec information - blindly follow stupid advices only because the source is google or what ever. I could point you to some really crazy stupid stuff on google's web fundamentals page written by so called google experts.

@serulux

This comment has been minimized.

Copy link

commented Jun 21, 2019

Hi,

Thanks for you answer. I've been testing the latest version of Lazysizes using Puppeteer without scroll events and it works well.

This all comes because quite a lot of Google's documentation stress the need of using the IO API and because they say in quite a few places that Lazysizes uses the API (without specifying that it's a different version). They're reviewing some documentation and they have already done some changes.

See more context here:

I hope you don't mind I have mentioned this thread.

Thanks for your work!

@aFarkas

This comment has been minimized.

Copy link
Owner

commented Jun 21, 2019

I've been testing the latest version of Lazysizes using Puppeteer without scroll events and it works well.

Yes this test is a good example. This artificial test is tailored in a way that lazy loaders with scroll events do not pass. Then they make the false claim that search bots work the same way. But this is not true. The fact that lazysizes (with and without IO) works with search bots has different reasons and in fact a normal IO based lazyloader would not work with search bots.

They're reviewing some documentation and they have already done some changes.

This is something that really hurts me. I tried to get some corrected information into google's web fundamentals some years ago. And the currently wrong information are pretty laughable if you think a little bit deeper about them. But as it turns out the ego of the author of these articles was too big to acknowledge any mistake. So downgrading the recommendation of a third party library is easy but correcting wrong information and acknowledging mistakes that a lot of developers are blindly following is quite hard. To be honest it's hard for me to overcome this double standard.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.