Infinite scroll “shortcut” makes unnecessary and erroneous AJAX requests

[JosKrause]

Note: This issue is registered upon request from stackoverflow here

Note: I am not the writer of that stackoverflow question, however I am repeatedly running into this issue, so I am creating this issue to give it visibility and hopefully a quick resolve, thanks.

Running into a problem while trying to implement Waypoints infinite scroll example from http://imakewebthings.com/waypoints/shortcuts/infinite-scroll/.

Here is a JSFiddle to demonstrate my issue: http://jsfiddle.net/jmankin/75g6cap2/5/

HTML

    <div class="infinite-container">
        <div class="infinite-item">Not much content</div>
    </div>

    <a class="infinite-more-link" 
        href="/gh/get/response.html/jermifer/jsfiddle/tree/master/waypoints-infinite/"
    >Loading...</a>

JS

    var waypoint = new Waypoint.Infinite({
         element: $('div.infinite-container')[0]
    });

In instances where the 1st "infinite-more-link" is "above the fold" of the viewport on page load (i.e. the "inifinite-item" content is too short to require scrolling), the script correctly makes an AJAX call to the link href and loads the requested content.

However, it then prematurely--and seemingly incorrectly--proceeds to make the AJAX call to the 2nd "infinite-more-link" even though that is "below the fold" when it loads.

Secondly, from then on, scrolling to the bottom of the page (what would technically now be the 2nd "infinite-item" content element) will cause an AJAX call to the originally requested URL (the one that the client explicitly addressed), which is completely baffling. Under normal circumstances, it does this over and over again. In jsFiddle, it just does it the once, but that still gives you an idea of what I mean.

(Note: I'm not able to know ahead of time the length of the content I'd be loading, which is why I can't guarantee that the user will have to scroll down to see the 1st "infinite-more-link.")

[imakewebthings]

@JosKrause Just letting you know that I see the behavior you're describing and will try to track down a cause when I can. If you learn anything new yourself in the meantime, please let me know.

[mattymess]

I'm having the same problem. I tweaked the infinite shortcut a bit to fit my specific use case, but every now and again, multiple requests will be made immediately, one after the other. It's tricky to debug and haven't figured out the issue yet. Did either of you make any headway?

[ProGFB]

Here's my code. I tested with the original against mine. Original code starts duplicating loaded content. Especially when you've scrolled alot and do a refresh in FF. FF wants to scroll to the same position and thus loads waypoints wildly. My script has no problems with that and continuing to scroll works as expected. I tested with two Strips of content. Each strip was only 200px high so that many loads can happen since many strips fit on the page simultaneously, all darning an infinite-more link.
TEST LINK: http://www.berglust.at/website/static/work/index3.php
'''
/* Private */
Infinite.prototype.setupHandler = function() {

    this.options.handler = $.proxy(function() {

        this.destroy()

        var loadURI = $(this.options.more).attr('href'),
            infiniteTrigger;

        /*console.log('URL:', loadURI )*/

        if (loadURI === undefined) {
            return
        }



        this.options.onBeforePageLoad()

        this.$container.addClass(this.options.loadingClass)

        if (this.$more) {
            infiniteTrigger = this.$more.remove();
        }

        $.get(loadURI, $.proxy(function(data) {
            var $data = $($.parseHTML(data), true)
            var $newMore = $data.find(this.options.more)

            var $items = $data.find(this.options.items)
            if (!$items.length) {
                $items = $data.filter(this.options.items)
            }

            this.$container.append($items)
            this.$container.removeClass(this.options.loadingClass)

            if (!$newMore.length) {
                $newMore = $data.filter(this.options.more)
            }
            if ($newMore.length) {
                //this.$more.replaceWith($newMore)
                this.$more = infiniteTrigger;
                $(this.options.element).after(this.$more)
                this.$more.attr('href', $newMore.attr('href'))
                this.waypoint = new Waypoint(this.options)
            } else {
                // this.$more.remove();
                this.destroy();
            }

            this.options.onAfterPageLoad($items)
        }, this))


    }, this)
}

'''

[CHH]

@ProGFB thanks for posting this. But somehow your changes make it so that it loads the second page, but then it won't load the third page and it keeps my more link visible. No errors in the dev console.

[ProGFB]

do you have a test link? Maybe your more-link in the loaded content isn't correctly placed.

[CHH]

@ProGFB I'll check that

[CHH]

@ProGFB it looks like the issue with your script is this line:

$(this.options.element).after(this.$more)

I'm using bootstrap navigation and it moves my next link out of the <li> which has the class I'm using to target the next link.

Markup:

<ul class="pagination js-infinite-scroll-enabled">
<li class="prev disabled"><span>← Previous</span></li>
<li class="active"><span>1 <span class="sr-only">(current)</span></span></li>
<li><a href="/app_dev.php/de/blog/?page=2">2</a></li>
<li><a href="/app_dev.php/de/blog/?page=3">3</a></li>
<li><a href="/app_dev.php/de/blog/?page=4">4</a></li>
<li><a href="/app_dev.php/de/blog/?page=5">5</a></li>
<li><a href="/app_dev.php/de/blog/?page=6">6</a></li>
<li class="next"><a href="/app_dev.php/blog/?page=2">Next →</a></li>
</ul>

After the first request it looks roughly like this:

<a href="/app_dev.php/blog/?page=3">Next →</a>
<ul class="pagination js-infinite-scroll-enabled">
<li class="prev disabled"><span>← Previous</span></li>
<li><span>1 <span class="sr-only">(current)</span></span></li>
<li><a href="/app_dev.php/de/blog/?page=2">2</a></li>
<li><a href="/app_dev.php/de/blog/?page=3">3</a></li>
<li><a href="/app_dev.php/de/blog/?page=4">4</a></li>
<li><a href="/app_dev.php/de/blog/?page=5">5</a></li>
<li><a href="/app_dev.php/de/blog/?page=6">6</a></li>
<li class="next"></li>
</ul>

new Waypoint.Infinite({
                element: $(".item-container")[0],
                more: ".item-wrapper > .pagination > .next > a",
                items: ".item-container > .item",
                onBeforePageLoad: function () {
                    $(".item-wrapper > .infinite-loader").show()
                },
                onBeforeInsertItems: function ($items) {
                    $items.css({opacity: 0, transform: 'scale(0.001)'})
                },
                onAfterPageLoad: function ($items) {
                    $(".item-wrapper > .infinite-loader").hide()
                    $i.imagesLoaded(function () {
                        $i.isotope("appended", $items)
                    })
                },
            })

[mathiasarens]

I am having a similar problem. If I zoom my browser window to 50% two additional XHR request with the exact same url are issued on the first page load. I would assume that one is enough. When I start scrolling then three XHR requests are send one first scroll triggger and seven on the second scroll trigger.

Here are the callstacks from Chrome's network tab of the first two XHR requests. The second request might be more interesting because it is superfluous I would guess. (jquery is 2.1.4, js files are uncompressed):

The callstack of the first XHR

jQuery.ajaxTransport.send   @   jquery.js:8630
jQuery.extend.ajax  @   jquery.js:8166
jQuery.each.jQuery.(anonymous function) @   jquery.js:8311
(anonymous function)    @   jquery.waypoints-infinite-3.1.1.js:36
jQuery.extend.proxy.proxy   @   jquery.js:512
Waypoint.trigger    @   jquery.waypoints-3.1.1.js:59
Group.flushTriggers @   jquery.waypoints-3.1.1.js:516
Context.refresh @   jquery.waypoints-3.1.1.js:425
Context.add @   jquery.waypoints-3.1.1.js:208
Waypoint    @   jquery.waypoints-3.1.1.js:43
Infinite    @   jquery.waypoints-infinite-3.1.1.js:25
(anonymous function)    @   productVariations.js:4 -> $(function() {
    new Waypoint.Infinite({
        element: $(".category-page")[0]
    });
});
jQuery.Callbacks.fire   @   jquery.js:3099
jQuery.Callbacks.self.fireWith  @   jquery.js:3211
jQuery.extend.ready @   jquery.js:3417
completed   @   jquery.js:3433

The callstack of the second XHR:

jQuery.ajaxTransport.send   @   jquery.js:8630
jQuery.extend.ajax  @   jquery.js:8166
jQuery.each.jQuery.(anonymous function) @   jquery.js:8311
(anonymous function)    @   jquery.waypoints-infinite-3.1.1.js:36
jQuery.extend.proxy.proxy   @   jquery.js:512
Waypoint.trigger    @   jquery.waypoints-3.1.1.js:59
Group.flushTriggers @   jquery.waypoints-3.1.1.js:516
Context.refresh @   jquery.waypoints-3.1.1.js:425
Context.refreshAll  @   jquery.waypoints-3.1.1.js:439
window.onload   @   jquery.waypoints-3.1.1.js:453

[ProGFB]

All I can contribute is that my changes have fixed the problems for me here: http://berglust.at/de/start
Unless you find issues that I haven't. Also, the code will be concatenated for this website in the near future.

[vjong]

I had the same issue and expanded the Waypoint.Infinite class as follows:

var infinite = new Waypoint.Infinite({
element: $('.infinite-container')[0],
onBeforeInsertItems: function() {
$('.infinite-more-link').remove();
}
})

This way, the link to the previous content is removed before adding the new url, meaning it can't be called anymore. Solved the problem for me for most cases.

I did have one page where I still had the issue, but when I increased the fetch size of the query it was gone.

[besworks]

I came across this bizarre issue while working for a client today. This is how I solved it. Relevant changes are commented.

  Infinite.prototype.setupHandler = function() {
    this.options.handler = $.proxy(function() {

      // The handler seemed to fire whenever it felt like it...
      // The onBeforePageLoad call would always happen whether there was any new content or not.
      // I added this check to see if there was actually any more content before doing anything.
      if (!this.$more) { return; }

      this.options.onBeforePageLoad();
      this.destroy();
      this.$container.addClass(this.options.loadingClass);

      $.get($(this.options.more).attr('href'), $.proxy(function(data) {
        var $data = $($.parseHTML(data, document, true));

        var $newMore = $data.find(this.options.more);

        this.$more.remove(); // always remove the existing more link before appending the new content
        this.$more = ($newMore.length) ? $newMore : null; // explicitly set the $more property to null if $newMore has a no length

        var $items = $data.find(this.options.items);
        if (!$items.length) {
          $items = $data.filter(this.options.items);
        }

        this.$container.append($items);
        this.$container.removeClass(this.options.loadingClass);

        // Replaced the original $newMore checks with this more succinct version
        if (this.$more) {
          this.waypoint = new Waypoint(this.options);
        }

        this.options.onAfterPageLoad();
      }, this))
    }, this)
  }

If you've ended up at this page as baffled as I was when I first saw this behavior then I hope this helps to save you some time.

[giovannipds]

Hello!

I've just tested quickly what @vjong said and later on @besworks' solution, and neither of them seemed to work for me.

@vjong's solution have kept the problem and @besworks have loaded just the first next page, disconsidering the others.

I'm still looking for a solution to the problem. Thanks for the contribution anyways, guys. Regards.

[ProGFB]

My waypoint solution is still working strong:
http://www.berglust.at/de/regioninfo_nationalpark-hohe-tauern

[giovannipds]

Hello!

I have done some testing this morning, and I may have found a way of fixing the bug. It may be not the must clever way, but it has worked.

When I was trying to check this.$more and this.$more.length it seemed that the object was still existing, but just wasn't on the DOM. So, based on this answer, about checking if the element's on the DOM, I've done this, before the line 32 of the Infinity.js:

...

  /* Private */
  Infinite.prototype.setupHandler = function() {
    this.options.handler = $.proxy(function() {

      // console.log('setupHandler is called... this.$more[0] = ');
      // console.log(this.$more[0]);
      // console.log('----------');
      if ( ! $.contains(document, this.$more[0])) {
        // console.log('Element is detached');
        this.destroy();
        return;
      }

      ...

It seemed to have solved the problem. The way I was testing, was, being in the middle of the pages (after there were loaded asynchronously) and refreshing the page, on Google Chrome 44.0.2403.130 m.

Hope to help anyone to implement a bug fix to this code somehow. I'll also be opening a pull request if the author @imakewebthings want to merge it to the code.

Best regards!

[besworks]

I too found issues with my solution. I gave the suggestion from @giovannipds a try and it was better but I was getting duplicates from the last page of content appearing if I reached the end of the content and reloaded the page. I did some more digging and discovered that the Infinite.prototype.destroy method didn't seem to really be destroying anything... I have come up with a new solution that really seems to solve everything.

Starting with a fresh copy of https://github.com/imakewebthings/waypoints/blob/master/lib/shortcuts/infinite.js

change line 68 from:
this.waypoint.destroy()
to:
this.waypoint.context.destroy()

Voila! Everything now works as expected. You can see a working demo at http://dev.besworks.net/test/infinite/

[imakewebthings]

Thank you everyone for your patience and many investigations. I'm working on getting this sorted this weekend.

[giovannipds]

Hello Caleb (@imakewebthings)! Any updates on your progresses?

Didn't tried @besworks's tip yet, but seems to be a smarter solution. Thanks for the contribution. :)

Cheers!

[imakewebthings]

@giovannipds Yes, I tracked down the cause. @besworks' last comment was very close to the root of the problem. I'll try to explain the bug and how I intend to fix it.

When the Infinite class is instantiated, we set this.waypoint to a new Waypoint:

waypoints/lib/shortcuts/infinite.js

Line 25 in 13ae054

this.waypoint = new Waypoint(this.options)

When a Waypoint is initialized, it immediately checks to see if it has already been crossed and triggers the handler in the "down" direction. As everyone has noted, this is where the bug appears, when the page is already at a point where it needs to load new content when Infinite is initialized. So the handler runs and as part of the handler, call this.destroy:

waypoints/lib/shortcuts/infinite.js

Line 33 in 13ae054

this.destroy()

The problem with destroy at this stage is that this.waypoint is still undefined. So this check comes out false and the first page waypoint never gets destroyed:

waypoints/lib/shortcuts/infinite.js

Line 67 in 13ae054

if (this.waypoint) {

What held me up from committing a fix last weekend was deciding how best to tackle this, and I've reached a conclusion that will end up killing two birds with one stone. All of those initial trigger checks during Waypoint instantiation will be no longer be synchronous. They will be delayed until the next requestAnimationFrame. This should also solve #375 that I had written off as too complicated for minimal gain.

I anticipate this will break exactly zero real world uses of Waypoints but it is going to be a fundamental change to the behavior and will be a bump in major version to 4.0.0. Expect that this weekend.

[imakewebthings]

Please give this a try with 4.0.0, everyone.

[slifin]

I'm having this issue on 4.0.0
on page load it loads in the second page immediately and then if you scroll a tiny bit all begins to ajax in all results possible sequentially

--- edit
I don't know if my dom structure is incorrect or something, this is the first time I'm using this library

[td234]

I am having the same problem with 4.0.1. When the first page is called, it subsequently calls every other page till the end. Only occurs when I have and "offset" value. CAN be either integer or percent, positive or negative.

var infinite = new Waypoint.Infinite({ element: $('#list-col .list-col-content'), onAfterPageLoad: function(direction) { var last = $('.page-loaded:last'); last.waypoint(function(direction) { if(direction == 'down') { if(window[window.GoogleAnalyticsObject]) { ga('set', 'page', last.data('url')); ga('send', 'pageview'); console.log('pageview ' + last.data('url')); } //history.pushState('', last.data('title'), last.data('url')); } }, { continuous: false }) }, offset: -50 });